dataesr/scientific-paragraphs-categorization huggingface dataset:
Conclusions We propose a modular semantic model able to accommodate different dimensions required for representing EO data: time, space, observation, source metadata, and provenance, together with a semantic ETL process for populating this model. We illustrated our approach by the integration of land cover data of a specific French winery geographic area, its administrative units, and its land registers. The feasibility of the approach and the capacity of the system has been demonstrated in the H2020 EO Big Data Hackathon (http://www. candela-h2020. eu/content/joint-hackathon-organizedeo-2-2017-projects). During the event, participants could access the project platform to integrate new land cover datasets and then query the triplestore to retrieve details of the administrative units or the parcels with their land cover.
All the tasks are realized through python scripts. As future work, we will consider other available datasets, such as land use, vegetation index, and change detection. Following the data triplification process that we defined, additional information can be attached to parcels throughout their life so that they can be referred to in the analysis process. We will consider, as well, integrating other land cover datasets, such as the one from  [33]. Finally, another direction is to consider big data scenarios (using Natura 2000 data for instance) and to evaluate the scalability of the approach, including the possibility of considering hybrid architecture and storage strategies (with relational, NoSQL and RDF storage supports). :GFObservation/LC_CESBIO__330300000A0480_11_1483228800_1514678400 a tuom:GFObservation; sosa:hasSimpleResult "0.
1"^^xsd:float; sosa:observedProperty :GFObservableProperty/LC_CESBIO_11. :GFObservation/LC_CESBIO__330300000A0480_31_1483228800_1514678400 a tuom:GFObservation; sosa:hasSimpleResult "0. 9"^^xsd:float; sosa:observedProperty:GFObservableProperty/LC_CESBIO_31. :GFObservableProperty/LC_CESBIO_11 tuom:name "culture ete"^^xsd:string. :GFObservableProperty/LC_CESBIO_31 tuom:name "foret feuillus"^^xsd:string. #Land cover observations for 2018 :GFObservationCollection/LC_CESBIO18_330300000A0480_1514764800_1546214400 a tuom:GFObservationCollection; a dcat:Dataset; a prov-o:Entity; tuom:observedPropertyType :GFObservablePropertyType/LC_CESBIO18; prov-o:wasDerivedFrom :RasterFile/LC_CESBIO_OCS_2018_CESBIO_tile; sosa:hasFeatureOfInterest :ParcelVersion/VX2018_330300000A0480; sosa:phenomenonTime :Interval/1514764800_1546214400; sosa:hasMember :GFObservation/LC_CESBIO18_330300000A0480_10_1514764800_ 1546214400; sosa:hasMember :GFObservation/LC_CESBIO18_330300000A0480_16_1514764800_ 1546214400.
Administrative units: French administrative units can be obtained from different open datasets, in particular, OpenStreetMap-based dataset (https://www. data. gouv. fr/en/datasets/decoupage-administratif-communal-francais-issu-d-openstreetmap/) and GeoZones dataset (https://www. data. gouv. fr/en/datasets/geozones/). The latter provides more details about administrative units than first but it is updated less frequently. For that reason, we chose the OpenStreetMap-based dataset, available as shapefiles. Land registers: Land register data is available from the French government data website (https://cadastre. data. gouv. fr/datasets/cadastre-etalab) in GeoJSON format or shapefiles.
Operational taxonomic units (OTUs) are usually treated as if they are internally uniform in environmental metabarcoding studies of microbial and macrobial eukaryotes, even when the OTUs are being used to infer biogeographic patterns. The OTUs constructed by the program Swarm have underlying network topologies in which nodes represent amplicons and edges represent 1 nucleotide differences between nodes. Such networks can be exploited to search for biogeographic patterns within each OTU. To do this, here we used an available protistan metabarcoding dataset consisting of the hypervariable V4 region of the 18S rRNA locus amplified from 77 lakes collected across Norway and Sweden. The 82 most abundant and wide-spread OTUs constructed by Swarm were evaluated using shortest path, assortativity, and geographical analyses.
g. , a minimum sequence similarity). These networks provide a first structuring of the data by partitioning it, since the continuity and discontinuity of resemblances between sequences as guided by chosen parameters generally produces distinct subgraphs, called connected components. Connected components display a variety of mathematical and topological properties that can be exploited in comparative investigations such as in: shortest path analyses, where the minimum distance in the network between nodes that share the same attribute (e. g. , geographic origin) is recorded; assortativity analyses, where the tendency of nodes that share the same attribute to be connected with each other in the network is assessed; and what we are here calling geographic analyses, where the distance of the geographic origin of nodes connected in the network is compared.
Such network analyses can easily be applied to OTUs that were constructed by the program Swarm  (Frédéric Mahé, Rognes, Quince, de Vargas, & Dunthorn, 2014 , 2015). Rather than using a global clustering threshold, Swarm uses a local clustering threshold with an iterative single-linkage approach to construct OTUs. Resulting OTUs therefore do not have a pre-determined radius of genetic similarity (i. e. , they are not clustered at global similarity values such as 97% or 98%). OTUs constructed by Swarm have underlying network topologies that can be exported using the -i and -j options. Forster et al. (2016)  exploited these network topologies underlying Swarm OTUs, by developing an approach to discover novel genetic diversity in already observed OTUs when sampling new environments. The same study also suggested that these networks' underlying Swarm OTUs can be exploited to look at within-OTU variation in a geographical context.
Underlying network topologies of Swarm OTUs structured by geographic variation may indicate allopatric speciation processes. Since, within such geographically structured Swarm OTUs, amplicons if the same geographic origin would be genetically more similar than amplicons of different geographic origin. One dataset that can be used to demonstrate  Forster et al. 's (2016)  suggested approach of evaluating within-OTU variation in a geographical context, is the planktonic protistan metabarcoding data from  Khomich et al. (2017). In their study,  Khomich et al. (2017)  originally sampled 77 homogeneous, low-productive boreal lakes across southern Norway and Sweden. DNA isolated from lake epilimnions were amplified with general eukaryotic primers amplifying the hypervariable-V4 region of the 18S-rRNA locus.
Data acquisition Epilimnic water samples were collected from selected lakes as previously described  (Khomich et al. , 2017). In brief, for each lake, 15 L of surface water (0-5 m) was filtered through a 100 µm filter and cells were accumulated on nine filters of 2 µm mesh size. Seventy-seven lakes were sampled and analyzed including: Gjersjøen, Halsjøen, Klämmingen, Milsjön, Näsrämmen, Saebufjorden, Skattungen, Storsjön, Torrsjøn, Vatsvatnet, Vinjevatn, Visten, Vostervatnet (Figure  S1 ). DNA was extracted from one to two filters of each lake and the V4 hypervariable region of the 18S rRNA was amplified with general eukaryotic primers  (Stoeck et al. , 2010). Raw sequencing data were obtained by 454 Roche pyrosequencing (GS FLX Titanium) and corresponding mapping files were accessed via Dryad (doi:10.
Clustering and OTU choice Raw reads were fully reprocessed through an automatic bioinformatic pipeline for HPC (DeltaMP v0. 4, https://github. com/lentendu/DeltaMP). The full bioinformatic procedure can be reproduced using the provided configuration file (File S1). Raw reads were demultiplexed by allowing a maximum of 2 mismatches on the 12 nucleotides long barcode sequences using QIIMEv1. 9. 1  (Caporaso et al. , 2010). Individual. sff files were converted into fasta and quality files. Primers were detected and removed from both ends using Cutadapt v1. 18  (Martin, 2011)  with an overlap of at least 2/3 of the primer sequence length and allowing up to 6 mismatches on the primer sequences. Only reads containing both primers were kept and reads starting with the reverse primer were reverse-complemented.
Reads were filtered using MOTHUR v1. 43. 0  (Schloss et al. , 2009). Reads shorter than 300 nucleotides and longer than 550 nucleotides, with any ambiguous base calls, with homopolymers longer than 8 nucleotides, and with an average Phred score below 20 were discarded. All high-quality filtered reads were dereplicated across the entire dataset to create amplicons, sorted by decreasing abundance, then clustered into operational taxonomic units (OTUs) using the fastidious option -f and -d 1 in Swarm v3. 0. 0  (Frédéric Mahé et al. , 2014 , 2015). Additionally, Swarm's -i and -j options were enabled to export the internal structure and the network of each OTU. The seed sequence of each Swarm OTU was used for taxonomic assignment against the PR2 reference database  (Guillou et al. , 2013 ) using VSEARCH v2.
13. 6 (Rognes, Flouri, Nichols, Quince, & Mahé, 2016). Chimeras were detected using VSEARCH and subsequently removed. The full OTU table (Dataset S1), the OTUs internal structures (Dataset S2), the OTUs network file (Dataset S3) and the dereplicated amplicon count table (Dataset S4) are available in the supplements. All downstream analyses were conducted in R v3. 5. 1 (R Core Team, 2018; File S1). Ten of the lakes were sequenced more than once; these technical replicates were pooled by lake. OTUs occurring as singletons or doubletons on the global level were removed from downstream analysis (19. 9% of all reads; 95. 5% of all OTUs). Basic diversity analyses were conducted using the Vegan package  (Oksanen et al. , 2016). Amplicons assigned to nonprotist hits were excluded; e. g. Streptophyta, Metazoa, Fungi, and unidentified Opisthokonta.
Network analyses For each widespread protist OTU, an undirected network was constructed using Swarm's internal structure file (i. e. output of option -i), which outputs the minimal spanning tree of each OTU network structure as an input for the R package Igraph v1. 2. 2  (Csardi & Nepusz, 2006). To retrieve the full spanning tree of the internal OTU network structure, additional edges linking two amplicons with one nucleotide difference were retrieved from Swarm's network file (i. e. output of option -j) and added to the network. The grafted edges added by the Swarm fastidious option and corresponding to two nucleotide mismatches were removed in order to keep consistent networks with one nucleotide difference edges only. In this final network, each amplicon is equivalent to one node and each single-linkage between two amplicons is equivalent to one edge representing a single nucleotide difference between the amplicons.
This value reflects a general trend observed from all OTUs in the samples. However, since biogeographic patterns in the same habitats may be observed for one OTU but not for another, we also conducted for each lake one separate assortativity analysis within each of the 82 OTUs. Within each Swarm OTU (and for the complete network of all Swarm OTUs), we tested if the observed assortativity for amplicons of the same lake fell within a 95% confidence interval of assortativity values obtained from 1,000 randomized networks containing the same number of nodes and the same node degree. Third, the assortativity analyses were further conducted in a framework including geographic distances between lakes. In order to take the occurrence of a single amplicon in multiple lakes into account, the centroid geographic position of each amplicon was calculated as the geographic centroid of all lakes in which an amplicon occurs using the package Sp v1.
Widespread OTUs The bioinformatic approach used here produced a similar protist OTU richness (312 ± 107; Figure  S1 ) and rarefaction results (Figure  S2 ) as originally published  (Khomich et al. , 2017). Among the 2 669 protist OTUs (240 646 reads), the 82 most abundant and widespread OTUs covered 67. 9% of total reads (163 382). The widespread OTUs represented on average 20% of each lake's OTU richness (± 4. 1 sd; Figure  S1 ). The taxonomy of the widespread OTUs reflects the total taxonomic composition of the dataset, with the most abundant groups being Cryptophyta, Dinoflagellata, Ochrophyta and Ciliophora (Figure  S3 ).
Network analyses can evaluate within-OTU biogeographic variation Using shortest path analyses, we first asked if amplicons from the same lake are connected directly to each other within the OTUs. Our results showed that the majority of amplicons in the metabarcoding dataset from the Norwegian and Swedish lakes, had indeed at least one neighboring amplicon from the same lake to which they were directly linked in the Swarm OTU networks. Analyses of confidence intervals confirmed some biogeographic structuring of the data, since the mean shortest path distance between two amplicons from the same lake was smaller than expected by chance. Despite this result, shortest path analyses here still emerged as only an indirect measure of biogeographic structuring within the OTUs. Though the results indicate that most amplicons of the same lake occurred at least in pairs (i.
The assortativity values observed in  Fondi et al. (2016)  on a global scale were also higher than those observed by our study on Norwegian and Swedish lakes. Both mentioned studies interpreted the assortative sequence groupings in their dataset as a result of adaptive evolutionary processes caused by environmental factors (habitat and niche affiliation, respectively). If assortative groupings of amplicons by geographic regions could be observed within an OTU, these observations may also be strong indications of evolutionary processes. One Swarm OTU resembles very fine-scaled genetic variation, even on an intraspecific level (D. Forster et al. , 2019). Our strategy aimed at testing if this fine-scaled genetic variation is non-randomly distributed but structured by geographic origin. Transferred to an evolutionary framework, one may detect allopatric speciation processes in which intraspecific subpopulations in different geographic regions are characterized by distinct haplotypes.
Even though few assortative values were obtained for some OTUs, these do not contradict the lack of biogeographic variation in the dataset. Instead the values corroborate and expand the results of the shortest path analyses: there may be scattered pairs or small groups of amplicons from the same lake that are linked within some OTUs, but there are no larger populations that dominated specific areas within any OTU. From the results of all types of network analyses we conclude that biogeographic distribution is not an underlying structuring factor of genetic diversity within the networks of the most abundant and widespread protist OTUs in lakes of southern Norway and Sweden. The lack of biogeographic variation recovered by the network analyses approach does not mean that there was generally no biogeographic variation in the dataset.
Wider applicability of within-OTU network analyses While the application of assortativity analyses for inferring biogeographic variation in metabarcoding datasets has been successfully demonstrated before  (Fondi et al. , 2016; D. Forster et al. , 2015) , our workflow is the first combination of assortativity analyses within-OTU networks created by Swarm. Our application is also the first to exploit Swarm's feature of fine-grained single-nucleotide differences for clustering instead of comparably coarsegrained clustering via fixed sequence similarity thresholds. The test for biogeographic variation presented here is a statistical expansion to evaluate Swarm clustering results. Commands for reproducing this step are available at https://github. com/lentendu/DeltaMP and do not demand additional pre-or post-clustering data treatment.
Order-constrained HAC. A simple and useful case of contiguity constraint is the case when the symmetric relation is a contiguity relation defined along a line. This special case of constraint is frequently encountered in genomics (where the contiguity relation is deduced from genomic positions along a given chromosome) and will be called order-constrained HAC (OCHAC) in the sequel. In this context, every cluster has exactly two neighbors (except for the two positioned at the beginning and the end of the line) and at step t of the algorithm, only n -t values of the linkage have to be computed (instead of (n -t)(n -t -1)/2 for standard HAC). This case is the one implemented in the R package adjclust and an efficient algorithm is described in  Ambroise et al. (2019)  for sparse datasets. In this specific case, constrained HAC can be seen as a heuristic to approximate the search of an optimal segmentation (i.
,. , on E. Under this assumption, the dissimilarity case is a simple extension of the original R p framework described in Section 2. Different versions of necessary and sufficient conditions for which an observed dissimilarity matrix is Euclidean have been obtained in  Schoenberg (1935) ;  Young and Householder (1938) ;  Krislock and Wolkowicz (2012). When such conditions do not hold, D is simply called a dissimilarity dataset, which is a particular case of proximity or relational datasets. Schleif and Tino (2015)  have proposed a typology of such datasets and described different approaches that can be used to extend statistical or learning methods defined for Euclidean data to such proximity data. In brief, the first main strategy consists in finding a way to turn a non-Euclidean dissimilarity into an Euclidean distance, that is the closest (in some sense) to the original dissimilarity.
Extension to similarity data Similarity data also aim at describing pairwise resemblance relations between the objects of Ω through a matrix of similarity (or proximity) measures S = (s ij ) 1≤i,j≤n. Even though the precise definition of a similarity matrix can differ within the literature (see e. g. ,  Hartigan (1967) ), it is generally far less constrained than kernel matrices. In most cases, the only conditions required to define a similarity is the symmetry of the matrix S 2  and the positivity of its diagonal. Since both similarities and kernels describe resemblance relations, it seems natural to try to extend the background of Section 3. 2 to similarity datasets by using Equation (  10 ). This allows the definition of a linkage, δ S , between clusters via sums of elements of S. However, this heuristic is not well justified since the quantity s ii + s jj -2s ij is not necessarily non-negative when S is not a positive definite kernel.
Monotonicity of Ward's linkage Ward's linkage corresponds to the variation of within-cluster inertia, so that the monotonicity of the linkage is ensured for Ward's standard HAC algorithm with Euclidean data. More generally,  Batagelj (1981)  gives necessary and sufficient conditions based only on the Lance-Williams coefficients that ensures monotonicity for a given linkage. These results apply to the extensions of HAC to non-Euclidean datasets and show that the monotonicity of the linkage values is always ensured for standard HAC with Ward's linkage. In addition,  Ferligoj and Batagelj (1982)  give necessary and sufficient conditions on the Lance-Williams coefficients to ensure the monotonicity of the linkage values in constrained HAC, for an arbitrary symmetric relational constraint. These conditions are not fulfilled for Ward's linkage.
One may then expect that the total within-cluster inertia at a given step of the algorithm is larger than for the unconstrained version that chooses the "optimal" merger at this step (that is, the merger with the smallest increase of the total within-inertia). In addition, the algorithm does not necessarily exhibit a clear and understandable monotonic evolution of the objective criterion, (m t ) t. However, it can be shown, even in a very simple example, that OCHAC can lead to better solutions in terms of within-cluser intertia, when the constraint is consistent to the spatial structure of the data. This fact is illustrated in Figure  2   3. In this example, 7 data points are displayed in R 2 with an order constraint illustrated by a line linking two points allowed to be merged. In this situation, (m t ) t is indeed non monotonic for OCHAC (bottom left figure) but leads to a better total withincluster inertia for k = 3 clusters (vertical green line), which is also more relevant for the data configuration (top figures).
Simulation HAC can be seen as a greedy algorithm to solve the problem of finding the partition with minimal within-cluster inertia ESS t of n objects into n -t classes, for each t = 1. n -1. It may be expected that the inertia of the partitions will be lower for HAC than OCHAC, since the possible mergers in OCHAC are chosen among a subset of the possible mergers in HAC. Can we quantify the impact of the order constraint on the quality of the partitions (as measured by ESS) obtained for HAC and OCHAC, depending on the strength of the actual order structure in the data? In this section, we address this question by analyzing Hi-C data  (Dixon et al. , 2012) , which present a strong order structure, as illustrated by Figure  5. We use a perturbation process to progressively break the consistency between the data structure and the constraint imposed in OCHAC.
(2012) 4 ). The downloaded Hi-C matrix contains 4,864 bins. It has been obtained with a bin size of 40kb and normalized using ICE  (Imakaev et al. , 2012). We further performed a logtransformation of the entries to reduce the distribution skewness prior clustering. In order to assess the influence of the data structure on the quality of the partitions obtained by OCHAC and standard HAC algorithms, we have used a perturbation process to progressively remove the strong diagonal in the original Hi-C map. The perturbation consists in swapping two entries, s ij and s i j of the matrix, in which (i, j) and (i , j ) have been randomly sampled with uniform probability among the pairs {(u, v), (u , v )} for which u ≤ v, u ≤ v and s uv + s u v > 0, where the last condition avoids swapping entries that are both zero.
For the original dataset, which presents an organization strongly consistent with the order constraint, the heights of standard HAC and OCHAC are very similar. However, interestingly, OCHAC improves the objective criteria (ESS t and m t ) for low perturbation levels (15%-30%) across a wide range of merging levels. More specifically, we compared the heights obtained for HAC and OCHAC at the merger number selected by the broken stick heuristic  (Bennett (1996) ; vertical lines in Figure  7 ). At these numbers of clusters or in their close neighborhood, ESS t is always smaller for OCHAC, which we interpret as more homogeneous clusterings for OCHAC than for HAC. The magnitude of the improvement achieved by OCHAC with respect to HAC depends on the perturbation level: for the original data, it is close to 5%, whereas it is much larger (25-30%) when the perturbation level is 15%-30%.
Thus, OCHAC has to be preferred in such contexts and will additionally result in a lower computational cost. The benefit of using a relevant constraint had already been observed by  Steinley and Hubert (2008) : their simulations proved that a relevant order constraint (in their case, obtained from the data) could improve the recovery of the true cluster structure (although possibly at the cost of a slight decrease in ESS t compared to the unconstrained version). For perturbation levels larger than 60%, the data structure is no more compatible with the constraint (see Figure  6 ) and standard HAC seems to perform globally better than OCHAC, as expected. In addition, in this extreme situation, OCHAC exibits very large reversals for m t (seen with the grey shadow in Figure  7 ), that are due to sudden breaks in the quality of the clusterings, induced by the constraint.