dataesr/scientific-paragraphs-categorization huggingface dataset:
INTRODUCTION AND MOTIVATION Generating continuous 3D models of urban environments, at ground level, is becoming an increasing need for a wide range of applications. Driving simulation engines, semantic perception for intelligent vehicles, trajectory planning and visual servoing for autonomous navigation are several applications requiring an automatic framework for producing accurate and scalable 3D models over large scale distances. Mobile Mapping Systems (MMS) equipped with active 3D sensors are well adapted for acquiring dense 3D measurements of the underlying surface, while driving in normal traffic conditions. Nevertheless, such a discrete representation must be further exploited in order to build a continuous surface via 3D modeling techniques. In addition, 3D modeling from MMS datasets requires scaling-up cartography which includes processing big datasets in an automatic fashion, designing consistent zoom levels for multi-scale mapping but also cloud computing for massive scalability of geo-specific 3D model databases.
In this research work we focus on the ground surface reconstruction theme which in dense urban environments is concerned with the road, sidewalk and ramp access areas. These sharp depth changes and geometrical details need to be preserved in order to cope with the accuracy required by the visual layer of simulator engines or for intelligent vehicles. When dealing with the surface reconstruction problem using 3D point clouds acquired by MLS sensors, several key issues must be addressed. Noise sources coming from the laser sensing device, external calibration and mobile acquisition (distance to the scanned surface, incidence angle, surface geometry and material type) must be carefully identified and modeled correspondingly. Smoothing the noise in MLS data while preserving sharpness is still a difficult task for the state-of-the-art surface reconstruction algorithms.
Furthermore, scalability issues must be addressed in order to deal with real-time rendering of big datasets acquired over large distances. In this paper we propose a fully automatic surface reconstruction framework for ground areas which copes with the aforementioned constraints imposed by MLS datasets. Our paper is organized as follows: Section 2 presents the existing solutions for ground surface reconstruction from 3D point clouds. Section 3 provides an overview of our framework which is driven by a ground segmentation module presented in Section 4. The ground points are exploited along with a novel surface reconstruction pipeline described in Section 5. Section 6 evaluates the performances of the proposed framework, while Section 7 presents quantitative results obtained over large scale distances.
Nevertheless, these methods do not deal with noise for recovering sharp depth features and they do not address scalability issues for enabling a detail-preserving surface reconstruction framework capable to process automatically big datasets acquired over large distances. Surface reconstruction algorithms based on interpolation were also reported  (Soucy and Laurendeau, 1995) ,  (Bernandini et al. , 1999). Nevertheless, interpolation-based methods do not cope with noisy datasets as those acquired by MMS. In  (Digne et al. , 2014) , authors introduce a volumetric approach. These methods require massive simplification (via edge and vertices collapsing operations) which is very ambiguous, leading to loss of detail and artifacts. In our research work we are interested in generating a triangular representation which is as close as possible to the scanned surface.
OVERVIEW OF THE PROPOSED METHOD The overall processing workflow of the proposed surface reconstruction method comes together with the global framework illustrated in Figure  1  which has as input a massive 3D point cloud acquired by a MMS. The dataset is first sliced into 3D chunks of N Mpts (Million points) each, where N denotes the number of 3D measurements recorded per chunk. According to the vehicle speed, the length of the surveyed area may vary. Figure  2  (a) illustrates an example of a 3D chunk acquired over a dense urban area situated in Paris, France. We do not make any assumption about the acquisition setup, so the input data can be supplied by different platforms  (Paparoditis et al. , 2012) ,  (Goulette et al. , 2006). When choosing a surface reconstruction method, the geometric properties of the underlying surface must be taken into account in order to design an adaptive framework, geometrically consistent with each object.
Long Triangles Elimination In order to eliminate long triangles from non-uniform boundary points, we perform statistics on the edge lengths and identify those with maximum length, noted emax. We identified that long edges correspond to emax ≈ δē, where ē denotes the mean length computed over all edges e j i belonging to the mesh MDT , i. e. over all triangles t j ∈ MDT , j = 1,. , Nt and for its corresponding edges e j i , i = {1, 2, 3}. The term δ denotes a proportionality factor. A triangle t j is eliminated if any of its edges e j i > emax, i = {1, 2, 3}. This criterion ensures that only long triangles belonging to the boundary are eliminated; moreover, since small triangles are not eliminated, holes can not be generated within the mesh. In practice, for several datasets acquired in urban areas by different MLS systems  (Paparoditis et al.
PERFORMANCE EVALUATION We evaluate the performances of the proposed framework in terms of accuracy, memory usage and computation time. Accuracy evaluation. We quantify the accuracy of the reconstructed surface with respect to several ground truth measurements which were performed manually on site (Cassette road, situated in Paris, France), mainly: the height of the sidewalk border, the height of the access and the road width, noted H sidewalk , Hramp and W road , respectively. Table  1  illustrates the ground truth and the reconstructed dimensions for dataset Urban ♯2 illustrated in Figure  5. It can be observed the reachable accuracy is better than 1. 5 cm. Dataset Urban ♯2 H sidewalk Hramp W road GT 10. 5 (cm) 2. 5 (cm) 3. 5 (m) Reconstruction 10. 1 (cm) 2. 3 (cm) 3. 514 (m) Table  1 : Accuracy evaluation of the ground surface reconstruction with respect to ground truth (GT) data for Urban ♯2.
Computation time. We evaluate our algorithm on a 64b Linux machine, equipped with 32 Gb of RAM memory and an Intel Core i7 running at 3. 40 GHz. Our method is implemented in C/C++ and exploits PCL  (Rusu and Cousins, 2011)  and VTK  (Schroeder et al. , 2006)  libraries. Table  2  illustrates the computation time obtained for the dataset Urban ♯2. We can observe that the decimation step is the most expensive phase, being related to the decimation factor fD. In this example, a maximum decimator factor was used f D = 90% for a mesh with 2 MTriangles, which results in 9 sec of computation time. Memory usage. resentation is more efficient than the point-based one, allowing to reduce the memory usage 3 times for the full resolution mesh and 20 times for a resolution mesh of r(MS) = 10%. These results show that the proposed surface reconstruction framework provides a memory efficient surface representation, while preserving geometric details.
Visual rendering. The frame frequency, measured in frames per second (FPS), allows to quantify the quality of a 3D model with respect to the visual rendering capability. The second row of Table  3  illustrates the frame frequency, noted νrate and measured using Cloud Compare (Girardeau-Montaut, 2013) for different surface representations (discrete and continuous). It can be observed that the point-based representation detains faster rendering capabilities than the full resolution mesh, which does not cope with real-time rendering requirements. In contrast, the decimated mesh exhibits real-time frame rates, while providing a continuous surface representation. Table  3 : Memory usage and frame frequency measures corresponding to the input chunk P and to the main outputs of the algorithm for dataset Urban ♯2 illustrated in Figure  5.
SCALING-UP DETAIL-PRESERVING GROUND MODELS We test the surface reconstruction algorithm on several chunks acquired over urban areas acquired with different MLS systems  (Paparoditis et al. , 2012) ,  (Goulette et al. , 2006) ; different smoothing parameters were applied for each MLS. Figure  7  illustrates the result obtained on several chunks corresponding to dataset Urban ♯2 acquired by a MLS equipped with a Riegl sensor  (Paparoditis et al. , 2012). Figure  7  shows the results obtained for 4 scans segments acquired over 217 m, containing 12 Mpts, from which 4 Mpts were classified as belonging to ground. It can be observed that due to the vehicle speed which may vary, the acquired 3D chunks have different lengths. By taking into account the computation time obtained for dataset Urban ♯2, in average, we process 3 Mpts for 50 m length of surveyed area in about 17 s.
CONCLUSIONS AND FUTURE WORK This paper presented Automatic Ground Surface Reconstruction (AGSR), a fully automatic framework for generating detail-  preserving and scalable ground surface reconstruction from MLS data in outdoor environments. The presented research work is mainly concerned with urban areas, where the ground is usually represented by quasi-flat zones; for off-road terrain modeling, adapted schemes must be designed. The proposed algorithm comes with a parallel scheme to be expanded at two levels: i) for running on series of ground chunks, and ii) at upper level, for reconstructing different classes, allowing to integrate adapted surface reconstruction frameworks for complex (non-planar) objets. The proposed method emphasizes the high potential of the MMS which, when combined with suitable frameworks, it allows to generate accurate and scalable 3D models in a fully automatic fashion.
This paper discusses the results of the LifeCLEF 2014 multimedia identification challenges with regards to the requirements of real-world ecological surveillance systems. In particular, we study the identification performances of the evaluated systems as a function of the ordinariness or rarity of the species in the dataset. This allows us to assess the ability of the underlying methods to be robust to heavily tailed distributions such as the ones encountered in real-world collections of life observations. Results show that all methods are more or less affected by the long-tail curse but that the best methods making use of classifiers with good discrimination capacities do resist the phenomenon pretty well.
Each task is based on large and real-world data and the measured challenges are defined in collaboration with biologists and environmental stakeholders in order to reflect realistic usage scenarios. As the purpose of this paper is to focus on a deeper analysis of the raw results of the 2014 campaign, we refer reader to the complete overview of the lab  [14]  for the details about each of the tasks including the data description, the metrics, the run formats, etc. We here only synthetise the main tasks: PlantCLEF 3  : an image-based plant identification task continues the three previous plant identification challenges of ImageCLEF in 2011  [7] , 2012  [8]  and 2013  [13]. The 2014 PlantCLEF dataset was composed of 60,962 pictures belonging to 19,504 observations of 500 species of trees, herbs and ferns living in a European region centered around France.
This data was collected by 1,608 members of TelaBotanica 4  , a French-speaking social network of 23,000 amateur and expert botanists. Each picture belongs to one of the 7 types of view reported in the meta-data (entire plant, fruit, leaf, flower, stem, branch, leaf scan) and is associated with a single plant observation identifier allowing to link it with the other pictures of the same individual plant (observed the same day by the same person). BirdCLEF 5  : an audio-based bird identification task based on the audio recordings collected by Xeno-canto 6  , a web-based community of bird sound recordists worldwide with about 1,500 active contributors that have already collected more than 180,000 recordings of about 9,000 species. Nearly 500 species from Brazilian forests are used in the BirdCLEF dataset totalling about 14,000 recordings produced by hundreds of users.
As a comparison, the previous largest bird species bioacoustic classification was the NIPS4B 2013 challenge with (only) 80 species from French Provence  [11]. FishCLEF: a video-based fish identification task based on the Fish4Knowledge 7  underwater video repository, which contains about 700k 10minutes video clips that were taken in the past five years to monitor Taiwan's coral reefs. More specifically, the FishCLEF dataset consists of about 3,000 videos with several tens of thousands of detected fish instances that were identified for the 10 most common species. 127 research groups worldwide registered to at least one task of the lab, of which 22 crossed the finish line by submitting runs (27 runs for the plant task, 29 runs for the bird task, 6 runs for the fish task). Details on the methods used in the runs and the results achieved by all teams are synthesised in the overview working notes of each task  [10, 9, 5].
The big problem with data Building effective multimedia analysis and machine learning techniques is unfortunately not the only side of the taxonomic gap problem. Whatever the used algorithms, the availability of rich and appropriate training data is actually equally challenging towards setting up powerful identification tools at large scales. If we look at the popular ImageNet dataset  [6] , widely used for the evaluation of large-scale image classification methods, it is essential to notice that the average number of training images per category is in the range 600-1200. And this is actually several orders of magnitude richer than most existing collections of multimedia life observations. Even the Encyclopedia Of Life 8  , which is the world's largest data centralization effort concerning multimedia data for life on Earth, does not have more than few images per class of interest for the vast majority of species.
This more generally reflects the heterogeneous knowledge that we have of plants and animals, with a huge volume of information on widespread and useful species for human beings, and very little information (in term of geographical distribution, morphological description, etc. ) on most of the plant species of a given area. A rather good average number of multimedia documents per taxon can therefore be misleading regarding the real coverage of all species. Figure  1  illustrates the distribution of the Bugwood 9  -ForestryImages 10  dataset which includes 187K images of about 18K plants species of economic concern. Similarly, Figures  3  and 4  present the long-tailed distribution of the number of observations per species Figure  1 : Long tail distribution of Bugwood-Forestry Images dataset within the plant and the bird datasets used for LifeCLEF 2014 challenges.
The histograms notably show that the plant dataset is more heavily tailed than the bird dataset. This is partially due to the natural abundance distribution of the species but also to the different characteristics of the social networks that collected the data (less contributors to the bird dataset but with a more homogeneous and wider expertise). But still for the plant dataset, the set of the kept species in LifeCLEF challenge is only the tip of the iceberg. This is illustrated by Figure  2 , which presents the distribution of the whole Pl@ntNet dataset  [12]  which in itself is the largest existing dataset for the French flora. As shown on the graph, the 500 hundred species of the PlantCLEF dataset (coloured in red) are mainly concentrated at the head of the distribution and therefore rather focus on the most common species, regardless the rich biodiversity existing in the country (estimated to be about 6000 plant species).
For each of the three tasks, we therefore split the species into 3 categories according to the number of observations populating these species in the datasets. The 3 resulting splits are illustrated in Figures  3, 4  and 5. For the three challenges, the category A of species (blue) corresponds to the most populated species (i. e. the tip of the distribution), the category B (red) corresponds to intermediate species with a relatively high number of observations, the category C (green) corresponds to the less populated species in the long tail. Note that for the fish task, as the total number of species is very low (restricted to the 10 most common species), we only included one species in the long tail category C. Therefore, the results will be statistically less relevant than for the bird and the plant task.
For the bird and the plant datasets, we used the same thresholds on the cumulative distributions to define the categories (cat. A is represented by the first 20 percent of the observations belonging to the most populated species, cat. B by the next 50 percent of the observations, cat. C by the least common 30 percent). Based on these relative abundance categories A, B and C, we computed a per-category score for all the runs submitted to LifeCLEF. This was done by Figure  3 : Distribution of PlantCLEF 2014 dataset -split in 3 ordinariness categories first computing the per-species official score of each run and then averaging the scores of the species that belong to the same category (details of the scores used for each task can be found in their respective CLEF working notes  [10, 9, 5] ).
e. none of the evaluated systems has a lower score on category A than on category B and this is also the case for category A vs. C and B vs. C). This is clearly not surprising and simply demonstrates that all participants use common statistical machine learning strategies whose performances are correlated with the class statistics. As it was not an objective of the measured challenges, none of the participants specifically tried to emphasize the less populated species or to balance the classes. This raises the question of whether it would be meaningful to foster rare species in the evaluation protocol of future LifeCLEF challenges (e. g. through a biodiversity-friendly evaluation metric or through a balanced distribution of the queries across the classes). On one hand, this would bias the evaluation because the natural dis-Figure  4 : Distribution of BirdCLEF 2014 dataset -split in 3 ordinariness categories tribution of the data somehow reflects the usage of a real-world identification system in which the most common species attract the most user requests  [12].
Now, the most important question is whether the methods used by the different participants are equally biodiversity-friendly or not. Let us first start with the bird task results (cf. Figure  6 ). For this challenge, we can observe that the overall ranking of the runs remains roughly the same whatever the category (A,B or C). This means that none of the methods is critically more affected by the long-tail issue than the other ones. Some variations can however be observed. The per-category scores of MNB TSA runs are notably more homogeneous than the ones of QMUL (with a variation coefficient of 0. 19 vs. 0. 28 for the best run of each team). That means that in addition to be better on average, the runs of MNB TSA are also more biodiversity-friendly. This confirms the good skills of the segment-probabilities features used by this group  [15]  as well as the good Figure  5 : Distribution of FishCLEF 2014 dataset -split in 3 ordinariness categories capacity of the ensemble of randomized decision trees they are using as classifier.
If we now look at the fish task results (cf. Figure  8 ), we can observe that here again the ranking of the runs is preserved for the three categories but that the variations in the homogeneity of the scores are much more accentuated. Whereas all runs have achieved comparable performances on the category A, the performances of the I3S runs clearly crashed on the less populated species leading to very bad biodiversity-friendliness values (variation coefficient greater than 1. 0). As discussed in the working note of the fish task  [5] , the lower performances of the I3S runs are mainly due to their fish detection algorithm that has a much lower recall than the ViBe  [2]  background modeling approach used in the baseline. Interestingly, we can see here that the deficit of recall is mainly concentrated on the less populated species in the dataset that are probably also the less visible ones in the video contents (smaller fishes and/or less numerous ones).
The results of the plant task (Figure  7 ) are probably the most informative ones concerning our biodiversity-friendliness analysis. The more heavily-tailed distribution of the plant dataset actually accentuates its impact on the methods that are the most sensitive to multi-class imbalanced problems. This is typically the case for the runs of SZTE, FINKI and PlantNet that result in high values of the variation coefficient and low classification scores for the species belonging to the long tail. The common point of all these runs is that they rely on instancebased classifiers that are more directly dependent on the feature density and thus more sensitive to imbalanced problems. The most affected runs are the ones of SZTE and FINKI that directly use a K-NN classifier on global visual features.
Metabarcoding is a powerful tool for exploring microbial diversity in the environment, but its accurate interpretation is impeded by diverse technical (e. g. PCR and sequencing errors) and biological biases (e. g. intra-individual polymorphism) that remain poorly understood. To help interpret environmental metabarcoding datasets, we investigated the intracellular diversity of the V4 and V9 regions of the 18S rRNA gene from Acantharia and Nassellaria (radiolarians) using 454 pyrosequencing. Individual cells of radiolarians were isolated, and PCRs were performed with generalist primers to amplify the V4 and V9 regions. Different denoising procedures were employed to filter the pyrosequenced raw amplicons (Acacia, AmpliconNoise, Linkage method). For each of the six isolated cells, an average of 541 V4 and 562 V9 amplicons assigned to radiolarians were obtained, from which one numerically dominant sequence and several minor variants were found.
Introduction High-throughput sequencing of phylogenetic markers (metabarcoding) is becoming the gold standard approach for exploring microbial diversity in the environment  [1, 2, 3]. The presence of the 18S rRNA across all eukaryotes, its extensive occurrence in public reference databases and the availability of generalist primers make this gene the best universal marker available to date for eukaryotes  [4, 5]. Metabarcoding of microbial eukaryotes typically targets the short variable regions V4 and V9 of the 18S rRNA gene  [2, 3]. From the reads generated (amplicons), definition of operational taxonomic units (OTUs) is classically used not only to identify taxonomic entities and describe community structure (e. g. diversity and richness), but also to assess the extent of the so-called ''rare biosphere''  [6, 7].
In addition to these biological concerns, PCR artifacts and sequencing errors that scale with the sequencing effort are known to artificially inflate diversity estimates  [20, 21]. Discriminating between natural amplicons and technical artifacts is definitively a challenge that has to be addressed for accurate interpretation of large datasets in molecular ecology. In this study, we investigated the intracellular diversity of the ribosomal barcodes V4 and V9 in eukaryotes using 454 pyrosequencing. We focused on two radiolarian taxa, Acantharia and Nassellaria, which are heterotrophic marine protists, from which no genomic data is available to date. Their large cells (100-500 mm in diameter) are supported by a mineral skeleton and can contain several nuclei  [22]. Acantharia and Nassellaria are important components of planktonic communities due to their abundance, predation, contribution to the vertical flux of organic matter, and indirectly as primary producers through symbiosis with microalgae  [23, 24, 25, 26].