dataesr/scientific-paragraphs-categorization huggingface dataset:
More specifically, we Fig. 4  Fscore results on the Flux Simulator simulations took the union of the multi-exon transcripts described in the three databases, while considering transcripts with the same internal exon/intron structure but with different length of the first or the last exon as duplicates. Reads were mapped to the reference transcriptome in order to restrict predictions to known genomic regions, and we perform independent analysis on the forward and reverse strands. All methods are run with default parameters. Figure  5  shows the Fscore per sample when FlipFlop, MiTie, and Cufflinks are run independently on each sample or when multi-sample strategies are used. Results on the forward and reverse strands are extremely similar. All methods give better results than their independent versions, and the performances of the multi-sample approaches increase with the number of used samples.
Illustrative examples We describe an example as a proof of concept that multisample FlipFlop with the group-lasso approach (1) can be much more powerful in some cases than its independent FlipFlop version, and than the merging strategy of Cufflinks/Cuffmerge. Figure  6  shows transcriptome assemblies of gene CG15717 on the first three modEN-CODE samples presented in the previous section, denoted as 0-2 h, 2-4 h and 4-6 h on the figure. For each sample, we display the read coverage along the gene, the junctions between exons, and the single-sample FlipFlop and Cufflinks predictions. At the bottom of the figure, we show the 6 RefSeq records as well as the multi-sample predictions obtained with FlipFlop or with Cuffmerge. A predicted transcript is considered as valid if all its exon/intron boundaries match a RefSeq record (✓ and ✗ denote validity or not).
The estimated abundances in FPKM are given on the right-hand side of each predicted transcript. Of note, the group-lasso predictions come with estimated abundances (one specific value per sample), whereas Cufflinks/Cuffmerge only reports the structure of the transcripts. For single-sample predictions, FlipFlop and Cufflinks report the same number of transcripts for each sample (respectively 2, 2 and 3 predictions for samples 0-2 h, 2-4 h and 4-6 h), with the same number of valid transcripts, except for the first sample where FlipFlop makes 2 good Fig. 5  Fscore results on the modENCODE data guesses against 1 for Cufflinks. This difference might be due to the fact that FlipFlop not only tries to explain the read alignement as Cufflinks does, but also the coverage discrepancies along the gene.
For multi-sample predictions, FlipFlop gives much more reliable results, with 4 validated transcripts (among 4 pre-dictions), while Cufflinks/Cuffmerge makes only 1 good guess out of 2 predictions. FlipFlop uses evidences from all samples together to find transcripts with for instance missing junction reads in one of the sample (such as the one with 30, 7 and 20 FPKM) or lowly expressed transcripts (such as the one with 0, 0. 5 and 2 FPKM). Cufflinks/Cuffmerge explains all read junctions but does not seek to explain the multi-sample coverage, which seems important in that example. Importantly, one can note that the results of multisample group-lasso FlipFlop are different from the union of all single-sample FlipFlop predictions (the union coincides here to the results of FlipFlop on the merged sample-data not shown).
Acknowledgements This work was supported by the  European Research Council  [ SMAC-ERC-280032  to J-P. V. , E. B. ]; the  European Commission  [ HEALTH-F5-2012-305626  to J-P. V. , E. B. ]; and the  French National Research Agency  [ ANR-09-BLAN-0051-04 ,  ANR-11-BINF-0001  to J-P. V. , E. B. ,  ANR-14-CE23-0003-01  to J. M, L. J. ].
The user is identified by a unique Personal Identification Code, considered as non-confidential in Estonia and based on Population Register. The main privacy protection mechanism is related to the management of user's attributes  [10, 11]. Austrian Citizen Card (CC). Massively rolled-out since 2006, Austrian eID is a flexible approach based on particular technology called Citizen Card concept (CC). The eID can be supported by various carriers such as mobile phone or chip cards (e. g. Social Security e-card). The CC and its carriers are not National ID documents, but are the official eID. As in Estonia, there is a central database called Central Residents Register. However, the Austrian legislation prohibits the use of these identifiers by service providers. The main privacy protecting mechanism is the use of sectorspecific pseudonymous authentication  [12, 13].
German eID. Launched in 2010, German credit-card format carrier fulfils two functions: eID, and machine-readable travel document and National Identity card (or "nPA" which stands for "neue Personalausweis, i. e. new passport). Here, national legislation does not allow central database of identifiers. The main privacy preserving mechanisms are: pseudonymous authentication, user controlled disclosure of attributes, no knowledge of user's activities by IDP, and mutual authentication between the user and the SP (the SP's validity is systematically verified)  [14, 15, 16]. SuisseID. Launched in 2010 as mostly private-sector initiative, SuisseID is used only for on-line authentication, and is not an ID document. Different form-factors (smartcard, usb-stick and mobile) are available. The privacy protection mechanisms are optional pseudonymous authentication and selective disclosure  [17, 18].
Multiple pseudonyms from one (or multiple) carrier(s). A more sophisticated mechanism is used in Austria and Germany. The eID deployer can generate software de-fined sector-specific pseudonyms, each of which is used in corresponding sector. In Austria for example, there are 26 distinct sectors, from social security to banking. It seems that there is no technical difficulty to increase the granularity from sectors-to service-specific pseudonyms. The SP identifies the user only as the pseudonym specific to the sector so that no cross-sector linkability is possible. In this approach, there is no "root" pseudonym or identifier. Indeed, German legislation prohibits central registry of citizen and unique identification numbers in general. Austrian eID system does rely on the Central Registry of Residents (CRR), but there are additional oneway hash mechanisms implemented at two stages: first to prevent to track back from sector-specific pseudonym to the sourcePIN stored in a separate container on the eID carrier, and second to prevent to track farther back from sourcePIN to the CRR.
In Austrian Citizen Card, few attributes are present in the eID deployer (namely name and date of birth). Centralized attributes. In this intermediate option, the eID deployer itself contains fewer personal data than the set managed by the issuing IDP. The role of IDP is complex as it deals both with the knowledge of attributes themselves, and with their assertion and disclosure to the SP: in some cases, IDP could disclose all attributes that it stores, while in other cases only a sub-set of attributes. The centralized design option is implemented in SuisseID, where the only mandatory attributes on the carrier are the SuisseID number and the name or the pseudonym. However, despite this minimal mandatory design, SuisseID carrier can also carry optional attributes, such as affiliation, e-mail, etc.
2. 2 The study of these emerging properties relies on the observation of systems of cities through harmonised spatio-temporal databases. Their systematic comparison leads the identification of empirical regularities and fosters theory building  (Bretagnolle et al. 2009; Pumain et al. 2015). Different modelling strategies have been pursued to study systems of cities: statistics, differential equations, synergetic, etc. The agent-based paradigm seems a promising one for geographers  (Bretagnolle et al. 2006; Heppenstall et al. 2012; Pumain & Sanders 2013) , because of its ability to capture the richness and diversity of cities  (Batty et al. 2012). First attempts at simulating the emergence of hierarchy in a settlement system were conducted with the Simpop model, the first to consider cities as agents  (Bura et al.
2. 3 The MARIUS project benefits from this double inheritance. First, it represents a new case study for the urban theory to be tested on: Soviet cities. We aimed at determining to what extent this system of cities can be considered generic and/or specific. Answering this question with standard statistical models and a harmonised database was the first step towards answering this question, but it showed that this system exhibits generic features (hierarchical distribution for example) as well as particularities (shape and level of hierarchy for example)  (Cottineau 2013; Cottineau 2014). It gave rise to the formulation of several hypotheses to be implemented incrementally as mechanisms.
2. 9 Model evaluation is acknowledged from the start in the model-building process: we use calibration and sensitivity analysis throughout the implementation of the models and compare simulations with empirical data collected about cities in the post-Soviet space [3]. By doing so, at each step of complexification of the model, we can estimate the extent of its contribution to the explanation of observed dynamics. In addition to expliciting the modelling process, EBIMM fits our research question, which is: to what extent do we need to particularize a model of system of cities' dynamics in order to reproduce the actual evolution of cities in the post-Soviet space? We illustrate the advantages of this approach in the following sections with the cornerstone MARIUS model. This cornerstone model relies on very generic mechanisms drawn from urban theory.
4. 9 We consider this new requirement to be an improvement, as it allows to identify the more interesting models with realistic microdynamics. However, in the best calibrated models, interactions are much reduced (by a very high distanceDecay and a very small economicMultiplier) and the macro results are less satisfactory (distanceToData = 12. 5319231911183; about 22 million people differing from the empirical distribution per census). Typically, cities exhibit very small values for supply and demand, and the trajectories are very chaotic (for the total urban population and the largest cities). At this point, we show that this most parsimonious model cannot satisfy both calibration objectives (no overflow and closeness to data). We verify this trade-off with the pareto front of the 62 best sets of parameters having finite values of overflow after more than 100.
5. 11 The range for the parameter estimating the decreasing effect of distance on interaction potentials (distanceDecay) is acceptable between 0. 4 and 1. 2. It is once again a value expected from the literature  (Fotheringham 1981; Baccaïni & Pumain 1998) , and a relatively low interval. It means that distance plays a negative but rather small effect on interurban interactions in the Soviet space as modeled with MARIUS. Figure  8. Profiles of MARIUS free parameters Red dots indicate the sets of parameters leading to a distance to data below our acceptance threshold for this model. We fixed this value at 1, close to the distance of the best calibrated model (0. 65) and yet restrictive enough. 5. 12 The most interesting result is obtained with the less documented parameter, wealthToPopulationExponent.
Mobile Mapping Systems (MMS) are now commonly acquiring lidar scans of urban environments for an increasing number of applications such as 3D reconstruction and mapping, urban planning, urban furniture monitoring, practicability assessment for persons with reduced mobility (PRM). MMS acquisitions are usually huge enough to incur a usability bottleneck for the increasing number of non-expert user that are not trained to process and visualize these huge datasets through specific softwares. A vast majority of their current need is for a simple 2D visualization that is both legible on screen and printable on a static 2D medium, while still conveying the understanding of the 3D scene and minimizing the disturbance of the lidar acquisition geometry (such as lidar shadows). The users that motivated this research are, by law, bound to precisely georeference underground networks for which they currently have schematics with no or poor absolute georeferencing.
A solution that may fit their needs is thus a 2D visualization of the MMS dataset that they could easily interpret and on which they could accurately match features with their user datasets they would like to georeference. Our main contribution is two-fold. First, we propose a 3D point cloud stylization for 2D static visualization that leverages a Principal Component Analysis (PCA)-like local geometry analysis. By skipping the usual and error-prone estimation of a ground elevation, this rendering is thus robust to non-flat areas and has no hard-to-tune parameters such as height thresholds. Second, we implemented the corresponding rendering pipeline so that it can scale up to arbitrary large datasets by leveraging the Spark framework and its Resilient Distributed Dataset (RDD) and Dataframe abstractions.
) caused by constructions. This often requires the registration of legacy local maps from relative or imprecise coordinates to a given spatial reference system. To assist this registration and provide such a reference map of the public space, we propose to derive a centimetric terrestrial orthographic view of the ground of the public space. This orthophotography-like dataset is proposed to be derived from a mobile mapping system for its precision and its surveying productivity which is similar to regular traffic speeds in urban environments, without the masking caused by trees and overhang structures experienced from higher sensor acquisitions (UAV, aerial. ). Lidar was also preferred to images as it provides a direct certified reading of depth measurements, while meeting the sampling density requirements of the targeted application.
Apart from this application, this end-product which we denote the (street-level) ortholidar has proved to be a valuable intermediate result that may be visualized very early in the production pipeline while being easy to interpret. As such its production is of prime interest for data quality inspection (manual validation of the dataset georeferencing) and standard image-based interaction, such as selection of ground control points or tie points for georeferencing. Thus, depending on the usage, the ortholidar may either be produced as (i) an image pyramid during a preprocess for optimal throughput or (ii) on-demand for low latency visualization of a given restricted extent only. While the high throughput qualities of (i) are easy to motivate in terms of a production pipeline, the low latency of (ii) would enable a user to peek interactively at intermediate results within the production pipeline without incurring the time costs of exporting a whole image pyramid.
Geospatial bigdata Eldawy and Mokbel (2014) note the explosion of geospatial datasets and the need for a distributed computing, analysis and visualization. The acquisition of lidar datasets in particular is increasing drastically both in terms of the number of surveys and in terms of the number of points per survey. It is thus natural to consider distributed computing frameworks in order to be able to cope with the volume of data and be able to guarantee a given velocity, either in terms of latency (for navigation and exploration purposes) or in terms of throughput (for postprocessing data quicker than they are acquired). van Oosterom et al. (2015)  recently benchmarked a number of possible management systems for point clouds in standard relational databases, distributed NoSQL databases and a file-based alternative.
They emphasize the respective strength and limitations of these systems. A comparatively new framework for distributed computing is Spark 1  , which extends the standard map-reduce Hadoop paradigm for in-memory processing. Spark introduced recently a Dataframe API into its Spark SQL library, which enables a columnar SQL handling of Spark's resilient distributed dataset (RDD) and performs Just-in-time SQL execution plan optimizations such as predicate pushdown  (Armbrust et al. , 2015). Point cloud visualization, also known as point-based rendering, is an entire field of research in itself. For instance,  Richter et al. (2015)  recently proposed to the geospatial community a lidar point cloud visualization approach that scales thanks to an out-of-core approach.
DISTRIBUTED PROCESSING OF POINT CLOUDS Lidar surveys are typically encoded in a number of files and may be seen as an ordered collection of records (the lidar point measurements) with both implicit attributes (Point ordering, filename. ) and explicit attributes (position in sensor coordinates, position in georeferenced coordinates using the sensor trajectory and calibration, amplitude, distance corrected reflectance, time. ). They are thus a perfect fit for database view where a table corresponds to a survey (possibly spanning multiple files), columns correspond to attributes and rows to lidar point samples. A naive RDBMS approach is likely not to be tractable for these huge datasets as RDBMSs tend to not scale well for tables consisting of billions of rows. A first approach  (Ramsey, 2014)  is to pack coherent patches of points into a single row to limit the number of rows and scale to larger datasets.
Spark Lidar IO In this work, we are exploring an alternative approach of using the Spark framework to scale out and distribute the processing of huge lidar datasets in a cloud computing context. Spark is a rising technology with a large momentum which is built on the Resilient Distributed Dataset (RDD) abstraction which distributes datasets in the distributed memory of a cluster, possibly spilling out to disk. RDDs are resilient to lost tasks or hosts due to its functional programming approach and their representation of a job as a Directed Acyclic Graph (DAG) of tasks. It provides 4 higher level distributed libraries: Streaming for micro batch processing, GraphX for graph computation, MLlib for machine learning which is used by  Liu and Boehm (2015)  for lidar point cloud classification, and SQL.
Spark SQL enables SQL operations on RDDs and external data sources by introducing a Dataframe API which is a columnar database-like view of a RDD which construction is analyzed and optimized at run-time. To enable the reading of a set of lidar files as a Spark Dataframe, we developed a scala library which provides: • Header readers for the commonly used PLY and LAS file formats. This enables the loading of the file metadata, the point record schema (name, types and offset of each attribute), and the binary section definition (given by a byte offset, a record count, and record length and stride in bytes). Note that extending this to other uncompressed binary formats is trivial and only requires the implementation of the specific header reader. • A PLY writer to write a partition (a portion of a RDD resident in the memory of a single node) to a (distributed) file system.
MMS Dataset The proposed approach has been deployed on a 10km long MMS dataset acquired by the IGN's Stereopolis vehicle  (Paparoditis et al. , 2012). This represents 339 tiles of 3 million points, thus 1017 million points in total. The tiles have already been processed by the dimensionality analysis, yielding uncompressed binary PLY files with many extra-attributes. In total, this dataset occupies 81. 439 GB on disk, 28. 476 GB of which are actually relevant for our processing  (7 attributes (x, y, r, e1z, e3z, d1, d2)  in single precision float format gives 28 bytes for each of the 1017 million points). This dimensionality analysis has been performed using a standalone unoptimized C++ executable that processes independently each tile in less than 3 minutes. This overall preprocessing took 2 hours when dispatched over 8 nodes.
Considering that this analysis is usually a basic preprocessing that is performed for subsequent processing steps, using it for visualization does not increase the overall computing costs. It however introduces a latency for visualization during its computation. Reducing or hiding the computing costs of this preprocessing is subject to future work, possibly by only performing this analysis on point clouds that have already been filtered by the viewing extent and undergo a pre-aggregation of point clusters. This dataset was uploaded to a HDFS distributed file system accessible by a Spark cluster with 8 slaves. We devoted 6 GB of memory to the driver and the executors to prevent out-of-memory issues. The overall computation of the tile pyramid took 41 min at a resolution of 1cm and 17min at a resolution of 2cm.
Sublinearity is due to the data-dependent nature of the algorithm, where tiles without points are not even considered. The cloud infrastructure provided by the IQmulus European project proposes an authenticated HTTP access to the HDFS directories. Thus the resulting PNG tiles may be served using a standard javascript webmapping html page for easy navigation in a web browser. We experimented the embedding of this Spark application in a scala http server using spray 2. Contrary to the Web-Map-Tile-Service (WMTS) style approach of the tile pyramid, the goal was to build an on-demand single image query service (similar to a Web-Map-Service -WMS). All the files were read at applica-tion start-up to compute the per-partition bounding boxes as a primary form of indexing in order to entirely cull partitions that are not visible in the query extent.
CONCLUSION Through this experiment, we have proved the possibility of handling massive point clouds using the Spark bigdata framework. While computing times are not ground-breaking compared to optimized distributed C++ code, we show that the abstraction provided by the Spark framework allows to express simply in a few lines of code algorithms that run distributed on a cloud in a resilient manner (e. g. robust to partition and executor losses). Future work will further investigate the coupling of the two proposed approaches to get the best of both: efficient direct access of huge precomputed areas using the tile-based approach and efficient and memory-efficient access to subsets using the partition-filtering approach. Furthermore, integrating this work with Geotrellis  (Kini and Emanuele, 2014) , a Spark-based geospatial raster library, would open new possibilities.
ACKNOWLEDGEMENTS This material is based upon work supported by the  FP7 IQmulus Integrating Project ,  FP7-ICT-2011-318787 ,  A High -volume Fu-sion and  Analysis Platform  for Geospatial Point Clouds, Coverages and Volumetric Data Sets, and the  TerraMobilita Project of the Cap Digital Business Cluster. We also thank  Electricité de France (EDF)  for their discussions and feedbacks on the proposed ortholidar visualization.
Here we present NeuroVault-a web based repository that allows researchers to store, share, visualize, and decode statistical maps of the human brain. NeuroVault is easy to use and employs modern web technologies to provide informative visualization of data without the need to install additional software. In addition, it leverages the power of the Neurosynth database to provide cognitive decoding of deposited maps. The data are exposed through a public REST API enabling other services and tools to take advantage of it. NeuroVault is a new resource for researchers interested in conducting meta-and coactivation analyses.