dataesr/scientific-paragraphs-categorization huggingface dataset:
Model uncertainties and performance Although equations have been carefully parameterised with the literature, some uncertainties in the correct estimation of soil erosion rates by RUSLE remain. Combined together, LS and C factors account for 94. 8% of the total variance in soil erosion predicted with 20th century land cover and WorldClim climate dataset. Sources of error in LS and C estimation include (1) rill to inter-rill soil erosion ratio a (equation 4), which was arbitrarily set without calibration, and (2) the choice of C-factor values (Table  1 ). However, average predicted rates of soil erosion for the 20th century (2. 16 tÁha À1 Áa À1 , 1-s range 0. 31-11. 58) are comparable to those predicted on similar temperate catchments with gentle slopes and moderate rainfall (e. g. ,  Van Rompaey et al.
It is slightly underestimated, probably as a consequence of the combined use of coarse, irregular SPM monitoring  (Coynel et al. , 2004)  with prediction of annual average water flow that omits hydrological extremes. Despite all these approximations, the results are globally relevant and validate the methodology of the study, especially regarding the parameterisation of equations. We emphasise that the correct prediction of present-day Charente sediment load is not an artefact related to the use of river SPM concentration data in SDR calculation (see section 3. 3). SPM data were obtained mostly on small tributaries (80% of sub-catchments smaller than 500 km 2 ) representative of local river conditions, and not of the global Charente catchment.
2% less erosive on average, resulting in a difference in sediment supply not greater than 8. 7%. Despite increased inter-annual variability (Fig. 3c ), slightly drier climate compensated for forest cover loss. There is a clear mismatch between the nearly stable evolution of predicted Charente river sediment load and the increase in silt content recorded in M7UC01 core (Fig. 1d ), which is at first sight counter-intuitive. The suitability of  Pauling et al. (2006)  paleoprecipitation  dataset could be questioned. In data-poor regions, few alternatives to gridded climate data exist, but spatiotemporal biases could undermine the confidence in results obtained from the misuse of such reconstructions  (Fortin and Gajewski, 2012). The application of these rainfall data to RUSLE should be restricted to periods and areas for which good to excellent reconstructive skills are reported, such as the 18-20th-century winters over the British Isles, the Iberian Peninsula, France, and Germany  (Pauling et al.
Acknowledgments We thank the invited editors  Philippe Joseph ,  Vanessa Teles , and  Pierre Weill  for giving us the opportunity to submit this work. We are indebted to  Dr. John Boyle  ( University of Liverpool ) and  Mr. Andrew Swales  ( NIWA ) for their meticulous review and inspiring comments, which have greatly helped us in the revision process. Unpublished Sr-Nd isotopic analyses were carried out by  Sidonie Revillon (SEDISOR). Geospatial data owned by the Institut geographique national (IGN) have been used according to the '' Enseignement et Recherche '' license attributed to  C. Poirier 's host institution.
In addition to the traditional Geographic Information System (GIS) data such as images and vectors, point cloud data has become more available. It is appreciated for its precision and true three-Dimensional (3D) nature. However, managing the point cloud can be difficult due to scaling problems and specificities of this data type. Several methods exist but are usually fairly specialised and solve only one aspect of the management problem. In this work, we propose a complete and efficient point cloud management system based on a database server that works on groups of points rather than individual points. This system is specifically designed to solve all the needs of point cloud users: fast loading, compressed storage, powerful filtering, easy data access and exporting, and integrated processing.
, 2013). However, using a file-based system has severe limitations. These systems are usually built around one file format, and are not necessarily compatible. Recent efforts have been made towards format conversion  2. Moreover, these systems are not adapted to share data and use it with several users simultaneously (concurrency). Hofle (2007)  proposed to use DataBase Management System (DBMS) to cope with concurrency. The DBMS creates a layer of abstraction over the file-system, with a dedicated data retrieval language (SQL), native concurrency capabilities (supporting several users reading/writing data at the same time), and the wrapping of user interaction into transactions that can be cancelled in case of errors. The DBMSs have also been used with image and vector data for a long time, and the possibility to define relations in the RDBMSs (Relational DBMS) offers a simple way to create robust data models.
Adding the capacity to create point clouds as services, DBMSs solve almost all the problems we face when dealing with point clouds. Usually, the database stores a great number of tables, and each table stores a point per row  (Lewis et al. , 2012; Rieg et al. , 2014). Such a database can easily reach billions of rows. Nevertheless, storing this many rows is problematic because DBMSs have a non-negligible overhead per row, which reduces the scaling possibilities, regarding the time it takes to create it, to index it, or in the final space it takes. These limitations have been studied and inspired NoSQL databases. Again, the proposed NoSQL databases are used to store individual points  (Martinez-Rubi et al. , 2014 , 2015; van Oosterom et al. , 2015; Wang et al. , 2014). NoSQL database are stripped DBMSs that have been specially tailored for massive and weakly relational data.
METHOD The proposed solution relies on a PostgreSQL (2014) RDBMS server using the PostGIS (2014) and pgPointCloud (2014) extensions. The key idea is to store groups of points into the server 3 https://github. com/hobu/greyhound table. Groups of points are called patches of point. An XML schema defines the size and nature of each attribute for each point type. The user can load data into the server by several common means (using major programming languages, Bash, SQL, Python), from any format of point cloud that can be expressed as a list of values. Point clouds are stored without loss and are compressed. The very sophisticated database indexes allow efficient filtering of the points. Point clouds can be used with vector and rasters and other point clouds. Metadata are integrated and exploited.
Furthermore, point clouds can be easily converted into other GIS data (vector/raster). Processing methods are directly accessible within the database; more can be added externally or internally. Attaining points from the database is also easy and can be done in several ways (whole files, specific points and streaming). Briefly, storing groups of points offers the advantages of generalisation (potentially more complex semantic objects), reduces the number of rows by several orders of magnitude, reduces index sizes, allows efficient compression, and offers a common framework for different types of points. Working on groups of points separates the filtering and retrieving of points. Groups can also be easily split or fused at any point after data loading. However, to obtain an individual point, we need to get the full group first.
Loading Writing data in a PostgreSQL RDBMS is standard. There are conceptually two kinds of solutions. Either the database reads the data ('server reads'), or a client connects to the server and writes the data ('client writes'). Clients exist in all major programming languages. DBMSs are build for concurrency, we illustrate both of these methods with parallelism. Parallel loading ('server reads') Our first loading method (Figure  2 ) reads point cloud files, convert them to a stream of attributes and writes them to temporary tables in the database. The database groups points into patches and adds the patches to the final point cloud table. This method mixes 'server read' and 'client sends' approaches. Distributed parallel loading ('Client sends') In previous method, the database performs the grouping of points into patches and the actual writing of patches into tables.
This information can be stored in the server, and leveraged in filtering (see Section 3. 5) or processing. Using several point clouds and other GIS data Point clouds are created by different sources, like stereo-vision, aerial Lidar, terrestrial Lidar, RGBZ device (Kinect), etc. The point cloud server mixes all this data, along with other GIS data (rasters and vectors). Vectors and rasters are stored and exploited using Post-GIS (2014). We can use geo-referenced point clouds together. For instance, a user asks for points in a given area. The user obtains points from a large 1 pt/m 2 aerial Lidar point cloud automatically complemented by a more detailed but very local 10 kpts/m 2 stereo-vision point cloud.
www. pdal. io Coverage map Using the server, we create metadata-like point clouds coverage (Figure  7 , 8), similarly to  (Lewis et al. , 2012, Figure 8). With this map, one can instantaneously and easily check what type of point cloud is available in a given area using a colour code (for instance). Point cloud as raster or vector In the spirit of generalisation (see Section 2. ), it is advantageous for some applications to convert points to other GIS data types, such as raster or vectors, directly within the database. We propose several in-base groups of points vector representation, such as bounding box, oriented bounding box, concave envelope similar to alpha shape  (Edelsbrunner et al. , 1983) , and 3D plans (Figure  9 ). These representations can be used to extract information at the patch level, accelerate filtering, enable large scale visualisation, etc.
Indexing Our system extensively uses indexes (BTree, RTree) that are native to PostgreSQL. We index patches (not points). Basically, these indexes answer in about 10ms to any filtering, such as 'What are the patches with f between. and. '; f can be anything, a spatial position, an attribute of the points, a function, etc. Indexes of functions are very powerful and can save a lot of space (no need to add an extra column). For instance, we may have a fast function f that gives a measure in [0; 1] of how much the patch looks like a vertical cylinder. Now, when looking for all the patches p that really resemble cylinder ( f (p) > 0. 8), for PostgreSQL also determines whether to use the indexes or not. In some cases, it will be faster to simply read all the data (sequential vs. random access). This decision is based on statistics on tables and a genetic optimisation.
Output Similarly to Section 2. 1, we divide the methods into two categories. The first family of solutions is that the server writes the points somewhere ('server writes'). The second family of solution is when the client reads the data from the server and do something with it ('client reads'). Using PostgreSQL drivers/connector ('client reads') PostgreSQL can be accessed using many programming languages, thus any PostgreSQL driver can be used to connect to the server and output points. This work-flow is very similar to what a classical processing program would do, 'open point cloud file, read points, do processing, write results' becomes 'connect to server, read points, do processing, write results on the server or elsewhere'. The additional capabilities are that the user does not have to read a whole file (or any files) if the user is interested in only a few points.
Using the point cloud server, the user can directly filter the point cloud to obtain only the points desired, and even use in-base processing or LOD to further change the points obtained. PLY File As a Service (PLYFAS) 'server writes' Our system can be used transparently with a file-based workflow. Indeed, users may already have legacy processing tools that work with files. Of course, these tools could be easily adapted to read points from the database and not from files, but users may want to use their usual tools as-is. For this case, we propose PLYFAS, an easy means to export points from the database and create a. ply file. The user can use the small PLYFAS API to request the database to create a ply file from any set of points. The user may simply want one of the exact original point cloud files that were loaded into the point cloud server.
However, the user has also access to much more power and can request a file with filtered points by any means introduced in Section 2. 3, or with the additional processing results of Section 2. 5. For instance, the user can request all the points in a given area that have been classified as 'building parts' with a given confidence, and that were sensed during the second week of March 2014. Massive parallel export ('client reads') We also designed a Python method to perform massive parallel export. Similarly to Section 2. 1, the goal is to reduce the work done on the server and increase the work done on the clients. In this version, the server sends raw binary uncompressed patches (groups of points), and the transformation to points is done by the client.
Processing Point Cloud with the Server Processing point clouds We think it is important to offer both points and adapted tools to users. Our system can be used for processing in two ways. The most classical is out-of-base processing. A client obtains the points, does something, and writes the results in the server. However, our system also offers in-base processing. Processing methods become very close to the data and can be reused or combined to create more complex methods. It is also more practical because the client does not have to install anything (all methods are on the server). An advantage of in-base processing is that it is easy to add new processing methods. These methods can be written for efficiency (C, Cpp) or using high level languages (Python, R) for very fast prototyping. The most useful in-base processing functions are fast and simple.
General System We design several experiments to test all parts of our point cloud server. All experiments have ample room for optimisation, and can be easily reproduced (open source tools). We use PostgreSQL 9. 3, PostGIS 2. 2, PgPointCloud 1. 0, Python 2. 7, and a recent (2014) version of Numpy and Scipy. Result at the system level Overall, we load several billion points into the point cloud server, perform several processing in and out of base (s to hour), extensively use simple and complex filtering (ms and s), convert points to images and vectors, and output points (≥ 100k pts/s). The entire system works as intended and is efficient and powerful enough to be used in research settings. Data sets used For this article, we use three data sets (including (IQmulus, 2014)). (See Figure  6 ).
Input Parallel loading ('server reads') In one night, we aim at loading the data sensed by a Lidar system during one day, which is a practical Lidar management requirement. The points are stored in files, over a gigabyte network. We uses a dedicated program to convert the points file into ASCII values (CSV). We modified the RPly library 5  for the. PLY point cloud, and use Lastool 6  for. las files. The ASCII values are streamed to a 'psql' process that is connected to the database. The 'psql' executes a 'COPY' SQL command that reads the ASCII streams and creates and fills a table with the values from the ASCII stream. When the file has been fully streamed, we use an SQL query to create points from attributes and group them into patches. These patches are inserted into the final patch table.
This pipeline (Figure  2 ) is executed several times in parallel, each pipeline working on a different file. Distributed parallel loading ('client sends') In this experiment, we use clients to send uncompressed patches to the server. The clients read point cloud files (. ply in our experiment, using the plyfile 7  Python module ). Then, each client groups the points into patches using a custom Python module. The patches are sent to the server through Python. The server compresses these patches and adds them to the final point cloud table. This experiment is a proof of concept; therefore, we limit the number of clients to one computer, using seven threads. Result We load Vosges and Paris data set through 'massive parallel loading', and stereo through 'out of database grouping' (Table  2  ).
For both methods, the bottleneck is not the CPU but the input/output (I/O). Indeed, the point files are read over the network, and the point tables are stored on the SSD, but the final patch table is stored on the regular disk, which also limits how many threads can write data on it at the same time. Result We load Vosges and Paris data set through 'massive parallel loading', and stereo through 'out of database grouping' (Table  2 ). For both methods, the bottleneck is not the CPU but the input/output (I/O). Indeed, the point files are read over the network, and the point tables are stored on the SSD, but the final patch table is stored on the regular disk, which also limits how many threads can write data on it at the same time.
Storing Point Cloud in Table Grouping points Points must be categorised into groups that will make sense for subsequent uses of the point cloud. Groups   of points must be big enough to have a tractable number of rows, but not too big because getting only one point still necessitates obtaining the entire group. We designed grouping rules so that the number of rows is less than a few millions -i. e. , fitting in server memory (Table  3 ). Moreover, this range of numbers of rows is classical for GIS software. We can afford to have very large groups as a result of the PostgreSQL TOAST 8 storage system. Grouping is done at data loading but can be changed at any time. Index creation is very fast (a few seconds to a few minutes), and the index size is ≤ 1 % of the point cloud size. Indexes are built on the patches and not on the points, and thus are several orders of magnitude smaller and much faster to build.
We estimate the size of indexes if we were to store one point per row rather than one patch per row (extrapolating index size and build time observed on 0. 01 to 10 million-row tables). See Table  3. Compressing point clouds Patches are compressed before storage. We compare loaded data-set space occupation on the server with original binary files on the disk. In our case, patches are compressed attribute-wise, with either a run-length, common bit removal, or zip strategy. Compressing efficiency widely varies depending on the data and the kind of attributes of the points. See Table  3. 4. Compressing and decompressing data introduces an overhead on data access. We estimate it by profiling the uncompress and compress functions. Again, the overhead is dependent on the type and number of attributes.
Point Clouds and Context Our point cloud server manages point clouds, as opposed to sets of points. First, we demonstrate the construction of several twodimensional (2D) vectorial visualisations of point clouds. We  We created several visualisations for the Paris dataset, ranging from 5MByte to 100kByte, each adapted to a different scale. (see Figure  7 ) • 1:25 to 1:1500: Precise, occlusions visible (∼1m). • 1:1500 to 1:15k: Help to understand road network structure (∼8m). As a proof of concept, we propose a coverage hexagonal grid (see Figure  8 ), conceptually identical to regular grid, with some benefits (see  (Sahr, 2011) ). The idea is to visualise both what was sensed and what remains to be sensed in a given area (here the whole of Paris), to help plan data-sensing missions. We fabricate an hexagonal grid over the extent of Paris (about30s), and remove the hexagons that are in buildings or too far from road (about 60s).
Using trajectory with point clouds We imported the Paris trajectory data (successive position of the Lidar sensor every few ms). In fact, using a constrained data model resulted in discovering errors in the raw trajectory data. Trajectory can be used for filtering point clouds (for instance, 3. 5). We demonstrate the use of trajectory for processing in the following scenario. The goal is to localise all the pedestrian crossings of the Paris dataset (few minutes). We (conceptually) walk along the trajectory, and every three metres we retrieve the patches closest to the trajectory. We use a crude marking-detection function on these patches (percent of points in given intensity range). Based on the score of the detection, we can favour recall or precision. The diminution factor illustrates how much less data we would need to process with the pedestrian crossing method that would follow detection (Table  7 ).
Output Using PostgreSQL drivers/connector ('client reads') We create a Python methods that works on a client computer. It reads uncompressed patches from the server and directly writes them to disk (saving it as Numpy double array). Using seven parallel workers, the result is in Table  8. Client reads data: Streaming to browser We performed a test of point cloud streaming to a WebGL application, using a Node. js server as the 'man in the middle'. The browser is set to a geographical position, and then requests the points around this position to the Node. js server. The Node. js server connects to the point cloud database to request the points. The point cloud server uses indexes to find patches and extract points that are then streamed to Node. js server through cursor use. The Node. js server compresses the point stream and sends it to the web browser.
Processing We demonstrate how easy it is to create new in-base processing methods. As such the methods are only cited to illustrate this. Some details may be found in  Cura (2014). In-base processing Fast prototyping is vital for wider point cloud use. We demonstrate the potential of using high level languages within the database to write simple processing methods. The experiment is not to create state-of-the-art processing methods, but to measure what a Python/R beginner can do in two weeks (designing methods and implementing). • * clustering points using Minimum Spanning Tree • * clustering points with DBSCAN  (Ester et al. , 1996)  • * extracting primitives (plans and cylinder) • * extracting a verticality index (using Independent Component Analysis) • # detecting facade footprint • # detecting cornerstones • # detecting road markings (*: directly working on patches, can be used out of the box on all point clouds; #: working on rasterised point clouds, need to use a point cloud to raster conversion method first) We also used the server for complex out-of-base processing (classifications).
DISCUSSIONS Storage in base Currently point cloud types are strongly constrained, thus adding or removing attributes is not immediate. 2 ) shows that data loading could be much faster. Most notably, directly reading the raw sensor data and streaming it to the database using PostgreSQL binary format would be much more efficient. We emphasise that a relatively recent initiative, PDAL 11  has gained maturity, and would be the ideal candidate to solve these two limitations. The PostgreSQL rule system would be the perfect candidate to use thousands of point clouds together. Metadata can be stored in our server, but a standard minimal data model would be necessary to facilitate exchanges, similar in spirit to the INSPIRE 12  European directive. Conjointly using vectors, rasters and point clouds offers a new world of possibilities.