dataesr/scientific-paragraphs-categorization huggingface dataset:
Generating optimized methods The approach works on a pair of methods, a non optimized method and its optimized version. However, a JIT compiler usually generates optimized methods at runtime while running a program and no optimized methods are available statically. Nowadays, many JIT compilers can be used as a library or directly from the programming language, allowing one to compile an optimized version of method statically if the JIT compiler is provided with a method and runtime type information. Virtual machines usually do not statically use the JIT compiler. However, several JIT compilers provide APIs to be used statically [RSB + 14, TWSC10]. Using the JIT compiler statically is very convenient for tests and static analysis. Even if the JIT compiler can be used statically to generate an optimized method, it needs runtime information that is usually extracted from the previous execution of the methods.
Firstly, the development team wanted to reduce the maintenance cost of a high performance virtual machine. Java's hotspot virtual machine was gifted with billions of dollars of investment, whereas the virtual machine we used has a limited number of engineers. This design reduced the maintenance cost because: • The optimized methods generated can reuse the baseline JIT compiler as a backend to generate machine code with limited changes: it was only needed to add support for hot spot detection and for a few extra bytecodes used only in optimized methods. • The bytecode to bytecode optimizer and deoptimizer can be implemented in the highlevel language, running in the same runtime than the programmer's application, in a similar way to the Graal JIT compiler available for Java's hotspot. Implementing these complex artifacts in a higher level language made them simpler to maintain.
For our optimizer, it means that this pointer has to be correctly restored when the stack is deoptimized. Closures are widely used in Smalltalk: in the Pharo Smalltalk Kernel, 19. 3% of compiled methods holds the bytecode to create a closure. Due to its reflective nature, the programmer can access stack frames at runtime as if they were objects  [DS84]  and edit the temporary variable values or exchange two objects identity by asking the virtual machine to switch all the pointers to one object to target other objects. The stack frame reification is the foundation for the debugger, the exception and the continuation implementations. Exchanging pointer identity is used to migrate instances when one adds at runtime an instance variable to a class and for database proxies. Although these two operations are important, they are uncommon.
Part of these commits consisted in important refactorings with results expected by the development team. A subset of this 3%, however, were real bugs that were introduced by the code committed. The approach identified such bugs to the development team allowing them to detect and fix the bugs early. Most of the reappearing bugs were related to the deoptimization of closures. It is normal since closures with their non-local semantics are the most complex part of Smalltalk execution model. Typically, the closure encapsulating environment was not correctly set. Raising the confidence level in the product. After 18 months of development, all the tested methods passed this validation. The approach led to an important confidence level in the optimizer, allowing to integrate the optimizer in the production VM with lower risks.
Symbolic execution of a lower code representation Symbolic execution of machine code. Our validation relies on a bytecode symbolic execution, and works fine to validate a runtime optimizing bytecode to bytecode compiler. Other optimizing JIT compilers such as Java's HotSpot, however, optimize the methods from bytecode to machine code and not to bytecode. For our approach, we need that the original and optimized version of the method share the same intermediate representation. In the case of Java's HotSpot, one could compare the machine code generated by the baseline JIT compiler and the optimizing JIT compiler. To validate dynamic deoptimization with these machine code versions of the method and our approach, one would need to build an abstract processor for one of the main machine code used, such as x86_64.
This abstract processor would be able to symbolically execute machine code routine with abstract values, to interrupt its execution and to let the user access the abstract stack for stack deoptimization and stack comparison. Such an abstract processor could be restricted by several constraints. Firstly, it needs only to be able to symbolically execute only the instructions generated by the JIT compiler, which are a subset of the available machine instructions. Secondly, as in our approach, it could ignore calls escaping the scope of the optimizations, using an abstract value instead. These constraints may reduce the complexity of an abstract processor, however, such an abstract processor is extremely difficult to implement. Current state-of-the-art machine code symbolic executors usually symbolically execute only part of the program, while the rest of the execution is done with concrete values.
For example, Cadar et al. ,  [CDE08]  concretize floating-point data to avoid their symbolic execution, others MMP + 12] concretize pointers to avoid symbolic execution of memory access, and lastly the S2E platform uses selective symbolic execution to automatically minimize the amount of code that has to be executed symbolically given a target analysis  [CKC11]. Hence, the symbolic executor required for our validation technique seems to require several improvements over the state-of-the-art executors, implying that the application of our approach on machine code would require a massive amount of work to be production-ready. Symbolic execution of a low level intermediate representation. As executing symbolically machine code is a tedious task, one may prefer to execute symbolically a low-level intermediate representation used by the JIT compiler.
Conclusion In this paper, we looked for a solution to validate the correctness of dynamic deoptimization in JIT compilers. This problem applies in any programming language featuring dynamic deoptimization in their execution environment, which is common for language with late binding and running on top of a virtual machine. We proposed a static approach to guarantee that the virtual machine can correctly deoptimize dynamically the runtime stack when needed. This approach relies on symbolic execution to access statically the runtime stack of methods. The solution was validated by an implementation which is now used to test our production code base.
IV. NUMERICAL IMPLEMENTATION OF THE MODEL The model described in the previous section is implemented in Matlab software TM to study the contact between fractal surfaces and smooth surface. The material used for the two surfaces is AISI 316 stainless steel (Young's modulus E = 193 GPa; Vickers hardness, Hv30 = 335; Poisson's Ratio ν = 0. 3). As mentioned above, the Weierstrass function (Eq. (  14 )) is used to generate fractal surface topographies with different fractal dimensions (see figure  1 ). The contact surfaces are discretized in 1024 points with (∆x = ∆x = 1 µm). In order to determine the asperity dimensions, the algorithm detailed in the Ref. 37 is used to determine the number of asperities. This allows calculating the mean values of length and height of asperities at different values of Ds.
Observational data For this study forest sites were retained if the following data were available: (a) short but intensive campaigns making flux and profile measurements within and/or below the tree canopy and, (b) multi-year monitoring of top-canopy fluxes. Through numerous regional projects such as CARBOEUROPE, AMERIFLUX, Fluxnet Canada, OZFLUX, ICOS and NEON, and efforts such as FLUXNET  (Baldocchi and Wilson, 2001) , multiple year-long time series are now commonly available especially for the temperate and boreal zones in Europe, Japan, Australia and North America. Site selection was thus mostly limited by the availability of within-canopy and below-canopy measurements. Eight flux observation sites (Table  2 ) met the aforementioned criteria, and represented various climates from the Mediterranean to the boreal zone and different vegetation types including broad-leaved summer green, broad-leaved evergreen and needle-leaved evergreen.
Data were thus missing from needle-leaved summer green vegetation such as Larch (Larix sp. ) and tropical vegetation, so it was not possible to cover all of the forest types that are considered in ORCHIDEE-CAN. The short intensive campaigns making measurements within-canopy and below-canopy usually extended for periods ranging from several days to a few weeks (Period I; Table  3 ). During intensive campaigns, vertical profile measurements of wind speed, temperature and atmospheric humidity were typically conducted. Such measurements were sometimes complemented with profile measurements of sensible and latent heat fluxes, as well as sub-canopy radiation measurements (Period II and III; Table  3 ). Furthermore, our parametrization and validation set-up required that top-canopy observations had to be available for periods exceeding one year (Period IV; Table  3 ).
A typical long-term set-up measured sensible and latent heat fluxes, longwave and shortwave incoming radiation, wind speed, atmospheric temperature and humidity. Parametrization and validation utilises the ORCHIDEE-CAN model simulations, and so climate forcing data were required to drive the simulations. Site-level weather observation, i. e. , shortwave incoming radiation, longwave incoming radiation, two dimensional wind speed, precipitation, snow, near-surface air pressure and specific humidity were reformatted and gapfilled using the method proposed by  Vuichard and Papale (2015). Weather observations are an integral part of both intensive campaigns and multi-year top-canopy flux monitoring. Hence, within a measurement site, flux, profile, and weather data were usually available at the same temporal resolution and over the same time periods.
Optimization procedure Parametrizing the scaling coefficients and weighting factors enabled us to simultaneously improve the match between the simulated and observed sub-canopy micrometeorology, including temperature and specific humidity when available, and between the simulated and observed top-canopy heat fluxes (LE and H). Within-canopy fluxes were also simulated but are not usually measured. The parametrization made use of an in-house optimization package called ORCHIDAS (ORCHIDEE Data Assimilation Systems;http://orchidas. lsce. ipsl. fr/). ORCHIDAS provides a range of numerical approaches for assimilating multiple data streams in ORCHIDEE. We used the maximum gradient approach to tune the parameters a 3 to a 10 , W br , and W sr for each study site independently. Over the course of several iterations, the optimization approach minimized the mismatch between the model output and the observations, using a gradient based algorithm called L-BFGS-B (Limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm with Bound constraints), which provides the possibility to prescribe boundaries for each parameter  (Byrd et al.
Experiment 1 (EXP1): Single-layer scheme with a prescribed canopy The first experiment was run at the site-level and made use of the default single-layer energy budget scheme. The energy budget scheme was driven by the observed climate forcing and the observed total LAI (Table  2 ). In this experiment, the vertical LAI profile was only used for the photosynthesis module in ORCHIDEE-CAN. Note that vertical LAI profiles cannot be used by the single-layer scheme and the results are therefore limited to the top-canopy fluxes. This experiment was used as the reference simulation to document the performance of the single-layer approach. Experiment 2 (EXP2): Single-layer scheme with a simulated canopy The second experiment was identical to the first experiment except that the LAI was now simulated by ORCHIDEE-CAN, rather than using the observed LAI.
Developing the multi-layer energy budget requires accurate field measurements for model calibration and validation. Here we were able to collect and make use of many of the few datasets that exist for intensive in-canopy profile time series measurements. We suggest that more intensive field campaigns, with soil water content observations, especially during the winter season would help in the development of a more reliable parametrization scheme for the within-canopy eddy diffusivity and soil-atmosphere interface conductance. For future model developments, adding an extra soil-atmosphere interface representation such as moss or herbs on the forest floor would be beneficial for a more complete multi-layer energy budget with the objective of describing the surface-atmosphere interface gas and water vapour exchanges.
Acknowledgements. YC, JR, MJM, JO, KN and SL were funded through  ERC starting  grant  242564  (DOFOCO), and AV was funded through  ADEME (BiCaFF). Geosci. Model Dev. Discuss. , doi:10. 5194/gmd-2016-26,2016 Manuscript under review for journal  Geosci. Model Dev. Published: 25 February 2016 c Author(s)  2016. CC-BY 3. 0 License.
The IQmulus infrastructure IQmulus is a research project funded by the European Commission running from November 2012 to October 2016. Within this project we have developed a software architecture for the processing of very large geospatial data. Figure  1  shows the main components of the IQmulus architecture. The users (typically GIS experts) use the web-based user interface to upload data to the distributed file system (HDFS). They define a workflow using a Domain-Specific Language (DSL). The workflow for the use case presented in this paper is described below. When the users press the Execute button the workflow is transferred to the JobManager, a component managing the infrastructure and the execution of workflows. The JobManager selects the compute nodes in the Cloud on which the workflow should be executed.
For simplicity, or if tile adjacencies are not available, tiles may be processed independently as the scale-adaptive property of  (Demantke et al. , 2011)  has the nice tendency of reducing the optimized scale of analysis so that it is fully contained in the current tile, thus minimizing the edge effects. Alternatively, if tile adjacencies are known, or if the dataset is re-partitioned into tiles (e. g. grid tiling), then it is easy to list the few number of tiles that may be read while computing the dimensionality features of a given tile. The proposed method when the dataset is sorted in sequential acquisition order (either because it has not been shuffled by a preprocess or because per-point acquisition timestamps are available) is to simply process each tile with a read access to the immediately preceding and succeeding tiles.
This has the combined advantage of (i) not requiring any retiling or shuffling as raw datasets are actually acquired and stored in time-sequential chunks rather than spatial partitions and (ii) that the processing is then almost insensitive to georeferencing issues, as georeferencing typically drifts in a way such that relative positioning errors are negligible within the short timespan of a few consecutive tiles. This last property is actually very important as the PCA is very sensitive to slight misalignments of lidar point clouds due for instance to repeated scans of the same street.
As the primary abstraction in Apache Spark, the resilient distributed dataset (RDD) plays a key role to organize data and achieve parallel computation. Our implementation can be summarized as a series of manipulations of RDDs including creating new RDDs, transforming existing RDDs, and performing operations on RDDs to generate results as shown in Figure  2. The input data is a point cloud with seven attributes, linearity, planarity, scattering, omnivariance, anisotropy, eigenentropy, and change of curvature,  (Demantke et al. , 2011). The output is point cloud data where each point got a class Id, indicating whether it is classified as belonging to the class 'tree' or the class 'non-tree'. The first RDD, RDD 1, is initialized from a list of file name strings through the operation parallelize.
Data access of the Fat Client In order to access the data provided by the processing services of the IQmulus infrastructure, we developed a data access module for the IQmulus Fat Client. The module utilizes Qt 5 for user authentication and data access via the web-based interface to the HDFS of the IQmulus infrastructure. More specifically, we developed a connection manager to establish secure communication with the IQmulus Data Access Service (DAS) and adapted the Webkit-based HTML engine provided by Qt (QWebView) to download and visualize data sets from the HDFS. The main components involved in HDFS data access in the IQmulus Fat Client are shown in Figure  5. In the following we provide a short overview of the functionality of the components of the data access module: DASConnector The DASConnector is responsible for configuring a secure connection to the IQmulus Data Access Service.
This requires the initialization of SSL parameters (we currently use TLS1/SSL3), loading the certificate of the IQmulus Data Access Service, and setting up security / functionality related properties of Qts HTML engine (e. g. configuration of JavaScript capabilities, WebGL, etc. ). The connection relevant settings are subsumed in a Qt class termed QNetworkAccessManager. A relevant property of this class is the internal handling of cookies, which are required during the authentication process of the IQmulus Data Access Service and must remain accessible for each connection request after authentication. The configured Net-workAccessManager is provided to the other components for secure access to the IQmulus Data Access Service. IQmulusWebbrowser The IQmulusWebbrowser is an extension of the HTML engine provided by Qt (QWebView).
Data description The workflow has been applied to a 10km-long mobile mapping dataset acquired by IGN's stereopolis vehicle,  (Paparoditis et al. , 2021)  acquired within an hour of operation in downtown Toulouse, France using a multi-return Riegl VQ-250 lidar scanner. Once georeferenced with all attributes this lidar dataset is constituted of 1 012 160 278 lidar points and 83GB.
The workflow in the IQmulus user interface In the main user interface developed in the IQmulus project users can edit processing workflows using a Domain-Specific Language (DSL). A DSL is a lean programming language that is targeted to a specific application domain or use case. We have developed a method to specify the grammar and syntax of such a DSL so that it is readable and understandable by domain experts. The method encompasses a domain analysis step to make sure the DSL uses the vocabulary and terminology known to the domain experts, cf. (Krämer, 2014). Figure  9  shows a screenshot of the workflow editor from the IQmulus main user interface. The screenshot shows the DSL script for the workflow described in this paper.
Visualization results. For the purpose of visualization of the resulting data set using the Fat Client, we developed a category viewer which performs a distinct color mapping of the points of different trees. Trees may be highlighted based on their ID and colors as well as point rendering parameters may be manually specified (see Figure  10 ). The visualization of the Toulouse data set allows for interactive navigation through the data. Due to the incremental refinement of the visualization (asynchronous download of additional data and subsequent rendering using an adjustable maximum number of rendered points) and the different constraints imposed by clientside hardware as well as network connection properties, it is hard to come up with general performance data. The performance of rendering the Toulouse data set using the Fat Client from local harddisk with different upper boundaries for rendered points is shown in Table  1.
Next steps At the current state of affairs, the full use case is implemented and running in the IQmulus infrastructure. A next step is to actually exploit the IQmulus visualization possibilities to analyze the geometric quality of the results in an interactive way. Simultaneously the computational performance of the workflow as a whole for different settings is under study using the implemented automated scalability testing framework. To verify the portability of the platform, the use case will also be evaluated on different mobile mapping data sets obtained by different systems in different road environments.
CONCLUSIONS In this paper we have presented results from the FP7 IQmulus project. The general infrastructure for storing, processing and visualizing big spatial data has been described and demonstrated on the so-called urban showcase. This showcase considers the efficient processing of a huge lidar mobile mapping data set consisting of 1 billion points sampling 10 km of urban streets in Toulouse. IQmulus is able to efficiently extract and visualize all points sampling the 4000+ individual trees sampled by the point cloud. More information on and results from the IQmulus project can be found on the IQmulus website, http://www. iqmulus. eu.
Background. Large scale metagenomic projects aim to extract biodiversity knowledge between different environmental conditions. Current methods for comparing microbial communities face important limitations. Those based on taxonomical or functional assignation rely on a small subset of the sequences that can be associated to known organisms. On the other hand, de novo methods, that compare the whole sets of sequences, either do not scale up on ambitious metagenomic projects or do not provide precise and exhaustive results. Methods. These limitations motivated the development of a new de novo metagenomic comparative method, called Simka. This method computes a large collection of standard ecological distances by replacing species counts by k-mer counts. Simka scales-up today's metagenomic projects thanks to a new parallel k-mer counting strategy on multiple datasets.
Results. Experiments on public Human Microbiome Project datasets demonstrate that Simka captures the essential underlying biological structure. Simka was able to compute in a few hours both qualitative and quantitative ecological distances on hundreds of metagenomic samples (690 samples, 32 billions of reads). We also demonstrate that analyzing metagenomes at the k-mer level is highly correlated with extremely precise de novo comparison techniques which rely on all-versus-all sequences alignment strategy or which are based on taxonomic profiling.
It better handles the comparison of large read sets and provides a read sub-set representation that facilitates result analyses and reduces the disk footprint. Seth et al. (2014)  used the notion of shared k-mers between samples for estimating dataset similarities. This is a slightly different problem as this was used for retrieving from an indexed database, samples similar to a query sample. More recently, two additional methods were developed to represent a metagenome by a feature vector that is then used to compute pairwise similarity matrices between multiple samples. For both methods, features are based on the k-mer composition of samples, but with a feature representing more than one k-mer and using only a subset of k-mers to reduce the dimension  (Ulyantsev et al. , 2016; Ondov et al.