dataesr/scientific-paragraphs-categorization huggingface dataset:
Meteorological observations The meteorological observations assimilated in WRF-FDDA include the standard measurements of 10-m wind, 2-m temperature and moisture fields from World Meteorological Organization (WMO) surface stations at hourly intervals and radiosondes at 12-houly intervals, as shown in Figure  2. There are five upper-air observations spread across the 9-km grid. However, none of them are located on the 3-and 1-km grids and only three WMO stations are available on the 1-km grid. The 20-minute wind profiles measured by the lidar are used in this study. The lidar is deployed in Lawrence, IN, about 15 km to the northeast of downtown Indianapolis. The lidar has operated and provided data continually since April 2013, with a 6-month gap at the end of 2015 when the system was temporarily removed and upgraded.
These profiles are used together to estimate the PBL depth. The 20-minute wind profiles measured by the lidar are also available for assimilation into WRF. In addition to the WMO and lidar observations, the winds, temperatures and moisture fields observed from the ACARS are also assimilated. The ACARS observations are available at times when observations are taken from commercial aircraft, and are distributed in the vertical from near-surface to cruising altitudes above the tropopause, and in the horizontal along the flight tracks. PBL data density is highest near major airports as aircraft ascend or descend.
The NARR analyses were downloaded from the Research Data Archive (RDA) maintained by the Computational and Information Systems Laboratory (CISL) at the National Center for Atmospheric Research (NCAR). In addition to assimilating observations during the model integration, the initial condition fields are further enhanced by sonde and surface data through the WRF objective analysis process, Obsgrid, using a modified Cressman analysis method  (Deng et al. 2009 , Rogers et al. 2013). The three-dimensional (3D) analyses and the surface analysis fields used for analysis FDDA are also enhanced by the objective analysis process and are defined at threehour intervals  (Deng et al. 2009) , which means that WRF solutions are nudged towards more accurate analyses than NARR on the WRF grid by including observations.
A calm wind threshold was used in this study to remove situations with very light winds (less than or equal to 1 m s -1 ) for the wind direction statistics calculation because the wind direction for near-calm wind is undefined. Note that due to the limited size of the 1-km grid, there are only three WMO surface stations and no sondes available within the grid, thus reserving a separate set of WMO observations for independent verification (as done in  Rogers et al. 2013 ) is not possible. Since the WRF FDDA technique uses a relaxation term in the model tendency equations rather than conform the model solution to an observation, one should not expect data assimilation to produce an exact match between the modeled and observed values, and spatial correlations in the nudging coefficients limit the extent of the observation influence.
Note that PBL depth is not assimilated, thus it is an independent variable for validation. WRF model can produce a diagnosed PBL depth from its PBL sub-model or PBL scheme. Several methods have been used to diagnose the PBL depth within the different PBL schemes that have been proposed in the literature, usually based on either the vertical thermal profile or the vertical profile of TKE as predicted by the specific PBL scheme. Here, we used the 2. 5-level MYNN scheme  (Nakanishi and Niino 2006)  which is a TKE-predictive scheme. We compare the model-predicted PBL-depth with the observed PBL depth to measure how well the model represents PBL processes. Nighttime comparisons of PBL depth were not performed due to the fact that modeled PBL depth in stable conditions is often less reliable and harder to define, and thus nighttime tower data are not currently utilized in our inverse CO 2 flux estimates.
Evaluation of model-predicted PBL depth using the Halo Doppler Lidar data The PBL depth for this study is manually estimated from the lidar observations for each 20-min time period, identified by large gradients in Signal-to-Noise Ratio (SNR) and the height where the vertical velocity variance becomes small (less than ~0. 1 m 2 s -2 ). NOAA is currently implementing and testing an algorithm to automate the PBL depth estimation process for the INFLUX data set. As an example, a comparison of PBL structures between WRF and the INFLUX lidar observations at Indianapolis for 29 and 30 August 2013 is shown in Figure  7. Generally, the model-predicted TKE structures are highly correlated with the lidar-observed vertical velocity variances and large gradients in SNR. The diurnal variation of the PBL structure can be clearly seen within both model output and observations.
(2016)  and kept identical across the four inversions. Therefore, the differences in the inverse emissions correspond to the impact of different transport model realizations, and more precisely the impact of the FDDA systems, assimilating various data sources. Figure  9  illustrates the different influence functions (or tower footprints) for the 12 tower locations for one single observation time. The extent and the main axes of the tower footprints vary significantly depending on the WRF-FDDA model results, which translates into varying spatial attributions of the observed atmospheric CO 2 mole fractions. Overall, four atmospheric inversions were performed for the period September-October 2013 producing five-day emissions estimates at 1 km resolution over the domain. Figure  10  shows the relative differences among the different transport solutions (influence functions) in space over the simulation domain (equivalent to the 1-km grid of WRF) represented by average pairwise differences.
INTRODUCTION OWL2  (Motik et al. , 2009)  allows users to write explicit, formal conceptualizations of domains models. It is build upon RDF and RDFS and has the same kind of syntax. As the main primitives of RDF/RDFS are related to the organization of the vocabularies in typed hierarchies using subclass and subproperty relationships, OWL 2 brings more language primitives besides those presented in its previous version to allow richer expressiveness. For example, in addition to the property unqualified cardinality restrictions available in OWL 1, in OWL 2, even the qualified cardinality restrictions are possible. Also, the assertions that an object property is symmetric or transitive provided by OWL 1 was supplemented in OWL 2 by the reflexive, irreflexive and asymmetric object properties assertions.
Skipping Previously Monitored Areas As shown in Figure  7 , to avoid monitoring the same area multiple times, all samples, which were taken within the area monitored during the exploration phase, are internally stored. For this purpose, PdUC maintains a list containing the location of the central position of all spirals with their respective radius to determine the monitored areas (as a circumference determined by a center and a radius). Next, in the exploration phase, all points inside these circles are omitted for the sake of celerity, as shown in line 9 of Algorithm 2. To prepare a suitable data environment, we have created various pollution distribution maps representing ozone levels to be used as inputs for testing. These pollution maps were generated using the R Graph tool  [43] , and following a Kriging-based interpolation  [44].
In particular, a Gaussian distribution is used to adjust the parameters coming from random data sources of ozone concentration. The actual values range between 40 and 180 ppb, thereby providing a realistic ozone distribution. Figure  9  shows some samples of the created maps, which have the highest pollution concentration (areas in red) located at completely different positions due to the stochastic scenario generation procedure adopted. Using the previously created data as input, we have run several simulations using OMNeT++, comparing our protocol against both the Billiard and Spiral mobility patterns. In the simulator, we have created a mobility model implementation of PdUC. In addition, to simulate the sampling process, we have configured OMNeT++ to periodically perform measurements taken from the pollution distribution map defined for the test.
Acknowledgment This work has been partially carried out in the framework of the  DIVINA Challenge Team , which is funded under the  Labex MS2T program. Labex   MS2T  is supported by the  French Government , through the program "Investments for the future" managed by the  National Agency for Research (Reference :  ANR-11-IDEX-0004-02 ). This work was also supported by the  "Programa Estatal de Investigación, Desarrollo e Innovación Orientada a Retos de la Sociedad ,  Proyecto I+D+I TEC2014-52690-R", the "Programa de becas SENESCYT de la República del Ecuador ", and the  Research Direction of University of Cuenca. The received funding do not lead any conflict of interests regarding the publication of the manuscript.
Background: In the last decade Photogrammetry has shown to be a valid alternative to LiDAR techniques for the generation of dense point clouds in many applications. However, dealing with large image sets is computationally demanding. It requires high performance hardware and often long processing times that makes the photogrammetric point cloud generation not suitable for mapping purposes at regional and national scale. These limitations are partially overcome by commercial solutions, thanks to the use of expensive and dedicated hardware. Nonetheless, a Free and Open-Source Software (FOSS) photogrammetric solution able to cope with these limitations is still missing. Methods: In this paper, the bottlenecks of the basic components of photogrammetric workflows -tie-points extraction, bundle block adjustment (BBA) and dense image matching-are tackled implementing FOSS solutions.
Background The generation of dense point clouds has been traditionally performed using active sensors like laser scanners due to the easiness, speed and ability to quickly capture millions of points. In the last decade, Photogrammetry has been living a second life pushed by the recent developments in Computer Vision, providing very dense and accurate point clouds compared to LiDAR techniques. Image-based approaches have the advantage of being cheaper than active sensors and they can provide colour and range information just in one acquisition. Photogrammetry is therefore replacing range techniques in many applications (such as archaeology, geology, etc. ) due to their reduced costs and the introduction of turnkey platforms such as unmanned aerial vehicles (UAVs) that makes possible the acquisition of large datasets.
The trend in the photogrammetric community is to adopt larger datasets, increasing the image overlaps and the extension of the surveyed area, and to increase the resolution of the model using larger image formats. Many commercial solutions (Autodesk 123D Catch  [1] , Agisoft PhotoScan  [2] , Pix4Dmapper  [3] , etc. ) and Free and Open-Source Software (FOSS) solutions (MicMac  [4] , Visu-alSFM  [5] , Bundler  [6] , openMVG  [7] , SFMToolkit  [8] , MVE  [9] , Theia  [10] , structure-from-motion  [11] , etc. ) have been developed in the last years showing good results in many applications. The results achieved by image matching techniques are so promising that some countries are studying the possibility to replace periodic LiDAR acquisitions with airborne photogrammetric acquisitions. However, most of the solutions concentrated on the generation of 3D models on relatively small areas, and solutions for point cloud generation at a regional or national level are still unluckily limited.
Only few commercial software solutions (such as Acute3D  [12] ) are able to process city-scale regions and they are usually coupled with expensive dedicated hardware to generate the final products in a reasonable amount of time. Other solutions have also been specifically tailored to deal with large city-scale images datasets  [13] [14] [15]. These solutions deal with clustered image sets, i. e. the total number of images is very large but even if the images belong to the same ares, these researches bypass this problem clustering the images in groups (for example images of a specific monument). In these approaches the technical challenge is more focused on the processing of the different chunks (to obtain the models of the different objects/clusters) and in their merging in a unique model, rather than in the processing of the whole block in a unique step.
The algorithms are implemented as FOSS tools. The FOSS photogrammetric tool MicMac has been adopted as starting point. The use of clusters has been foreseen as possible solution to distribute the jobs on many machines, making possible the processing of big datasets. The paper is organized as follow: the theoretical background and the problem statement are discussed in "Theoretical background and problem statement" section. The general structure of the MicMac tool is presented in "Methods" section while the algorithms are discussed in "Tie-points reduction algorithms" section and Distributed computing section. "Implementation" section contains implementation details. Experiments of test datasets are reported in "Results and discussion" section while the conclusions and future developments are discussed in "Conclusions" section.
The DSM can be therefore used to generate the true-orthophoto. The orthophoto is an orthogonal representation of the scene where the projective deformation of the images are removed exploiting the information given by the DSM. Photogrammetric workflows are nowadays automated processes and the human interaction is limited. The development of large format cameras  [20]  and the increase in the image overlaps has led to the increase in the amount of data captured on the same area. This has resulted in a wider computational request to process large image datasets. The possibility to parallelize the process on CPUs and GPUs has just partially solved this issue as some steps cannot be easily divided in different processes and the steps are often too time consuming to make the whole process affordable and timely.
Tie-points reduction algorithms The memory requirements for the BBA are in some cases simply unaffordable for large datasets. We propose an elegant solution that consists of running a tie-points reduction before the BBA. The tie-points reduction decreases the number of tie-points effectively decreasing the memory requirements of the BBA. The general idea is that a sufficient number of well distributed tie points in the overlapping images is sufficient to assure a stable and reliable image orientation, as it was commonly done in the traditional analogue Photogrammetry. Two different algorithms of tie-points reduction have been developed. A similar idea of tie-points reduction has been already introduced in a study conducted by Mayer  [24]. However, in that case the tie-points reduction was part of the BBA.
On the other hand, the proposed solutions are intended as a preliminary step to run before the BBA. In this way, the memory requirements of the BBA are decreased since the beginning preventing the need for large memory use. In the following subsections the overview of the two different algorithms is given. The first algorithm performs the reduction using only the information provided by the tie-points in image space, while the second approach exploits the information provided by the tie-points both in image and object space to assure a better distribution in both spaces. For an extended description and a study on the impact in the quality of the produced point cloud using the reduced set of tie-points, the reader can refer to the online report of the project "Improving Open-Source Photogrammetric Workflows for Processing Big Datasets" funded by the Netherlands eScience Center  [25].
Tie-points extraction The current single-machine implementation available in the Tapioca command of MicMac is the following: • For each image of the dataset, the set of corresponding overlapping images is first determined. This process can be eased by the image geo-tag, reducing the number of homologous images to a small set. If the geo-tag is not available, then the whole dataset needs to be considered; • For each image the features are extracted; • For each stereopair, the features are matched to determine the candidate tie-points. The extraction and matching process are performed in parallel exploiting all the cores of the machine. However, this process is insufficient to process very large datasets. The distributed computing solution is depicted in Fig. 4. The algorithm splits the list of stereopairs (image pairs) in chunks of N stereopairs where N is configured by the user: the first chunk contains the stereopairs from the position 0 to N-1 of the original list, the second chunk contains the stereopairs from the position N to 2N -1 of the original list and so on.
Dense image matching The current single-machine implementation available in MicMac to generate the dense colored point cloud is the following: (i) the dense image matching is run and a dense depth map, i. e. a Digital Surface Model (DSM), is initially generated; (ii) using the DSM and the image spectral information, a true-orthophoto is then produced; (iii) the true-orthophoto and the DSM are finally used to generate the colored dense point cloud. The process is generally divided in tiles and each of them is processed independently by a core. The GPU can be also adopted for this parallel process. However, the process can take very long times on a single machine. For a large dataset, a distributed computing solution is therefore required. The distributed computing dense image matching algorithm is shown in Fig.
Implementation The tie-points reduction algorithms are implemented as stand-alone commands within the MicMac tool. The names of the commands are RedTieP and OriRedTieP. The distributed computing algorithms for tie-points extraction and dense image matching are implemented in pymicmac (https://github. com/ImproPhoto/pymicmac). Pymicmac is a Python interface for MicMac workflows execution. It allows to define through a XML file photogrammetric workflows to be executed with MicMac. A sequence of MicMac commands is specified in the XML file. During the execution of each command pymicmac also monitors the usage of CPU, memory and disk. Pymicmac can also define parallel commands via XML, using a set of commands to be executed in parallel. This is what is used for the distributed computing tie-points extraction and the distributed computing dense image matching.
Results and discussion A set of experiments has been executed using the datasets reported in Table  1. The two datasets have been specifically chosen to assess the performances of the tie-points reduction and the distributed computing. The first datased was acquired by an Unmanned Aerial Vehicle over a urban area in Gronau (Germany). The image overlap is quite irregular with 3 cm GSD. This dataset was chosen to assess the effectiveness of the tie-point reduction with highly overlapping and convergent image configurations. The second dataset is part of the acquisition yearly performed by the Waterschapshuis in The Netherlands: the image overlap is 60% alongtrack and 30% acrosstrack with 10 cm GSD. The number and size of the images prevent their processing on a single machine and it therefore represents a challenging dataset for the distributed computing task.
Tie-points extraction The Gronau dataset has been used considering two different configurations: (i) the single-machine processing and (ii) the cluster processing. In both cases the feature extraction process has been supported by the geo-tag information. In the first case, this operation was executed with MicMac, i. e. using Tapioca, in a Ubuntu 14. 04 Virtual Machine (VM) with 4 cores. Tapioca took 292,318. 26 seconds, used all the available cores, and the average used memory was 13% out of the available 13. 4 GB. Figure  8  (left) depicts the CPU and memory usage of Tapioca. In the distributed computing solution, the DAS 4 cluster  [28]  in TU Delft was used. The cluster has 32 nodes (only 23 are currently usable), and each node has 16 hyperthreaded cores at 2. 4 GHz (512 hyper-threaded cores in total, only 368 can be currently used) and 24 GB RAM (768 GB in total, only 552 GB can be currently used).
Tie-points reduction and image orientation The online report  [25]  of the project "Improving Open-Source Photogrammetric Workflows for Processing Big Datasets" contains a thorough study of the impact of a reduced set of tie-points in relation to the quality of the produced final point cloud. In that study various datasets and values for the options of the tie-points reduction  commands were tested. In this paper, only some meaningful comparison are presented. In particular, the comparison of the BBA results, i. e. Tapas in MicMac, using all the tie-points or a reduced set of tie-points are considered. The two commands RedTieP and OriRedTieP are tested. As in the previous section, the Gronau dataset has been used. In order to assess the quality of the orientations, the BBA residuals have been measured on the tie-points re-projection, on Ground Control Points (GCPs) and Check Points (CPs).
The GCPs have been distributed using an uneven and very sparse distribution in order to better assess the possible deformations on the CPs due to the reduction of the used tie-points. Additionally, the number of iterations to converge the process has been recorded as well. The MicMac command GCPBascule is used to absolutely orient the images in the reference system and measure the residuals on the GCPs and CPs. GCPBascule performs a simple 7-parameter transformation where the parameters are estimated in a least square approach. This tool was preferred to Campari (traditional BBA, with GCP embedded in the process) because it does not introduce any nonlinear effect in the model built by the photogrammetric data (i. e. the output of Tapas) and it allows a more frank estimate of the goodness of the photogrammetric models.
Conclusions An innovative approach devoted to the efficient and reliable photogrammetric processing of large datasets has been presented in this paper. The bottlenecks in the basic components of photogrammetric workflows when dealing with large datasets have been pointed out at first. For each of them an efficient solution has been conceived using the FOSS tool MicMac. The tie-points extraction and the dense image matching (point cloud generation) are similar problems. In these cases, the amount of data to be processed is very large and it cannot be reduced. The positive aspect is that the process can be parallelized and distributed on large clusters, as demonstrated in this paper. On the other hand, the BBA is not easily divisible and an algorithm to optimize the amount of processed data has been developed.
The developed algorithms allow the processing on user-grade machines, preventing the need for more powerful devices on large datasets. The achieved tests have provided encouraging outcomes in all the developed tools, showing that the developed algorithms decrease the computational time. This paper represents a proof of concept and many details will be improved in the future in order to further optimize the whole process. The tie-points reduction would greatly benefit from the division of the two tasks (feature extraction and feature matching), reducing the number of stereopairs that have to be matched in the cluster processing. The dense point cloud generation would need some improvement in the radiometric equalization of the point cloud color. The Zwolle dataset was challenging due to the high number of (large format) images processed.