dataesr/scientific-paragraphs-categorization huggingface dataset:
A recent survey classifies Bag Of Words techniques as a third category  [15]. Bag Of visual Words (BoW) is one of the most common approach in robotics. The BoW approach is the result of an analogy with words in textual retrieval: concepts developed for large full-text search engines are extended to image retrieval in large database and object recognition  [16, 17]. Each image is represented as a document containing words (the features). The set of visual words, or statistics obtained from this set, is therefore coded as an intermediates representation of the image information, which is sometimes called Mid-Level features  [18]. A request on a corpus can be executed with one or several words from the set and a ranking is established according to their distinctiveness across the documents. Bag Of Words and these techniques are globally efficient, but their complexity can be a burden for embedded systems and fast computation requirements.
However, they handle the feature point detection with various algorithms, SIFT (Scale Invariant Feature Transform)  [31]  and FAST (Features from Accelerated Segment Test)  [32] , and various hand tuned parameters and state that it is still an open issue. In this way, their CNN has been trained and evaluated on datasets whose image pairs are rectified (same pixels resolution and strictly the same point of view). In our work, we make the hypotheses that point of view of the same scene could vary. Furthermore, we evaluate our approach on datasets with major appearance changes in order to estimate its robustness against long-term changes. In this paper, we justify the use of Harris feature detector and propose a new feature descriptor which improves experimental results dedicated to the visual place recognition problem and its extension to multimodality.
2. We show that corner detectors are more suitable to multimodal datasets than patch-based methods such as Differences-of-Gaussians. We draw the relevant consequences and propose a new approach to describe the corners' neighborhood, we call PHROG (Plural Histograms of Restricted Oriented Gradients). We introduce the multiscale approach in Section 3. 3. Specificities are given on the description pattern used in Section 3. 3 and how we extract information with a HOG-like description in Section 3. 4. In Section 3. 5, we explain how a few additional computational steps can improve the features similarity evaluation. Finally, Section 3. 6 presents explanation on the Bag-of-Words retrieval process we use. The method overview is represented in Figure  2.
These sets of connected pixels are generally called blobs. As observed in  [22] , objects in infrared images look different than in color images. Objects globally loose their textures: for example, a screen printing with many colors and shapes seems homogeneous in infrared images (Figure  3 ). Another noteworthy example is that of vegetation which appears much darker in visible imagery than in infrared one (a landscape looks "snowy" as in Figure  4 ). Knowing what happens globally to textures across different spectral ranges, we have decided to validate the detection step itself in order to choose the best detection approach, namely corner detection or patch of interest detection. We have taken pairs of images from the visible-LWIR dataset introduced in  [38]. Each pair has been adjusted in order to make viewpoints and resolutions of both images (infrared and visible) exactly the same.
The tuning of the quality ratio can provide significant improvements on repeatability for some image pairs. Globally results are better when a low quality level (equal to 0. 0001) is chosen. Minimum distance between two pixels selected as features is a less determinant criterion on repeatability and we have chosen to keep a minimal distance of 2 pixels in order to detect less points while preserving the repeatability. We define a tolerance of two pixels in positions returned by each algorithm. Figure  7  gives the repeatability results according to each pair of the visible-LWIR dataset  [22]. For each image pair, the repeatability is far better with a corner-like detection approach. We assume that these results are due to the fact that object's shapes are mostly the same in both modalities whereas textures tend to differ.
We use Precision-Recall curves and their AUC (Area Under the Curve) as an evaluation of the performance of PHROG with different scale parameters. We give, in Figure  8 , an example of the results obtained on the VPrice dataset (which is introduced in Section 4. 4). We find that 5 scale levels are appropriate because the computation of supplementary levels brings no significant benefits on matching results and is a good compromise for the memory cost of our proposal. We make no further corner refinement or additional image interpolation.
Bag-of-Words Retrieval A common strategy used for image retrieval is to compute a Bag-of-Words as explained in  [16]. Used data are divided into two sets: the first composes the memory and the second one is referred to as "live" or "online" sequence because this information is usually acquired progressively during the localization process. Bag-of-Words retrieval takes advantage of a preprocessing step applied on images from the memory part of the dataset. A global schema of the Bag-of-Words process is presented in Figure  11. All feature points are first extracted from all images from the memory. A K-means algorithm then separates the whole descriptors space in K clusters (from 1000 to 8000 clusters according to the different test cases). The set of each cluster's mean descriptor is outlined as the vocabulary.
All the descriptors are then quantized relatively to this vocabulary. Each word of the vocabulary is subsequently weighted by a TF-IDF score (Term Frequency-Inverse Document Frequency). The calculation of TF-IDF scores is a product of two terms: • tf i,j (Term Frequency) is defined as follows (Equation (  6 )): tf i,j n i,j ∑ |I| k=1 n k,j (6) where n i,j is the number of occurrences of the word having index i in the dictionary, in the image having index j in the memory database, and |I| is the total number of images in the corpus. • idf i (Inverse Document Frequency) is defined as follows (Equation (  7 )): idf i log |I| |{i j : w i ∈ i j }| ( 7 ) where |I| is the total number of images in the corpus and |{i j : w i ∈ i j }| is the number of images where the word w i occurs. Clustering in descriptors space (K-means clustering).
Hard quantization Visual words histogram Figure  11. Bag-of-Words approach: a "learning dataset" is first used to cluster extracted features in the descriptor space. Features are then quantified according to nearest cluster. The resulting representation of the images are visual words histograms. Thus, scores tf-idf i,j = tf i,j × idf i allow us to specify words which are the most relevant both in a given image from the memory and in the whole database. During the processing of the "on-line" sequence, we extract features from each frame and quantify them relatively to the dictionary computed with memory sequence. The most relevant words found in the current image figure out the closest image from the memory.
PHROG Applied to the Visual Place Recognition Problem As mentioned in the introduction, several works and developments have been made to deal with the long-term visual place recognition problem. These works focus on two main approaches: the first one concerns with the image description itself and tried studies to improve the image matching one by one. The other way considers a sequence and not only a single image. Thus, temporal coherence allows to envision a filtering on faulty image matching. Works presented in this paper remain to the first one. Our aim is to improve the matching of two images of the same scene when sensors with different spectral ranges are used. In order to evaluate our method and usual well-known methods in the literature, we use several datasets. Some come from recent available anterior works in the community.
We have prepared our own additional dataset to evaluate the different techniques against increased constraints and difficulties. For each test case, we take one modality from a dataset (being visible or infrared) and build a codebook off-line. We refer to this subset as the "memory". After a few experimentations, we have determined that codebooks composed of 1000 words are a good compromise between efficiency and computing speed. We refer to the other subset as the "live" sequence. It is composed of images from the other modality composing the dataset. In other words, for each test case, the codebook is built thanks to a single modality and the image retrieval step is always performed with another modality. We try to match each image composing the live subset by founding the closest image in the memory according to the histograms of words in each image.
If both memory image returned by the algorithm and live image come from the same pair, we consider the test as a true positive, else as a false positive. We experiment this method on following cases presented in the next parts. We give for each one the ratio between true positives and total number of live images in the dataset. We also show in Figure  12  an example where our proposed algorithm fails because of dominant aliasing between both scenes (shape of the railroad, building on the right side, background with mountains, etc. ).
Experimental Setup We have led the experimentations on a desktop computer running Ubuntu 16. 04 LTS with an Intel Core i7 processor and 8 GiBytes of RAM. We have limited to 10, 000 the number of feature points detected in each frame. We have given details about the description pattern above (Figure  9 ), and the HOG modification. A PHROG descriptor vector is thus 2 times smaller (64 values) than a SIFT one. With such parameters, computing a codebook of 1000 visual words lasts between one hour and two hours depending on the considered dataset.
Experimentations on Visible and Near-InfraRed Images The first dataset comes from EPFL (École Polytechnique Fédérale de Lausanne in Switzerland) and is presented in  [40]. It is composed of several sorted subsets: country, field, forest, indoor, mountain, oldbuilding, street, urban and water. Each subset is composed of about 50 images pairs. Each pair is comprised of a visible image and a near-infrared (NIR) counterpart. Images from each pair have been rectified by the authors so that viewpoints and resolutions are strictly the same. A pair example is given in Figure  13. We choose to focus on urban, street and country subsets which are closer to robotics and navigation use-cases. The country subset was indeed used in  [22]  as the learning dataset for the whole experimentations. We choose alternatively the NIR images and the visible set as the memory and its live counterpart.
We give results in Table  1 , Figures  14 15 16 for each subset and detector-descriptor association and we show an example of a confusion matrix obtained on the urban dataset with PHROG (Figure  17 ). We can easily see that the diagonal of the confusion matrix presents the lowest distances between images: datasets have been synchronized in order to compute performance in a simple way. It allows us to consider that images associated on the diagonal are the truth and the others are distances computed for false matchings. This configuration of the confusion matrices makes possible the computation of Precision-Recall (PR) curves and their AUC (Area Under The Curve). PR curves have been widely used to evaluate search processes: a perfect classifier should present a PR curve going from (0, 1) to (1, 0) points with an area as close as possible to 1.
With such consideration, the best method from a set is the one which presents the highest AUC. As it can be seen, NIR images are not so different from visible images. It appears that image retrieval with traditional features gives very good matching ratios (see urban and street datasets). By the way, results on country dataset are more disparate: one possible explanation is that vegetation is much more present in this dataset and hence the gradient reversal problem appears more frequently than with building and inorganic matters. PR curves show that PHROG is the best on "urban" and "street" datasets: its AUC is higher by at least 9% than other evaluated methods. On the "country" dataset, PHROG does not hit the best AUC but its precision remains the best when recall is low: it is significant because it means that if the searching process returns only one result, PHROG gives the best answer.
Experimentations on Visible and Long-Wavelength InfraRed Images This part handles infrared images from a spectral range much further from visible spectrum than the previous one. We used the visible and Long-Wavelength InfraRed dataset introduced in  [22, 38]. This spectral range is appreciated for its thermal response and its possible usage as night vision system. This dataset encompasses outdoor view scenes of the Barcelona campus. Images from this dataset have also been rectified by their authors so that resolutions and viewpoints are the same. An instance from the dataset is given in Figure  18. We try two situations by switching the modality used as the memory. The results are summed up in Table  2  and Figure  19  presents the PR curves. We can observe that matching ratio are slightly better when the LWIR subset is used as the source of the codebook composition.
We suppose that less textured images enhance the computation of the visual words dictionary. Besides, our descriptor PHROG demonstrates its value on this dataset. PHROG gives significantly better results than the other methods with exactly the same parameters than on the EPFL dataset. However, we can note that AUC are very low (below 5%), even if the precision of PHROG is good when the recall is low. It means that the distances between images for true and false matchings are very close from each other but discriminant enough when there is a need to retrieve only one image. The further spectral ranges are, higher the differences between two frames are and the less efficient the usual methods become.
Experimentations on VPrice Dataset In this part, we use a dataset which is composed of images picked up from the VPrice dataset (https://roboticvision. atlassian. net/wiki/pages/viewpage. action?pageId=14188617); an example from this dataset is given in the Figure  20. It has been designed to evaluate long-term place recognition. Both sequences from the dataset have been taken using sensors dealing with the visible spectra but at two different times so that seasonal changes are preponderant. Compared to the two previous datasets, images are not rectified so that two images from the same pair have a different point of view. References  [7, 17]  present interesting results on this dataset but give no details on feature extractors performance on single images: they only introduce an average accuracy of their respective method on the whole sequences matching.
Experimentations on our Visible-SWIR Dataset The last test case implies our own dataset. It has been made with a visible camera and a SWIR sensor. This dataset is much more challenging because image resolution and viewpoints are not the same. Evermore, one subset has been acquired several months later so that vegetation appearance is very different (with and without leaves). Figure  22  is an instance of this dataset. The SWIR set, with the smallest resolution, has been used as the memory first and the two modalities have been switched then. Results are presented in Table  4  and Figure  23. Our method shows again good results compared to the usual descriptors. A notable remark has to be made concerning the choice of the memory: results are better when the SWIR composes the memory than the visible one.
Whereas a few works emphasize on these issues, we have considered in this paper the constraints introduced by the sensor invariance need, particularly when robotics systems embed cameras with different spectral ranges. We have studied image retrieval approaches between a visible dataset and an infrared one, being NIR, SWIR or LWIR. We have made these experimentations along with the typical constraints studied in the long-term visual place recognition literature: viewpoint and image resolution changes, long-term appearance evolution and illumination variations of the scene. The developed work in this paper proposes a new feature description method: based on a scoping study, we have determined that feature detectors dealing with contrasted shapes like Harris are more appropriated to a cross-modality repeatability.
We have proposed a description pattern with a modified HOG feature which maximizes repeatability across different spectral ranges. This feature is then extracted at several scales in order to face the viewpoint changes. We have evaluated our proposal on several well-known datasets used for multimodal feature benchmarking and one dedicated to visual place recognition. We have proposed another dataset with a SWIR sensor whose attributes (resolution for example) are more challenging. In that way, our paper presents experimental results on four kind of cameras with different spectral ranges, ensuring that our proposal do not over-fit a particular couple of visible/IR sensors but is rather generalized. We have noticed that some methods can be discriminant enough with a Bag-of-Word approach and are relevant for long-term localization.
We present a high-resolution atmospheric inversion system combining a Lagrangian Particle Dispersion Model (LPDM) and the Weather Research and Forecasting model (WRF), and test the impact of assimilating meteorological observation on transport accuracy. A Four Dimensional Data Assimilation (FDDA) technique continuously assimilates meteorological observations from various observing systems into the transport modeling system, and is coupled to the high resolution CO 2 emission product Hestia to simulate the atmospheric mole fractions of CO 2. For the Indianapolis Flux Experiment (INFLUX) project, we evaluated the impact of assimilating different meteorological observation systems on the linearized adjoint solutions and the CO 2 inverse fluxes estimated using observed CO 2 mole fractions from 11 out of 12 communications towers over Indianapolis for the Sep.
2016). The Weather Research and Forecasting (WRF) model is a state-of-the-science community-supported numerical weather prediction (NWP) and atmospheric simulation system. WRF has been used worldwide for both research and operational applications  (Skamarock et al. 2008). Its ability to simulate atmospheric processes relevant to atmospheric transport and dispersion has been tested widely  (Cintineo et al. 2014 , Clark et al. 2015 , Coniglio et al. 2013 , Hariprasad et al. 2014 , Rogers et al. 2013 , Lauvaux et al. 2013 , Karion et al. 2015). In addition to its advanced numerical scheme and continuously upgraded array of model physics parameterizations, WRF has a four dimensional data assimilation (FDDA) capability implemented by Penn State University  (Deng et al. 2009 ) that allows meteorological observations to be continuously assimilated, allowing WRF to produce dynamic analyses at user-desired resolution.
Rogers et al. (2013)  investigated the effect of various FDDA strategies on the accuracy of the WRF-simulated mesoscale features over the Central Valley of California. The optimal FDDA configuration identified in  Rogers et al. (2013)  has been used in many recent modeling studies involving studying GHGs  (Lauvaux et al. 2013 , Karion et al. 2015). Towards that end, we test the impact of assimilating various meteorological data on the transport model accuracy for the INFLUX region, and the effect of assimilating these meteorological data sources on the inverse CO 2 fluxes derived for the city. Since wind speed, wind direction, and the depth of mixing in the PBL are expected to be critical to accurate and precise GHG flux estimation, the regional operational weather observation network was enhanced with a commercial compact Halo Streamline Doppler lidar  (Pearson et al.
2009 ) capable of continuous observations of those key atmospheric variables. The lidar data complement the standard WMO rawinsondes (only available at 00 and 12 UTC), and aircraft measurements from the commercial airline program Aircraft Communications Addressing and Reporting System (ACARS,  Anderson 2010). We evaluate the performance of the inversion system over 2 months (September-October 2013) using WRF atmospheric simulations that assimilate different combinations of data sources from WMO surface stations, the lidar, and in-situ data from ACARS, and assess the overall impact on WRF performance for each of the meteorological instrumentation used in our assimilation system. Finally, we conduct several sensitivity studies to assess the impact of these different meteorological data assimilation choices on the inverse CO 2 flux estimates obtained over the two-month period.
2004 ) is used. The Noah LSM is a four-layer soil temperature and moisture scheme and includes plant root zone, evapotranspiration, soil drainage, and runoff, taking into account vegetation categories, monthly vegetation fraction, soil texture, and snow and ice cover. The WRF modeling system has four dimensional data assimilation (FDDA) capabilities that allow continuous assimilation of meteorological observations during the model simulation, unlike intermittent approaches used in variational data assimilation techniques. For retrospective applications, FDDA can be used in numerical models to produce accurate dynamic analyses at the desired temporal and spatial resolution. FDDA has been widely used in studying atmospheric transport and dispersion processes  (Deng et al. 2004 , 2006 , Rogers et al.
e. producing the analytical flux solution with its associated error covariances (i. e. Kalman method)  (Tarantola 2005). Prior CO 2 emissions estimates within the inner domain are drawn from the Hestia 2012 emissions product  (Gurney et al. 2012). Hestia is a high-resolution (i. e. , building-level) inventory-based data product created to estimate anthropogenic CO 2 emissions for the Indianapolis area. It combines several datasets such as energy consumption, traffic data, industrial productivity, and electricity generation from the power plant, with models such as a building energy model. Forwards-in-time atmospheric CO 2 simulations based on emissions from the Hestia dataset within WRF-Chem are only used to compare to the mole fractions estimated with WRF-LPDM. Time periods with large discrepancies between forwards and adjoint atmospheric CO 2 mole fractions reveal unusually difficult transport conditions, and these times are excluded from the inverse analyses.
Model configuration The WRF modeling system used in this research is based on WRF V3. 5. 1, released in September 2013. The INFLUX WRF configuration consists of three nested grids with 9-/3-/1km horizontal resolutions (Figure  1a ), with the focus on the 1-km grid. The topographic and landuse database needed to initialize the WRF model is based on the U. S. Geological Survey (USGS) 30-second terrain and 24-category landuse. As indicated by the landuse distributions (Figure1b, c, d), a significant fraction of the 1-km grid is characterized as urban (Figure  1d ). In the vertical, fifty nine (59) vertical terrain-following layers are used, with the center point of the lowest model layer located ~7 m above ground level (AGL). The thickness of the layers increases gradually with height, with 24 layers below 850 hPa ( ~1550 m above sea level).