dataesr/scientific-paragraphs-categorization huggingface dataset:
Related Work Currently, there are two prevailing techniques for fine-grained image recognition. The first approach is known as localization classification subnetworks, and the other one is end-to-end feature encoding. The two-stage method (localization classification subnetworks), in particular, depends on the object and part localization annotations, region proposal networks  [21] , or attention mechanisms to acquire discriminatory areas, which are subsequently fed to the classifier. Study  [22]  built object and part detectors using bounding box datasets to find the most effective local semantic parts, and afterward applied a classifier to retrieve final classifications. Ref. [23]  generated portions using segmentation and a posture graph, followed by moving them to a classification model.
In terms of conclusion, each of those techniques attempts to use object-level or local-level details to eliminate unnecessary information, then feed the relevant information to the classifier for the classification task. Considering the complex two-stage pipeline and tedious and resource-intensive datasets, the present work emphasizes end-to-end feature encoding through deep learning neural networks to identify minute differences within subcategories. This strategy relies on maximizing classification outcomes through improved feature representations. A couple of research studies  [29, 30]  proposed paired interaction learning approaches to gather semantic differences. The authors of  [31]  employed the self-attention mechanism to retrieve discriminative features. Study  [11]  suggested a hierarchical architecture that performs cross-layer bilinear pooling.
Dataset The Stanford car dataset, having 16,185 images of 196 classes and sometimes extended to 208 classes, is used for the experiment. This dataset contains images of vehicle brands with significantly identical visual appearances and is one of the few benchmark datasets that are specifically designed for fine-grained image recognition. The dataset images, which are in JPEG format and contain different sizes, were first converted into Yolov5 format and then split into train, validation, and test sets. Figure  4  shows the dataset visualization.
Experimental Environment Our training setup involved a Windows 10 64-bit operating system with a 13th Gen Intel(R) Core(TM) i5-13400 processor, 32 GB of RAM, NVIDIA GeForce RTX 3060 Ti GPU, and Python 3. 9 with the Pytorch framework. To optimize the performance of our model and to compare and analyze it with existing Yolov5 models, we trained all the models with SGD and ADAM optimizers.
Experimental Environment Our training setup involved a Windows 10 64-bit operating system with a 13th Gen Intel(R) Core(TM) i5-13400 processor, 32 GB of RAM, NVIDIA GeForce RTX 3060 Ti GPU, and Python 3. 9 with the Pytorch framework. To optimize the performance of our model and to compare and analyze it with existing Yolov5 models, we trained all the models with SGD and ADAM optimizers. Table  1  displays multiple experimental hyperparameters. The number of correctly detected objects refers to true positives; false positives are wrongly identified as targets, and false negatives are the number of undetected objects. If the predicted bounding box of an object differs from the ground truth, this is not evidence that the detection is incorrect; therefore, intersection over union (IoU) is a frequent approach, where intersection over union is the ratio of the detected bounding box over the ground truth (bounding box).
mAP = ∑ 1 n Average Precision (9) We trained our proposed model using both SGD and ADAM optimizers along with the Yolov5l, Yolov5x, Yolov7, and Yolov8 models on the Stanford car training data set. The training results after every epoch are displayed in Figure  5. We observed that during the first 100 epochs, all eight models' precision increased gradually, while the recall dropped and fluctuated until the 50th epoch. However, the mAP@50 and mAP@50:95 remained stable and were gradually increasing, which shows that the instability during the initial stage of the training did not affect the performance of the models. Our proposed models' accuracy peaked at 0. 934 on the 220th epoch followed by the Yolov8 at 0. 919, while both models maintained high recall at 0. 89 and 0. 88, respectively, considering stable training where the model was improving at every epoch.
Comparison with State of the Art We compared our model with some of the existing state-of-the-art fine-grained image recognition models. Most of the models have utilized the VGG-19 and Resnet-50 backbones and are weakly supervised, where no training annotations were employed. The comparison is based on the accuracy achieved using the benchmark Stanford car dataset. Our method achieved 93. 4 percent accuracy with an improved CSP-Darknet53 backbone; a comparison can be seen in Table  5.
Conclusions In this paper, we proposed a one-stage fine-grained object recognition model based on the Yolov5 object detector. We improved the backbone of the existing Yolov5 model to effectively capture global dependencies and enabled the model to learn relationships between distant regions. This improved the model's ability to understand context and capture long-range spatial relationships in an image, which are important aspects of the fine-grained recognition task. We also replaced the Yolov5 detection heads with three transformer heads using the discriminative feature maps from transformer encoder blocks. To evaluate model improvement after adding the transformer encoder blocks, we used the famous Stanford car dataset, which is a benchmark dataset for fine-grained recognition consisting of 16,185 highly similar images of 196 different classes of vehicles, which has been updated to 208 classes.
The impact of the environment on health is usually studied in a segmented manner, with a focus on a single source, pollutant, or exposure medium. To better understand spatial health inequalities, it is necessary to adopt multi-dimensional approaches to comprehensively describe the environment, especially at the territorial level. Clustering methods, which allow for the development of territorial typologies, are particularly interesting for this purpose. By simplifying complex datasets, these methods may reveal spatial patterns and geographical phenomena that would otherwise be difficult to observe. Based on the existing literature, there is a clear need for large-scale territorial typologies that comprehensively address the physical and outdoor environment. A robust and transposable framework was developed and applied to 3,041 municipalities in Northern France using open environmental data.
It consists of five main steps: data collection, data selection, data preparation, cluster analysis, and cluster interpretation. This methodology allows for the development of an environmental classification of municipalities by identifying the primary environmental profiles represented in the study area. Cluster detection was performed based on 39 spatialized indicators that describe the level of environmental contamination (air, water, soil), the level of pollutant emissions, the proximity to emission sources, the land use, the agricultural practices, and the degree of naturalness in every municipality. As a result, municipalities were allocated into one of the seven following environmental profiles: (i) Dense urban centers; (ii) Peripheral urban municipalities; (iii) Intensive agricultural municipalities under urban influence; (iv) Intensive agricultural municipalities beyond urban influence; (v) More extensive and diversified agricultural municipalities; (vi) Municipalities with predominant livestock activities and significant natural areas; (vii) Municipalities with predominant natural areas: forests, wetlands, and water surfaces.
g. land use, agricultural practices, naturalness). Exploring these multiple spatial datasets to identify and describe complex territorial profiles requires the implementation of multidimensional analysis strategies. Clustering techniques are statistical methods that can be used on large and complex datasets to group together individuals that share common traits  (Madhulatha, 2012). Cluster analysis has been used in many fields of science to analyze spatial data in various contexts, such as agriculture  (Alvarez et al. , 2018) , economy  (Segessemann and Crevoisier, 2016) , healthcare  (Wendt, 2009) , or energy  (Weinand et al. , 2019). By aggregating information, these methods reveal geographical patterns and spatial phenomena that would otherwise be difficult to observe. In the field of environmental studies, such approaches have been commonly employed.
Methodological framework The key methodological steps can be summarized as follows: data collection, data selection, data preparation, cluster analysis and clusters interpretation (Fig. 1 ). Data collection. To describe the physical and outdoor environment at the municipality level, potential data sources were first identified. Then, when appropriate, raw data were collected and processed to be included in the dataset. Data selection. Before initiating any statistical processing, spatial indicators were sorted out to limit the redundancy of information and to give similar importance to the different phenomena described. Also, when several options were available to describe the same phenomenon, decisions were made to select the most appropriate and representative indicator. This preliminary step is crucial as it contributes to the quality and interpretability of the results.
Data preparation. Since data can be expressed in very different units and/or generated by different producers for different purposes and according to different reference systems, standardization is necessary to ensure that indicators are comparable  (Böhringer and Jochem, 2007; Saib et al. , 2015). Moreover, because of the large number of indicators involved, it is recommended to reduce the dataset with principal component analysis (PCA) before clustering  (Everitt, 1979). Cluster analysis. Using clustering algorithms, the objective was to obtain a sufficiently complex and informative classification with, at the same time, some consistent and easy-to-interpret clusters. Clusters interpretation. Once the clusters were identified, a descriptive analysis was conducted to describe and characterize the environmental profile associated with each cluster.
Data collection Sources of data. The method's transposability and potential adoption by other researchers and stakeholders were favored by prioritizing the use of open-access and nationally available data. Data sources to be included were selected based on three inclusion criteria: (i) comprehensive and available data on a national scale; (ii) data adapted to the territorial context; (iii) data available or calculable at the municipality level. Moreover, the following exclusion criteria were applied: (i) data not describing the physical and outdoor environment; (ii) data not related to environmental health issues.
Data management and transformation: calculation of indicators. To help structure the dataset, indicators were allocated into the seven following sub-dimensions: contamination levels, emission levels, location of emission sources, land use, agricultural practices, naturalness, climate. Three types of indicators can be distinguished: -indicators which did not require any particular manipulation (the unchanged raw data could be included in the dataset). -indicators which required elementary transformations. In particular, in many cases, the raw data was divided by the municipality's surface area or population. This step was imperative to subsequently compare the municipalities with each other. -indicators for which more complex processing was required, such as distance calculations, surface calculations or rescaling.
Depending on the case, the manipulations were carried out using QGIS software (QGIS Development Team, 2021) and R software  (R Core Team, 2021). Missing data were managed with one of the two following methods: (i) if the missing value resulted from the previous merger of several municipalities into a new one, the value of the newly created municipality was attributed to the corresponding municipalities that have been merged. (ii) if there was no history of a merger or creation of a municipality, the values of neighboring municipalities (i. e. those sharing a border) were identified, and the median of these values was assigned to the municipality with missing data  (Bennett et al. , 1984).
Data selection Expert selection. Beforehand, the balance and consistency of the phenomena described in the dataset were validated by experts to ensure an adequate characterization of the physical environment. When several options were available, choices were made to select the most adapted and representative indicators. Mathematical selection: bivariate correlation analysis. Then, a second selection based on bivariate correlation analysis was used to identify redundant variables and thus summarize the major information contained in the dataset  (Jolliffe, 1972; King and Jackson, 1999). The maximum threshold for the Pearson correlation coefficients was fixed at 0. 70  (Dormann et al. , 2013; United States Environmental Protection Agency, 2020). Indicators were sorted out so that in absolute value, no Pearson correlation coefficient exceeded 0.
Data preparation Standardization of indicators. Each of the indicators was standardized according to the recommended z-score methodology before using PCA  (Marzi et al. , 2018). Thus, all indicators were converted into a common scale with a mean equal to zero and a standard deviation equal to 1. This method is less sensitive to extreme values than other methodologies and maintains correlation structures between indicators  (Kolenikov and Angeles, 2004; OECD et al. , 2008). Principal Component Analysis. PCA was applied to the final dataset to define the subspace that best represents the diversity of individuals (i. e. municipalities). Then, for the rest of the analysis, only the principal components with an eigen value above 1 were retained.
This graph provides a quick overview and enables the identification of the most atypical indicators for each cluster, that is to say, the indicators that best characterize each cluster in comparison with the entire study area. For example, if a municipality shows a z-score of zero for a given indicator, it corresponds to the exact average of that indicator across the study area. For a z-score of +1 (or -1), it represents one standard deviation above (or below) the average. Also, the typology was represented on a map using QGIS software (QGIS Development Team, 2021).
Data collection Sources of data. Initially, 106 environmental data sources were identified. Many sources have been excluded due to an insufficient spatial coverage (e. g. , concentrations of pesticides in the air). Another major reason was simply the irrelevance of the data to the environmental-health issue (e. g. , seismic risks). In the end, 24 different data sources were validated and exploited. Of these sources, all but one were openly accessible. The only exception is a database that provides a composite indicator called the High Natural Value (HNV) indicator  (Pointereau et al. , 2010 , Pointereau et al. , 2007)  which assesses the environmental quality of agricultural practices at the municipality level. The producer of the database was contacted, and the data was freely provided without restrictions.
Data management and transformation: calculation of indicators. In all, 113 environmental indicators were included in the dataset and allocated as follows: contamination levels (n = 24); emission levels (n = 13); location of emission sources (n = 22); land use (n = 15); agricultural practices (n = 23); naturalness (n = 6); climate (n = 10). In terms of temporal resolution, whenever possible, several years of data were considered. It concerns 63 out of 113 indicators. For these indicators, the calculation was performed for each available year of data, and the resulting average was computed and included in the dataset.
Data selection Expert selection. To analyze the metallic contamination of soils, the Soil Quality Monitoring Network  (Arrouays et al. , 2003)  provides data on both total concentrations and concentrations of the extractable phase. Mean concentrations were calculated at the municipal level for both options. However, for several metals, data on the extractable phase was missing. Thus, for reasons of comparability, only those indicators that describe the total concentration have been retained. To study proximity to wastewater treatment plants, two indicators were developed: one that considered all plants regardless of compliance, and another that only accounted for non-compliant plants. However, noncompliance data was only available for a single year (2017). Consequently, to be as representative as possible, the general indicator was retained.
In the case of waste treatment and management facilities, ten proximity indicators that described the distance to specific types of facilities (such as incinerators or storage facilities) were excluded from our analysis, due to scattered spatial distribution. Instead, only one general proximity indicator was retained to account for all types of waste treatment and management facilities. Sometimes, multiple data sources were available to study the same phenomenon. In such cases, the indicator generated from the most reliable and exhaustive database was selected. This approach was taken for several topics, including livestock farming and polluted sites and soils. Climate data were not considered because such indicators may have exhibited irrelevant factors such as the seafront/ inland gradient  (Shortt et al.
, 2010 (Shortt et al. , , 2012)). Finally, 67 indicators were retained among the 113 initially included in the dataset. Mathematical selection: bivariate correlation analysis. Correlations between contamination indicators were handled first. As the three considered air pollutants (PM 2. 5 , PM 10 and NO 2 ) were highly correlated, only the PM 2. 5 contamination indicator was kept. In light of current scientific knowledge, it seemed more appropriate to study PM 2. 5 due to its diverse emission sources and significant implications for public health  (Nan et al. , 2023). Similarly, for soil contamination, significant correlations were found among various trace metals. Zinc was selected as a suitable synthetic marker for four other highly correlated elements: copper, cadmium, chromium, and nickel.
Four indicators of soil contamination were finally retained: arsenic, mercury, zinc and thallium. For emission levels, pollutants already described via contamination data were excluded as a priority. Only data on the following pollutants were retained: sulfur dioxide, non-methanic volatile organic compounds, ammonia, arsenic and greenhouse gases. For land use, the proportion of artificial land was kept in the dataset rather than the proportion of agricultural land, because agricultural land use is described much more precisely by crop type. The total pesticides and the insecticides indicators were kept to assess phytosanitary pressure, while the herbicide and fungicide indicators, which exhibited a strong correlation with total pesticides, were excluded. In the end, the final dataset comprises 39 indicators.
Final dataset and dimensionality reduction (PCA) The 39 spatial indicators of the final dataset were distributed between the sub-dimensions as follows: contamination levels (n = 10); emission levels (n = 5); location of emission sources (n = 5); land use (n = 11); agricultural practices (n = 5); naturalness (n = 3). At the end of the PCA, only the first ten principal components were retained for the following analysis. Cumulatively, it accounts for 62. 10% of the total variance in the dataset.
Discussion This research aimed to develop a robust and transferable framework for distinguishing territorial profiles of the physical and outdoor environment. A multivariate methodology using open data and unsupervised clustering algorithms was developed and applied at the municipal level in northern France. Based on 39 spatial indicators, seven territorial profiles were identified in the study area. The data used described the contamination levels of the environment (air, water, and soil), pollutant emissions, emission source locations, land use, intensity of agricultural practices, and degree of naturalness.
Access to comprehensive and reliable environmental data The framework proposed in this study is based on the collection of multiple environmental databases, to describe various dimensions of the physical environment. Reusing open environmental data is a major challenge. Firstly, a significant volume of open data was collected to characterize the environment at the municipality level. The initial data set included 113 spatialized environmental indicators generated from 24 different databases. However, accessibility to sufficiently robust data for creating spatialized indicators is still challenging, as the environment is only partially described today. The availability and quality of these data, and their management are discussed here. Contamination levels. Regarding environmental contamination, air pollution models are well documented by the European Environment Agency  (Horálek et al.
, 2021)  or the National Institute of Industrial Environment and Risks  (Real et al. , 2022)  for regulated pollutants (particles, nitrogen oxides and ozone). However, many other pollutants are not even described, including trace metals, hydrocarbons or pesticides. To characterize soil contamination, the available data exclusively concern the concentration of trace elements in agricultural and forest soils. The Soil Quality Monitoring Network (RMQS;  Arrouays et al. , 2003)  is the only available database covering the national territory. The RMQS is based on the monitoring of 2,240 agricultural sampling sites uniformly distributed across the national territory according to a square mesh of 16 km per side. After spatial interpolation with a resolution of 1*1 km, concentrations data is aggregated at the municipal level.
It should be kept in mind that this model could be inaccurate at a fine scale, especially in urban areas. Furthermore, these data reflect more the background levels of natural soils than anthropogenic contamination. Other soil quality measurements are however carried out, notably as part of urban development projects. However, there is no database that lists all these data in France. Concerning the aquatic environment, since the presence of rivers or lakes is very heterogeneous across France, the focus was placed on the drinking water distribution network, which covers the entire territory (SISE-Eaux database; Ministère des Solidarités et de la Santé, 2023). Very few indicators could finally be included in the data set. Conductivity, turbidity, pH, as well as the presence of chlorine or enterobacteria are the only indicators that could be retained, despite data concerning trace elements or polycyclic aromatic hydrocarbons.
, 2021). Its calculation relies on the concept of naturalness, which was previously defined  (Guetté et al. , 2018). To assess territorial naturalness, this definition distinguishes three criteria: (i) the biophysical integrity of land cover, which measures the deviation of a landscape from its state in the absence of anthropogenic transformations; (ii) the spontaneity of processes, which evaluates the extent of human intervention in natural dynamics; (iii) spatial continuity, corresponding to the degree of landscape connectivity. The naturalness index is computed by summing the values of three sub-indicators developed for each criterion. Furthermore, another indicator (BIODIV_RESERVOIRS) was developed to evaluate the presence of biodiversity reservoirs in municipalities. The data utilized for this indicator were derived from a national program conducted in France, which aimed to identify and map biodiversity reservoirs  (Billon et al.