dataesr/scientific-paragraphs-categorization huggingface dataset:
The state vector xl C represents the logarithmic-scale combined CH 4 profile product (i. e. the MUSICA IASI CH 4 data updated with the TROPOMI XCH 4 observation). The superscript " l " used with xl C and xl I indicates the use of the logarithmic scale. Here and in the following, we will mark scalars, vectors, or matrix operators that are in logarithmic scale by the superscript " l ". The matrix L is the operator for the transformation of differentials or small changes (as given by averaging kernels or error covariances) from the logarithmic to the linear scale (for more details, see Appendix C). The column vector m is the Kalman gain operator, and it is given by   2 ) are analogous to Eqs. (B9) and (B10). As demonstrated in Appendix B, this kind of Kalman filter application has a strong similarity to an optimal estimation retrieval that uses a combined IASI and TROPOMI measurement vector (synergetic use of level 1 data).
Acknowledgements. This research has largely benefitted from funds of the  Deutsche Forschungsgemeinschaft  (provided for the two projects MOTIV and TEDDY with IDs/Geschäftszeichen 290612604/GZ: SCHN1126/2-1  and 416767181/GZ: SCHN1126/5-1 , respectively) and from support by the  European Space Agency  in the context the "Sentinel-5p+Innovation ( S5p+I) -Water Vapour Isotopologues  ( H2O-ISO )" activities. Furthermore, we acknowledge funds from the  Ministerio de Economía y Competividad from Spain  for the project  INMENSE  ( CGL2016-80688-P ). An important part of this work was performed on the supercomputers ForHLR and HoreKa funded by the  Ministry of Science, Research and the Arts Baden-Württemberg  and by the  German Federal Ministry of Education and Research. We also acknowledge the contribution of  Teide High-Performance Computing  facilities.
Teide  HPC  facilities are provided by the  Instituto Tecnológico y de Energías Renovables  (ITER), S. A (https://teidehpc. iter. es/en/home/,last access:  5 July 2022 ). The TROPOMI data processing was carried out on the  Dutch National e-Infrastructure  with the support of the SURF cooperative. The presented material contains modified Copernicus data (2017, 2019). The Eureka TCCON measurements were made at the  Polar Environment Atmospheric Research Laboratory (PEARL)  by the  Canadian Network  for the  Detection of Atmospheric Change , primarily supported by  NSERC ,  ECCC , and  CSA. The East Trout Lake TC-CON station is supported by  CFI ,  ORF , and  NSERC. The Karlsruhe TCCON station has been supported by the  German Bundesministerium für Wirtschaft und Energie (BMWi)  via  DLR  under grants  50EE1711A  to E and by the  Helmholtz Society  via the research program ATMO.
The Park Falls and Lamont TCCON sites are supported by and would like to acknowledge  NASA 's  Carbon Cycle Science Program  (grant no. NNX17AE15G ) and  OCO-2  and  OCO-3  projects (primary grant no. NNN12AA01C ), respectively. The Burgos and Rikubetsu TCCON sites are supported in part by the  GOSAT  series project. Burgos is supported in part by the  Energy Development Corp. , the  Philippines. Funding for the Edwards TCCON station is provided by  NASA's Earth Science Division. Nicholas M. Deutscher  is funded by  ARC   Future Fellowship   FT180100327. Darwin and Wollongong TCCON stations are supported by  ARC  grants  DP160100598 ,  LE0668470 ,  DP140101552 ,  DP110103118 , and  DP0879468  and  Darwin  through  NASA  grants  NAG5-12247  and  NNG05-GD07G. The Lauder TCCON programme is core-funded by  NIWA through New Zealand's Ministry of Business, Innovation and Employment.
The Trainou AirCore measurements have been supported by  CEA ,  CNES ,  UVSQ , IPSL, and the  EU H2020   RINGO  project (GA no. 730944 ) and are part of the  French consortium for Aircore measurements (LMD, LSCE, GSMA, CNES). The Sodankylä TC-CON and AirCore measurements have been supported via the  ESA   FRM4GHG  project (under grant agreement no. ESA-IPLPOE-LGcl-LE-2015-1129 ) and the  EU H2020   RINGO  project. The CH 4 observations at Jungfraujoch were established as part of the  Swiss National Air Pollution Monitoring Network  and are supported through  ICOS-CH , which is funded by the  Swiss National Science Foundation  and in-house contributions. We would like to thank our colleague Thomas von Clarmann for his strong support in revising the equations in the Appendix of this paper. We thank  Michela Giusti  in the Data Support team at  ECMWF  for retrieving and providing comments about the CAMS data.
We acknowledge the support by the  Deutsche Forschungsgemeinschaft and the Open Access Publishing Fund of the Karlsruhe Institute of Technology. Financial support. This research has been supported by the  Deutsche Forschungsgemeinschaft  (project  MOTIV  (grant no. 290612604 ), and project  TEDDY  (grant no. 416767181 )), the  Ministerio de Economía y Competitividad  (grant no. CGL2016-80688-P ), the  European Space Agency  (grant nos. 4000127561/19/I-NS  and  ESA-IPLPOE-LG-cl-LE-2015-1129 ), the  Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg , and the  Bundesministerium für Bildung und Forschung  for funding in the context of the ForHLR and HoreKa supercomputing infrastructure. The article processing charges for this open-access publication were covered by the  Karlsruhe Institute of Technology (KIT).
Data availability. The MUSICA IASI data are described in (Schneider et al. , 2022)  and can be accessed at https://doi. org/10. 35097/408 (Schneider et al. , 2021a)  and https://doi. org/10. 35097/412 (Schneider  et  al. ,  2021b). The TROPOMI XCH 4 data used in this study are described in Lorente et al. (2021a)  and can be accessed at https://doi. org/10. 5281/zenodo. 4447228 (Lorente et al. , 2021b). The TCCON data are available via the TCCON data archive, hosted by CaltechDATA (Total Carbon Column Observing Network Team, 2014, https://doi. org/10. 14291/TCCON. GGG2014). For Trainou AirCore data, please contact Michel Ramonet (michel. ramonet@lsce. ipsl. fr), and for Sodankylä AirCore data, please contact Huilin Chen (huilin. chen@rug. nl). The Jungfraujoch GAW surface in situ CH 4 data are available via the World Data Centre for Greenhouse Gases (WDCGG) and can be directly accessed at https://doi.
We propose a novel Mean-Shift method for data clustering, called Robust Mean-Shift (RMS). A new update equation for point iterates is proposed, mixing the ones of the standard Mean-Shift (MS) and the Blurring Mean-Shift (BMS). Despite its simplicity, the proposed method has not been studied so far. RMS can be set up in both a kernel-based and a nearest-neighbor (NN)-based fashion. Since the update rule of RMS is closer to BMS, the convergence of point iterates is conjectured based on the Chen's BMS convergence theorem. Experimental results on synthetic and real datasets show that RMS in several cases outperforms MS and BMS in the clustering task. In addition, RMS exhibits larger attraction basins than MS and BMS for identical parametrization; consequently, its kernel variant requires a lower aperture of the kernel function, and its NN variant a lower number of nearest neighbors compared to MS or BMS, to achieve optimal clustering results.
I. INTRODUCTION Data clustering is a type of unsupervised learning which consists of automatically grouping data points having similar characteristics into identified clusters without training sample points. It is a central task in various application fields such as medicine, genomics, content-based image and video indexing, and Big Data mining to cite a few. Clustering is also increasingly relevant in the context of Artificial Intelligence, in order to unveil the existence of underlying complex structures in datasets  [2] , and especially in applications where little or no training data is available. Despite several decades of research, clustering remains a challenging task for many applications because of the increasing size (number of data points) and dimensionality (number of features) of modern datasets.
d. f. ). MS relies on kernel density estimation (KDE), a non-parametric way to estimate a p. d. f. from data samples  [25] ,  [26]. In MS, each point of the dataset is moved iteratively by a small amount (the so-called mean shift) until convergence to some stationary point, i. e. a local mode of  VOLUME 10, 2022  This work is licensed under a Creative Commons Attribution 4. 0 License. For more information, see https://creativecommons. org/licenses/by/4. 0/ the estimated p. d. f. MS has first been used as an unsupervised data clustering method, for which the retained local modes after convergence of the point iterates serve as cluster representatives (or exemplars). A connected component post-processing stage  [26]  is therefore necessary after the convergence is achieved to assign a cluster label to each of the original data points.
This method, which we name Robust Mean-Shift (RMS) is a hybridization of the standard Mean-Shift (MS) algorithm  [25] , and of the so-called Blurring Mean-Shift (BMS) method  [27]. It is worth recalling that BMS was actually first proposed in  [11] , as pointed out in  [26]. Our algorithm is based on iteratively moving updates of initial data points, similar to MS and BMS, but the update equation of RMS fundamentally differs from both methods. The proposed RMS approach proposed in this work has several valuable properties: • We find experimentally that RMS requires a lower bandwidth parameter (in the kernel-based variant) and a lower number of nearest neighbors (in the NN variant) than MS and BMS to achieve comparable or even better results in the clustering task; this is especially interesting to speed up the computation of point iterates, and in the case of the NN-RMS variant, to reduce the size of the NN graph compared to the ones required by the NN variants of MS and BMS; • Compared to MS and BMS, RMS generally performs better, as evidenced experimentally through the analysis of various datasets; • In most experiments, RMS converges faster than BMS, the latter being proved to converge faster than MS  [35] ; • The (classical) kernel-based RMS can be easily turned to a K -nearest neighbor (K NN) algorithm, similarly to MS and BMS  [30] ,  [31] ; • The NN variant of RMS does not require a termination threshold, unlike the NN variant of BMS.
The paper is organized as follows. Section II provides a brief overview of related works, including kernel-based and K NN-based Mean-Shift (MS) approaches published so far. In Section III, we introduce the proposed clustering method and explain how it relates to MS and BMS. The convergence of RMS in the kernel-based framework is then discussed in Section IV. Section V describes the NN-based variant of RMS. An experimental study of RMS and its comparison with other similar clustering approaches on various datasets is provided in Section VI. Conclusions and perspectives of this work are given in Section VII.
A. MEAN-SHIFT (MS) MS can be used to partition a dataset by assigning each data point a label corresponding to the unique point it converges to after some (expected) finite number of iterations of an update equation. More precisely, with the above notations, the update equation of MS writes, ∀i: y (t+1) i = N k=1 f x k , y (t) i. x k N j=1 f x j , y (t) i. (1) With the above assumption on the kernel function f , the updated points {y (t+1) i } i=1,. ,N are obtained as a convex linear combination (i. e. with non-negative coefficients summing up to unity) of the initial data points {x i } i=1,. ,N.
C. DISCUSSION 1) MAIN ADVANTAGES AND DRAWBACKS As pointed out in  [26]  MS and BMS algorithms have several advantages compared to other clustering techniques, among which: • their parametrization is limited to the choice of an appropriate kernel function, and a single bandwidth (or aperture) parameter for this kernel function; • they allow to discover non-convex clusters; • they can automatically determine the number of clusters, depending on the chosen bandwidth parameter; • the algorithms are totally deterministic. These advantages make a significant difference with respect to classical clustering methods like k-means or fuzzy c-means for which none of the last three items above is ensured. However, MS-based methods have two main drawbacks which are the lack of scalability to large datasets, and a high sensitivity of clustering performances for high-dimensional datasets with respect to the bandwidth parameter  [26].
2) CONVERGENCE OF MS-BASED METHODS Since the early works on MS-based clustering, and until very recently, the convergence of MS and BMS has been studied. A comprehensive study of the convergence of both MS and BMS algorithms is summarized in  [26] , but specific results were provided for MS in  [25] ,  [35] , and for BMS in  [1] ,  [27]. For MS, the convergence of moved points to local modes is ensured theoretically and practically in the general case. However the situation may vary from one kernel to another as for the number of iterations required to convergence: the latter is proven in a finite number of steps for the Epanechnikov kernel  [25] , but infinite for the Gaussian kernel, moreover with a linear convergence rate  [37]. For BMS, the convergence issue depends on the kernel aperture: for large ones which encompass the whole dataset X , convergence is ensured to a unique mode, whereas for narrower ones with finite support, the BMS update rule allows the iterates to converge quickly to well-separated distinct modes during the first steps.
3) K NN-MS BASED METHODS Choosing the optimal set of hyperparameters (kernel function and bandwidth) for a specific task can become very challenging. This is why, since the early works on Mean-Shift-based density estimation and data clustering  [11] ,  [12] , the use of nearest neighbors as an alternative to the standard kernel approach has been proposed by many researchers. Indeed, the NN approach (i) does not require to specify an underlying parametric function, so that only one parameter K is required; and (ii) the K NN principle makes it possible to maintain the relationship between data points located far from each other, especially on the external border of clusters. In this sense, the NN-based framework is data-adaptive, contrarily to the kernel-based one. In  [11] , a mean-shift estimate calculated from K nearest neighbors was proposed as a natural way to automatically adapt the density estimation to its local variations.
,N , it is expected that iterates remain 'bound' to the initial data points, therefore avoiding convergence to a unique mode for broad kernels, which is a known drawback of BMS  [35] ; • The kernel-based update rule in Eq. (  3 ) can be easily modified to involve nearest neighbors, similarly to NNGA  [31]  and the graph-theoretic approach in  [12]. Surprisingly, this update equation has not been reported so far in the literature to the best of our knowledge. Figure  1  illustrates with a simple example the differences between MS, BMS and RMS. For the three methods, the same kernel parametrization is used, i. e. f (u, v) = exp - u -v 2 σ 2 , ( 4 ) with bandwidth parameter σ = 0. 15. The dataset is drawn randomly from three 2-D normal distributions with identical diagonal covariance matrices, centered at [0, 0], [0, 1],  [1, 1].
V. NEAREST-NEIGHBOR RMS Similarly to MS and BMS, the kernel-based RMS algorithm can be cast to a nearest-neighbor-based algorithm, by setting f as a variable-radius flat kernel based on the K NNs: f KNN (x, y) = 1 if x ∈ KNN(y) 0 else. (6) This yields the following nearest-neighbor robust Mean-Shift (NN-RMS) update rule: y (t+1) i = N k=1 f KNN x k , y (t) i. y (t) k N j=1 f KNN x j , y (t) i = 1 K k:x k ∈KNN(y (t) i ) y (t) k (7) The corresponding algorithm is detailed in Algorithm 1. Notice that, contrarily to the kernel-based RMS approach, the new one is point-wise (local), i. e. its radius around one data point y j is equal to the distance to its K th NN in X. By doing so, one can expect that NN-RMS better captures the local complexity of the data distribution, compared to the kernel-based version.
Algorithm 1 NN-RMS Input: X = {x m } , x m ∈ R n , m = 1,. , N ; the dataset K , the number of NNs; Output: The vector of points' labels c = [c 1 ,. , c N ] t ; the set of cluster exemplars E ; 1) Initialize Y (0) = y (0) m = {x m } , m = 1,. , N ; t = 1; y (1) m = y (0) m ; 2) Core loop while t = 1 ∨ y (t) m = y (t-1) m do for m = 1 : N do j = [j 1 , j 2 ,. j K ] = KNN(X , y (t) m ); y (t+1) m = 1 K K k=1 y (t) j k ; c (t+1) m = j 1 ; t ← t + 1; end for end while 3) Set E as the set of unique indices in c (t) ; 4) Remap E to cluster labels in [1.
C], C = |E |; Contrarily to NN-BMS, NN-RMS does not update anymore after a small number of iterations. This can be explained easily: the KNN search in NN-RMS operates so as to find the NNs of the current updates within the set of original data points, which remain static. On the one hand, this is different from NN-BMS in which the KNN search is performed within the set of current updates, which do continuously move along the iterations. On the other hand, the KNN search is similar to NN-MS (or NNGA  [31] ), but since NN-RMS is essentially a BMS algorithm (because next updates are weighted sums of the current moving points), its convergence is faster than MS. Indeed, NN-RMS stops if all the moving points have reached the condition that they share the same NNs within the original dataset, even if the last current iterates are close to each other, but distinct.
C. SELECTED VALIDATION CRITERIA To allow the comparison of different results, we used the following cluster validation criteria: • Since all the datasets include a ground truth labeling as an external data for cluster assessment, the overall accuracy (OA), average accuracy (AA), kappa index, can be obtained after optimal pairing of cluster labels with the actual ground truth classes owing to the Munkres assignment algorithm  [39] ; • The purity and normalized mutual information (NMI) indices  [40]  are also based on the relationship between the ground truth and the predicted labels, but do not require label reassignment; • The consistency violation ration (CVR)  [41]  is a clustering index based on an information-theoretic concept, which is also fitted to non-convex clusters; a lower CVR indicates a better clustering result.
Contrarily to the previous criteria, the CVR index does not require the ground truth labels, which makes it useful to assess the results of unsupervised classification. In Figure  5  The results obtained on synthetic datasets are given in Table  2. In addition to the cluster indices mentioned above, we also added the number of output clusters, the computation time and the number of iterations for each method used for comparison. For each dataset, the clustering task was performed with two groups of methods, namely NN-based and kernelbased methods. The values displayed in Table  2  correspond to the specific values of K (for NN-based methods) or σ (for the kernel-based methods) providing the best kappa index. Concerning the NN-based methods, it can be seen that NN-RMS provided the best kappa indices on five over the nine datasets (Aggregation, DataS1, S4, Unbalance and Worms2d).
Furthermore, NN-RMS provided the second best kappa on three other datasets (Flame, Spiral and Birch1). One important issue of this comparison is that these best kappa results for NN-RMS were obtained for the lowest number of NNs among all the compared methods on eight over the nine datasets. This result can be considered as significant since most of the optimal values of K for the other methods are often more than twice the optimal values for NN-RMS. It is also noticeable that NN-RMS found the correct number of clusters in seven over nine datasets, and that this number is very close to the actual one in the remaining cases. With regard to kernel-based Mean-Shift methods, over the nine datasets, G-RMS again outperforms G-MS and G-BMS on six datasets (Flame, Spiral, Aggregation, DataS1, S4 and Worms2d).
Also, in all cases, G-RMS never requires a higher Gaussian kernel aperture than G-MS and G-BMS to achieve the results with the best kappa indices, and for most datasets the optimal aperture is well below the ones provided by G-MS and G-BMS. This finding is in accordance with the results of the NN-based MS methods. Note that for the R15 dataset, all methods (NN-or kernel-based) perform equally well, except NN-MS and NN-MedShift with lower kappa, NMI and purity, and higher CVR. 2) REAL DATA Table  3  provides the clustering results obtained on the real datasets described above. Here, only NN-based MS methods were considered, as well as kNNDPC, GWENN and NN-MedShift. Again, these results correspond to the optimal parameter K in terms of output kappa index. The variety in data size (from N = 150 to 2310), dimensionality (up to n = 20), and cluster shape makes it difficult to draw clear conclusions from these observations.
Here, over the six datasets, in terms of kappa index, NN-RMS performs better than the other methods in two cases (Banknote and Segment), whereas NN-BMS is better for three others (Iris, Ecoli and Attface). However, NN-RMS was able to provide the correct number of clusters in all cases, except for the Ecoli dataset for which all the methods failed. This is particularly true for the Attface dataset, despite the relative high dimensionality (n = 20) and low populated classes (only 10 data points per class) which makes it a challenging clustering problem. Finally, similarly as above for the synthetic datasets, the best results in terms of kappa index reported for NN-RMS in all cases correspond to lower values of K than that of the other methods.
It is interesting to notice that all four methods can discover two additional clusters with respect to the available ground truth. These clusters are visually  coherent both from the spectral viewpoint as can be seen from the composite image, and from the spatial viewpoint since the additional segments in the maroon and light blue regions of the ground truth map follow the same spatial structure as the labeled ones. Despite the high dimensionality of the dataset, here again, NN-RMS provides the best overall clustering result among the four methods, still with the lowest number of nearest neighbors K.
RMS has several advantages over MS and BMS: • For a same kernel bandwidth (in the kernel-based implementation) or number of NNs (in nearest-neighborbased implementation), RMS shows larger basins of attraction than MS and BMS. One consequence is that the size of the NN graph required to achieve the same number of clusters is smaller for NN-RMS than for NN-MS and NN-BMS. • Experimental results on synthetic and real datasets show that RMS in most cases outperforms MS and BMS in the clustering task. • Though the RMS update equation is closer in spirit to BMS, their NN-based versions have different behaviors when the iterates get close to their fixed-point limit: whereas NN-BMS iterates continue to evolve until the mean squared error between the current and the previous iterates reach a specified small value, NN-RMS stops as soon as all iterates share the same set of NNs within the original dataset.
, we illustrate the evaluation of the results obtained by applying NN-MS, NN-BMS, NN-RMS, and NN-MedoidShift to the Flame dataset, as a function of the number of NNs K in the range [6, 50]. The results clearly show that NN-RMS provides an estimation of the correct number of clusters close to the actual one for a much lower number of NNs K. In comparison, NN-MS gives a very high number of clusters, as well as NN-BMS and NN-MedoidShift. The latter reaches the correct number of clusters for a higher number of NNs than NN-RMS, and for NN-BMS even higher. The OA index shown on Figure 5-(b) reveals the potential of NN-RMS to quickly approach the ideal partition for very low K. On Figure 5-(c), the low CVR indices within a large range of K confirms the robustness of NN-RMS with respect to the selection of the parameter K.