dataesr/scientific-paragraphs-categorization huggingface dataset:
An example of their application is presented in  [48] , where the authors aimed to retrieve the number of groups in real reference datasets using the Gap statistic method; they obtained mitigated results though, mostly due to the parameters to set: when correctly setting the parameters, they were able to retrieve the correct number of groups, but, when the parameters slightly differed from their optimal values, they obtained poorer results, mostly due to close and overlapping groups. Nonetheless, although being deemed reliable  [49] , this technique suffers from two major drawbacks: its computational heaviness, and its empirical approach. Since several values of the number of clusters must be tested, it could be considered a more sophisticated method to select the best partitioning among others, rather than a true way to estimate the number of clusters in a dataset.
Finally, in order to estimate the number of clusters, the authors of  [50]  proposed the ECD test, a semi-empirical approach which operates in a split-and-merge fashion. To that end, the authors began by creating a cluster dataset (only once) using a large value for the number of clusters, resulting in possibly very small groups, which are later characterized by their Empirical Cumulative Distributions (ECDs), which are finally compared using the Hausdorff distance and linked to one another using region-growing clustering. By comparing the ECDs of the different groups, the idea is to regroup the clusters in the same region of the feature space (which are assumed to have similar features, and therefore close ECDs), which are eventually merged to empirically estimate the borders of the distinct regions of the space.
The ECD test aims to estimate the true number of regions of the feature space, value which serves as an estimate for the optimal number of clusters. This technique has the advantage of performing the clustering only once, is computationally efficient, since it relies on simple comparisons (the ECDs and the Hausdorff distance), and proved to be accurate in the identification of the borders of the feature space of a real industrial dataset. All these works aim to estimate the "optimal" number of clusters for a clustering algorithm to build, either by following a semi-empirical approach or by a simpler selection of the best partitioning among a set; they are summarized in Table  1  to provide an overview of their advantages and limitations. However, some unsupervised clustering techniques are specifically crafted to perform without this parameter, in order to avoid the intrinsic difficulty one may encounter in estimating it.
Probably the most common category of unsupervised clustering techniques which build the clusters automatically is Hierarchical Clustering  [51, 52]. Such algorithms typically regroup data, or groups of data, by proximity or similarity in a hierarchical fashion. For instance, with a given dataset, the distance between every pair of data is computed, and the data of the pair with the lowest distance are linked together, forming the very first cluster; then, the procedure is repeated, but this time replacing the two previous data by their mean. As a consequence, all data are linked to each other in a hierarchical fashion, and the clusters can be obtained by stopping the linkage procedure somewhere. Knowing where to cut the link between the data/groups is the major drawback of hierarchical clustering, since it relies on a threshold (cut when two data/groups are at a distance higher than this threshold).
In practice, Hierarchical Clustering was used in  [53]  to extract the manufacturing bottlenecks of a real-world production system from the event logs only, allowing the operators to identify the corresponding bottlenecks in the real system; it was also used by  [54]  in a more Industry 4. 0-oriented context to propose a hybrid cut criterion in the scope of the design of a Decision Support System (DSS) for 3D printers. Another relevant unsupervised clustering algorithm which does not require the number of clusters to build is Region Growing  [55]. The general approach of such techniques is to draw a point from the database, aggregate its neighbors to it, thus forming a true cluster, and repeat the aggregation with any data contained within; once there is no more data close enough to any data in the cluster, it is considered fully built and another one is started following the exact same procedure, starting from any yet-unprocessed data.
Another very popular algorithm is the Density-Based Spatial Clustering of Applications with Noise (DBSCAN)  [58]  algorithm, which operates somehow by region growing, but with a dedicated initialization procedure. Indeed, according to this method, the density of a cluster is defined as the number of data contained within a given topology ball. Therefore, the algorithm searches for local groups of data whose density meets a given parameter (e. g. , if a datapoint has enough neighbors at a distance below this parameter, then it is assimilated as the cluster's seed), regroups all of them to form a "proto-cluster", before expanding it step-by-step by considering the neighborhood of the added data. Among other applications, DBSCAN was used in  [59]  to detect and segment the planar objects of a 3D point cloud, and it obtained good results on both academic and real datasets.
The main methods of unsupervised clustering introduced in this section are summarized in Table  2 , where their advantages and limitations are shortly mentioned to provide an overview of these methods. This detailed literature review shows that many methods exist to, on the one hand, estimate the optimal number of groups present in a given dataset, and, on the other hand, to propose clustering algorithms which would overcome that limitation, by operating in an iterative fashion. Nonetheless, most of the already-existing techniques are either a sophisticated manner to select the best partitioning among others (e. g. , the Elbow method), are computationally heavy (e. g. , the silhouette coefficients), or replace that knowledge by another as of a threshold. As such, there currently exists no reliable and non-parametric solution to automatically identify and extract the true regular states (the behaviors) of real unknown systems, whence the proposal of the SPRADA method.
Required Already-Existing Materials for the Proposed Methodology The proposed methodology, the SPRADA method, aims to identify the main region of an unknown feature space, by using a recursive and self-assessed approach. The proposed solution does not introduce new concepts sensu stricto, for it is essentially a methodology, i. e. , how to use already-existing tools to carry out the proposed behavioral identification. As a consequence, in order to carry out this study, some concepts must be introduced, in particular unsupervised clustering and the metrics to quantify the integrity of the clusters. These topics are the matter of the two following subsections. In all the following, d refers to any mathematical distance (e. g. , Euclidean), the data instances are represented by x, and the database is denoted as D and is defined by (1), D = {x i } i∈[[1,N]] , (1) with N = |D| the cardinal (size) of the dataset, the final set of clusters is defined by (2), C = {C k } k∈[[1,K]] , ( 2 ) where K is the total number of clusters, with any cluster C k typically defined by (3), C k = x ∈ D argmin k ∈[[1,K]] d(x, p k ) = k , (3) where p k is the representative of cluster C k , outputted by the learning stage.
Probably the most well-known unsupervised clustering algorithm is the K-Means (KM)  [64]  algorithm. Albeit naive, it is also the most intuitive fashion to regroup data. It starts by drawing K points from the database, the barycenters, then it associates any point of the database to the closest barycenter to form the groups, the clusters. Once they are built, the barycenters are updated as the clusters' means, and the procedure is repeated. The learning stops when any criterion is achieved, for instance a maximal number of iterations, or once the barycenters change very little from a round to the next. However, the K-Means has two limitations: it is not computationally efficient, and is only able to separate linearly separable groups (at least, in an Euclidean space). An accurate alternative to the K-Means is the Self-Organizing Maps (SOMs)  [65]  algorithm, which operate similarly, but adds a notion of neighborhood to the learning.
Indeed, there are still K points drawn first and assimilated to the clusters' barycenters, but now they are virtually linked to each other in the form of a grid. Then, the learning procedure consists of statistically learning from the dataset. To that end, a datapoint is randomly drawn, and every barycenter is attracted to it, by applying a learning function updating the position of the barycenters, to bring them closer to the data drawn. The update is the strongest for the closest barycenter, and its strength diminishes as the farther away the barycenters get, and the less connected to the first one they are (i. e. , the update is stronger for the direct neighbors of the closest barycenter, is weaker for the neighbors of the neighbors, and so on). Although aging, the SOMs are still very popular for their accuracy and efficiency: for instance, they found practical applications in  [66]  to evaluate the quality of groundwater, and in  [67]  to track the metaloids contained in soils.
The SOMs are very accurate, are computationally efficient (due to their statistical approach), and proved able to distinguish most regular states from real industrial datasets  [30] , which makes them appealing for the identification of the main regions of industrial processes with the SPRADA method. The SOMs are very general, work well in many cases, but some local adjustments may increase their accuracy and representativeness: Geographical SOM (Geo-SOM) to take into account spatial dependency  [68] , Time-Adaptive SOM (TASOM) to use time-adaptive parameters  [69] , Growing SOM (GSOM) to use a self-growing map  [70] , etc. A useful alternative may be the Bi-level SOM (BSOM), proposed in  [71]. This variant was originally crafted to identify the true regular states of a real industrial system, by somehow averaging several SOMs.
The closer to +1, the closer the data to its neighbors and the more distant from the other clusters, and vice-versa for the closer to -1. The silhouettes generally provide a good indicator of the integrity of the whole partitioning, but take a long time to compute (complexity in O(N 2 )), and there are too many coefficients to consider them all, thus, usually, only statistics of them are used (such as the average of the silhouettes of the data comprised by a cluster). Nonetheless, the computational complexity of the silhouette coefficients makes them ill-suited to very large datasets, such as industrial ones. ∀x i ∈ C k , avg(x i ) = 1 |C k | -1 ∑ x j ∈C k \{x i } d(x i , x j ) ( 4 ) ∀x i ∈ C k , dis(x i ) = min k ∈[[1,K]] k =k    1 |C k | ∑ x j ∈C k d(x i , x j )    (5) ∀x i ∈ C k , sil(x i ) = dis(x i ) -avg(x i ) max{dis(x i ), avg(x i )} (6) Another relevant metric to estimate the integrity of the clusters is the Average Standard Deviation (AvStd), which was the original metric used in the T-DTS  [73].
It is a simple and relatively unrepresentative measure of data scattering, even though it was sufficient to test and assess the recursive decomposition principle of the method. The Average Standard Deviation is aptly named: it is defined as the average of all the standard deviations of a dataset, computed along every dimension. Indeed, when dealing with a n-dimensional space, the traditional standard deviation does not really make sense, since it is typically computed with scalar values, not vectors; nonetheless, with real systems, especially when using machine-learning-based tools, the input data are very often vectors. Therefore, the AvStd computes the standard deviation σ (i) , defined by  (7) , of the data of a given set (e. g. , the database D or any cluster C k ) along every dimension i of the space, with m the dataset's mean (or m k for cluster C k ), and eventually returns the mean of all these values, as expressed by  (8).
This is close to the Physics' specific mass, thus, to obtain a true density, this value is eventually normalized by a reference, such as the maximal values found among all the clusters. Applied to both academic and real datasets, the Hyper-Density proved to be more accurate than the AvStd in the identification of the issuing clusters. Several hyper-volumes may be considered to estimate the volume of a group of n-dimensional data, but the hyper-sphere appears to be the most democratic one; according to  [75] , the hyper-volume V (n) of a hyper-sphere of radius r is given by (  9 ), V (n) r = r n. π n/2 Γ n 2 + 1 , ( 9 ) where Γ is the Gamma function. The radius r of the hyper-sphere may be estimated as half the span of the dataset S of mean m, or simply by the maximal distance from any point and that mean, as expressed by  (10).
Proposed Solution: the SPRADA Methodology The proposed methodology can be summarized as an optimizing decomposition of a space, by recursively analyzing the integrity of the clusters and deciding whether or not a new stage of partitioning could improve the results. That idea is intuitive, and was investigated about two decades ago, resulting in a multi-modeling method called the Tree-like Divide To Simplify (T-DTS)  [73] , whose first aim was to reduce the complexity of real, dynamic and nonlinear processes by locally splitting their feature space into smaller and more homogeneous regions  [76] , which was later extended in  [77]  to the modeling of such systems by locally modeling any of their identified regions. This method was first crafted to approximate functions, or to model complex systems, by recursively decomposing a database, then building local models upon each cluster, and eventually connecting all of them in some fashion to provide a global model.
Recursive Decomposition Principle The proposed SPRADA method relies on a recursive decomposition of a database. To that end, a database D is first split into a set of several clusters using any unsupervised clustering, and the integrity of every of them is then computed using any characterizing metric. Then, every cluster whose integrity is judged as low is split, and the integrity of the so-built groups is analyzed, and a new step of split is once again applied if needed. This procedure is repeated until the integrity of any cluster is good enough, or stops when a group contains only one single data. As a consequence, the split process resembles a tree, whose root is the full database, and the leaves are the clusters. The general operating principle of the recursive decomposition is depicted in Figure  1.
The last concern is to choose the value to which set the split criterion to. To answer that question, to assume the "Self-Parameterized" property of the proposed method, one may make a weak assumption: at least one region of the feature space is salient and is correctly isolated by clustering. From experience, this is often true, even with real industrial data: there is almost always at least one state which stands out, for instance the steady state, since the system should behave this way most of the time, thus the database should comprise a lot of data pertaining to it. Therefore, since this assumption is very often true, this cluster may serve as a reference, since it should have the best integrity among all, and can as such provide an idea of what "good integrity" means with the current database (whichever may it be).
As a consequence, as a split criterion, one may consider using a percentage of that maximal value, thus providing a dynamic and self-adaptive split criterion. A good value may be around 10% in general, but it depends on the inner scattering of the database. The pseudo-code of the recursive decomposition is given as Algorithm 1 to help the reader understand the different steps introduced until now. In this example, it is the Hyper-Density which is used as split criterion, but it may be replaced by any other metric; one should only pay attention to the "sense" of the criterion: it can be the higher or the lower, depending on the metric. For the Hyper-Density and the silhouettes, the closer to +1, the better, but, for the AvStd, the closer to 0, the better. Therefore, the word "higher" in the if statement on line 3 may be replaced my the word "lower" if needed.
The recursive decomposition principle is fairly intuitive, and has the advantage of greatly diminishing the impact of a wrong value set for the objective of clusters, if any. For instance, assuming the database actually features K clear regions, and that one partitions it using the K-Means such that K < K , thus some regions will be merged (best case), or even wrongly split (worst case). Nevertheless, assuming that merging two dissimilar regions with each other will lead to a unique cluster with a low integrity is not a so strong assumption; as a consequence, the problematic cluster would be detected and split to cluster it (more) correctly. However, this may lead to numerous small clusters, and the two regions considered may be split into many pieces, leading to inconsistent groups, likely poorly meaningful.
Merging of the Clusters Recursive split has the bad trend of generating numerous clusters, which may comprise possibly very few data; as a consequence, it may be hard to use the results as they are, with no further refinement. Therefore, the proposed SPRADA method includes a dedicated step of merging, in order to rebuild the regions of the feature space in a split-and-merge fashion. Indeed, the appearance of very small clusters is generally due to the split of a larger, not highly compact nor consistent regular state (behavior); in such a case, i. e. , when a same behavior is split into several clusters, their respective data should have close features, at least to some extent (especially compared to that of the other behaviors), and may therefore be compared and regrouped fairly easily. To that end, the SPRADA method will use the merging procedure introduced in the ECD test  [50] , which proved to be accurate when comparing (and merging) clusters obtained with real industrial datasets.
Second, any metric may be used as split criterion, but the Hyper-Density seems to be a good choice, for it proved to be accurate when characterizing real industrial processes  [79]. The silhouettes may also be considered, but they are long to compute, thus they may not be adapted to large industrial datasets. Notice that the Hyper-Density does not take into account the relationship between the clusters, but the recursive approach of the proposed decomposition overcomes that limit. Additionally, the split criterion may be set to 10% of the maximal Hyper-Density found among the clusters obtained at the initial partitioning. Third, for the final merging by region growing, a similar threshold may be obtained by searching for the maximal MHD between every pair of ECDs (representing the most distant clusters), and using a ratio of this value.
Presentation of the Database First, it should be mentioned that this work takes place within the European project HyperCOG, which aims to study the Industry 4. 0-orientated cognitive plant. To that end, some of the partners develop a CPS to connect the production units and management tools with each other, whilst the others work on the automatic extraction of knowledge so as to build some form of intelligence which may help operators manage and control their systems in a semi-supervised fashion. One important topic of the project is anomaly diagnosis, i. e. , detecting, identifying and investigating how to correct such anomalies. As discussed in Section 1, this can be performed using dedicated tools oriented toward (automatic) anomaly diagnosis, but also by identifying the regular behaviors of the system, since an anomaly should be any differing state, whence the proposal of the SPRADA methodology.
One of the industrialists involved in the project HyperCOG is Solvay, a chemistry plant located in La Rochelle, France, specialized in rare earth specialty products. In order to test and assess the solutions developed during the project, they provided the partners with several datasets recorded in real situations. One of them is a two-month long record of fourteen key sensors, operating at a frequency of one data every five minutes, for a total of 17,568 samples. Over that period, the processes worked correctly most of the time, although some local events occurred nonetheless (e. g. , short maintenance or true anomalies). Notice that the data of two of these sensors were almost the same than that of two other sensors, which have therefore been removed to avoid redundancy. Additionally, the sensors have been normalized between 0 and 1, and their names and dimensions have been randomized for confidentiality concerns.
Application of the SPRADA Method to the Real Industrial Dataset In order to assess the proposed methodology, it is applied to the industrial dataset of Figure  3. In this subsection, in order to illustrate the operating principle of the SPRADA method, only the Self-Organizing Maps will be used for clustering, and the Hyper-Density for the quantification of the integrity of the clusters. The impact of the clustering algorithm and the quantification metric will be discussed in Section 5. 4. In this subsection, the SPRADA method is performed in the feature space spanned by the twelve sensors of the database, using all of them during every stage of clustering and quantification. The clustering method used throughout this study is a 3 × 3 SOM, and the quantification metric is Hyper-Density with a hyper-sphere used as delimiting shape.
To provide an idea on the possible results of the complete recursive decomposition, Figure  5  depicts the outputted set of clusters for the twelve sensors of the database. This partitioning corresponds to the six splits of Figure  4 , for a total of fifty clusters. This figure is barely explainable, for they are too many clusters (the reason why a special focus on a single sensor will be made in the next subsection); nonetheless, the will behind this is to show that the recursive decomposition tends to produce many clusters, sometimes comprising many data (as those represented by the blue, orange, green and red crosses on the figure), but many other comprise very few; as a consequence, it is generally not possible to make sense of the clusters as they currently are, for they do not really represent anything in their present format.
As such, the SPRADA method will be applied to the dataset of Figure  3 , with each of these six variants. Moreover, even though the twelve sensors are used to carry out this study, only Sensor 12 will be depicted for the sake of clarity and readability. Notice that, since the number of clusters may differ from one test to another due to the stochastic property of unsupervised clustering, the results presented in the following have been selected so as to be compliant with Table  5. In practice, this means that, for a given pair of clustering methods and quantification metrics, the SPRADA method has been performed several times until it obtained a partitioning whose number of clusters is close to the typical number of groups one may expect when considering that pair: it is that partitioning that is presented and discussed thereafter.