dataesr/scientific-paragraphs-categorization huggingface dataset:
The following solution respects the result of disassociation on different levels: • What has been disassociated in two distinct clusters should not be re-associated. • Associations from different record chunks of the same cluster can only be re-associated. • The associations preserved in a record chunk should not be altered or re-associated between them. They already passed the k m -anonymity test for anonymization. ALgorithm 5 takes as input the disassociated dataset T * , and starts by reconstructing a neighbor cluster C of each cluster C from T * (Line 1). C i is the cluster containing the result of the gradual re-association between the record chunks of C i. At first, C i is loaded with the records of the first record chunk R C 0 (Line 2). Further, the original number of records in a cluster affects deeply the re-association where empty records show the weakness of the associations between the record chunks.
The two selected itemsets, r 0 and r 1 , are merged together (Line 11) and moved to the temporary cluster ReconstructedRecord (Line 12), waiting for all the itemsets in R C j to be merged with other itemsets from C i in a similar way (Line 8-15). The generated itemsets in ReconstructedRecord are added to the neighbor cluster C i for the merge with the next record chunk (Line 16). The union of all the generated clusters form the neighbor dataset of the disassociated dataset (Line 18). In Section 5, we present a set of experiments to evaluate the result in terms of neighborhood generation. Algorithm 5 Probabilistic wheel re-association. Input: T * Output: T 1: for each cluster C i in T * do 2: C i = R 0 C i 3: for k = 0, k++, while |C i | -|C i | > 0 do 4: C i = C i ∪ / 0 5: end for 6: for each record chunk R j C i from cluster C i such that j > 0 do 7: ReconstructedRecord = / 0 8: while R j C i > 0 do 9: r 0 = SELECTRECORD(C i ) 10: r 1 = SELECTRECORD(R j C i ) 11: r 0 = r 0 ∪ r 1 12: ReconstructedRecord = ReconstructedRecord ∪ r 0 13: C i = C i \ r 0 14: R j C i = R j C i \ r 1 15: end while 16: C i = C i ∪ ReconstructedRecord 17: end for 18: T = T ∪C i 19: end for Algorithm 6 SelectRecord Function 1: function SELECTRECORD(R C ) 2: Assign counts count r to the distinct records r in R i C.
Analysis of Re-associated Datasets in terms of Distance How can a re-associated dataset be evaluated in terms of neighborhood generation? A suitable distance provides precise indicators about the similarity between objects. As presented in section 3, tree edit distance provides a solution to assess the similarity between set-valued datasets. Using this metric, it is easy to assess, first the distance between two re-associated datasets and second the distance between a re-associated dataset and the original dataset. For two datasets to be neighbors the distance between them should be small, in other words, few modifications should be made to make them similar. Neighborhood is generated when multiple re-associated datasets are close to each other. It is interesting to investigate the process of neighborhood initiation by generating re-associated datasets using the same/distinct privacy constraints.
However, using the tree edit distance to evaluate the distance between the generated dataset and the original one, can cause a privacy breach in practice. Let us consider a very basic example where the distance is really low, almost zero: in this case, the public can be sure that they have the original dataset. Yet, we stress the idea that with a big dataset and domain, probabilistically it is very rare to regenerate the original dataset. Therefore, the distance between a re-associated dataset and the original one should not be leaked in any sense to the public, increasing the awareness of an attacker to the background data. The next section presents experiments evaluating the neighborhood generation and the preservation of the data utility for analysis. Despite our realization of the privacy threat that might occur when calculating the distance with the original dataset, we are going to use the tree edit distance on the original dataset, to prove the efficiency of the probabilistic wheel re-association algorithm in terms of neighborhood generation.
Evaluating the Neighborhood of Set-valued Datasets We start our investigation from the Probabilistic Wheel Re-association algorithm to see how well can it generate a neighborhood from the original dataset. Despite our awareness in terms of privacy worries arising from the calculation of the tree edit distance with the original dataset. Our main objective in these experiments is to evaluate realistically the efficiency of the proposed algorithm; and this cannot be done without the original dataset. To evaluate the neighborhood of the generated datasets, disassociation on BMS1 with different k, m and δ constraints is applied and then multiple re-associated datasets for each combination are generated. TED is then calculated between every re-associated dataset and the original one. The implementation of APTED, developed in  [23, 24] , is finally used to calculate the TED.
The results for the following analysis, are represented in two formats: First, Table  4  represents the average TED calculated between multiple re-associated datasets and the original dataset and the standard deviation of TED, under the same privacy constraints (same k, m and δ). • Are datasets generated from the same disassociated result reflecting a neighborhood? • What is the effect of the privacy constraints on the distance? Second, Figure  5  is a visual representation of the calculated distances for five samples of each combination of k, m and δ, to make more sense of the distances calculated. For a more concrete understanding, we plot the calculated TED relatively to the average TED by δ, without using a scale. In the following subsections we analyze the represented results to answer respectively the following questions: 5.
1. 1. The Neighborhood Generation TED evaluates the transformations that a disassociated dataset has to undergo to transform itself into a re-associated dataset. In this analysis, we are searching for datasets that are relatively in the same distant range from the original dataset. This information is enough to distinguish neighborhoods of datasets. It is notable from the data in Table  4  that datasets generated from the same privacy constraints (k, m, δ), through the probabilistic wheel re-association algorithm, fall under the same distance range, where TED is almost equivalent for the same k, m, δ combination. This means that from the same disassociated dataset, we can generate datasets that need relatively the same amount of transformations to retrieve the starting point (i. e. , the original dataset), where the standard deviation of the TED calculated is fairly small.
Data Privacy and Neighborhood Distance In this section, we investigate the effect of the level of privacy on the neighborhood generation. From the first analysis, we recognized that the same privacy constraint can generate re-associated datasets that have a close radius to the original dataset. What is shockingly interesting is that higher privacy (k = 4 and m = 4) generate neighbor datasets closer to the original dataset than lower privacy constraints (k = 2 and m = 2) as shown in Figure  5. This can be explained by the reality that higher privacy is harder to achieve while disassociating the original dataset. In fact, imposing an extra check on associations between items in a cluster, will force dissimilar subsets to be separated into distinct record chunks, thus creating less diversity in a record chunk.
Accordingly, the probabilistic wheel re-association algorithm will have less diverse subsets to choose to re-associate together between record chunks. We can notice another behavior related to the maximum cluster size δ, smaller δ generate datasets that are closer to the original dataset and this is due to the fact that when fewer records are allowed in a cluster, diversity in terms of associations is minimized. Therefore, giving less choice to probabilistic wheel re-association by imposing higher privacy (k, m), and lower number of records to re-associate (smaller δ) can lead to generating datasets that are less distant from the original dataset. To conclude, the above experiments prove the efficiency of the Probabilistic Wheel Re-association algorithm for its ability to generate different solutions, yet taking into account the privacy level imposed in the disassociation phase.
Neighbor Datasets and Knowledge Discovery Creating a neighbor dataset is useless if it is not a synthetic representation of the original dataset. We dedicate this section to validate our approach for data analysis faithfulness between the original dataset and its neighbors. To do so, we examine the widely used techniques for mining set-valued data, the frequent itemsets and frequent association rules. Apriori is an algorithm for frequent itemset mining and association rule learning. It proceeds by identifying the frequent individual items in the database and extending them to larger and larger itemsets as long as those itemsets appear sufficiently often in the database. We benefit from the implementation of the apriori algorithm in ARULES, developed in  [25, 26] , to investigate the frequent itemsets and association rules representation in neighbor datasets.
Mining Frequent Itemsets In order to go deeper in the analysis of the neighbor datasets, we turn our attention to the most frequent itemsets. The discovery of frequent itemsets help decision-makers develop new strategies by gaining insights into which items are frequently found together. For this investigation, we use the support metric, defined as the proportion of transactions in the dataset D which contains an itemset X. support(X ) = |X| |D|. ( 10 ) Table  5  presents the itemsets with support greater or equal to 0. 008. We are able to extract 17 itemsets, representing the most frequent itemsets. To compare the accuracy of knowledge extraction between the original and re-associated dataset, we compared their ranks in terms of frequency. Despite the drop of the count of those itemsets, we can notice that they reappear in the re-associated dataset in the same rank range with mild changes in the ranking, yet they are still the 17 most frequent itemsets in the re-associated dataset.
We further tested the three most frequent itemsets, shown in Table  6. We could not detect their presence with the same minimum support, so we dropped it from 0. 008 to 0. 005, which means they are less frequent in the dataset. However, we found that even for less frequent itemsets we can retrieve them in almost the same rank range. This shows that a neighbor dataset is a faithful representation of the original dataset, by preserving the most frequent itemsets in the same rank range even with the drop in the support.
Mining Association Rules The discovery of association rules draws another view on the neighbor datasets. In their default definition, association rules represent a correlation between itemsets. An association rule is defined as an implication of the form X =⇒ Y where X and Y are two disjoint itemsets, X ∩Y = / 0. They illustrate a concept of correlation between the two itemsets, where the presence of itemset X shows a strong presence of itemset Y. The discovery of association rules is a more complex characterization of data that depends fundamentally on the discovery of frequent itemsets. We searched for the most frequent association rules by their support, which is relative to their count and the size of the dataset. For this experiment, we extracted the 50 most frequent association rules in the original dataset with a support greater than 0.
001. Then, we calculated the average support for those association rules in the generated re-associated datasets, T. From Figure  6  we can see that the support of the most frequent association rules is well-preserved through probabilistic wheel re-association. This shows that neighbor datasets are a faithful representation of the original dataset, by preserving reflecting close support. We can notice a loss in 5 Association Rules (support =0); in fact, this is not a real loss this is reflecting association rules that have a support less than the minimum threshold(0. 001) after re-association. However, if the need is to preserve predefined associations, ant-driven clustering, investigated thoroughly in Section 2, can be used.
Conclusion Anonymization is challenging when data must be analyzed, studied and explored afterwards. Unstructured data complicate the anonymization process, since in a very general way data items may vary tremendously in structure and value without following a pattern. Set-valued data is a type of unstructured data representing multi-sets of sets. In this work, we are interested in publishing an anonymized set-valued dataset, ready for future accurate analysis. In this context, set-valued data is considered for isolated data, without temporality, and without semantics. This scenario is a generalization of specific data values, which can be considered for a broader investigation of set-value data. Disassociation is an anonymization technique for set-valued datasets, presented by  [2]. It guarantees k m -anonymity for associations without altering the value of the data items.
It proceeds by grouping the data records in different clusters and then vertically separate items of the records when k m -anonymity is not verified, thus creating ambiguity on the level of associations for the items that are vertically separated in a cluster. This paper is an extension of the work done in  [1] , where the utility-privacy trade-off in a disassociated dataset is studied deeply. The loss of associations for aggregate analysis is considered for the theoretical study. We came to the conclusion that the loss of the utility is directly linked to the clustering process of disassociation. Driven by this problem we propose in  [1] , UGAC, to drive the process of clustering for a set of records representing predefined utility rules. As a continuity of the previous work, we tackle the problem of knowledge extraction in a disassociated dataset.
We know from the aggregate analysis that it is hard to evaluate itemsets that are split on multiple record chunks. The problematic becomes: how can we re-associate the itemsets from the disassociated result while staying faithful to anonymization and knowledge extraction at the same time, in set-valued datasets? To solve this problem, we define a general notion of similarity between datasets: neighbor datasets. Neighbor datasets are datasets that are not copies but synthetic representation, that leads to trustworthy data analysis. To standardize our notion of a neighborhood, we need a distance to assess it. Up to our knowledge, there exists no specific metric for set-valued data which is mathematically defined as a multiset of sets. Our first contribution in this work is the formalization of the datasets into trees and the use of tree edit distance that calculates the number of transformations needed to move from one tree to another.
This way we are able to calculate a distance between two multisets of sets, also known as datasets. Our second contribution is an algorithm that intuitively generates neighbor datasets. We propose a probabilistic wheel re-association algorithm to generate a re-associatied dataset from the result of disassociation of the original dataset. Finally, we test our utility guided ant-based clustering and probabilistic wheel re-association algorithms to evaluate their efficiency especially for knowledge extraction and data analysis in the context of general set-valued data. From the experiments, we can see that re-associated datasets create a neighborhood to the original dataset that depends on the privacy level imposed by disassociation. Despite the perturbation and noise added to the support of itemsets, probabilistic wheel re-association is able to generate synthetic datasets respecting the overall data representation of the itemsets and association rules in the data.
This is extremely interesting for prediction and decision-making analysis, as statistical exploration will lead to a very close hypothesis. On another side, when applying the same privacy constraints for disassociating the original dataset, the probabilistic wheel re-association algorithm generates re-associated datasets almost at the same distance from the original dataset. This reflects the fact that our approach respects the privacy imposed through the process. We can conclude that probabilistic wheel re-association is a faithful algorithm for knowledge extraction and data analysis over anonymized set-valued datasets. The preservation of predefined utility rules is necessary when we want to ensure their representation over a threshold. We run a set of experiments with the utility guided ant-based clustering algorithm, UGAC, to see how well can it preserve the utility rules.
UGAC is compared with the classical clustering technique, k-means, and normal horizontal disassociation for various properties of utility rules. The results show that UGAC, compared to the other two solutions, is able to decrease the information loss for the utility rules without increasing the information loss of other associations in the cluster. Combining the two solutions, we can say that knowledge extraction and data analysis is exceptionally valid on anonymized datasets when transformed into neighbors to the original dataset. In future work, we intend to use the Tree Edit Distance on the set of generated neighbor datasets, to decide which re-association is the most representative of the datasets for publication, being the centroid, or in other words the least distance to all the other datasets.
Funding: : This work is funded by the  Labex ACTION program  (contract  ANR-11-LABX-01-01 ), by the  Interreg   RESponSE  project, by the  Hubert Curein CEDRE  Programme number  40283YK , by the  National Council for Scientific Research in Lebanon CNRS-L , and by the  Antonine University. Computations have been performed on the supercomputer facilities of the  Mésocentre de calcul de Franche-Comté.
This raises the problem of how to analyze so many points using traditional computer and statistical methods, to reach a clinically relevant diagnosis. Here again, artificial intelligence can contribute in comparing all the patient's clinical and cephalometric data with a wide range of cases to automatically provide an accurate cephalometric diagnosis  [53] [54] [55]. The cephalometric analysis process, from the placement of landmark points to diagnosis, would thereby be fully automated. For surgical simulations, the use of displacement ratios is difficult to generalize and approximation models are mathematical estimates. Training a machine learning algorithm on multiple postoperative 3D models would allow precise simulations based on factual data and as many parameters as possible such as age, sex, ethnic origin, surgical technique… Machine learning algorithms require databases for training, allowing them to provide results based on "acquired experience".
A standardization of practices is therefore required along with inter-center cooperation to assemble databases of pre-and postoperative 3D cephalometric data for a wide range of patients, as well as any clinical information relevant to the surgery or follow-up (age, weight, ethnicity, sex…). Databases nearly as complete as this are already available for craniofacial research  [32][56]. Databases such as these, so large that they are beyond current analytical capabilities, are referred to as big data. They can be tackled by machine learning techniques involving artificial neural networks. To reach a diagnosis, physicians rely on their experience or at best on knowledge drawn from cohort studies. A neural network trained on a database exploits the experience drawn from an uncountable number of cases in populations with variable characteristics, treated by a multitude of physicians using different methods.
[5]  based on a hybrid model constructed a training formwork of the ELM model. They adopted a time window features scaling as a preprocessing and appropriate features selection step to guarantee an accurate prediction of RUL. Ordóñez et al. [6]  proposed a hybrid autoregressive model combined with an improved SVM using genetic algorithm to build several estimation algorithms for early RUL prediction. Chen et al. [7]  used a SVM-based similarity approach for RUL prediction with the same C-MAPPS dataset. It should be mentioned that all cited works could be classified as a hybrid model that aims to accurately predict the RUL by an unsupervised training or preprocessing of training data before a "single-batch" supervised training. However, time-varying data and parameter update is not considered in these cases, which is incompatible with data-driven prediction models able to address dynamic health deterioration of equipment according to internal or environmental conditions.
• An OS-ELM is introduced. Tikhonov regularization is involved to reduce structural risk and over-fitting by minimizing the norm of the training weights. Dynamic forgetting of old data, USS and TD error objective functions are all integrated into both SAEs and OS-ELM to achieve better accuracy and adaptability to newly arrived data. The proposed approach is applied to the public dataset of the turbofan engine  [20] , where obtained results compared to other recent used approaches in the literature show higher performance. The remainder of this work is organized as follows: In Section 2, a general description of the turbofan engine dataset is presented. Section 3 elaborates on the proposed approach used in this paper. Section 4 illustrates and discusses the experimental results where the performances of the proposed learning scheme are showcased and compared to other data-driven methods.
Dataset Description The C-MAPSS dataset consists of four different subsets, where each dataset consists of multiple multivariate time series and describes different operating conditions and faults modes for different engines. Each dataset is divided into training and testing sets. In our work, we have used the subset FD001 and only a single operating condition and one fault mode are considered. At the beginning of each time series, the engine is considered as normally functioning according to initial conditions and at a certain level, it starts deteriorating gradually by losing its performances until the failure. Each dataset contains 26 columns which are considered as the input features of the prediction model, each column contains information about engine number, time cycles, sensors measurements, and operational conditions of the operating engine  [21].
Dataset Preparing The preparation of the dataset is firstly initialized with an appropriate feature selection where only the most significant and sensitive features will be chosen. Each one must describe a certain attitude of degradation level or gradual time-varying behavior in each time cycle. Measurements that do not represent any variation during operating cycles will not contribute to the learning process. In fact, their noncorrelation with the desired RUL can disturb the training model and makes it more sensitive to over-fitting and other ill-posed problems. In our experiments, we chose only 10 parameters for RUL prediction whose indices are: {7, 8, 9, 12, 14, 16, 17, 20, 25, 26}. After that these features are scaled in the interval [0-1] using a min-max normalization procedure before training or testing.