## 4 Self-Tuning Exploratory Data Analytics
### Joint-Approach
#### 4.1.2 K-Means Algorithm  
In the joint-approach, the singular value decomposition is applied to data to cut down the dimensions of the data prior to the learning process. Since the different document-concept vectors can be clustered, the learning process implements the K-means algorithm. The difference between clustering and LSA is that clustering algorithms assign each document to a specific cluster, while LSA assigns a set of topics to each document. Still, a K-means algorithm applied after the singular-value decomposition improves the results, as shown in [1; 32]. We have decided to implement the K-means clustering because it is an easy algorithm to implement that has good performance and which converges quickly, all while providing good results [44; 45]. Moreover, the performance of the algorithm is still being researched in order to obtain better and better results [46], which would allow us easy adaptability in the case of new and better-performing techniques. ESCAPE manages to discover groups of documents that share a similar topic by self-assessing the quality of the found clusters. It uses an algorithm to automatically configure the cluster analysis activity through an analysis of different quality metrics to evaluate the obtained partitions. To this end, several configurations have been tested by ESCAPE, modifying the specific-algorithm parameter (i.e., number of desired clusters). **Self-Tuning Clustering Evaluation. **  
After the formation of the \(K\) clusters from the collection of textual documents, it is necessary to corroborate the clustering results with three indicators obtained from the computation of the silhouette [47]. The silhouette index gauges, from a qualitative point of view, the similarity of an element with respect to its own cluster (cohesion), and compared to other clusters (separation). The silhouette varies from \(-1\) to \(+1\). If the silhouette has a high value, it means that the object is cohesive to its own cluster and well-separated from the neighboring cluster. In order to estimate the cohesion and separation of each cluster set, the solutions found are compared through the calculation of different silhouette-based indices to measure t. Then, the best three configurations, which identify a proper division of the original collection, are chosen. ESCAPE exploits three versions of the standard silhouette index to assess the quality of the discovered cluster set: (i) the weighted distribution of the silhouette index (WS) [1]; (ii) the average silhouette index (ASI) [48]; and (iii) the global silhouette index (GSI) [48]. Specifically, the WS index indicates the amount of documents in each positive bin that are properly weighted with an integer value \(w\in[1;10]\) (the highest weight is given to the first bin [1; 2; 3; 4; 5; 6; 7; 8; 9], and so on) and normalized within the sum of all the weights. It is better to have distributions with a positive asymmetry (i.e., more documents have silhouette values belonging to the higher bins) instead of those with a majority of lower silhouette values (negative skewness). ASI gives an overview of the average silhouette of the entire cluster set, while GSI is able to take into account the possible imbalance number of elements in each cluster. If these indicators have higher values, it means that there is a better clustering validity. A detailed description of all the computations of these metrics is reported in Section 5. We apply a rank function for each quality index to estimate the cohesion and separation of each cluster set. The rank assigned to each quality index may vary from 2 (assigned to the solution with the highest silhouette index) to the Kmax (assigned to the solution with the lowest silhouette index).