# An Empirical Evaluation of Document Embeddings and Similarity Metrics for Scientific Articles
## 7 Task II-Clustering
### Analysis of Results  
To illustrate the results, we have used a set of dendrograms obtained for each combination of measure and embedding, except for the ones that were too costly to calculate, as mentioned earlier. Notice that the coloring of the dendrograms has been left to the judgment of the function dendrogram() from the _SciPy_ library. Moreover, we edited the images to slightly stretch the charts so that they fit in a single column. We also changed the verbose labels for abbreviations (which were already introduced in the previous section). For the SCS case, both w2v and fT (see Figure 7) offer a similar performance. The super clusters obtained are positively consistent (slightly less with fT) and the small ones have some inconsistencies. For w2v, the Bicycle Sharing Systems set is combined with the Sports Analytics set. However, both fall in the same super cluster, which is acceptable. On the other hand, for fT, both previous cases are better clustered, but Ambient Occlusion, Immersive Analytics, and Molecular and Biological Visualization sets are combined more dubiously. Even with these small conflicts, both produced an excellent result for the Transport Visualization Methods, Rankings Visualization, and Mobile Rendering Systems. Gv, which works with statistical information obtained using the context window approach, as explained previously, is the one that works better generally. Obtained super clusters are completely consistent. As a counterpart, small clusters exhibit several mistakes larger than the others. When using SCS as a distance, its performance appears to be good for a less detail-oriented clustering. Ignoring the one-element-cluster, which constitutes a \"single-large-cluster\", in w2v and fT (see Figure 8), the RWMD case is, in general, slightly worse in comparison with SCS. Figure 7: Task II–clustering. Comparing w2v + SCS and fT + SCS shows that their behavior is similar, and that the results are good for some categories (e.g., TM, Rank, and MR.). Colors denote clusters that have been determined automatically by the plotting algorithm, and the threshold can be fine-tuned if necessary. The consistency at the third and fourth levels is high and approximately the same as for SCS, but at the fifth level, the clusters become inconsistent. Notice that, for both cases, the behavior is the same, just changing the conformation of the clusters. Remarkably, fT + RWMD separates well Bicycle Sharing Systems and Sports Analytics sets, while in SCS, this does not occur. On the other hand, w2v + RWMD clusters well Immersive Analytics and Ambient Occlusion sets, both highly impaired in SCS. These cases report the difficulty behind the clustering of papers from subareas of the same thematic area. The information captured by the embeddings, and its use by the distance measures, directly impacts their performances, dramatically varying the low-level results. GV + RWMD (see Figure 9a) has generally the same performance as GV + SCS, but is more mixed-up and forked at small clusters. This seems to be a direct consequence of RWMD, given that the performance of GV with SCS is more consistent, steady, and better. Figure 8: Task II. Clustering using w2v + RWMD and fT + RWMD shows less consistency in clustering: some groups are broken and grouped with other ones (e.g., TM or MR). Colors are assigned automatically by the dendrograms drawing algorithm. The threshold could be changed. The combination that behaves more consistently for small and big clusters is d2v + CS. In the experiments, only two errors can be spotted: one document from the Immersive Analytics set, placed in the Molecular and Biological Visualization cluster, and one document from the Molecular and Biological Visualization set, located in the Mobile Rendering cluster. NRC is logical in big clusters but, on the other hand, less consistent in small clusters, as shown in Figure 10. It separates papers from the same sets over many levels, being the measure that exhibits a stronger inclination in this direction. Figure 9: Task II.