# An Empirical Evaluation of Document Embeddings and Similarity Metrics for Scientific Articles
## 3 Related Work  
Although specific methods tailoring a concrete domain may yield better results than unsupervised methods (e.g., [4; 5]), we are interested in general models widely applicable to a range of fields. One of the key factors is that training learning models for specific domains or tasks is very costly regarding energy and environmental effects [6]. There is a significant amount of research involving texts comparison. However, the number of ways that text can be processed and compared varies greatly. Most typical forms of text similarity involve small documents, ranging from single words to one sentence or a paragraph. These are simpler to evaluate. One of the challenges we find when addressing evaluation problems is the difficulty to find baselines to compare. For example, some datasets have been built to represent word similarity and relatedness. However, they are fairly small, with some hundreds of words [7]. Furthermore, there are no equivalents for different languages. When addressing longer documents, researchers evaluate their success by custom datasets, and by manually analyzing some results. For example, [PERSON] et al. [8] compare Wikipedia articles, and for a selection of results, they check whether the nearest neighbors make sense. [PERSON] et al. [9] compare patents, and they use the hierarchy of classes created by USPTO, patent citations, and patents rejections list (as they expect to be more similar than patent citation) to assess their results. [PERSON] [7] use academic articles from PubMed, and they create a custom dataset for testing, where the similarity is obtained from common references. Finally, [PERSON] created a synthetic testing dataset by taking research articles and iteratively adding pseudo-random sentences from one document to another [10]. This way, the documents become more and more similar. Though we might not find this kind of scenario in the real world often, it may be suitable for evaluating measuring similarities, for example when the problem to solve involves ranking the similarity among a set of texts. We can find in the literature several papers that compare the behavior of different vector space models when used for similarity detection between texts. Most of those comparisons are restricted to a few vector representations. Furthermore, they usually do not compare similarity measures: commonly cosine similarity is the one used. For short texts, several papers focus on word or sentence embeddings. For example, [PERSON] and [PERSON] compare _word2 vec_ and _doc2 vec_ in two tasks (duplicate questions and sentences' similarity) that involve only short paragraphs [11]. [PERSON] et al. [12] presented a method for sentence embedding based on a weighted averaging of word vectors. Their method outperforms other popular methods to detect sentence to sentence similarity. Other approaches that create sentences embeddings are due to [PERSON] et al. [13] or [PERSON] et al. [14]. For longer texts, [PERSON] et al. [8] compare _doc2 vec_[15], also known as paragraph vectors, with Latent Dirichlet Allocation (LDA) [16] for Wikipedia articles search and _arXiv_ papers. They found that _doc2 vec_ works better than LDA for finding nearest neighbors on Wikipedia. It also generates a lower number of false positives. In both cases, the similarity measure used is cosine similarity. On the other hand, given the large space of parameterization possibilities, as well as the problems to be addressed, previous research has shown contradictory results when analyzing the performance of different embeddings, such as _GloVe_ versus _word2 vec_ (cf. [17; 18]). It is, however, widely accepted, that these systems outperform other classical approaches, such as Pointwise Mutual Information, for some tasks [19]. [PERSON] et al. [9] compare patents using vector spaces built using TF-IDF and _doc2 vec_. Though the latter works better, to achieve the best results, fine-tuning is necessary. Given the costs involved, TF-IDF may be a sensible choice in many scenarios due to its performance and lower costs.