# An Empirical Evaluation of Document Embeddings and Similarity Metrics for Scientific Articles
## 4 Embeddings and Measures
### Similarity Measures  
In NLP, the most common measures dealing with text are cosine similarity and soft cosine similarity. Both of them work directly over word or document embeddings. Although other measures have also been used, such as the Jacquard similarity, or the Euclidean distance, they are far less popular than the ones described next. **Cosine Similarity** is a measure of similarity between two non-zero vectors. CS makes a judgment of orientation, not magnitude, over the vectors. This is because it computes the angle of both and uses this angle as a measure of similarity. This measure is probably one of the most common similarity measures used and studied in the literature for areas such as data mining, information retrieval, or text mining. It can be calculated as follows:  
\[\cos\theta=\frac{A\cdot B}{\|A\|\cdot\|B\|}=\frac{\sum_{i=1}^{n}A_{i}\cdot B_{ i}}{\sqrt{\sum_{i=1}^{n}A_{i}^{2}}\sqrt{\sum_{i=1}^{n}B_{i}^{2}}} \tag{1}\]  
where \(A_{i}\) and \(B_{i}\) are components of vectors \(A\) and \(B\). The higher the absolute value, the more similar the elements should be. One of the problems of this measure is the fact that only angles between vectors are used, not magnitudes. If magnitudes are important, the cosine similarity maybe not completely useful. As for complexity, cosine similarity can be calculated in \(\mathcal{O}(n)\), where \(n\) is the dimensionality of the vector. It is important to remarkthat, although it is a similarity measure, it is not a proper metric because the triangle inequality does not hold, and it is defined for positive values only. **Soft Cosine Similarity** is a generalization of the CS [34]. When objects are represented in a Vector Space Model (VSM), they are represented as vectors of values of features, corresponding each feature to a dimension in the VSM. The traditional CS considers VSM features as independent or in an orthonormal basis, i.e., there is no relation between the different features. In many contexts, this is not realistic, and SCS tries to take this into account to strengthen the measure. To consider the similarity or relation between features the authors propose a free-context formulation, due to the feature-nature dependency, to compute the relation between the features. This relation is represented as a matrix that can be precalculated or evaluated in real-time following whichever measure or system. For example, in NLP context the authors use the Lavenshtein distance (String metric for measuring the distance/similarity between two strings by counting the minimum number of operations needed to transform a string into another one.) and hint at the possibility of using the measures available through WordNet (Lexical database of semantic relations between words.). [PERSON] et al. [34] found that SCS works better than CS for many tasks using the Lavenshtein distance as a measure of similarity between features. **Word Mover's Distance**[23] is a distance measure introduced that works over text documents. It is based on the well-known Earth Mover's Distance (or EMD), introduced by [PERSON] et al. [35]. WMD starts from the same point as SCS: the not real orthogonality of the feature vectors. Furthermore, as well as SCS, for the specific use in NLP, WMD tries to supply the necessity of considering the relation or distance between individual words when comparing two text documents. This measure takes advantage of the fact that word embeddings preserve the semantic relations in vector operations. Using it as a base, WMD, in general terms, represents text documents as a weighted point cloud of embedded words. It measures the distance between two documents as the minimum cumulative distance that words from document A need to travel to match exactly the point cloud of document B. Specifically, WMD, such as EMD, reduces the problem of the distance to a transportation problem. To calculate the distances of two documents, a flow matrix \(T\in\mathbb{R}^{(n\times n)}\) represents the cost of traveling from each word to the rest.