# An Empirical Evaluation of Document Embeddings and Similarity Metrics for Scientific Articles
## 6 Task I-Validation
### Data  
To perform the validation experiment, two accumulative-incremental-by-addition datasets will be created; one for research papers, which is the main focus of this project,  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **w2v** & **fT** & **GV** & **d2v** & **ELMo** \\ \hline
**Vector size** & & 300 & & 1024 \\ \hline
**Window size** & & 7 & 5 & - \\ \hline
**Min vocabulary count** & & 4 & & - \\ \hline
**Corpus count** & & 398,387 & & 793471 \\ \hline
**Corpus total words** & & 4,366,988 & & 1B \\ \hline
**Training epochs** & & 10 & & - \\ \hline
**Architecture** & CSG & CSG & - & PV-DM & - \\ \hline \hline \end{tabular}
\end{table}
Table 3: Summary of the models’ training parameters, where applicable.  
and one for Ph.D. theses, to explore the performance of the combinations beyond the length of 10 to 20 pages that most papers usually have.  
An accumulative-incremental-by-addition dataset consists of taking two source documents, which will be called _base document_ (\(d_{b}\)) and _destination document_ (\(d_{d0}\)), respectively. Then, in step one, we add several words or sentences, randomly picked, from \(d_{b}\) to \(d_{d0}\). This results in a modified destination document (\(d_{d1}\)). We perform this step several times, but each step we take as the destination, the result of the previous step. That is, the step \(i\) of this procedure uses \(d_{di}\) document as destination one, and generates \(d_{di+1}\). The result is a set of document pairs that, with the intuitive definition of similarity, should be increasingly similar. In our case, we add sentences, to preserve the context of the words, which is an important aspect for the embeddings that capture words' context. This creation process is based on the premise that the more sentences the destination document receives, the more similar to the base document should be. Furthermore, the decrease/increase should be monotonous, since the previous sentences are accumulated at each step.  
For the papers' dataset, the same previously listed research groups from our University (UPC) will be used. To systematically evaluate the combinations for different scenarios, one base paper from one group (with acronym CDIF) will be tested with other five papers from the same research group (named CDIF1 to CDIF5 in the figures), and the same base paper will be tested with other five papers from five different research groups (with acronyms ANT, ADR&M, UMA, AIEM, and IDEAI). The list of selected papers can be found at Appendix B. At each step, 10 new sentences are copied from the base paper to the destination one. The result is a set of 11 documents ranging from 0 to 100 transferred sentences.  
Following the same idea, the Ph.D. theses set has been created, taking as a base the thesis \"Influence of agricultural practices on the Microbiome and the Antibiotic Resistance Gene complement in soils, plants, and crops\" by [PERSON], and as a destination, the thesis \"Deep Learning architectures applied to wind time series multistep forecasting\" by [PERSON]. Unlike the set of papers, for the theses, the steps go from 0 to 1000 transferred sentences, with 10 sentences copied each step. We proceeded this way because both theses have more than 150 pages with very diverse and differentiated information, which appears to be hard to break just by passing a few sentences.