# User Representation Learning for Social Networks:
## 5 Experimental Results and Discussion
### Comparison of Different PWE Models  
Table 12 compares the use of publicly available PWE models in the user representation task introduced in Section 4.3. In addition, experimental results for the commonly favored Glove model are listed separately in Table 13 (see Table 8 Cases 1-2). Glove embeddings are available in various vector formats in terms of their vector sizes and the corpora on which they were trained. Table 13 shows how these properties affect the performance of the user embeddings. The F1 scores are the weighted average of the F1 scores of all groups.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**PWE MODEL** & **Dimension** & **Accuracy** & **F1** \\ \hline Fast Text & 300 & 0.810 & **0.809** \\ Number Batch & 300 & 0.810 & 0.807 \\ Google News & 300 & 0.720 & 0.713 \\ \hline \hline \end{tabular}
\end{table}
Table 12: Comparison of the PWE models.  
Figure 3: 2d view of the best model (accuracy 97.5%).  
The performances of the Word2 Vec models that were trained on Dataset A from scratch (Table 8 Case-3) with different min count parameters are presented in Table 14. The dimension size is chosen to be 100 for these models. The best word2 vec model gives the better accuracy compared with results in Tables 12 and 13.  
Table 15 shows the performance of the contextualized word embeddings BERT and ELMO. A document pool embedding is created by stacking BERT, ELMO, and Glove embeddings using the Fair NLP framework (see Table 8 Cases 4-5).  
Figure 4 depicts the F1 scores obtained in TF-IDF-based user representation experiments (see Table 8 Case-6). The number of maximum features is the size of the vocabulary, which also determines the size of the user embeddings. Thirty experiments are conducted with five different training datasets for six different numbers of features. It can be seen that datasets A and C perform better than the other datasets. The detailed accuracy and F1 scores for these experiments are reported in Table A4 (see Appendix C).  
Glove, FastText, Google News, and NumberBatch are publicly available, ready-to-use PWEs. They are trained on different datasets with different algorithms and hyperparameters. Therefore, it is difficult to compare their performance on this task under exactly the same conditions. However, the comparison of the different Glove embeddings shows that dimension size, dataset, and number of tokens have some effect on the accuracy of the user embeddings. It is also shown that GoogleNews embeddings do not perform well for this task. FastText and NumberBatch and Glove give similar accuracy results. The pre-trained Bert model also performs as well these models but it does not outperform them. Considering the high computation and time cost caused by the big dimension sizes of Bert and Elmo, these models were not proved as effective alternatives to traditional PWEs without a fine-tuning process.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Ambedding Model** & **Dimension** & **Accuracy** & **F1** \\ \hline Bert & 3072 & 0.815 & **0.814** \\ Elmo & 3072 & 0.770 & 0.765 \\ Bert-Elmo-Glove (Stacked) & 3072 + 3072 + 100 & 0.815 & 0.812 \\ \hline \hline \end{tabular}
\end{table}
Table 15: Contextualized embedding experiments.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Corpus Source** & **Vocab Size** & \begin{tabular}{c} **Number of** \\ **Tokens** \\ \end{tabular} & **Dimension** & **Accuracy** & **F1** \\ \hline Wikipedia & \(400\times 10^{3}\) & \(6\times 10^{9}\) & \begin{tabular}{c} 100 \\ 200 \\ 300 \\ \end{tabular} & \
\begin{tabular}{c} 0.835 \\ 0.815 \\ 0.800 \\ 0.800 \\ \end{tabular} &
\begin{tabular}{c} 0.828 \\ 0.815 \\ 0.797 \\ 0.797 \\ 0.799 \\ 0.835 \\ 0.840 \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 13: Glove hyper-parameter search.