## 6 Results
### Sentence Evaluation Tasks  
This subsection shows the results of proposed approaches on the SentEval library. In order to make fair comparisons, it was decided to use the same parameters that are described on the repository page. Hence, for tasks that require classification models, batch size is set to 64, [PERSON] is used as optimizer, number of epochs is set to 4, and each experiment is done by using a _k-fold_ (\(k=10\)) cross-validation method. We also evaluate the performance of different _sBert_ sentence embedding models in order to show that the used dimensionality reduction technique is capable of performing without substantial loss from its original model (sBert\({}_{768}\)). Since our reduced _sBert_ vectors (sBert\({}_{reduced}\)) have 300 dimensions, we find it appropriate to compare it with a 384 dimensional, trained on 12-layered architecture model (sBert\({}_{384}\)). To assess the quality of _sBert_ embeddings, we utilized a script implemented using the Sentence-Transformers python package. For evaluating _FastText_ sentence embeddings, we used the _bag-of-words_ approach. For a detailed comparative analysis with other models in the literature, refer to [14; 23], which provides extensive benchmarks across various embedding architectures by using the same datasets. Table 3 presents the results for classification tasks. Refined _FastText_ word embeddings, updated through a _distance minimization_ method, underperform when compared to the original FastText\({}_{subset}\) across all tasks. Moreover, employing the _distance minimization_ model alone does not lead to performance gains. Conversely, our second proposal, the _distance and perplexity minimization_ model and its updated _FastText_ embeddings demonstrate improved outcomes on _CR_, _MPQA_, _SST2_, and _MRPC_ datasets in comparison to the FastText\({}_{subset}\). This performance improvement is partly due to the _Word Domain Coverage_ metric included in the table, which assesses the overlap between the training data vocabulary used in transferring knowledge and that of the classification datasets in SentEval. Generally, datasets with higher domain coverage scores, indicating more vocabulary overlap, correspond to better results. However, the MRPC dataset represents an outlier in this trend. Unlike other datasets that primarily require the construction of a classifier, _MRPC_ also demands the comparison of sentence pairs to determine semantic equivalence, which is a distinct task. Consequently, the success on _MRPC_ is not solely attributable to word domain overlap but also to the model's ability to discern detailed semantic relationships. Our findings from Table 4 support this nuanced view. \begin{table}
\begin{tabular}{c c c c} \hline
**Model** & **Semantic Accuracy** & **Syntactic Accuracy** & **Total Accuracy** \\ \hline sBert\({}_{768}\) & 30.77\% & 21.83\% & 22.01\% \\ \hline FastText\({}_{subset}\) & **98.72**\% & **80.20**\% & **80.58**\% \\ \hline FastText\({}_{updated}\) by & 1.28\% & 0.12\% & 0.14\% \\ Distance Min & & & \\ \hline FastText\({}_{updated}\) by & 69.87\% & 52.67\% & 53.02\% \\ Distance and PP Min & & & \\ \hline \end{tabular}
\end{table} 
Table 2: Google Analogy Task results. sBert\({}_{768}\) corresponds to averaged sub-word results, FastText\({}_{subset}\) denotes the original embeddings, and FastText\({}_{updated}\) covers the embeddings enhanced by Distance Min and Distance and PP Min methods. Table 4 delineates the results for semantic textual similarity (STS) tasks. Mirroring the trends observed in classification tasks, embeddings updated via the _distance minimization_ method continue to underperform. However, when these embeddings are further processed through the _distance minimization_ model, there is a notable improvement, with performance exceeding that of the original _FastText_ (FastText\({}_{subset}\))embeddings in five out of seven STS tasks, indicative of a superior mean performance.