## 5 Experimental Procedure  
In this section, the experimental framework is employed to validate the efficacy of our proposed methodologies. Since the information transfer is carried out between two different models, it is necessary to minimize the number of parameters to be learned. Instead of having the network learn the dimensional differences between _sBert_ sentence vectors and _FastText_ sentence (word) vectors (sBert sentence vectors have 768 dimensions whereas _FastText_ embeddings have 300 dimensions. This situation forces the proposed models to output 300 dimensional vectors which requires an additional feed-forward network for eliminating the dimensional difference), we utilized the orthogonal mapping process to reduce the dimensions of _sBert_ sentence vectors to the dimensions of _FastText_ vectors. This operation requires calculation of sentence vectors from both approaches for input-output relations. Then, by calculating a simple rotation matrix, the dimensions of _sBert_ are reduced to the same size of _FastText_. This process can be referred to as _supervised dimensionality reduction_[22]. We named the resulting vectors as _sBert_\({}_{\textit{Rel}}\) vectors. The test values for the _sBert_ vectors obtained from this process are also given in the Section 6.  
The _distance minimization_ method aims to transfer knowledge by aligning the sentence embeddings generated by the encoder model with the _sBert_ sentence vectors. To achieve this, the combination of Euclidean and cosine distance metrics are utilized as a loss function. The weights for each distance function are shared equally. The _distance and perplexity minimization_ experiment demonstrates the effectiveness of incorporating context information to enhance the transfer of information between _sBert_ sentence vectors and _FastText_ word embeddings. The key difference from the previous method is the usage of a decoder substructure.  
For both the _distance minimization_ and _distance and perplexity minimization_ methods, architectures consisting of a single layer with the following configurations were constructed: The embedding layers utilize 300-dimensional vectors sourced directly from _FastText_ word embeddings. Each model employs 4 attention heads and generates 300-dimensional vectors for sentence vector comparison, analogous to _sBert_ representations, which are also projected into a 300-dimensional space. During the training phase, extensive hyperparameter optimization was conducted, revealing that a learning rate of 3 \(\times\) 10\({}^{-4}\) was optimally consistent across all models. This learning rate, in conjunction with the use of the Adam optimizer and a batch size of 128, resulted in the most effective training outcomes. All experiments are performed using an Nvidia RTX A4500 GPU.  
In order to make a fair comparison, we randomly selected 300 thousand sentences from the mentioned WMT corpora. The dataset was divided into a training set comprising 285 thousand sentences and a test set containing 15 thousand sentences.  
The validation results of each proposed method are given in Table 1. It is clear that the _distance and perplexity_ minimization technique converges to a better cosine result.  
Both proposed methods can generate sentence vectors in two ways: Either by averaging the updated _FastText_ word embedding (bag-of-words approach), or by generating through the model. Although, the updated _FastText_ word embeddings will be evaluated through word analogy test sets as well as sentEval evaluation sets. The model outputs will be evaluated only on sentence evaluation tasks.