## 3 Methodology
### Methods
#### 3.2.2 Word Embeddings  
Word Embeddings transform text data into numerical representation, the so-called vectorization. According to [PERSON] [35], word embedding, also known as distributed representations of words, is the term used to represent the technique where individual words are represented into real-value vectors. These vectors often have a dimension number in the tens or even thousands scale. Each word is mapped to one vector, representing a sentence in a list of these vectors. The mapping of a word to a vector can be done through dictionaries. This is better than using sparse word representations on the scale of thousands or even millions of dimensions [36]. TfidfVectorizer (scikit-learn.org/stable/modules/generated/sklearn.feature_extraction. text.TfidfVectorizer, accessed on 27 September 2022) is a method in the scikit-learn framework that enables the conversion of raw documents into a matrix of TF-IDF features. TF-IDF is the combination of the term frequency (TF) metric that represents the number of times a term occurs in a document versus the total number of terms in a document. In contrast, the inverse document frequency (IDF) represents the number of documents that contain the term [37]. The setting tested for this word embedder that works best for this work was **ngram_range = (1,1)**, where the first value indicates the minimum amount of grams to take into consideration and the second value the maximum, in which, by making them (1,1), we are solely utilizing unigrams. Word2 Vec (github.com/tmikolov/word2 vec, accessed on 26 September 2022) is a popular word-embedding technique, developed by [PERSON] [38]. It provides two methods to achieve this task, either by using a continuous bag-of-words (CBOW) [39] or skip-gram model (SG) [38]. The CBOW method takes the context of each word as the input and tries to predict the word corresponding to the context, while the SG method inverts the CBOW method, using the target word as the input and trying to predict the context. According to the author, CBOW is faster and has better representations for more frequent words, while SG works well with a small amount of data and represents rare words well [40]. In addition, the original GitHub implementation, the Word2 Vec method, is in a commonly used framework called Gensim (radimrehurek.com/gensim/models/word2 vec, accessed on 26 September 2022). We used the **word2 vec-google-news-300** (huggingface.co/fse/word2 vec-google-news-300, accessed on 27 September 2022) pre-trained vectors with an activated binary mode max length equaling 200. Global vectors for word representation (GloVe) is the unsupervised learning algorithm for this task created by Stanford (nlp.stanford.edu/projects/glove/, accessed on 26 September 2022) [41]. It is available on GitHub (github.com/stanfordnlp/GloVe, accessed on 26 September 2022), where they supply pre-trained models for the task, depending on the requirements. Using the pre-trained models means the GloVe model becomes a static dictionary, since we obtain the word and its vector representation by downloading a pre-trained model [42]. In our work, we used **glove.6B.200** (nlp.stanford.edu/projects/glove/, accessed on 26 September 2022) with 6 B tokens, 400 K vocab, uncased, and 200 dimensions in which we maintained that dimensions preset (max length equaling 200). FastText is a library for efficiently learning word representation and sentence classification created by Meta Research (opensource.fb.com, accessed on 26 September 2022; github.com/facebookresearch, accessed on 26 September 2022) [43]. It is available on GitHub (github.com/facebookresearch/fastText, accessed on 26 September 2022), where they offer their state-of-the-art model for English word vectors and word vectors for 157 additional languages. It diverges from Word2 Vec by using subword information on word similarity tasks to improve its results.