## 3 Methodology
### Feature Extraction
#### 3.4.2 Word Embedding  
Word embedding is a widely used method in deep learning for generating vector representations of words. It is especially valued in sentiment analysis and classification tasks for its ability to capture both syntactic and semantic relationships between words [20]. The principle of word embedding is based on the idea that words with similar contexts will be represented by similar vectors in a vector space. For example, the words \"man\" and \"woman\" often appear in similar contexts and are represented by closely related vectors. Similarly, the words \"queen\" and \"king\" are also close to each other and share a semantic relationship with the words \"man\" and \"woman\". With this vector representation, logical operations can be performed between words, such as \"man + women \(-\) king = queen\". Among the most widely used feature extraction methods are Word2 Vec, GloVe, and FastText. In this paper, we are applying two of these techniques, Word2 Vec and FastText, to our datasets as shown in Table 4.  
Word2 Vec model is proposed by [PERSON] as a vector representation approach that can capture natural language's syntactic and semantic meaning. It reflects the relationships between words by grouping them into the same vector space [21]. Word2 Vec proposes two architectures for generating word embeddings: continuous bag of words (CBOW) (Figure 3) and skip-gram (Figure 4). Both are based on a three-layer neural network. Word2 Vec was utilized across the three datasets in our study. For the English dataset, we employed the standard pre-trained Word2 Vec model based on the Google News corpus, which consists of 300-dimensional English word vectors ([[https://code.google.com/archive/p/word2](https://code.google.com/archive/p/word2) vec/]([https://code.google.com/archive/p/word2](https://code.google.com/archive/p/word2) vec/), accessed on 13 November 2024). In the case of the French dataset, we used frWac2 Vec ([[https://fauconnier.github.io/](https://fauconnier.github.io/)]([https://fauconnier.github.io/](https://fauconnier.github.io/)), accessed on 13 November 2024) from the frWac corpus, constructed from websites with a.fr domain, featuring 200-dimensional word vectors. For the Arabic dataset, we extracted features using the ArWordVec model [22], which was developed from a substantial collection of tweets covering various subjects, resulting in 300-dimensional vectors. \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Models** & **English Word Embedding** & \multicolumn{2}{c}{**French Word Embedding**} & \multicolumn{2}{c}{**Arabic Word Embedding**} \\ \cline{2-7}
**Configuration** & **Word2 Vec** & **FastText** & **frWac2 Vec** & **FastText** & **ArWordVec** & **FastText** \\ \hline
**Vector size** & 300 & 300 & 200 & 300 & 300 & 300 \\
**Approaches** & CBOW & CBOW & CBOW & CBOW & CBOW & CBOW \\
**Window size** & 5 & 5 & Not Specified & 5 & 3 & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Configurations of word embedding models. Figure 4: The architecture of the skip-gram model. Figure 3: The architecture of the CBOW model. FastText embedding is a model developed by Facebook researchers in 2016, trained in multiple languages, including English, French, and Arabic. It extends the Word2 Vec model by considering subword information, allowing it to better capture the meanings of rare or misspelled words [23]. FastText is an advancement of Word2 Vec, capable of producing high-quality word vectors through its mechanism that breaks down each word into character n-gram. Unlike Word2 Vec, where each word is represented as a bag of words, FastText represents each word as a bag of n-gram characters. This approach allows for better capturing of suffixes, prefixes, and other linguistic and lexical forms. The word \"Hello\" with a trigram could be decomposed into: <He\",\"Hel\",\"ell\",\"llo\", \"lo\"> where <He and lo- represent, respectively, the beginning and end of the word. We applied the FastText model to the three datasets.