# Sentiment Analysis on Algerian Dialect with Transformers
## 4 Classification of Algerian YouTube Comments Using Transformers
### Model Inference
In this task, we simply allow only one punctuation mark between two words. 8. Removing stop words: Arabic and some Algerian stop words were removed entirely in this task. 9. Removing redundant words: Some comments are basically spam, repeating the word dozens of times. This task only allows two redundant words as a maximum. 10. Balancing the dataset: As discussed in Section 3.2, 22.4% of the total labeled samples are neutral sentiments, 35.5% are negative, and 42.1% are positive; this task takes the lowest percentage to apply to all classes and discards the others so that we end up with a balanced dataset. 11. Normalizing special Arabic characters: Some Arabic characters need to be normalized, such as replacing!, and! with! Removing emojis reduced the F1-score a little bit. Generally, emojis help determine the sentiment of the sentence, and task 5 proves this, where we add spaces between emojis to help the tokenizer, improving the F1-score by 0.004. \begin{table}
\begin{tabular}{c l c c} \hline \hline
**Task ID** & **Preprocessing Task** & **F1-Score** & **Accuracy** \\ \hline / & No Preprocessing & 0.7764 & 0.8078 \\ \hline
**1** & **Removing elongation** & **0.7781** & **0.8086** \\
**2** & **Replacing URLs, phone numbers, and emails** & **0.7789** & **0.8094** \\
**3** & **Removing HTML** & **0.7812** & **0.8112** \\
**4** & Removing emojis & 0.7763 & 0.8074 \\
**5** & **Inserting spaces between emojis** & **0.7804** & **0.8107** \\
**6** & **Converting Latin characters to Arabic** & **0.7788** & **0.8076** \\
**7** & **Removing redundant punctuation** & **0.7786** & **0.8087** \\
**8** & **Removing stop words & 0.6440 & 0.6881 \\
9 & Removing redundant words & 0.7758 & 0.8072 \\
10 & Balancing the dataset & 0.7653 & 0.7680 \\
**11** & **Normalizing special Arabic characters** & **0.7776** & **0.8094** \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Evaluation metrics using various different preprocessing tasks. Removing stop words reduced the performance significantly. A loss of about 0.13 is seen in the F1-score, and that is totally expected, as stop words include negation words and other similar words that definitely help influence the sentiment. For instance, the word \"\(\cup\)\"(Not) is considered a stop word, so a sentence like \"\(\cup\)\"\(\cup\)\"\(\cup\)\"\(\cup\)\" (This is not beautiful at all) will be transformed into \"\(\cup\)\" (Beautiful/Nice) since all other words are stop words, and the meaning is totally reversed. Therefore, stop word removal surely does not work in the sentiment analysis task. Balancing the dataset also reduced the performance, and that is because a lot of samples were lost. As expected, accuracy is a more reliable metric in this case and is very close to the F1-score. After experimenting with different preprocessing tasks, only tasks 1, 2, 3, 5, 6, 7, and 11 (highlighted ones in Table 8) were candidates to use to obtain the best combination between them. Table 4 shows the highest-performing combinations. As highlighted in Table 9, the combination of preprocessing tasks 2, 3, 5, and 11 performed the best with scores of 0.7838 and 0.8174 for F1-score and accuracy, respectively. ## 6 Conclusions and Future Work

In conclusion, this research aimed to make a significant contribution to the field of sentiment analysis on the Algerian dialect. By creating a custom and relatively large dataset extracted from Algerian YouTube channels and manually annotated by the research team, we were able to train a state-of-the-art deep learning model for natural language processing called BERT. Our results showed that BERT achieved an F1-score of 78.3% and an accuracy of 81.7% on the testing set, demonstrating the effectiveness of our approach and the potential of using BERT for sentiment analysis on the Algerian dialect. The ability to infer sentiment from any Algerian text provides a valuable tool for understanding the opinions and emotions of the population.