# Towards Sentiment Analysis for Romanian Twitter Content
## 4 Experiments and Results
### Discussion and Further Work  
Experiments presented in Section 4.1 show that a standard method such as Bernoulli Naive Bayes employed on the classical TD-IDF encoding supplies results that fit our media surveillance needs. The performance of the Bernoulli NB classifier is slightly better than the ones of other classifiers, being in a narrow margin below a BERT-based classifier. Applying evolutionary optimization for the hyperparameter search allows us to improve the performance of all classifiers by 1% to 3%. Bernoulli NB has the advantage of very fast inference times for novel instances, being also easily retrainable, to accommodate for the volatility of the discussed topics. In contrast, although the BERT-based classifier indeed produces the state-of-the-art results in terms for both accuracy and weighted F1-measure, its needs in terms of hardware resources and computational time make it infeasible for our practical needs. With the final experiments presented in Section 4.2, we demonstrate that the selected classification model is suitable for our project production environment with general discussion topics, with a performance superior or at least equivalent to the already existing classifiers for the Romanian language although: (i) we learned a limited language model \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Encoding** & **Classifier** & **Accuracy** & **Weighted F1-Measure** \\ \hline TF-IDF & Bernoulli NB & 0.6500 & 0.6311 \\ \hline Word2 Vec & LSTM & 0.5833 & 0.5518 \\ \hline Sentiment.ro & & 0.4750 & 0.4699 \\ \hline Multilingual BERT & & 0.6583 & 0.6338 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of various models (accuracy and weighted F1-measure) on a Romanian dataset with aviation industry-specific tweets. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Encoding** & **Classifier** & **Accuracy** & **Weighted F1-Measure** \\ \hline TF-IDF & Bernoulli NB & 0.4833 & 0.4942 \\ \hline Word2 Vec & LSTM & 0.4583 & 0.4429 \\ \hline Sentimetric.ro & & 0.4917 & 0.4730 \\ \hline Multilingual BERT & & 0.5583 & 0.5417 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance of various models (accuracy and weighted F1-measure) on a Romanian dataset with general tweets. from a very specific dataset of only about 15,000 tweets; (ii) Romanian knowledge was produced with a public automatic translation service; and (iii) model learning was performed on domain-specific knowledge--the airline industry. Therefore, we confirm that using datasets for Romanian NLP tasks constructed by automatic translation from English could be a solution, especially for SA. If no extensive language model exists, and if computing time and hardware resources represent a barrier, we recommend the use of the very classical TF-IDF encoding with a simple classifier such as the Bernoulli NB. In our case, classical ML methods proved to be more robust to generalization than DL-based methods. This might be due to the fact that the classic models used the TF-IDF features which do not take into account semantic and syntactic structures, thus being less affected by the automatic translation process when compared to Word2 Vec. If one would opt for a wide-scope hyperparameter search, we suggest to employ evolutionary optimization as being capable of fine tuning the classifiers in a reasonable time. For our future work, we would like to repeat the experiment presented in Section 4.2 with the following differences: using considerably more texts in order to be more representative, defining standardized labeling rules, employing the help of more human annotators, and excluding texts which proved to be hard to label even by humans. Having more texts which are better labeled by more human annotators should improve the accuracy of all the presented models. Additionally, the sentiment class distribution of the dataset could be balanced using various oversampling techniques. The models can be retrained on the balanced data following the same methodology presented in this work and their performance re-evaluated.