# Towards Reliable Baselines for Document-Level Sentiment Analysis in the Czech and Slovak Languages
## 2 Related Work
Slovak language, historically younger and restricted to a smaller population than Czech, exhibits a similar developmental pattern: [PERSON] and [PERSON] [66] created the first corpus of 1588 Slovak Facebook posts, relatively evenly assigned by human annotators into five classes, from strongly positive to strongly negative. The authors then used it to evaluate four most popular approaches to sentiment analysis (Naive Bayes, Maximum Entropy, Support Vector Machines, and Lexicon-based) in combination with various data pre-processing steps (with or without emotion normalization, diacritics reconstruction, lemmatization, and negation handling). Unlike [PERSON] et al. [50], however, they only used word unigrams as features, i.e., the bag of words representation of each Facebook post. Despite this limitation, they too observed that Maximum Entropy algorithm performed best, and that the envisaged data pre-processing steps were indeed important. (A somewhat puzzling exception was the damaging effect of their attempt at proper negation handling on all the tested approaches except the lexicon-based one.) One year later, [PERSON], [PERSON] and [PERSON] [67] used a real-life dataset consisting of 5318 Slovak reviews of various services assigned into seven categories, from most negative (\(-3\)) to most positive (\(3\)), in order to compare the accuracy of a deep neural network against a baseline SVM model. Here again, as in earlier such attempts in Czech, the results were ambiguous: although their best deep learning model (code-named M2) slightly outperformed the best SVM model in fine-grained sentiment classification into seven classes (by about 0.22 of a percentage point), it lagged by more than two percentage points behind SVM in classification into three classes (positive, negative, neutral). In their subsequent work [68], the same authors achieved further significant improvement of more than six percentage points in accuracy on the same dataset by a combination of deep contextualized word representations based on pre-trained version of Embeddings from Language Models (ELMo) for Slovak language with Bi-LSTM (Bi-Directional Long Short-Term Memory) and attention mechanism. Recently, the SlovakBERT model with RoBERTa architecture [45] heralded the arrival of the new state of the art in Slovak, demonstrating further modest improvements even over the model based on Bi-LSTM with ELMo and attention mechanism. To be able to objectively judge the extent to which these new techniques outstrip the past ones, we need to revisit the earlier classical machine learning experiments representing the historical \"baseline\" and rectify some of their apparently erroneous claims and reportedresults. As part of this effort, we provide here the following main contributions to sentiment analysis in the Czech and Slovak languages:  
* We validate and correct the main baseline results for Czech reported by [PERSON] et al. [50] on their three datasets;
* We show that their online product review dataset from Mall.cz contains more than 18% of non-trivial duplicates and must therefore be de-duplicated before analysis;
* We demonstrate that, without deduplication, the macro F1-measure results for the Mall.cz dataset are inflated by more than 19 percentage points and thus completely unreliable;
* We establish that part-of-speech-related features have no damaging effect on machine learning algorithms, contrary to the claim made by [PERSON] et al.