# Transferring Sentiment Cross-Lingually within and across Same-Family Languages
## 8 Results and Discussion  
### Results  
We conducted experiments on each of the datasets described in Section 5 using the methodology described in Section 6. We reported on the accuracy and macro F1 evaluation of five-class and three-class classifications. To verify the performance of one model over the other, we performed statistical testing using the almost stochastic order significance test [118,119] implemented by [PERSON] et al. [120] and the random approximation test [121]. We ran the two tests for each model, as well as the corresponding five- and three-class metrics (F1) scores. We only have three-class scores for Czech, Russian, and Russian (Latin) because the datasets use three-label tagging schemes. Table 5 displays the results for the best-performing language pairs that are statistically significant. Each combination that outperformed the others on any of the four metrics has been listed. Bulgarian + English performed best for the five-class metric, while Czech + Bulgarian performed best for the three-class metric. Bulgarian + English and Bulgarian + Czech have no significance over each other for 3-class F1 but are statistically significant over the other combinations, like Bulgarian + Croatian. We did not find any decently performing non-Bulgarian combinations in the top 10 list. In a five-class setting, the combination of Croatian + English produced higher scores. The combination of Croatian + Czech, Croatian + Polish, and Croatian + Bulgarian data proved advantageous in a three-class setting. One observation was made that the combination of Croatian + Bulgarian performed statistically similar to Croatian + Bulgarian (Latin) data. The baseline in Czech performs statistically better as compared to all the other cases. Combining Czech with other languages does not help Czech, which is a language with a lot of training instances. Similar to Czech, adding other languages to English does not help in a 5-class setting, but we see a significant improvement in the 3-class F1 with the Czech 3-class data combination. For the 5-star classification, we see no significant improvement for the combinations of Bulgarian (Latin) + Polish and Russian (Latin) + Polish. For the 3-class, the Polish baseline performs better, and it too belongs to a larger data instance; adding more training instances does not help. The combination of other language data with Polish during training leads to a drastic drop in performance. When Russian is combined with either Croatian or Bulgarian, the results are about the same in both metrics, and statistically, this combination does better than others. The combination of Slovak + English performs better for 5-class F1, while Croatian + Slovak + Slovene works best for 3-class F1. For Slovene, a combination with a high-resource language provided better performance compared to the baseline. The best results were obtained when Russian data were converted to the Latin script and combined with English data. Russian (Latin) + English, Russian (Latin) + Polish, and Russian (Latin) + Czech perform similarly but significantly better statistically than other combinations. However, the Latin version of the Bulgarian dataset did not outperform its Cyrillic counterpart. Bulgarian (Latin) + English was the highest scoring combination for the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Language** & **Acc-3** & **F1-3** \\ \hline Bulgarian & 67.80 (0.0076) & 69.42 (0.0046) \\ Croatian & 62.37 (0.004) & 57.47 (0.0053) \\ Czech & 83.82 (0.0037) & 83.76 * (0.0033) \\ English & 68.15 (0.0076) & 67.85 (0.0100) \\ Polish & 87.70 (0.0033) & 87.57 * (0.0039) \\ Russian & 71.43 (0.0013) & 70.20 (0.0030) \\ Slovak & 81.60 (0.0057) & 79.75 (0.0017) \\ Slovene & 59.13 (0.0180) & 59.97 (0.0307) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Baseline three-class classification scores are averaged over five-fold runs. The standard deviation is presented in the brackets to the right.