# Comparing the Quality and Speed of Sentence Classification with Modern Language Models
## 4 Results and Discussion
### Results of the Main Study
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{2}{c}{**MCC Score**} & \multicolumn{2}{c}{**Training Time (TT)**} \\ \hline
**Training Regime** & **MPD** & **TREC6** & **MPD** & **TREC6** \\ \hline p5 lr.1 mlr.0001a.5 & 0.475 \(\pm\) 0.042 & 0.923 \(\pm\) 0.007 & 0.051 \(\pm\) 0.008 & 0.362 \(\pm\) 0.037 \\ p10 lr.1 mlr.0001a.5 & 0.477 \(\pm\) 0.054 & 0.923 \(\pm\) 0.01 & 0.079 \(\pm\) 0.018 & 0.56 \(\pm\) 0.087 \\ p15 lr.1 mlr.0001a.5 & 0.477 \(\pm\) 0.044 & 0.925 \(\pm\) 0.007 & 0.103 \(\pm\) 0.011 & 0.699 \(\pm\) 0.094 \\ p20 lr.1 mlr.0001a.5 & 0.472 \(\pm\) 0.029 & 0.928 \(\pm\) 0.005 & 0.145 \(\pm\) 0.026 & 0.761 \(\pm\) 0.057 \\ p25 lr.1 mlr.0001a.5 & 0.468 \(\pm\) 0.03 & 0.931 \(\pm\) 0.008 & 0.158 \(\pm\) 0.029 & 0.91 \(\pm\) 0.067 \\ p20 lr.1 mlr.001a.4 & 0.473 \(\pm\) 0.043 & 0.928 \(\pm\) 0.008 & 0.1 \(\pm\) 0.02 & 0.438 \(\pm\) 0.057 \\ p20 lr.1 mlr.01a.5 & 0.475 \(\pm\) 0.056 & 0.92 \(\pm\) 0.015 & 0.082 \(\pm\) 0.02 & 0.367 \(\pm\) 0.078 \\ p5 lr.2 mlr.008a.5 & 0.437 \(\pm\) 0.05 & 0.92 \(\pm\) 0.015 & 0.042 \(\pm\) 0.01 & 0.167 \(\pm\) 0.031 \\ p10 lr.05 mlr.001a.5 & 0.477 \(\pm\) 0.055 & 0.916 \(\pm\) 0.012 & 0.063 \(\pm\) 0.011 & 0.359 \(\pm\) 0.042 \\ p15 lr.1 mlr.001a.5 & 0.471 \(\pm\) 0.049 & 0.929 \(\pm\) 0.005 & 0.086 \(\pm\) 0.01 & 0.618 \(\pm\) 0.096 \\ p10 lr.3 mlr.01a.3 & 0.484 \(\pm\) 0.027 & 0.898 \(\pm\) 0.012 & 0.051 \(\pm\) 0.013 & 0.119 \(\pm\) 0.02 \\ p25 lr.15 mlr.001a.5 & 0.463 \(\pm\) 0.04 & 0.934 \(\pm\) 0.007 & 0.132 \(\pm\) 0.025 & 0.645 \(\pm\) 0.112 \\
**p20 lr.1 mlr.002a.5** & **0.487 \(\pm\) 0.046** & **0.933 \(\pm\) 0.005** & **0.098 \(\pm\) 0.014** & **0.269 \(\pm\) 0.02** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Impact of the learning-rate-related parameters on the framework performance for the TREC6 set and the MPD given a mini-batch size = 8, hidden size = 256, shuffle = True. Compared variants were named with use of abbreviations: p—patience, lr—initial learning rate, mlr—minimal learning rate, a—annealing. For example, p5 lr.1 mlr.0001a.5 stands for patience = 5, initial learning rate = 0.1, minimal learning rate = 0.0001, and annealing factor = 0.5. The final training regime selected for the final LM comparison is highlighted in bold. The results for the MPD allowed us to visualize different qualities of various LMs grouping sentences by classes. Figure 7 presents a visualization based on a principal component analysis, which  
Figure 5: Comparison of framework performance with selected language models on the TREC6 data set. Explanation of the abbreviated names of LMs: S—scalar mix, L—large model version. Example notation RoBERTa LS stands for RoBERTa large scalar mix model. Figure 6: Comparison of framework performance with selected language models on the MPD. Explanation of the abbreviated names of LMs: S—scalar mix, L—large model version. Example notation RoBERTa LS stands for RoBERTa large scalar mix model. indicates that the Glove-based framework was unable to clearly divide classes, whereas the RoBERTa large scalar mix LM was able to distinguish groups with significantly higher quality.