## 5 Experiments and Results
### The Influence of Corpus Characteristics on the ADR Detection Accuracy  
First of all, we conducted experiments on the latest version (RDRS-2800) of our corpus that contains 2800 texts, obtained by extension of the first version RDRS-1660 (containing 1660 texts) so as to assess the dependence of ADR detection accuracy on the number of \begin{table}
\begin{tabular}{c c c c} \hline
**Topology and Features** & **ADR** & **Medication** & **Disease** \\ \hline \multicolumn{4}{c}{**Model A—Influence of features**} \\ \hline ELMo + PoS & 26.2 \(\pm\) 3.0 & 72.9 \(\pm\) 0.6 & 46.6 \(\pm\) 0.9 \\ \hline ELMo + ton & 26.6 \(\pm\) 3.9 & 73.5 \(\pm\) 0.5 & 47.3 \(\pm\) 1.0 \\ \hline ELMo + Vidal & 26.8 \(\pm\) 1.0 & 73.2 \(\pm\) 1.1 & 45.8 \(\pm\) 1.2 \\ \hline ELMo + MESHRUS & 27.4 \(\pm\) 2.2 & 73.3 \(\pm\) 1.5 & 46.5 \(\pm\) 1.2 \\ \hline ELMo + MESHRUS-2 & 27.4 \(\pm\) 0.9 & 73.1 \(\pm\) 0.4 & 46.7 \(\pm\) 1.4 \\ \hline \multicolumn{4}{c}{**Model A—Topology modifications**} \\ \hline ELMo with 3-layer LSTM & 28.2 \(\pm\) 5.1 & 74.7 \(\pm\) 0.7 & 51.5 \(\pm\) 1.8 \\ \hline ELMo with CRF & 28.8 \(\pm\) 2.7 & 73.2 \(\pm\) 1.1 & 46.9 \(\pm\) 0.4 \\ \hline \multicolumn{4}{c}{**Model A—Best combination**} \\ \hline ELMo with 3-layer LSTM and CRF + ton, PoS, MESHRUS, MESHRUS-2, Vidal & 32.4 \(\pm\) 4.7 & 74.6 \(\pm\) 1.1 & 52.3 \(\pm\) 1.4 \\ \hline \multicolumn{4}{c}{**XLM-RoBERTa**} \\ \hline XLM-RoBERTa-large & 40.1 \(\pm\) 2.9 & 79.6 \(\pm\) 1.3 & 56.9 \(\pm\) 0.8 \\ \hline \end{tabular}
\end{table} 
Table 11: The accuracy (by the F1-exact metric) of recognizing entities of different types in the first version of our corpus (RDRS-1600) using models with different features and topology. ADR mentions. Such direct expansion of the corpus (see RDRS-1600 and RDRS-2800 in Table 12) results in an increase in the ADR identification precision by 13% for ADR, 6% for disease, and 4% for medication. Figure 11 presents the results of training Model B on different fractions of the training set of RDRS-2800, and shows that the ADR detection accuracy stops growing when the training set reaches 80% of its size. Similar behavior is observed for the accuracy of recognizing other entity types (see Table 13). Figure 11: Dependence of the ADR recognition accuracy (by the F1-exact metric) on the size of the training set for different tags in RDRS-2800. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Corpus** & **RDRS-2800** & **RDRS-1600** & **RDRS-1250** & **RDRS-610** & **RDRS-1136** & **RDRS-500** \\ \hline Number of reviews & 2800 & 1659 & 1250 & 610 & 1136 & 500 \\ \hline Number of reviews containing ADR & 625 & 339 & 610 & 610 & 610 & 177 \\ \hline Percentage of reviews containing ADR & 0.22 & 0.2 & 0.49 & 1 & 0.54 & 0.35 \\ \hline Number of ADR entities & 1778 & 843 & 1752 & 1750 & 1750 & 709 \\ \hline Average number of ADR per review & 0.64 & 0.51 & 1.4 & 2.87 & 1.54 & 1.42 \\ \hline Number of reviews containing Indication & 1783 & 955 & 670 & 59 & 154 & 297 \\ \hline Total number of entities & 52,186 & 27,987 & 21,807 & 3782 & 6126 & 9495 \\ \hline Number of Indication entities & 4627 & 2310 & 1518 & 90 & 237 & 720 \\ \hline Ratio of ADR to Indication entities & 0.38 & 0.36 & 1.15 & 19.44 & 7.38 & 0.98 \\ \hline F1-exact of ADR detection & \(52.8\pm 3.8\) & \(40.1\pm 2.9\) & \(61.1\pm 1.5\) & \(71.3\pm 3.4\) & \(68.6\pm 3.3\) & \(61.6\pm 2.9\) \\ \hline Saturation \((\cdot 10^{3})\) & 4.25 & 3.41 & 9.77 & 72.57 & 42.99 & 9.08 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Subsets of the RDRS corpus with respect to the number and proportion of ADR mentions, and their ADR detection accuracy. Note also that direct expansion from 1600 to 2800 mentions gives only a small increase in the average number of ADR mentions per review (0.22 versus 0.2).