## 5 Results

Table 2: Results for VUA-SEQ (Sequential) and VUA-18 (variant of VUA-SEQ from [PERSON] et al. ; cf. Section 3.2 for details). \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Model** & **Precision** & **Recall** & **F1** \\ \hline \multirow{6}{*}{VUA-VERB} & MDGI-J-S & 78.8\% & 71.5\% & 75.0\% \\  & MDGI-J & 78.9\% & 70.9\% & 74.7\% \\  & RNN\_HG & 69.3\% & 72.3\% & 70.8\% \\  & RNN\_MHCA & 66.3\% & 75.2\% & 70.5\% \\  & RoB\_BASE & 76.9\% & 72.8\% & 74.7\% \\  & RoB\_SEQ & 79.2\% & 69.8\% & 74.2\% \\  & DeepMet & **79.5\%** & 70.8\% & 74.9\% \\  & MelBERT & 78.7\% & 72.9\% & **75.7\%** \\ \hline \multirow{6}{*}{VUA-VERB} & MsW\_base & 62.1\% & 77.1\% & 68.8\% \\  & MsW\_cos & 60.9\% & **77.7\%** & 68.3\% \\ \cline{1-1}  & MsW\_sbert & 67.2\% & 77.7\% & 72.0\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results for VUA-VERB. Although, as mentioned before, in the tables we quote the results published by [PERSON] et al. in [10], at the same time we would like to share the results of MelBERT's reruns for both VUA-20 and VUA-18 that we have conducted ourselves using the same set of random seeds as well as the same GPU. These are as follows: VUA-20 (Precision: 75.8%, Recall: 69.6%, F1: 72.6%); VUA-18 (Precision: 79.9%, Recall: 77.3%, F1: 78.6%). Additionally, we have compared these F1-scores with the F1-scores achieved by MsW_base and MsW_cos, using two-tailed \(t\)-test. The value achieved was \(p>0.05\), meaning that the differences are not statistically significant. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Genre** & **Model** & **Precision** & **Recall** & **F1** \\ \hline \multirow{8}{*}{Acad} & RNN\_HG & 76.5\% & 83.0\% & 79.6\% \\  & RNN\_MHCA & 79.6\% & 80.0\% & 79.8\% \\  & RoB\_BASE & 88.1\% & 79.5\% & 83.6\% \\  & RoB\_SEQ & 86.0\% & 77.3\% & 81.4\% \\  & DeepMet & **88.4\%** & 74.7\% & 81.0\% \\  & MelBERT & 85.3\% & 82.5\% & 83.9\% \\ \cline{2-5}  & MsW\_base & 85.3\% & 82.4\% & 83.8\% \\  & MsW\_cos & 84.4\% & **83.5\%** & **84.0\%** \\  & MsW\_sbert & 85.6\% & 81.0\% & 83.2\% \\ \hline \multirow{8}{*}{Conv} & RNN\_HG & 63.6\% & 72.5\% & 67.8\% \\  & RNN\_MHCA & 64.0\% & 71.1\% & 67.4\% \\  & RoB\_BASE & 70.3\% & 69.0\% & 69.6\% \\  & RoB\_SEQ & 70.5\% & 69.8\% & 70.1\% \\  & DeepMet & **71.6\%** & 71.1\% & **71.4\%** \\  & MelBERT & 70.1\% & 71.7\% & 70.9\% \\ \cline{2-5}  & MsW\_base & 69.2\% & **73.8\%** & 71.4\% \\  & MsW\_cos & 70.2\% & 72.7\% & **71.4\%** \\  & MsW\_sbert & 68.0\% & 73.5\% & 70.6\% \\ \hline \multirow{8}{*}{Fict} & RNN\_HG & 61.8\% & 74.5\% & 67.5\% \\  & RNN\_MHCA & 64.8\% & 70.9\% & 67.7\% \\  & RoB\_BASE & 74.3\% & 72.1\% & 73.2\% \\  & RoB\_SEQ & 73.9\% & 72.7\% & 73.3\% \\  & DeepMet & **76.1\%** & 70.1\% & 73.0\% \\  & MelBERT & 74.0\% & 76.8\% & 75.4\% \\ \cline{2-5}  & MsW\_base & 73.6\% & **77.6\%** & **75.5\%** \\  & MsW\_cos & 73.3\% & 77.1\% & 75.2\% \\  & MsW\_sbert & 72.9\% & 77.3\% & 75.0\% \\ \hline \multirow{8}{*}{News} & RNN\_HG & 71.6\% & **76.8\%** & 74.1\% \\  & RNN\_MHCA & 74.8\% & 75.3\% & 75.0\% \\  & RoB\_BASE & 83.5\% & 71.8\% & 77.2\% \\  & RoB\_SEQ & 82.2\% & 74.1\% & 77.9\% \\  & DeepMet & **84.1\%** & 67.6\% & 75.0\% \\  & MelBERT & 81.0\% & 73.7\% & 77.2\% \\ \cline{2-5}  & MsW\_base & 82.2\% & 76.1\% & **79.0\%** \\  & MsW\_cos & 81.5\% & 76.0\% & 78.7\% \\  & MsW\_sbert & 81.6\% & 74.3\% & 77.8\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results for Genres. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Model** & **Precision** & **Recall** & **F1** \\ \hline \multirow{6}{*}{MOH-X} & DeepMet & 79.9\% & 76.5\% & 77.9\% \\  & MelBERT & 79.3\% & 79.7\% & 79.2\% \\ \cline{2-5}  & MsW\_base & 82.3\% & **80.0**\% & **80.8**\% \\ \cline{2-5}  & MsW\_cos & 81.0\% & 80.0\% & 80.2\% \\ \cline{2-5}  & MsW\_sbert & **83.3**\% & 77.7\% & 80.1\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results for MOH-X ([PERSON] et al. [2016] dataset).