# Parallel Bidirectionally Pretrained Taggers as Feature Generators
## 3 Results  
The evaluation results (in the form of weighted \(f_{1}\)-scores) for each standalone tagger were averaged out for each outer cross-validation fold and are presented in Table 3. The upper section of the table shows the results grouped by tagger and fold, while the lower section of the table shows the average, minimum and maximum \(f_{1}\)-scores, as well as the performance stability score (calculated as the ratio of the minimum and maximum scores) for each tagger. Figure 4: Outer and inner loops of nested cross-validation. In the outer loop, 20% of the annotated sentences were used as the test set. In the inner loop, the remaining 80% were split in a 4:1 ratio, resulting in 64% for the training set and 16% for the tuning set for each run. For each of the five folds, we also calculated a total average score across all standalone taggers and extracted the best score (among the standalone taggers) in each fold to be used as a baseline for testing the relative performance levels of the four composite methods (_voting_, _bidding_, _scoring_ and _stacking_). The results of these tests are presented in Table 4 as a comparison of the baseline and unidirectionally built composite methods and in Table 5 as a comparison of the baseline and bidirectionally built composite methods. The lower sections of these two tables also present the same compiled metrics mentioned in the description provided for Table 3. \begin{table}
\begin{tabular}{c|c c|c c c c} \hline \hline \multirow{2}{*}{**Fold**} & \multicolumn{2}{c}{**Standalone**} & \multicolumn{2}{c}{**Standalone**} & \multirow{2}{*}{**Voting**} & \multirow{2}{*}{**Bidding**} & \multirow{2}{*}{**Scoring**} & \multirow{2}{*}{**Stacking**} \\  & **Average** & **Best (Baseline)** & & & & \\ \hline \#1 & 0.9458 & 0.9519 & 0.9584 & 0.9608 & **0.9718** & **0.9718** \\ \#2 & 0.9252 & 0.9538 & 0.9495 & 0.9589 & 0.9677 & **0.9700** \\ \#3 & 0.9280 & 0.9550 & 0.9490 & 0.9597 & 0.9668 & **0.9715** \\ \#4 & 0.9459 & 0.9556 & 0.9503 & 0.9597 & 0.9715 & **0.9721** \\ \#5 & 0.9245 & 0.9547 & 0.9612 & 0.9611 & 0.9677 & **0.9715** \\ \hline average & 0.9339 & 0.9546 & 0.9537 & 0.9600 & 0.9691 & **0.9714** \\ minimum & 0.9245 & 0.9538 & 0.9490 & 0.9589 & 0.9668 & **0.9700** \\ maximum & 0.9459 & 0.9556 & 0.9612 & 0.9611 & 0.9718 & **0.9721** \\ stability & 0.9734 & 0.9964 & 0.9881 & 0.9955 & 0.9979 & **0.9989** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of the derived baseline (left) and averaged weighted \(f_{1}\)-scores for each of the outer five folds and each of the four tested composition methods built on the unidirectionally trained standalone taggers (right). The best result for each fold and each additional metric is shown in bold. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Fold** & **spaCy** & **spaCy-I** & **TreeTagger** & **TreeTagger-I** & **Stanza** & **Stanza-I** \\ \hline \#1 & 0.9465 & 0.9483 & 0.9519 & 0.9539 & 0.9389 & 0.9275 \\ \#2 & 0.9457 & 0.9467 & 0.9538 & 0.9522 & 0.8760 & 0.8911 \\ \#3 & 0.9470 & 0.9428 & 0.9550 & 0.9532 & 0.8819 & 0.9254 \\ \#4 & 0.9436 & 0.9467 & 0.9556 & 0.9540 & 0.9386 & 0.8839 \\ \#5 & 0.9448 & 0.9458 & 0.9547 & 0.9535 & 0.8741 & 0.8431 \\ \hline average & 0.9455 & 0.9461 & 0.9542 & 0.9534 & 0.9019 & 0.8942 \\ minimum & 0.9436 & 0.9428 & 0.9518 & 0.9522 & 0.8741 & 0.8431 \\ maximum & 0.9470 & 0.9483 & 0.9556 & 0.9540 & 0.9389 & 0.9275 \\ stability & 0.9964 & 0.9942 & 0.9961 & 0.9981 & 0.9310 & 0.9090 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of the averaged weighted \(f_{1}\)-scores for each of the outer five folds and each of the six tested standalone taggers (top), and a comparison of their average, minimal, and maximal scores as well as their stability, calculated as the ratio of minimum to maximum scores (bottom).