File size: 3,167 Bytes
da6967f d63f342 da6967f 592f960 da6967f a40f97a da6967f d63f342 da6967f 3c755b0 da6967f d63f342 da6967f d63f342 3c755b0 d63f342 3c755b0 d63f342 da6967f d63f342 da6967f d63f342 3c755b0 d63f342 3c755b0 d63f342 da6967f 3c755b0 da6967f d63f342 da6967f d63f342 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
---
license: mit
language: uk
---
# roberta-large-wechsel-ukrainian
[`roberta-base`](https://huggingface.co/roberta-base) transferred to Ukrainian using the method from the NAACL2022 paper [WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models](https://aclanthology.org/2022.naacl-main.293/).
# Evaluation
Evaluation was done on [lang-uk's ner-uk project](https://github.com/lang-uk/ner-uk), the Ukrainian portion of [WikiANN](https://huggingface.co/datasets/wikiann) and the [Ukrainian IU corpus from the Universal Dependencies project](https://github.com/UniversalDependencies/UD_Ukrainian-IU). Evaluation results are the mean of 5 runs with different seeds.
__Validation Results__
| | lang-uk NER (Micro F1) | WikiANN (Micro F1) | UD Ukrainian IU POS (Accuracy) |
|:-------------------------------------------------|:-------------------------|:-------------|:-------------------------|
| roberta-base-wechsel-ukrainian | 88.06 (0.50) | 92.96 (0.08) | 98.70 (0.05) |
| roberta-large-wechsel-ukrainian | __89.27 (0.53)__ | __93.22 (0.15)__ | __98.86 (0.03)__ |
|
| roberta-base-scratch-ukrainian* | 85.49 (0.88) | 91.91 (0.08) | 98.49 (0.04) |
| roberta-large-scratch-ukrainian* | 86.54 (0.70) | 92.39 (0.16) | 98.65 (0.09) |
|
| dbmdz/electra-base-ukrainian-cased-discriminator | 87.49 (0.52) | 93.20 (0.16) | 98.60 (0.03) |
| xlm-roberta-base | 86.68 (0.44) | 92.41 (0.13) | 98.53 (0.02) |
| xlm-roberta-large | 86.64 (1.61) | 93.01 (0.13) | 98.71 (0.04) |
__Test Results__
| | lang-uk NER (Micro F1) | WikiANN (Micro F1) | UD Ukrainian IU POS (Accuracy) |
|:-------------------------------------------------|:-------------------------|:-------------|:-------------------------|
| roberta-base-wechsel-ukrainian | 90.81 (1.51) | 92.98 (0.12) | 98.57 (0.03) |
| roberta-large-wechsel-ukrainian | __91.24 (1.16)__ | __93.22 (0.17)__ | __98.74 (0.06)__ |
|
| roberta-base-scratch-ukrainian* | 89.57 (1.01) | 92.05 (0.09) | 98.31 (0.08) |
| roberta-large-scratch-ukrainian* | 89.96 (0.89) | 92.49 (0.15) | 98.52 (0.04) |
|
| dbmdz/electra-base-ukrainian-cased-discriminator | 90.43 (1.29) | 92.99 (0.11) | 98.59 (0.06) |
| xlm-roberta-base | 90.86 (0.81) | 92.27 (0.09) | 98.45 (0.07) |
| xlm-roberta-large | 90.16 (2.98) | 92.92 (0.19) | 98.71 (0.04) |
\*trained using the same exact training setup as the wechsel-\* models, but without parameter transfer from WECHSEL.
# License
MIT |