benjamin commited on
Commit
188fb14
1 Parent(s): aa68794

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -9,7 +9,7 @@ language: uk
9
 
10
  # Evaluation
11
 
12
- Evaluation was done on [lang-uk's ner-uk project](https://github.com/lang-uk/ner-uk), the Ukrainian portion of [WikiANN](https://huggingface.co/datasets/wikiann) and the [Ukrainian IU corpus from the Universal Dependencies project](https://github.com/UniversalDependencies/UD_Ukrainian-IU).
13
 
14
  __Validation Results__
15
 
@@ -17,8 +17,10 @@ __Validation Results__
17
  |:-------------------------------------------------|:-------------------------|:-------------|:-------------------------|
18
  | roberta-base-wechsel-ukrainian | 88.06 (0.50) | 92.96 (0.08) | 98.70 (0.05) |
19
  | roberta-large-wechsel-ukrainian | __89.27 (0.53)__ | __93.22 (0.15)__ | __98.86 (0.03)__ |
 
20
  | roberta-base-scratch-ukrainian* | 85.49 (0.88) | 91.91 (0.08) | 98.49 (0.04) |
21
  | roberta-large-scratch-ukrainian* | 86.54 (0.70) | 92.39 (0.16) | 98.65 (0.09) |
 
22
  | dbmdz/electra-base-ukrainian-cased-discriminator | 87.49 (0.52) | 93.20 (0.16) | 98.60 (0.03) |
23
  | xlm-roberta-base | 86.68 (0.44) | 92.41 (0.13) | 98.53 (0.02) |
24
  | xlm-roberta-large | 86.64 (1.61) | 93.01 (0.13) | 98.71 (0.04) |
@@ -29,13 +31,15 @@ __Test Results__
29
  |:-------------------------------------------------|:-------------------------|:-------------|:-------------------------|
30
  | roberta-base-wechsel-ukrainian | 90.81 (1.51) | 92.98 (0.12) | 98.57 (0.03) |
31
  | roberta-large-wechsel-ukrainian | __91.24 (1.16)__ | __93.22 (0.17)__ | __98.74 (0.06)__ |
 
32
  | roberta-base-scratch-ukrainian* | 89.57 (1.01) | 92.05 (0.09) | 98.31 (0.08) |
33
  | roberta-large-scratch-ukrainian* | 89.96 (0.89) | 92.49 (0.15) | 98.52 (0.04) |
 
34
  | dbmdz/electra-base-ukrainian-cased-discriminator | 90.43 (1.29) | 92.99 (0.11) | 98.59 (0.06) |
35
  | xlm-roberta-base | 90.86 (0.81) | 92.27 (0.09) | 98.45 (0.07) |
36
  | xlm-roberta-large | 90.16 (2.98) | 92.92 (0.19) | 98.71 (0.04) |
37
 
38
- \*trained using the same exact training setup as the wechsel-\* models, but without parameter transfer.
39
 
40
 
41
  # License
 
9
 
10
  # Evaluation
11
 
12
+ Evaluation was done on [lang-uk's ner-uk project](https://github.com/lang-uk/ner-uk), the Ukrainian portion of [WikiANN](https://huggingface.co/datasets/wikiann) and the [Ukrainian IU corpus from the Universal Dependencies project](https://github.com/UniversalDependencies/UD_Ukrainian-IU). Evaluation results are the mean of 5 runs with different seeds.
13
 
14
  __Validation Results__
15
 
 
17
  |:-------------------------------------------------|:-------------------------|:-------------|:-------------------------|
18
  | roberta-base-wechsel-ukrainian | 88.06 (0.50) | 92.96 (0.08) | 98.70 (0.05) |
19
  | roberta-large-wechsel-ukrainian | __89.27 (0.53)__ | __93.22 (0.15)__ | __98.86 (0.03)__ |
20
+ |
21
  | roberta-base-scratch-ukrainian* | 85.49 (0.88) | 91.91 (0.08) | 98.49 (0.04) |
22
  | roberta-large-scratch-ukrainian* | 86.54 (0.70) | 92.39 (0.16) | 98.65 (0.09) |
23
+ |
24
  | dbmdz/electra-base-ukrainian-cased-discriminator | 87.49 (0.52) | 93.20 (0.16) | 98.60 (0.03) |
25
  | xlm-roberta-base | 86.68 (0.44) | 92.41 (0.13) | 98.53 (0.02) |
26
  | xlm-roberta-large | 86.64 (1.61) | 93.01 (0.13) | 98.71 (0.04) |
 
31
  |:-------------------------------------------------|:-------------------------|:-------------|:-------------------------|
32
  | roberta-base-wechsel-ukrainian | 90.81 (1.51) | 92.98 (0.12) | 98.57 (0.03) |
33
  | roberta-large-wechsel-ukrainian | __91.24 (1.16)__ | __93.22 (0.17)__ | __98.74 (0.06)__ |
34
+ |
35
  | roberta-base-scratch-ukrainian* | 89.57 (1.01) | 92.05 (0.09) | 98.31 (0.08) |
36
  | roberta-large-scratch-ukrainian* | 89.96 (0.89) | 92.49 (0.15) | 98.52 (0.04) |
37
+ |
38
  | dbmdz/electra-base-ukrainian-cased-discriminator | 90.43 (1.29) | 92.99 (0.11) | 98.59 (0.06) |
39
  | xlm-roberta-base | 90.86 (0.81) | 92.27 (0.09) | 98.45 (0.07) |
40
  | xlm-roberta-large | 90.16 (2.98) | 92.92 (0.19) | 98.71 (0.04) |
41
 
42
+ \*trained using the same exact training setup as the wechsel-\* models, but without parameter transfer from WECHSEL.
43
 
44
 
45
  # License