go-inoue commited on
Commit
9be3527
1 Parent(s): 109e4b7

Update experimental results

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -140,10 +140,10 @@ We follow the original English BERT model's hyperparameters for pre-training, un
140
 
141
  | Task | Dataset | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
142
  | -------------------- | --------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
143
- | NER | ANERcorp | MSA | 80.2% | 66.2% | 74.2% | 82.4% | 82.3% | 82.0% | 82.3% | 80.5% |
144
- | POS | PATB (MSA) | MSA | 97.3% | 96.6% | 96.5% | 97.4% | 97.4% | 97.4% | 97.4% | 97.4% |
145
- | | ARZTB (EGY) | DA | 90.1% | 88.6% | 89.4% | 90.8% | 90.3% | 90.5% | 90.5% | 90.4% |
146
- | | Gumar (GLF) | DA | 97.3% | 96.5% | 97.0% | 97.1% | 97.0% | 97.0% | 97.1% | 97.0% |
147
  | SA | ASTD | MSA | 76.3% | 69.4% | 74.6% | 76.9% | 76.0% | 76.8% | 76.7% | 75.3% |
148
  | | ArSAS | MSA | 92.7% | 89.4% | 91.8% | 93.0% | 92.6% | 92.5% | 92.5% | 92.3% |
149
  | | SemEval | MSA | 69.0% | 58.5% | 68.4% | 72.1% | 70.7% | 72.8% | 71.6% | 71.2% |
@@ -156,10 +156,10 @@ We follow the original English BERT model's hyperparameters for pre-training, un
156
  ### Results (Average)
157
  | | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
158
  | -------------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
159
- | Variant-wise-average<sup>[[1]](#footnote-1)</sup> | MSA | 81.9% | 75.3% | 79.9% | 83.2% | 82.9% | 83.1% | 83.0% | 82.1% |
160
- | | DA | 73.5% | 71.1% | 72.1% | 73.5% | 73.1% | 73.4% | 73.3% | 73.1% |
161
  | | CA | 79.8% | 80.9% | 79.6% | 79.7% | 79.9% | 80.0% | 79.7% | 79.8% |
162
- | Macro-Average | ALL | 78.2% | 74.0% | 76.6% | 78.9% | 78.6% | 78.8% | 78.7% | 78.2% |
163
 
164
  <a name="footnote-1">[1]</a>: Variant-wise-average refers to average over a group of tasks in the same language variant.
165
 
 
140
 
141
  | Task | Dataset | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
142
  | -------------------- | --------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
143
+ | NER | ANERcorp | MSA | 80.8% | 67.9% | 74.1% | 82.4% | 82.0% | 82.1% | 82.6% | 80.8% |
144
+ | POS | PATB (MSA) | MSA | 98.1% | 97.8% | 97.7% | 98.3% | 98.2% | 98.3% | 98.2% | 98.2% |
145
+ | | ARZTB (EGY) | DA | 93.6% | 92.3% | 92.7% | 93.6% | 93.6% | 93.7% | 93.6% | 93.6% |
146
+ | | Gumar (GLF) | DA | 97.3% | 97.7% | 97.9% | 97.9% | 97.9% | 97.9% | 97.9% | 97.9% |
147
  | SA | ASTD | MSA | 76.3% | 69.4% | 74.6% | 76.9% | 76.0% | 76.8% | 76.7% | 75.3% |
148
  | | ArSAS | MSA | 92.7% | 89.4% | 91.8% | 93.0% | 92.6% | 92.5% | 92.5% | 92.3% |
149
  | | SemEval | MSA | 69.0% | 58.5% | 68.4% | 72.1% | 70.7% | 72.8% | 71.6% | 71.2% |
 
156
  ### Results (Average)
157
  | | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
158
  | -------------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
159
+ | Variant-wise-average<sup>[[1]](#footnote-1)</sup> | MSA | 82.1% | 75.7% | 80.1% | 83.4% | 83.0% | 83.3% | 83.2% | 82.3% |
160
+ | | DA | 74.4% | 72.1% | 72.9% | 74.2% | 74.0% | 74.3% | 74.1% | 73.9% |
161
  | | CA | 79.8% | 80.9% | 79.6% | 79.7% | 79.9% | 80.0% | 79.7% | 79.8% |
162
+ | Macro-Average | ALL | 78.7% | 74.7% | 77.1% | 79.2% | 79.0% | 79.2% | 79.1% | 78.6% |
163
 
164
  <a name="footnote-1">[1]</a>: Variant-wise-average refers to average over a group of tasks in the same language variant.
165