go-inoue commited on
Commit
ed18037
1 Parent(s): c45ebff

Update experimental results

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -126,10 +126,10 @@ We follow the original English BERT model's hyperparameters for pre-training, un
126
 
127
  | Task | Dataset | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
128
  | -------------------- | --------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
129
- | NER | ANERcorp | MSA | 80.2% | 66.2% | 74.2% | 82.4% | 82.3% | 82.0% | 82.3% | 80.5% |
130
- | POS | PATB (MSA) | MSA | 97.3% | 96.6% | 96.5% | 97.4% | 97.4% | 97.4% | 97.4% | 97.4% |
131
- | | ARZTB (EGY) | DA | 90.1% | 88.6% | 89.4% | 90.8% | 90.3% | 90.5% | 90.5% | 90.4% |
132
- | | Gumar (GLF) | DA | 97.3% | 96.5% | 97.0% | 97.1% | 97.0% | 97.0% | 97.1% | 97.0% |
133
  | SA | ASTD | MSA | 76.3% | 69.4% | 74.6% | 76.9% | 76.0% | 76.8% | 76.7% | 75.3% |
134
  | | ArSAS | MSA | 92.7% | 89.4% | 91.8% | 93.0% | 92.6% | 92.5% | 92.5% | 92.3% |
135
  | | SemEval | MSA | 69.0% | 58.5% | 68.4% | 72.1% | 70.7% | 72.8% | 71.6% | 71.2% |
@@ -142,10 +142,10 @@ We follow the original English BERT model's hyperparameters for pre-training, un
142
  ### Results (Average)
143
  | | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
144
  | -------------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
145
- | Variant-wise-average<sup>[[1]](#footnote-1)</sup> | MSA | 81.9% | 75.3% | 79.9% | 83.2% | 82.9% | 83.1% | 83.0% | 82.1% |
146
- | | DA | 73.5% | 71.1% | 72.1% | 73.5% | 73.1% | 73.4% | 73.3% | 73.1% |
147
  | | CA | 79.8% | 80.9% | 79.6% | 79.7% | 79.9% | 80.0% | 79.7% | 79.8% |
148
- | Macro-Average | ALL | 78.2% | 74.0% | 76.6% | 78.9% | 78.6% | 78.8% | 78.7% | 78.2% |
149
 
150
  <a name="footnote-1">[1]</a>: Variant-wise-average refers to average over a group of tasks in the same language variant.
151
 
126
 
127
  | Task | Dataset | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
128
  | -------------------- | --------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
129
+ | NER | ANERcorp | MSA | 80.8% | 67.9% | 74.1% | 82.4% | 82.0% | 82.1% | 82.6% | 80.8% |
130
+ | POS | PATB (MSA) | MSA | 98.1% | 97.8% | 97.7% | 98.3% | 98.2% | 98.3% | 98.2% | 98.2% |
131
+ | | ARZTB (EGY) | DA | 93.6% | 92.3% | 92.7% | 93.6% | 93.6% | 93.7% | 93.6% | 93.6% |
132
+ | | Gumar (GLF) | DA | 97.3% | 97.7% | 97.9% | 97.9% | 97.9% | 97.9% | 97.9% | 97.9% |
133
  | SA | ASTD | MSA | 76.3% | 69.4% | 74.6% | 76.9% | 76.0% | 76.8% | 76.7% | 75.3% |
134
  | | ArSAS | MSA | 92.7% | 89.4% | 91.8% | 93.0% | 92.6% | 92.5% | 92.5% | 92.3% |
135
  | | SemEval | MSA | 69.0% | 58.5% | 68.4% | 72.1% | 70.7% | 72.8% | 71.6% | 71.2% |
142
  ### Results (Average)
143
  | | Variant | Mix | CA | DA | MSA | MSA-1/2 | MSA-1/4 | MSA-1/8 | MSA-1/16 |
144
  | -------------------- | ------- | ----- | ----- | ----- | ----- | ------- | ------- | ------- | -------- |
145
+ | Variant-wise-average<sup>[[1]](#footnote-1)</sup> | MSA | 82.1% | 75.7% | 80.1% | 83.4% | 83.0% | 83.3% | 83.2% | 82.3% |
146
+ | | DA | 74.4% | 72.1% | 72.9% | 74.2% | 74.0% | 74.3% | 74.1% | 73.9% |
147
  | | CA | 79.8% | 80.9% | 79.6% | 79.7% | 79.9% | 80.0% | 79.7% | 79.8% |
148
+ | Macro-Average | ALL | 78.7% | 74.7% | 77.1% | 79.2% | 79.0% | 79.2% | 79.1% | 78.6% |
149
 
150
  <a name="footnote-1">[1]</a>: Variant-wise-average refers to average over a group of tasks in the same language variant.
151