Commit
·
7c6b3b6
1
Parent(s):
514ef0d
Update README.md
Browse files
README.md
CHANGED
@@ -61,10 +61,10 @@ training_args = TrainingArguments(
|
|
61 |
### Eval results
|
62 |
The model was evaluated using the binary test sets for MultiNLI, ANLI, LingNLI and the binary dev set for Fever-NLI (two classes instead of three). The metric used is accuracy.
|
63 |
|
64 |
-
mnli-m-2c | mnli-mm-2c | fever-nli-2c | anli-all-2c | anli-r3-2c | lingnli-2c
|
65 |
-
|
66 |
-
|
67 |
-
|
68 |
|
69 |
## Limitations and bias
|
70 |
Please consult the original DeBERTa paper and literature on different NLI datasets for potential biases.
|
|
|
61 |
### Eval results
|
62 |
The model was evaluated using the binary test sets for MultiNLI, ANLI, LingNLI and the binary dev set for Fever-NLI (two classes instead of three). The metric used is accuracy.
|
63 |
|
64 |
+
dataset | mnli-m-2c | mnli-mm-2c | fever-nli-2c | anli-all-2c | anli-r3-2c | lingnli-2c
|
65 |
+
--------|---------|----------|---------|----------|----------|------
|
66 |
+
accuracy | 0.925 | 0.922 | 0.892 | 0.676 | 0.665 | 0.888
|
67 |
+
speed (text/sec, GPU, 128 batch) | 6.0 | 6.3 | 3.0 | 5.8 | 5.0 | 7.6
|
68 |
|
69 |
## Limitations and bias
|
70 |
Please consult the original DeBERTa paper and literature on different NLI datasets for potential biases.
|