ReaderBench
commited on
Commit
•
f0587eb
1
Parent(s):
fb77fbb
update readme
Browse files
README.md
CHANGED
@@ -52,7 +52,7 @@ print(tokenizer.decode(text[0]))
|
|
52 |
|
53 |
### Training Statistics
|
54 |
|
55 |
-
| Version | Number of parameters | Number of epoch | Duration of an epoch |
|
56 |
|:-------:|:--------------------:|:---------------:|:--------------------:|:----------:|:----------:|:---:|
|
57 |
| Base | 124M | 15 | 7h | 1024 | 72 | 22.96 |
|
58 |
| Medium | 354M | 10 | 22h | 1024 | 24 | 17.64 |
|
@@ -125,8 +125,8 @@ print(tokenizer.decode(text[0]))
|
|
125 |
|RoBERT-small | - | 30.84 | 45.17 |
|
126 |
|RoBERT-base | - | 53.52 | 70.04 |
|
127 |
|RoBERT-large | - | 55.46 | 69.64 |
|
128 |
-
|mBERT | - | 72.7 |
|
129 |
-
|XLM-R Large | - |**83.6
|
130 |
|RoGPT2-base | Greedy | 23.69 | 35.97 |
|
131 |
|RoGPT2-base | Beam-search-4 | 24.11 | 35.27 |
|
132 |
|RoGPT2-medium | Greedy | 29.66 | 44.74 |
|
|
|
52 |
|
53 |
### Training Statistics
|
54 |
|
55 |
+
| Version | Number of parameters | Number of epoch | Duration of an epoch | Context size | Batch size | PPL |
|
56 |
|:-------:|:--------------------:|:---------------:|:--------------------:|:----------:|:----------:|:---:|
|
57 |
| Base | 124M | 15 | 7h | 1024 | 72 | 22.96 |
|
58 |
| Medium | 354M | 10 | 22h | 1024 | 24 | 17.64 |
|
|
|
125 |
|RoBERT-small | - | 30.84 | 45.17 |
|
126 |
|RoBERT-base | - | 53.52 | 70.04 |
|
127 |
|RoBERT-large | - | 55.46 | 69.64 |
|
128 |
+
|mBERT | - | 59.9 | 72.7 |
|
129 |
+
|XLM-R Large | - |**69.7** |**83.6** |
|
130 |
|RoGPT2-base | Greedy | 23.69 | 35.97 |
|
131 |
|RoGPT2-base | Beam-search-4 | 24.11 | 35.27 |
|
132 |
|RoGPT2-medium | Greedy | 29.66 | 44.74 |
|