Mihai-Dan MAŞALA (25095)
commited on
Commit
•
b460ae6
1
Parent(s):
f774d4f
Update README
Browse files
README.md
CHANGED
@@ -13,11 +13,13 @@ language:
|
|
13 |
Pretrained model on Romanian language using a masked language modeling (MLM) and next sentence prediction (NSP) objective.
|
14 |
It was introduced in this [paper](https://www.blank.org/). Three BERT models were released: RoBERT-small, RoBERT-base and RoBERT-large, all versions uncased.
|
15 |
|
16 |
-
Model
|
17 |
-
|
18 |
-
RoBERT-small | 19M
|
19 |
-
RoBERT-base | 114M
|
20 |
-
RoBERT-large | 341M
|
|
|
|
|
21 |
|
22 |
|
23 |
All models are available:
|
13 |
Pretrained model on Romanian language using a masked language modeling (MLM) and next sentence prediction (NSP) objective.
|
14 |
It was introduced in this [paper](https://www.blank.org/). Three BERT models were released: RoBERT-small, RoBERT-base and RoBERT-large, all versions uncased.
|
15 |
|
16 |
+
| Model | Weights | L | H | A | MLM accuracy | NSP accuracy |
|
17 |
+
|--------------|:---------:|:------:|:------:|:------:|:------------:|:------------:|
|
18 |
+
| RoBERT-small | 19M | 12 | 256 | 8 | 0.5363 | 0.9687 |
|
19 |
+
| RoBERT-base | 114M | 12 | 768 | 12 | 0.6511 | 0.9802 |
|
20 |
+
| RoBERT-large | 341M | 24 | 1024 | 24 | 0.6929 | 0.9843 |
|
21 |
+
|
22 |
+
|
23 |
|
24 |
|
25 |
All models are available:
|