Dongsung commited on
Commit
fd743d5
1 Parent(s): 42f4eb9

Update model description

Browse files
Files changed (1) hide show
  1. README.md +10 -1
README.md CHANGED
@@ -1,5 +1,4 @@
1
  ---
2
- license:
3
  tags:
4
  - generated_from_trainer
5
  datasets:
@@ -13,6 +12,16 @@ model-index:
13
 
14
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the squad_v2 dataset.
15
 
 
 
 
 
 
 
 
 
 
 
16
  ## Training and evaluation data
17
 
18
  Trained and evaluated on the [squad_v2 dataset](https://huggingface.co/datasets/squad_v2).
 
1
  ---
 
2
  tags:
3
  - generated_from_trainer
4
  datasets:
 
12
 
13
  This model is a fine-tuned version of [roberta-base](https://huggingface.co/roberta-base) on the squad_v2 dataset.
14
 
15
+ ## Model description
16
+
17
+ RoBERTa is based on BERT pretrain approach but it t evaluates carefully a number of design decisions of BERT pretraining approach so that it found it is undertrained.
18
+
19
+ It suggested a way to improve the performance by training the model longer, with bigger batches over more data, removing the next sentence prediction objectives, training on longer sequences and dynamically changing mask pattern applied to the training data.
20
+
21
+ As a result, it achieves state-of-the-art results on GLUE, RACE and SQuAD and so on on.
22
+
23
+ Paper link : [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://arxiv.org/pdf/1907.11692.pdf)
24
+
25
  ## Training and evaluation data
26
 
27
  Trained and evaluated on the [squad_v2 dataset](https://huggingface.co/datasets/squad_v2).