Commit
•
788d397
1
Parent(s):
5981d71
Update README.md
Browse files
README.md
CHANGED
|
@@ -12,6 +12,8 @@ license: apache-2.0
|
|
| 12 |
|
| 13 |
The model was pre-trained using T5's denoising objective on [C4](https://huggingface.co/datasets/c4), subsequently additionally pre-trained using [REALM](https://arxiv.org/pdf/2002.08909.pdf)'s salient span masking objective on [Wikipedia](https://huggingface.co/datasets/wikipedia), and finally fine-tuned on [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions).
|
| 14 |
|
|
|
|
|
|
|
| 15 |
Other community Checkpoints: [here](https://huggingface.co/models?search=ssm)
|
| 16 |
|
| 17 |
Paper: [How Much Knowledge Can You Pack
|
|
|
|
| 12 |
|
| 13 |
The model was pre-trained using T5's denoising objective on [C4](https://huggingface.co/datasets/c4), subsequently additionally pre-trained using [REALM](https://arxiv.org/pdf/2002.08909.pdf)'s salient span masking objective on [Wikipedia](https://huggingface.co/datasets/wikipedia), and finally fine-tuned on [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions).
|
| 14 |
|
| 15 |
+
**Note**: The model was fine-tuned on 100% of the train splits of [Natural Questions (NQ)](https://huggingface.co/datasets/natural_questions) for 10k steps.
|
| 16 |
+
|
| 17 |
Other community Checkpoints: [here](https://huggingface.co/models?search=ssm)
|
| 18 |
|
| 19 |
Paper: [How Much Knowledge Can You Pack
|