data2vec squad
This is a testing fine tuned data2vec model in the squad dataset, any improvements and suggestions are welcome!Intended use
Question AnsweringTraining results
| Epoch | Training Loss | Validation Loss |
|---|---|---|
| 1 | 1.015800 |
0.997690 |
| 2 | 0.804400 | 0.950322 |
Hyperparameters
- evaluation_strategy="epoch"
- learning_rate=2e-5
- per_device_train_batch_size=15
- per_device_eval_batch_size=15
- num_train_epochs=2
- weight_decay=0.01
Frameworks and libraries used:
- transformers
- datasets
- evaluate
- Downloads last month
- 8