File size: 427 Bytes
0559d41 c8d0776 |
1 2 3 4 5 6 7 |
# T5-base pretrained on CORD-19 dataset
The model has been pretrained on text and abstracts from the CORD-19 dataset, using a manually implemented denoising objetive similar to the original T5 denoising objective.
Model needs to be finetuned on downstream tasks.
Code avaliable in github: [https://github.com/manueldeprada/Pretraining-T5-PyTorch-Lightning](https://github.com/manueldeprada/Pretraining-T5-PyTorch-Lightning). |