bioformer-8L pretrained on 164,179 COVID-19 abstracts (from LitCovid website) for 100 epochs.
In our evaluation, this pretraining process leads to improved performance on the multi-label COVID-19 topic classification task (BioCreative VII track 5).
- Downloads last month
- 2