---
language:
- {en}  # Example: fr
tags:
- {fill-mask}  # Example: audio
---


This model was pretrained from scratch on a custom vocabulary on Pubmed, Clinical trials corpus, and a small subset of Bookcorpus