File size: 417 Bytes
cb7bdd6 |
1 2 3 4 5 6 7 8 |
To produce BioELECTRA, we pretrain ELECTRA on a corpus of over 20 million abstracts from PubMed.
How to use the generator in transformers:
from transformers import ElectraForMaskedLM, ElectraTokenizerFast
import torch
generator = ElectraForMaskedLM.from_pretrained("molly-hayward/bioelectra-base-generator")
tokenizer = ElectraTokenizerFast.from_pretrained("molly-hayward/bioelectra-base-generator") |