moll_ha
commited on
Commit
•
cb7bdd6
1
Parent(s):
434fe23
added readme
Browse files
README.MD
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
To produce BioELECTRA, we pretrain ELECTRA on a corpus of over 20 million abstracts from PubMed.
|
| 2 |
+
|
| 3 |
+
How to use the generator in transformers:
|
| 4 |
+
|
| 5 |
+
from transformers import ElectraForMaskedLM, ElectraTokenizerFast
|
| 6 |
+
import torch
|
| 7 |
+
generator = ElectraForMaskedLM.from_pretrained("molly-hayward/bioelectra-base-generator")
|
| 8 |
+
tokenizer = ElectraTokenizerFast.from_pretrained("molly-hayward/bioelectra-base-generator")
|