Update README.md

README.md

@@ -9,23 +9,25 @@ widget:
 
 ## PubMedELECTRA-base (abstracts only)
 
-Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. [Recent work](https://arxiv.org/abs/2007.15779) shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.
+Pretraining large neural language models, such as BERT and ELECTRA, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. [Recent work](https://arxiv.org/abs/2007.15779) shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.[Followup work](https://arxiv.org/abs/2112.07869) explores alternate pretraining strategies and the impact of these on performance on the BLURB benchmark. 
 
-This PubMedBERT is pretrained from scratch using _abstracts_ from [PubMed](https://pubmed.ncbi.nlm.nih.gov/). This model achieves state-of-the-art performance on several biomedical NLP tasks, as shown on the [Biomedical Language Understanding and Reasoning Benchmark](https://aka.ms/BLURB).
+This PubMedELECTRA is pretrained from scratch using _abstracts_ from [PubMed](https://pubmed.ncbi.nlm.nih.gov/).
+
+PubMedELECTRAlLarge is not able to be made available on HuggingFace and can be downloaded from [here](https://hanoverdev.blob.core.windows.net/electralargerelease/pubmed_electra_large/).
 
 ## Citation
 
-If you find PubMedBERT useful in your research, please cite the following paper:
+If you find PubMedELECTRA useful in your research, please cite the following paper:
 
 ```latex
-@misc{pubmedbert,
-  author = {Yu Gu and Robert Tinn and Hao Cheng and Michael Lucas and Naoto Usuyama and Xiaodong Liu and Tristan Naumann and Jianfeng Gao and Hoifung Poon},
-  title = {Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing},
-  year = {2020},
-  eprint = {arXiv:2007.15779},
+@misc{https://doi.org/10.48550/arxiv.2112.07869,
+  doi = {10.48550/ARXIV.2112.07869},
+  url = {https://arxiv.org/abs/2112.07869},
+  author = {Tinn, Robert and Cheng, Hao and Gu, Yu and Usuyama, Naoto and Liu, Xiaodong and Naumann, Tristan and Gao, Jianfeng and Poon, Hoifung},
+  keywords = {Computation and Language (cs.CL), Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  title = {Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing},
+  publisher = {arXiv},
+  year = {2021},
+  copyright = {arXiv.org perpetual, non-exclusive license}
 }
 ```
-
-<a href="https://huggingface.co/exbert/?model=microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract&modelKind=bidirectional&sentence=Gefitinib%20is%20an%20EGFR%20tyrosine%20kinase%20inhibitor,%20which%20is%20often%20used%20for%20breast%20cancer%20and%20NSCLC%20treatment.&layer=10&heads=..0,1,2,3,4,5,6,7,8,9,10,11&threshold=0.7&tokenInd=17&tokenSide=right&maskInds=..&hideClsSep=true">
-	<img width="300px" src="https://cdn-media.huggingface.co/exbert/button.png">
-</a>