Renames PubMedBERT to BiomedBERT
Browse files
README.md
CHANGED
@@ -7,15 +7,17 @@ widget:
|
|
7 |
- text: "[MASK] is a tyrosine kinase inhibitor."
|
8 |
---
|
9 |
|
10 |
-
##
|
|
|
|
|
11 |
|
12 |
Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. [Recent work](https://arxiv.org/abs/2007.15779) shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.
|
13 |
|
14 |
-
This
|
15 |
|
16 |
## Citation
|
17 |
|
18 |
-
If you find
|
19 |
|
20 |
```latex
|
21 |
@misc{pubmedbert,
|
|
|
7 |
- text: "[MASK] is a tyrosine kinase inhibitor."
|
8 |
---
|
9 |
|
10 |
+
## MSR BiomedBERT (abstracts only)
|
11 |
+
|
12 |
+
*NOTE: This model was previously named "PubMedBERT (abstracts only)".*
|
13 |
|
14 |
Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on general domain corpora, such as newswire and Web. A prevailing assumption is that even domain-specific pretraining can benefit by starting from general-domain language models. [Recent work](https://arxiv.org/abs/2007.15779) shows that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.
|
15 |
|
16 |
+
This BiomedBERT is pretrained from scratch using _abstracts_ from [PubMed](https://pubmed.ncbi.nlm.nih.gov/). This model achieves state-of-the-art performance on several biomedical NLP tasks, as shown on the [Biomedical Language Understanding and Reasoning Benchmark](https://aka.ms/BLURB).
|
17 |
|
18 |
## Citation
|
19 |
|
20 |
+
If you find BiomedBERT useful in your research, please cite the following paper:
|
21 |
|
22 |
```latex
|
23 |
@misc{pubmedbert,
|