kiddothe2b commited on
Commit
1253859
1 Parent(s): c392946

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -2
README.md CHANGED
@@ -9,5 +9,4 @@ tags:
9
 
10
  # Biomedical Longformer (base)
11
 
12
- This is a derivative model based on [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract) BERT model developed in the work "Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
13
- " by [Gu et al. (2020)](https://arxiv.org/abs/2007.15779). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following [Beltagy et al. (2020)](https://arxiv.org/abs/2004.05150) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).
 
9
 
10
  # Biomedical Longformer (base)
11
 
12
+ This is a derivative model based on [microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract) BERT model developed in the work "Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing" by [Tinn et al. (2021)](https://arxiv.org/abs/2112.07869). All model parameters where cloned from the original model, while the positional embeddings were extended by cloning the original embeddings multiple times following [Beltagy et al. (2020)](https://arxiv.org/abs/2004.05150) using a python script similar to this one (https://github.com/allenai/longformer/blob/master/scripts/convert_model_to_long.ipynb).