Li commited on
Commit
95fda1d
1 Parent(s): 3fd1e6c

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -4,10 +4,11 @@ language:
4
  license: apache-2.0
5
  ---
6
 
 
7
 
8
- Bioformer is a lightweight BERT model for biomedical text mining. Bioformer uses a biomedical vocabulary and is pre-trained from scratch only on biomedical domain corpora. Our experiments show that Bioformer is 3x as fast as BERT-base, and achieves comparable or even better performance than BioBERT/PubMedBERT on downstream NLP tasks.
9
 
10
- Bioformer has 8 layers (transformer blocks) with a hidden embedding size of 512, and the number of self-attention heads is 8. Its total number of parameters is 42,820,610.
11
 
12
  **The usage of Bioformer is the same as a standard BERT model. The documentation of BERT can be found [here](https://huggingface.co/docs/transformers/model_doc/bert).**
13
 
 
4
  license: apache-2.0
5
  ---
6
 
7
+ NOTE: `bioformer-cased-v1.0` has been renamed to `bioformer-8L`. All links to `bioformer-cased-v1.0` will automatically redirect to `bioformer-8L`, including git operations. However, to avoid confusion, we recommend updating any existing local clones to point to the new repository URL.
8
 
9
+ Bioformer-8L is a lightweight BERT model for biomedical text mining. Bioformer uses a biomedical vocabulary and is pre-trained from scratch only on biomedical domain corpora. Our experiments show that Bioformer is 3x as fast as BERT-base, and achieves comparable or even better performance than BioBERT/PubMedBERT on downstream NLP tasks.
10
 
11
+ Bioformer-8L has 8 layers (transformer blocks) with a hidden embedding size of 512, and the number of self-attention heads is 8. Its total number of parameters is 42,820,610.
12
 
13
  **The usage of Bioformer is the same as a standard BERT model. The documentation of BERT can be found [here](https://huggingface.co/docs/transformers/model_doc/bert).**
14