Ahmed Elnaggar commited on
Commit
d604cdc
1 Parent(s): a97c308

Update prot-t5-xl from elnaggar

Browse files
Files changed (3) hide show
  1. README.md +3 -3
  2. pytorch_model.bin +2 -2
  3. pytorch_model_723k.bin +3 -0
README.md CHANGED
@@ -3,10 +3,10 @@ language: protein
3
  tags:
4
  - protein language model
5
  datasets:
6
- - BFD
7
  ---
8
 
9
- # ProtT5-XL-BFD model
10
 
11
  Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in
12
  [this paper](https://doi.org/10.1101/2020.07.12.199554) and first released in
@@ -87,7 +87,7 @@ The details of the masking procedure for each sequence are as follows:
87
 
88
  ### Pretraining
89
 
90
- The model was trained on a single TPU Pod V2-256 for 600 thousand steps in total, using sequence length 512 (batch size 2k).
91
  It was trained using ProtT5-XL-BFD model as an initial checkpoint, rather than training from scratch.
92
  It has a total of approximately 3B parameters and was trained using the encoder-decoder architecture.
93
  The optimizer used is AdaFactor with inverse square root learning rate schedule for pre-training.
 
3
  tags:
4
  - protein language model
5
  datasets:
6
+ - UniRef50
7
  ---
8
 
9
+ # ProtT5-XL-UniRef50 model
10
 
11
  Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in
12
  [this paper](https://doi.org/10.1101/2020.07.12.199554) and first released in
 
87
 
88
  ### Pretraining
89
 
90
+ The model was trained on a single TPU Pod V2-256 for 991.5 thousand steps in total, using sequence length 512 (batch size 2k).
91
  It was trained using ProtT5-XL-BFD model as an initial checkpoint, rather than training from scratch.
92
  It has a total of approximately 3B parameters and was trained using the encoder-decoder architecture.
93
  The optimizer used is AdaFactor with inverse square root learning rate schedule for pre-training.
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6f2fb2f30a5956893af81f094f6a376d7218b7e7ae8feb861a7536f5ad2bebbb
3
- size 11275562628
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c06141d42e93c12b5f6d95c701952156bd4079661f5f2d981d0d2036ba96dae9
3
+ size 11275562724
pytorch_model_723k.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6f2fb2f30a5956893af81f094f6a376d7218b7e7ae8feb861a7536f5ad2bebbb
3
+ size 11275562628