SARS-Cov-2-S-Pro
The SARS-CoV-2-S-Pro is a protein model trained on a masked language modeling objective, resulting from the unsupervised fine-tuning of the ESM-2 (https://huggingface.co/facebook/esm2_t33_650M_UR50D) protein language model. Its fine-tuning dataset is sourced from SARS-Cov-2 S protein sequences in the NCBI Virus database (as of February 20, 2025). The original dataset comprises 2,584,107 protein sequences from different species, reduced to 134,555 sequences after removing duplicates and those with 100% identity.
The SARS-CoV-2-S-Pro model is suitable for downstream applications related to SARS-Cov-2 viruses. For detailed information on the model’s training data, please refer to the accompanying paper.
Multiple SARS-CoV-2-S-Pro checkpoints are available on the Hub, with varying model sizes. Generally, larger models offer higher accuracy but require more memory and training time.
| Checkpoint name | Num layers | Num parameters |
|---|---|---|
| .SARS_Cov_2_S_Pro_t36_3B_esm2 | 36 | 3B |
| SARS_Cov_2_S_Pro_t33_650M_esm2 | 33 | 650M |
| SARS_Cov_2_S_Pro_t30_150M_esm2 | 30 | 150M |
| SARS_Cov_2_S_Pro_t12_35M_esm2 | 12 | 35M |
| SARS_Cov_2_S_Pro_t6_8M_esm2 | 6 | 8M |
Results on Epoch 15:
Training Metrics:
Training Loss: 0.0122
Training Perplexity: 1.0123
Evaluation Metrics:
Evaluation Loss: 0.0129
Evaluation Perplexity: 1.0130
- Downloads last month
- 2