Update README.md

README.md

@@ -2,9 +2,9 @@
 tags:
 - biology
 ---
-# AIDO.Protein 16B 
+# AIDO.Protein-16B 
 
-AIDO.Protein stands as the largest protein foundation model in the world to date, trained on 1.2 trillion amino acids sourced from UniRef90 and ColabFoldDB. 
+AIDO.Protein-16B is a protein language model, trained on 1.2 trillion amino acids sourced from UniRef90 and ColabFoldDB. 
 
 By leveraging MoE layers, AIDO.Protein efficiently scales to 16 billion parameters, delivering exceptional performance across a vast variety of tasks in protein sequence understanding and sequence generation. Remarkably, AIDO.Protein demonstrates exceptional capability despite being trained solely on single protein sequences. Across over 280 DMS protein fitness prediction tasks, our model outperforms previous state-of-the-art protein sequence models without MSA and achieves 99% of the performance of models that utilize MSA, highlighting the strength of its learned representations.
 
@@ -23,7 +23,7 @@ More architecture details are shown below:
 |Vocab Size|44 | 
 | Context Length |2048 |
 
-## Pre-training of AIDO.Protein 16B
+## Pre-training of AIDO.Protein-16B
 Here we briefly introduce the details of pre-training of AIDO.Protein 16B. For more information, please refer to [our paper](https://www.biorxiv.org/content/10.1101/2024.11.29.625425v1)
 ### Data
 Inspired by previous work, We initially trained AIDO.Protein with 1.2 trillion amino acids sourced from the combination of Uniref90 and ColabeFoldDB databases. Given the effectiveness of Uniref90 for previous protein language models and the observed benefits of continuous training on domina-specific data for enhancing downstream task performance, AIDO.Protein is further trained on an additional 100 billion amino acids from Uniref90. 
@@ -60,12 +60,19 @@ We assess the advantages of pretraining AIDO.Protein 16B through experiments acr
 
 
 ## How to Use
-### Build any downstream models from this backbone
+### Build any downstream models from this backbone with ModelGenerator
+For more information, visit: [Model Generator](https://github.com/genbio-ai/modelgenerator)
+```bash
+mgen fit --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
+mgen test --model SequenceClassification --model.backbone aido_protein_16b --data SequenceClassificationDataModule --data.path <hf_or_local_path_to_your_dataset>
+```
+
+### Or use directly in Python
 #### Embedding
 ```python
-from genbio_finetune.tasks import Embed
-model = Embed.from_config({"model.backbone": "proteinfm"}).eval()
-collated_batch = model.collate({"sequences": ["ACGT", "AGCT"]})
+from modelgenerator.tasks import Embed
+model = Embed.from_config({"model.backbone": "aido_protein_16b"}).eval()
+collated_batch = model.collate({"sequences": ["HLLQ", "WRLD"]})
 embedding = model(collated_batch)
 print(embedding.shape)
 print(embedding)
@@ -73,9 +80,9 @@ print(embedding)
 #### Sequence Level Classification
 ```python
 import torch
-from genbio_finetune.tasks import SequenceClassification
-model = SequenceClassification.from_config({"model.backbone": "proteinfm", "model.n_classes": 2}).eval()
-collated_batch = model.collate({"sequences": ["ACGT", "AGCT"]})
+from modelgenerator.tasks import SequenceClassification
+model = SequenceClassification.from_config({"model.backbone": "aido_protein_16b", "model.n_classes": 2}).eval()
+collated_batch = model.collate({"sequences": ["HLLQ", "WRLD"]})
 logits = model(collated_batch)
 print(logits)
 print(torch.argmax(logits, dim=-1))
@@ -83,44 +90,33 @@ print(torch.argmax(logits, dim=-1))
 #### Token Level Classification
 ```python
 import torch
-from genbio_finetune.tasks import TokenClassification
-model = TokenClassification.from_config({"model.backbone": "proteinfm", "model.n_classes": 3}).eval()
-collated_batch = model.collate({"sequences": ["ACGT", "AGCT"]})
+from modelgenerator.tasks import TokenClassification
+model = TokenClassification.from_config({"model.backbone": "aido_protein_16b", "model.n_classes": 3}).eval()
+collated_batch = model.collate({"sequences": ["HLLQ", "WRLD"]})
 logits = model(collated_batch)
 print(logits)
 print(torch.argmax(logits, dim=-1))
 ```
 #### Regression
 ```python
-from genbio_finetune.tasks import SequenceRegression
-model = SequenceRegression.from_config({"model.backbone": "proteinfm"}).eval()
-collated_batch = model.collate({"sequences": ["ACGT", "AGCT"]})
+from modelgenerator.tasks import SequenceRegression
+model = SequenceRegression.from_config({"model.backbone": "aido_protein_16b"}).eval()
+collated_batch = model.collate({"sequences": ["HLLQ", "WRLD"]})
 logits = model(collated_batch)
 print(logits)
 ```
-#### Protein-Protein Interaction
-
-#### Or use our one-liner CLI to finetune or evaluate any of the above!
-```
-gbft fit --model SequenceClassification --model.backbone proteinfm --data SequenceClassification --data.path <hf_or_local_path_to_your_dataset>
-gbft test --model SequenceClassification --model.backbone proteinfm --data SequenceClassification --data.path <hf_or_local_path_to_your_dataset>
-```
-For more information, visit: [Model Generator](https://github.com/genbio-ai/modelgenerator)
-
-
-# Or use our one-liner CLI to finetune or evaluate any of the above
-
-For more information, visit: Model Generator
 
 # Citation
 Please cite AIDO.Protein using the following BibTex code:
 ```
-@inproceedings{Sun2024mixture,
-title={Mixture of Experts Enable Efficient and Effective
-Protein Understanding and Design},
-author={Ning Sun, Shuxian Zou, Tianhua Tao, Sazan Mahbub, Dian Li, Yonghao Zhuang, Hongyi Wang, Le Song, Eric P. Xing},
-booktitle={NeurIPS 2024 Workshop on AI for New Drug Modalities},
-year={2024}
+@inproceedings{sun_mixture_2024,
+	title = {Mixture of Experts Enable Efficient and Effective Protein Understanding and Design},
+	url = {https://www.biorxiv.org/content/10.1101/2024.11.29.625425v1},
+	doi = {10.1101/2024.11.29.625425},
+	publisher = {bioRxiv},
+	author = {Sun, Ning and Zou, Shuxian and Tao, Tianhua and Mahbub, Sazan and Li, Dian and Zhuang, Yonghao and Wang, Hongyi and Cheng, Xingyi and Song, Le and Xing, Eric P.},
+	year = {2024},
+    booktitle={NeurIPS 2024 Workshop on AI for New Drug Modalities},
 }
 ```