Package: scgpt
scGPTThis is the official codebase forscGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI.!UPDATE: We have released several new pretrained scGPT checkpoints. Please see thePretrained scGPT checkpointssection for more details.[2023.11.07]As requested by many, now we have made flash-attention an optional dependency. The pretrained weights can be loaded on pytorch CPU, GPU, and flash-attn backends using the sameload_pretrainedfunction,load_pretrained(target_model, torch.load("path_to_ckpt.pt")). An example usage is alsohere.[2023.09.05]We have release a new feature for reference mapping samples to a custom reference dataset or to all the millions of cells collected from CellXGene! With the help of thefaisslibrary, we achieved a great time and memory efficiency. The index of over 33 millions cells only takes less than 1GB of memory and the similarity search takes less than1 second for 10,000 query cells on GPU. Please see theReference mapping tutorialfor more details.Online appsscGPT is now available at the following online apps as well, so you can get started simply with your browser!Run thereference mapping app,cell annotation appand theGRN inference appwith cloud gpus. Thanks to theSuperbio.aiteam for helping create and host the interactive tools.InstallationscGPT works with Python >= 3.7.13 and R >=3.6.1. Please make sure you have the correct version of Python and R installed pre-installation.scGPT is available on PyPI. To install scGPT, run the following command:pipinstallscgpt"flash-attn<1.0.5"# optional, recommended# As of 2023.09, pip install may not run with new versions of the google orbax package, if you encounter related issues, please use the following command instead:# pip install scgpt "flash-attn<1.0.5" "orbax<0.1.8"[Optional] We recommend usingwandbfor logging and visualization.pipinstallwandbFor developing, we are using thePoetrypackage manager. To install Poetry, follow the instructionshere.$gitclonethis-repo-url
$cdscGPT
$poetryinstallNote: Theflash-attndependency usually requires specific GPU and CUDA version. If you encounter any issues, please refer to theflash-attnrepository for installation instructions. For now, May 2023, we recommend using CUDA 11.7 and flash-attn<1.0.5 due to various issues reported about installing new versions of flash-attn.Pretrained scGPT Model ZooHere is the list of pretrained models. Please find the links for downloading the checkpoint folders. We recommend using thewhole-humanmodel for most applications by default. If your fine-tuning dataset shares similar cell type context with the training data of the organ-specific models, these models can usually demonstrate competitive performance as well. A paired vocabulary file mapping gene names to ids is provided in each checkpoint folder. If ENSEMBL ids are needed, please find the conversion atgene_info.csv.Model nameDescriptionDownloadwhole-human (recommended)Pretrained on 33 million normal human cells.linkbrainPretrained on 13.2 million brain cells.linkbloodPretrained on 10.3 million blood and bone marrow cells.linkheartPretrained on 1.8 million heart cellslinklungPretrained on 2.1 million lung cellslinkkidneyPretrained on 814 thousand kidney cellslinkpan-cancerPretrained on 5.7 million cells of various cancer typeslinkFine-tune scGPT for scRNA-seq integrationPlease see our example code inexamples/finetune_integration.py. By default, the script assumes the scGPT checkpoint folder stored in theexamples/savedirectory.To-do-listUpload the pretrained model checkpointPublish to pypiProvide the pretraining code with generative attention maskingFinetuning examples for multi-omics integration, cell type annotation, perturbation prediction, cell generationExample code for Gene Regulatory Network analysisDocumentation website with readthedocsBump up to pytorch 2.0New pretraining on larger datasetsReference mapping examplePublish to huggingface model hubContributingWe greatly welcome contributions to scGPT. Please submit a pull request if you have any ideas or bug fixes. We also welcome any issues you encounter while using scGPT.AcknowledgementsWe sincerely thank the authors of following open-source projects:flash-attentionscanpyscvi-toolsscibdatasetstransformersCiting scGPT@article{cui2023scGPT,title={scGPT: Towards Building a Foundation Model for Single-Cell Multi-omics Using Generative AI},author={Cui, Haotian and Wang, Chloe and Maan, Hassaan and Pang, Kuan and Luo, Fengning and Wang, Bo},journal={bioRxiv},year={2023},publisher={Cold Spring Harbor Laboratory}}