Package: scETM
scETM: single-cell Embedded Topic ModelA generative topic model that facilitates integrative analysis of large-scale single-cell RNA sequencing data.The full description of scETM and its application on published single cell RNA-seq datasets are availablehere.This repository includes detailed instructions for installation and requirements, demos, and scripts used for the benchmarking of 7 other state-of-art methods.ContentsscETM: single-cell Embedded Topic ModelContents1 Model Overview2 Installation3 UsageData formatA taste of scETMp-scETMTransfer learningTensorboard Integration4 Benchmarking1 Model Overview(a)Probabilistic graphical model of scETM. We model the scRNA-profile read count matrix yd,gin cell d and gene g across S subjects or studies by a multinomial distribution with the rate parameterized by cell topic mixture θ, topic embedding α, gene embedding ρ, and batch effects λ.(b)Matrix factorization view of scETM.(c)Encoder architecture for inferring the cell topic mixture θ.2 InstallationPython version: 3.7+
scETM is included in PyPI, so you can install it bypipinstallscETMTo enable GPU computing (which significantly boosts the performance), please installPyTorchwith GPU supportbeforeinstalling scETM.3 UsageA step-by-step scETM tutorial can be found inhere.Data formatscETM requires a cells-by-genes matrixadataas input, in the format of an AnnData object. Detailed description about AnnData can be foundhere.By default, scETM looks for batch information in the 'batch_indices' column of theadata.obsDataFrame, and cell type identity in the 'cell_types' column. If your data stores the batch and cell type information in different columns, pass them to thebatch_colandcell_type_colarguments, respectively, when calling scETM functions.A taste of scETMfromscETMimportscETM,UnsupervisedTrainer,evaluateimportanndata# Prepare the source dataset, Mouse Pancreasmp=anndata.read_h5ad("MousePancreas.h5ad")# Initialize modelmodel=scETM(mp.n_vars,mp.obs.batch_indices.nunique(),enable_batch_bias=True)# The trainer object will set up the random seed, optimizer, training and evaluation loop, checkpointing and logging.trainer=UnsupervisedTrainer(model,mp,train_instance_name="MP",ckpt_dir="../results")# Train the model on adata for 12000 epochs, and evaluate every 1000 epochs. Use 4 threads to sample minibatches.trainer.train(n_epochs=12000,eval_every=1000,n_samplers=4)# Obtain scETM cell, gene and topic embeddings. Unnormalized cell embeddings will be stored at mp.obsm['delta'], normalized cell embeddings at mp.obsm['theta'], gene embeddings at mp.varm['rho'], topic embeddings at mp.uns['alpha'].model.get_all_embeddings_and_nll(mp)# Evaluate the model and save the embedding plotevaluate(mp,embedding_key="delta",plot_fname="scETM_MP",plot_dir="figures/scETM_MP")p-scETMp-scETM is a variant of scETM where part or all of the the gene embedding matrix ρ is fixed to a pathways-by-genes matrix, which can be downloaded from thepathDIP4 pathway database. We only keep pathways that contain more than 5 genes.If it is desired to fix the gene embedding matrix ρ during training, let trainable_gene_emb_dim be zero. In this case, the gene set used to train the model would be the intersection of the genes in the scRNA-seq data and the genes in the gene-by-pathway matrix. Otherwise, if trainable_gene_emb_dim is set to a positive value, all the genes in the scRNA-seq data would be kept.Transfer learningfromscETMimportscETM,UnsupervisedTrainer,prepare_for_transferimportanndata# Prepare the source dataset, Mouse Pancreasmp=anndata.read_h5ad("MousePancreas.h5ad")# Initialize modelmodel=scETM(mp.n_vars,mp.obs.batch_indices.nunique(),enable_batch_bias=True)# The trainer object will set up the random seed, optimizer, training and evaluation loop, checkpointing and logging.trainer=UnsupervisedTrainer(model,mp,train_instance_name="MP",ckpt_dir="../results")# Train the model on adata for 12000 epochs, and evaluate every 1000 epochs. Use 4 threads to sample minibatches.trainer.train(n_epochs=12000,eval_every=1000,n_samplers=4)# Load the target dataset, Human Pancreashp=anndata.read_h5ad('HumanPancreas.h5ad')# Align the source dataset's gene names (which are mouse genes) to the target dataset (which are human genes)mp_genes=mp.var_names.str.upper()mp_genes.drop_duplicates(inplace=True)# Generate a new model and a modified dataset from the previously trained model and the mp_genesmodel,hp=prepare_for_transfer(model,hp,mp_genes,keep_tgt_unique_genes=True,# Keep target-unique genes in the model and the target datasetfix_shared_genes=True# Fix parameters related to shared genes in the model)# Instantiate another trainer to fine-tune the modeltrainer=UnsupervisedTrainer(model,hp,train_instance_name="HP_all_fix",ckpt_dir="../results",init_lr=5e-4)trainer.train(n_epochs=800,eval_every=200)Tensorboard IntegrationIf a Tensorboard SummaryWriter is passed to thewriterargument of theUnsupervisedTrainer.trainmethod, the package will store.4 BenchmarkingThe commands used for runningHarmony,Scanorama,Seurat,scVAE-GM,scVI,LIGER,scVI-LDare available in thescriptsfolder.