Package: dinglab-pollock
PollockPollock is a tool for single cell classification. Pollock is available in both Python, R, and as a command line toolInstallationRequirementsOS:macOS 10.12.6 (Sierra) or laterUbuntu 16.04 or laterWindows 7 or later (not tested)Anaconda/CondaWorking installation of conda is required. Note this is not required if using Docker.To installPollock is available to run in a Docker image (see below) or can be installed with Conda.If running without Docker, follow the installation instructions below.First, download the Pollock repogitclonehttps://github.com/ding-lab/pollock.gitThen, create a conda environment from the environmental file within the Pollock repository and activate the conda environment.cdpollock
condaenvcreate--fileenv.yaml
condaactivatepollock
pipinstall.If you intend to run Pollock off .RDS Seurat single cell objects you will also need to install the rpollock R library with the following command. For additional information about running with R, see the Usage - R section below.R-e"Sys.setenv(TAR = system('which tar', intern = TRUE)); devtools::install_github('https://github.com/estorrs/rpollock')"NOTE: tensorflow requires a fair amount of space to build correctly. In some clusters the tmp/ directory does not have enough space for tensorflow to build. If you run pollock and get an error that tensorflow is not available you will have to install it manually using a directory with enough space (> 2GB should be sufficient) with the following command.TMPDIR=<path/to/directory>pipinstall--cache-dir=<path/to/directory>--build<path/to/directory>tensorflow==2.1.0UsagePollock uses deep learning to make cell type predictions. At it's core, pollock is a variational autoencoder (VAE) paired with a random forest classifier.With pollock, there are a selection of cell type classification modules that have been trained on a variety of single cell RNA-seq datasets. Any of these modules can be used to classify your single cell data.Additionally, if you have annotated single cell data, pollock can also be used to train a new module based on the given cell types.ModulesThere are a variety of modules available for cell type classification. The modules and training datasets can be found on Zenodo athttps://zenodo.org/record/5514140#.YYACrNZKhhEThe following is a list of available pretrained modules:scRNA-seqdisease_specific_brca_scRNAseqdisease_specific_cesc_scRNAseqdisease_specific_hnscc_scRNAseqdisease_specific_melanoma_scRNAseqdisease_specific_mmy_scRNAseqdisease_specific_pdac_scRNAseqgeneral_scRNAseqpanimmune_scRNAseqHCA_bone_marrow_scRNAseqsnRNAseqdisease_specific_brca_snRNAseqdisease_specific_gbm_snRNAseqdisease_specific_ccrcc_snRNAseqgeneral_snRNAseqsnATACseqdisease_specific_brca_snATACseq_gene_activitydisease_specific_brca_snATACseq_motifdisease_specific_ccrcc_snATACseq_gene_activitydisease_specific_ccrcc_snATACseq_motifdisease_specific_gbm_snATACseq_gene_activitydisease_specific_gbm_snATACseq_motifgeneral_snATACseqYou can also create new modules with pollock (see training section below)TutorialsPython APImodule training tutorial on pbmc datasetprediction and feature explaination with an existing modulemodule examinationR APIThere is an R library rpollock that comes installed with pollock that allows you to train a module and make predictions directly from R.Note: rpollock is dependent on the R library reticulate, which will sometimes prompt for a python install location. If this occurs, run the below code to find out the location of your python installation. It will output<path/to/python/executable>whichpython3When running R you will need to have this line at the very start of your script (before your library imports)reticulate::use_python("<path/to/python/executable>")example usage of rpollock on pbmc3kThis notebookis a python script walking over the information that is contained in each module. Though it is in python, all this information is saved in a json file so everything done in that notebook can also be done in R.Command line toolusage:pollock[-h][--module-filepathMODULE_FILEPATH][--seurat-rds-filepathSEURAT_RDS_FILEPATH][--scanpy-h5ad-filepathSCANPY_H5AD_FILEPATH][--counts-10x-filepathCOUNTS_10X_FILEPATH][--min-genes-per-cellMIN_GENES_PER_CELL][--txt-output][--output-prefixOUTPUT_PREFIX][--explain-filepathEXPLAIN_FILEPATH][--background-filepathBACKGROUND_FILEPATH][--predicted-keyPREDICTED_KEY][--background-sample-sizeBACKGROUND_SAMPLE_SIZE][--cell-type-keyCELL_TYPE_KEY][--alphaALPHA][--epochsEPOCHS][--latent-dimLATENT_DIM][--n-per-cell-typeN_PER_CELL_TYPE]modesource_typeArgumentsmodeWhat task/mode is pollock to perform. Valid arguments are:trainpredictexplainsource_typeInput source type. Possible values are: from_seurat, from_10x, from_scanpy.module_filepathIf in prediction mode, this is the filepath to module to use for classification. Pretrained modules can be downloaded herehttps://zenodo.org/record/5155939#.YQqxbxNKi-YIf in training mode, this is the filepath where pollock will save the trained module.If in explain mode, this is the filepath to the module to use to explain the given pollock predictions.mode specific arguments--seurat-rds-filepath SEURAT_RDS_FILEPATHA saved Seurat RDS object to use as input. Raw RNA-seq (i.e. not normalized) countsmustbe stored in @assays$RNA@counts. Note that this is where raw rna-seq counts will be stored by most Seurat single cell workflows by default.--scanpy-h5ad-filepath SCANPY_H5AD_FILEPATHA saved .h5ad file to use as input. scanpy expression matrix (.X attribute in the anndata object) must be raw expression counts (i.e. not normalized)--counts-10x-filepath COUNTS_10X_FILEPATHCan only be used with predict mode. Results of 10X cellranger run to be used for classification. There are two options for inputs: 1) the mtx count directory (typically at outs/raw_feature_bc_matrix), and 2) the .h5 file (typically at outs/raw_feature_bc_matrix.h5).specific to train mode--cell-type-key CELL_TYPE_KEYThe key to use for training the pollock module. The key can be one of the following: 1) A string representing a column in the metadata of the input seurat object or .obs attribute of the scanpy anndata object, or 2) filepath to a .txt file where each line is a cell type label. The number of lines must be equal to the number of cells in the input object. The cell types must also be in the same order as the cells in the input object. By default if the input is a Seurat object pollock will use cell type labels in @active.ident, or if the input is a scanpy anndata object pollock will use the label in .obs["leiden"].--alpha ALPHAThis parameter controls how regularized the VAE is. .0001 is the default. If you increase alpha the cell embeddings are typically more noisy, but also more generalizable. If you decrease alpha the cell embeddings are typically less noisy, but also less generalizable.--epochs EPOCHSNumber of epochs to train the neural net for. Default is 20.--latent-dim LATENT_DIMSize of hidden layer in the B-VAE. Default is 25.--n-per-cell-type N_PER_CELL_TYPEThe number of cells per cell type that should be included in the training dataset. Typically this number will be somewhere between 500-2000. The default is 500. If you have a particular cell type in your dataset that has a low cell count it is usually a good idea not to increase n_per_cell_type too much. A good rule of thumb is that n_per_cell_type should be no greater than the minimum cell type count * 10.optional arguments specific to predict mode--min-genes-per-cell MIN_GENES_PER_CELLThe minimun number of genes expressed in a cell in order for it to be classified. Only used in 10x mode--txt-output TXT_OUTPUTIf included output will be written to a tab-seperated .txt file. Otherwise output will be saved in the metadata of the input seurat object (.rds) or scanpy anndata object (.h5ad)--output-prefix OUTPUT_PREFIXFilepath prefix to write output file. Extension will be dependent on the inclusion of --output-txt argument. By default the extension will be the same as the input object type. Default value is "output"specific to explain mode--explain-filepath EXPLAIN_FILEPATHFilepath to seurat .rds object or scanpy .h5ad anndata object containing cells to be explained. Expression data must be raw counts (i.e. unnormalized). Larger numbers of cells to explain will mean a longer run time. For reference, running ~100 cells with a background sample size of ~100 cells results in a runtime of approximately 15 minutes. Path to predicted cell type labels is specified by the --predicted-key--background-filepath BACKGROUND_FILEPATHFilepath to seurat .rds object or scanpy .h5ad anndata object containing cells to use for background samples in model explaination. Expression data must be raw counts (i.e. unnormalized). This object will be sampled to --background-sample-size cells. See --background-sample-size for more details.optional arguments specific to explain mode--predicted-key PREDICTED_KEYThe key holding pollock predictiosn to use for explaining the given input data. The key can be one of the following: 1) A string representing a column in the metadata of the input seurat object or the .obs attribute of the scanpy anndata object, or 2) filepath to a .txt file where each line is a cell typeexample basic usagepredict modeAn example of cell type prediction on a Seurat .RDS objectpollockpredictfrom_seurat--module-filepath<path_to_module_directory>--seurat-rds-filepath<filepath_to_RDS_object>--output-prefixoutputAn example of cell type prediction on a Seurat .RDS object, but writing to a txt file instead of an RDS objectpollockpredictfrom_seurat--module-filepath<path_to_module_directory>--seurat-rds-filepath<filepath_to_RDS_object>--output-prefixoutput--txt-outputAn example of cell type prediction on a scanpy .h5ad objectpollockpredictfrom_scanpy--module-filepath<path_to_module_directory>--scanpy-h5ad-filepath<filepath_to_scanpy_h5ad>--output-prefixoutputAn example of cell type prediction on cellranger outputpollockpredictfrom_10x--module-filepath<path_to_module_directory>--counts-10x-filepath</filepath/to/cellranger/outs/raw_feature_bc_matrix>--output-prefixoutputtrain modeAn example of training a model on a Seurat .RDS object that has cell type labels in @active.idents slot. Note this is where cell type labels are typically stored in Seurat workflows.pollocktrainfrom_seurat--module-filepath<path_to_write_output_module>--seurat-rds-filepath<filepath_to_RDS_object>An example of training a model on a Seurat .RDS object that has cell type labels stored in a metadata column named "my_special_cell_types".pollocktrainfrom_seurat--module-filepath<path_to_write_output_module>--seurat-rds-filepath<filepath_to_RDS_object>--cell-type-keymy_special_cell_typesAn example of training a model on a Seurat .RDS object where cell type labels are in a file.pollocktrainfrom_seurat--module-filepath<path_to_write_output_module>--seurat-rds-filepath<filepath_to_RDS_object>--cell-type-key<filepath_to_cell_labels>An example of training a model on a Seurat .RDS object with custom model hyperparamterspollocktrainfrom_seurat--module-filepath<path_to_write_output_module>--seurat-rds-filepath<filepath_to_RDS_object>--alpha.0001--epochs20--latent-dim25--n-per-cell-type500An example of training a model on a scanpy .h5ad object that has cell type labels stored in a column in .obs named "my_special_cell_types".pollocktrainfrom_scanpy--module-filepath<path_to_write_output_module>--scanpy-h5ad-filepath<filepath_to_h5ad_object>--cell-type-keymy_special_cell_typesexplain modeNote: explain mode can have excessive runtimes for large numbers of cells, so we recommend downsampling the number of cells in the inputs to <1k cells for faster runtimes.The explain object contains cells to be explained, the background arguments contains cells to be sampled as background.An example of explaining a model for a Seurat .RDS object that has cell type labels in @active.idents slot. Note this is where cell type labels are typically stored in Seurat workflows.pollockexplainfrom_seurat--explain-filepath<path_to_explain_seurat_object>--background-filepath<path_to_background_seurat_object>--module-filepath<path_to_pollock_module>--output-prefix<path_to_write_output>An example of explaining a model on a Scanpy .h5ad object that has cell type labels in column named 'cell_type' in .obs dataframe.pollockexplainfrom_scanpy--explain-filepath<path_to_explain_h5ad>--background-filepath<path_to_background_h5ad>--module-filepath<path_to_pollock_module>--predicted-keycell_type--output-prefix<path_to_write_output>DockerDocker images are available for Pollock. To pull the latest Pollock docker image run the following:dockerpullestorrs/pollock-cpu:0.1.2example basic usage of comand line tool within a docker containerWhen using docker, the input and ouput file directories need to be mounted as a volume using the docker -v argument.Below is an example of predicting cell types from within a docker container. Sections outlined by <> need to be replaced. Note file and directory paths in the -v flag must be absolute. For more examples of how the pollock command line tool is used see the above usage examples.dockerrun-v</path/to/directory/with/seurat/rds>:/inputs-v</path/to/output/directory>:/outputs-v</path/to/modules/directory/>:/modules-testorrs/pollock-cpu:0.1.2pollockpredictfrom_seurat--module-filepath/modules/<module_name>--seurat-rds-filepath/inputs/<name_of_seurat_rds_file>--output-prefix/outputs/outputTestingTo run Pollock tests navigate to the tests/ directory and runpytest-vvtest_pollock.py