Package: sc-instant
InSTAnTInSTAnTis a toolkit to identify gene pairs which are d-colocalized
from single molecule measurement data e.g. MERFISH or SeqFISH. A gene
pair is d-colocalized when their transcripts are within distance d
across many cells.This repository contains implementation of PP Test and CPB test and demo
on a U2OS dataset. The dataset can be downloaded from here (Moffit et
al., 2016, PNAS ) -http://zhuang.harvard.edu/MERFISHData/data_for_release.zipUPDATE: Added support for AnnData input/output.We recommend using ourenvironment.ymlfile to create a new conda environment to avoid issues with package incompatibility.conda env create -f environment.ymlThis will create a new conda environment with the nameinstantand has all dependancies installed.First we will initialise the Instant class object. This object will allow us to calculate the proximal pairs and find global colocalized genes. The primary argument isthreadswhich controls the number of threads the program uses. If you run into memory issues with the default settings, we suggest setting theprecision_modetolow. The argumentsmin_intensityandmin_areaare used only for MERFISH data preprocessing and can be skipped otherwise.obj = Instant(threads = threads, precision_mode = 'high', min_intensity = 10**0.75, min_area = 3)To load MERFISH data, we use the functionpreprocess_and_load_data().preprocess_and_load_data()is used to preprocess and load MERFISH dataonly. The final data is stored as a pandas DataFrame and the following columns['gene', 'uID', 'absX', 'absY']. Below is an example table for a 2D data (if the data is 3D,absZcolumn is also expected) -geneuIDabsXabsYAKAP112-1401.666-2956.618SIPA1L33-1411.692-2936.609THBS1925-764.6989-1604.828obj.preprocess_and_load_data(expression_data = f'data/u2os/rep3/data.csv', barcode_data = 'data/u2os/codebook.csv')If the data has been preprocessed, we can load it like below.obj.load_preprocessed_data(data = f'data/u2os_new/data_processed.csv')Since, subcellular spatial transcriptomic data is generally present as.csvfile containing all the transcripts, the primary input format is.csv. However, we have included the ability to format the data into an AnnData object and save the results of the subsequent analysis in that object. We convert the input file to an AnnData object and save it with the same name into the same directory. All the subsequent analysis will be updated and saved into the same file. We also provide the functionality to save any of the results seperately as individual files.Note- The function also supports loading an AnnData object. Currently, we only accept files in.h5adformat. Since, Anndata objects are not natively designed for subcellular datasets, we expect the.csvfile containing the transcript information to be present inadata.uns['transcripts']. If you wish to run differential colocalization, cell type labels are required, which are expected inadata.obswhile for spatial modulation, cell locations are required, which are expected inadata.uns['cell_locations']. If these files are not present in the AnnData object, they must be supplied to the specific funtions seperately. If an AnnData object is provided during this loading, all the subsequent outputs are saved and updated in the specified file.obj.load_preprocessed_data(data = f'data/u2os_new/data.h5ad')The dataframe is loaded in the object variabledfand can be accessed throughobj.df. After the data has been loaded, we can calculate each cell's proximal gene pairs using therun_ProximalPairs()function. The following arguments are used byrun_ProximalPairs()distance_threshold:(Integer)Distance threshold at which to consider 2 genes proximal.min_genecount:(Integer)Minimum number of transcripts in each cell.pval_matrix_name:(String)(Optional)if provided saves pvalue matrix using pickle at the input path.gene_count_name:(String)(Optional)if provided saves gene expression count matrix using pickle at the input path.obj.run_ProximalPairs(distance_threshold = 4, min_genecount = 20, 
    pval_matrix_name = f"data/u2os/rep3/rep3_pvals.pkl", 
    gene_count_name = f"data/u2os/rep3/rep3_gene_count.pkl")Proximal Pairs calculation has 3 variants -run_ProximalPairs()- Designed for 2D subcellular spatial transcriptomics data.run_ProximalPairs3D()- Designed for 3D subcellular spatial transcriptomics data with continuos z-axis.run_ProximalPairs3D_slice()- Designed for 3D subcellular spatial transcriptomics data with discrete/sliced z-axis.
(Note- For AnnData objects. These results are stored inadata.uns['pp_test_pvalues'].)All subsequent analysis requirerun_ProximalPairs()to be run first to generate the p-value matrix for all cells.Next, we can use the output to find which gene pairs are significantly colocalized globally using therun_GlobalColocalization()function. The following arguments are used byrun_GlobalColocalization()alpha_cellwise:(Float)Pvalue signifcance threshold (>alpha_cellwise are converted to 1). Default = 0.05.min_transcript:(Float)Gene expression lower threshold. Default = 0.high_precision:(Boolean) High precision pvalue. Expect a longer compute time. Default = False.glob_coloc_name:(String)(Optional)if provided, saves the global colocalization matrix as a CSV at the input path.exp_coloc_name:(String)(Optional)if provided, saves the expected colocalization matrix as a CSV at the input path.unstacked_pvals_name:(String)(Optional)if provided, saves a more interpretable global colocalization matrix as a CSV at the input path.obj.run_GlobalColocalization(
    high_precision = False, 
    alpha_cellwise = 0.05,
    glob_coloc_name = f"data/u2os/rep3/global_colocalization.csv", 
    exp_coloc_name = f"data/u2os/rep3/expected_colocalization.csv", 
    unstacked_pvals_name = f"data/u2os/rep3/unstacked_global_pvals.csv")The final outputs are 3 CSV files -Global colocalization:  contains pairwise significance value of d-colocalizationgene5830417i10rikAatfAbcc1Abhd25830417i10rik10.31750611Aatf0.317506110.416612Abcc11110.055185Abhd210.4166120.0551850.744798Expected colocalization:  contains pairwise significance value of expected d-colocalizationgene5830417i10rikAatfAbcc1Abhd25830417i10rik1.0689861.9787190.6869940.999343Aatf1.9787194.8778251.699752.344674Abcc10.6869941.699750.7952510.857116Abhd20.9993432.3446740.8571161.358516Unstacked colocalization: contains pairwise significance value of d-colocalization in an interpretable formatg1g2gene_id1gene_id2p_val_condExpected colocColoc. cells(Threshold<0.05)Present cellsfrac_cellsPrpf8, Polr2aPrpf8Polr2a-2.5E-143.5632211121630.687117Col1a1, Fn1Col1a1Fn1-2.4E-145.3501361071790.597765Fbln2, Fn1Fbln2Fn1-1.7E-144.00996741770.418079Col1a1, Fbln2Col1a1Fbln2-1.7E-144.629268731770.412429Col1a1, BgnCol1a1Bgn-1.6E-145.360849711780.398876(Note- For AnnData objects, only the unstacked file is saved inadata.uns['cpb_results'].)Next, we will use InSTAnT's spatial modulation analyses to find spatially modulated gene pairs. We use therun_spatial_modulation()function for this. The following arguments are used byrun_spatial_modulation()inter_cell_distance:(Float)Maximum distance between cells at which they are considered proximal.cell_locations:(String)(Optional)Path to file contains locations for each cell. Should be in sorted order. If not provided, cell locations are expected to be provided inadata.uns['cell_locations']in the AnnData file specified during initialization.spatial_modulation_name:(String)(Optional)Path and name of the output Excel file.alpha:(Float)(Optional)p-value significance threshold (>alpha_cellwise are converted to 1). Default = 0.01.randomize:(Boolean)(Optional)Shuffle cell locations. Default = False.obj.run_spatial_modulation(f"data/u2os/rep3/cells_locations.csv", inter_cell_distance = 100, spatial_modulation_name = f"data/u2os/rep3/spatial_modulation.csv")Thecell_locationsfile should be a.csvfile with the uID of the cells in sorted order and as the index of the file. The next 2 columns should be the x and y position of that cell respectively. (if provided in the AnnData file during initialization, cell locations are expected inadata.uns['cell_locations'])uIDx_centroidy_centroid10545785.5785699.61810595757.255770.5710675781.675238.70610685784.0235177.07610695772.4065173.24710715757.6525815.921The output is an Excel file containing the gene pairs and the log-likelihood ratio of their spatial modulation. Below is an example -g1g2gene_id1gene_id2llrw_h1p_g_h1p_g_h0MALAT1, MALAT1MALAT1MALAT1113.20890.5240390.8032850.805375FASN, TLN1FASNTLN161.483020.4046230.8087790.808774COL5A1, THBS1COL5A1THBS158.721940.3916710.7291870.733704MALAT1, SRRM2MALAT1SRRM247.825710.3532540.4106440.409021COL5A1, FBN2COL5A1FBN247.626710.3481130.5747130.576769(Note- For AnnData objects. These results are stored inadata.uns['spatial_modulation'].)Lastly, we will calculate the cell type specificity of InSTAnT categorized d-colocalized gene pair. We call it Differential Colocalization and the functionrun_differentialcolocalization()is used for it. There are 3 different modes in which it can be run -1va: Compares colocalization for genes in the input cell type vs all other cell types.1v1: Compares colocalization for genes in the input cell type 1 vs input cell type 2.ava: Compares colocalization for genes for all cell types vs all other cell types.Arguments:cell_type:(String)Cell type to calculate differential colocalization for. Is ignored if mode == "ava".cell_labels:(String)(Optional)Path to file contains cell type for each cell. If not provided, cell labels are expected to be provided inadata.obsin the AnnData file specified during initialization.file_location:(String)(Optional)Directory in which to store output files. if mode == "ava", creates a new directory in this path to store all results.cell_type_2:(String)(Optional) Cell type 2 to calculate differential colocalization for. Required if mode == "1v1".mode:(String)Either "1va" (One cell type vs All cell types), "1v1" (One cell type vs one cell type) or "ava" (All cell types vs All cell types).alpha:(Float)(Optional)p-value significance threshold (>alpha_cellwise are converted to 1). Default = 0.01.alpha_dc: (Float) (Optional) pvalue signifcance threshold for unconditional differential colocalization. Default = 5e-6.folder_name: *(String)(Optional)if mode == "ava", folder name inside the specified path in which to store results for each cell type. Default = "differential_colocalization".obj.run_differentialcolocalization(cell_type = None, mode = "a2a", 
     cell_labels = f"data/u2os/rep3/cell_labels.csv", 
     file_location = f"data/u2os/rep3/",
     folder_name = "differential_colocalization")Thecell_labels.csvfile must have 2 columnsuIDandcell_typewhich denote the cell number/ID and the type of the cell respectively.uIDcell_type83434278343127834472774692791749278313127The function will create a folder namedfolder_namein thefile_locationlocation. The folder will contain a folder for each of the cell types selected to be analyzed and for each cell type contains a single file will all relevant outputs.{cell_type}_unstacked.csvg1,g2p_uncondp_cond_g1p_cond_g2p_g1_expressionp_g2_expressionmin_g1g2g1_rankg2_rankmin_rankSlc17a6, Syt42.27E-756.75E-086.67E-645.8E-3041.48E-175.8E-3041221Cbln1, Slc17a62.58E-586.29E-145.01E-122.1E-1315.8E-3045.8E-304211Gabra1, Slc17a61.01E-536.75E-455.89E-064.39E-135.8E-3045.8E-3042711Cbln1, Gpr1656.64E-391.5E-086.27E-432.1E-1310.9996422.1E-1312952Cbln1, Syt44.29E-361.55E-101.35E-322.1E-1311.48E-172.1E-1312222(Note- For AnnData objects. These results are stored inadata.uns['differential_colocalization'].adata.uns['differential_colocalization']is a dictionary with keys based on the analysis done. For1vaandava, the key is{cell_type}and for1v1, the key is{cell_type)_vs_{cell_type_2}).