Package: blksheep
BlackSheepA tool for differential extreme-value analysisInstallationWith pippipinstallblksheepWith condacondainstall-cbiocondablksheepRequirements (automatically taken care of with pip and conda)pandasnumpymatplotlibseabornscipyscikit-learnstatsmodelsDocumentationhttps://blacksheep.readthedocs.io/en/latest/index.htmlUsageIn pythonimportblacksheep# Read in datavalues_file=''#insert values file hereannotations_file=''#insert annotations file herevalues=deva.read_in_values(values_file)annotations=deva.read_in_values(annotations_file)# Binarize annotation columnsannotations=blacksheep.binarize_annotations(annotations)# Run outliers comparative analysisoutliers,qvalues=blacksheep.deva(values,annotations,save_outlier_table=True,save_qvalues=True,save_comparison_summaries=True)# Pull out resultsqvalues_table=qvalues.dfvis_table=outliers.frac_table# Make heatmaps for significant genesforcolinannotations.columns:axs=blacksheep.plot_heatmap(annotations,qvalues_table,col,vis_table,savefig=True)# Normalize valuesphospho=blacksheep.read_in_values('')#Fill in file hereprotein=blacksheep.read_in_values('')#Fill in file hereCommand line interfaceExampleblacksheepbinarizeannotations.tsv--output_prefixannotations_test
blacksheepdevavalues.csvannotations_test.binarized.tsv--output_prefixtest\--write_outlier_table--write_comparison_summaries--write_gene_list\--make_heatmapsFull helpJust make the outliers table:usage:blacksheepoutliers_table[-h][--output_prefixOUTPUT_PREFIX][--iqrsIQRS][--up_or_down{up,down}][--ind_sepIND_SEP][--do_not_aggregate][--write_frac_table]values

Takesatableofvaluesandconvertstoatableofoutliercounts.

positionalarguments:valuesFilepathtoinputvalues.Columnsmustbesamples,genesmustbesitesorgenes.Only.tsvand.csvaccepted.

optionalarguments:-h,--helpshowthishelpmessageandexit--output_prefixOUTPUT_PREFIXOutputprefixforwritingfiles.Defaultoutliers.--iqrsIQRSNumberofinterquartileranges(IQRs)aboveorbelowthemediantoconsideravalueanoutlier.Defaultis1.5IQRs.--up_or_down{up,down}Whethertolookforupordownoutliers.Choicesareupordown.Defaultup.--ind_sepIND_SEPIfsitelabelshaveaparentmolecule(e.g.agenenamesuchasATM)andasiteidentifier(e.g.S365)thisisthedelimiterbetweenthetwoelements.Defaultis---do_not_aggregateUseflagifyoudonotwanttosumoutliersbasedonsiteprefixes.--write_frac_tableUseflagifyouwanttowriteatablewithfractionofvaluespersite,persamplethatareoutliers.Willnotbewrittenbydefault.Usefulforvisualization.Binarize the columns in an annotations table.**Warning: do not include non-categorical columns, or columns you don't want binarized. You'll
end up with a huge un-wieldly table. **usage:blacksheepbinarize[-h][--output_prefixOUTPUT_PREFIX]annotations

Takesanannotationtablewheresomecolumnsmayhavemorethan2possible
values(notincludingempty/nullvalues)andoutputsanannotationtablewith
onlytwovaluesperannotation.Propagatesnullvalues.

positionalarguments:annotationsAnnotationtablewithsamplesasrowsandannotationlabelsascolumns.

optionalarguments:-h,--helpshowthishelpmessageandexit--output_prefixOUTPUT_PREFIXOutputprefixforwritingfiles.Defaultoutliers.Compare all the groups described in columns of an annotation table using outlier countsusage:blacksheepcompare_groups[-h][--output_prefixOUTPUT_PREFIX][--frac_filterFRAC_FILTER][--write_comparison_summaries][--iqrsIQRS][--up_or_down{up,down}][--write_gene_list][--make_heatmaps][--fdrFDR][--red_or_blue{red,blue}][--annotation_colorsANNOTATION_COLORS]outliers_tableannotations

Takesanannotationtableandoutliercounttable(outputofoutliers_table)andoutputsqvaluesfromastatisticaltestthatlooksforenrichmentof
outliervaluesineachgroupintheannotationtable.Foreachvalueineach
comparison,theqvaluetablewillhave1column,ifthereareanygenesinthatcomparison.

positionalarguments:outliers_tableTableofoutliercounts(outputofoutliers_table).Mustbe.tsvor.csvfile,withoutlierandnon-outliercountsascolumns,andgenes/sitesasrows.annotationsTableofannotations.Mustbe.csvor.tsv.Samplesasrowsandcomparisonsascolumns.Comparisonsmusthaveonlyuniquevalues(notincludingmissingvalues).Iftherearemoreoptionsthanthat,youcanusebinarizetopreparethetable.

optionalarguments:-h,--helpshowthishelpmessageandexit--output_prefixOUTPUT_PREFIXOutputprefixforwritingfiles.Defaultoutliers.--frac_filterFRAC_FILTERTheminimumfractionofsamplespergroupthatmusthaveanoutlierinagenetoconsiderthatgeneintheanalysis.Thisisusedtopreventahighnumberofoutliervaluesin1samplefromdrivingalowqvalue.Default0.3--write_comparison_summariesUseflagtowriteaseparatefileforeachcolumnintheannotationstable,withoutliercountsineachgroup,p-valuesandq-valuesineachgroup.--iqrsIQRSNumberofIQRsusedtodefineoutliersintheinputcounttable.Optional.--up_or_down{up,down}Whetherinputoutliertablerepresentsupordownoutliers.Neededforoutputfilelabels.Defaultup--write_gene_listUseflagtowritealistofsignificantlyenrichedgenesforeachvalueineachcomparison.Ifused,needanfdrthresholdaswell.--make_heatmapsUseflagtodrawaheatmapofsignficantlyenrichedgenesforeachvalueineachcomparison.Ifused,needanfdrthresholdaswell.--fdrFDRFDRcutofftouseforsignficantlyenrichedgenelistsandheatmaps.Default0.05--red_or_blue{red,blue}If--make_heatmapsiscalled,colorofvaluestodrawonheatmap.Defaultred.--annotation_colorsANNOTATION_COLORSFilewithcolormaptouseforannotationheaderif--make_heatmapsisused.Musthavea'value color'formatforeachvalueinannotations.Anyvaluenotrepresentedwillbeassignedanewcolor.Make heatmaps visualizing enriched genes for each group in an annotation tableusage:blacksheepvisualize[-h][--output_prefixOUTPUT_PREFIX][--annotations_to_showANNOTATIONS_TO_SHOW[ANNOTATIONS_TO_SHOW...]][--fdrFDR][--red_or_blue{red,blue}][--annotation_colorsANNOTATION_COLORS][--write_gene_list]comparison_qvaluesannotationsvisualization_tablecomparison_of_interest

Usedtomakecustomheatmapsfromsignificantgenes.

positionalarguments:comparison_qvaluesTableofqvalues,outputfromcompare_groups.Mustbe.csvor.tsv.Hasgenes/sitesasrowsandcomparisonvaluesascolumns.annotationsTableofannotationsusedtogenerateqvalues.visualization_tableValuestovisualizeinheatmap.Samplesascolumnsandgenes/sitesasrows.Usingoutlierfractiontableisrecommended,butoriginalvaluescanalsobeusedifnoaggregationwasused.comparison_of_interestNameofcolumninqvaluestablefromwhichtovisualizesignificantgenes.

optionalarguments:-h,--helpshowthishelpmessageandexit--output_prefixOUTPUT_PREFIXOutputprefixforwritingfiles.Defaultoutliers.--annotations_to_showANNOTATIONS_TO_SHOW[ANNOTATIONS_TO_SHOW...]Namesofcolumnsfromtheannotationtabletoshowintheheaderoftheheatmap.Defaultisallcolumns.--fdrFDRFDRthresholdtousetoselectgenestovisualize.Default0.05--red_or_blue{red,blue}Colorofvaluestodrawonheatmap.Defaultred.--annotation_colorsANNOTATION_COLORSFilewithcolormaptouseforannotationheader.Musthavealinewith'value color'formatforeachvalueinannotations.Anyvaluenotrepresentedwillbeassignedanewcolor.--write_gene_listUseflagtowritealistofsignificantlyenrichedgenesforeachvalueineachcomparison.Run the whole pipeline: call outliers, perform comparisons on all groups in an annotation table
, optionally make heatmaps for each group.usage:blacksheepdeva[-h][--output_prefixOUTPUT_PREFIX][--iqrsIQRS][--up_or_down{up,down}][--do_not_aggregate][--write_outlier_table][--write_frac_table][--ind_sepIND_SEP][--frac_filterFRAC_FILTER][--write_comparison_summaries][--fdrFDR][--write_gene_list][--make_heatmaps][--red_or_blue{red,blue}][--annotation_colorsANNOTATION_COLORS]valuesannotations

Runswholeoutlierspipeline.Hasoptionstooutputeverypossibleoutput.

positionalarguments:valuesFilepathtoinputvalues.Samplesarecolumnsandgenes/sitesarerows.Only.tsvand.csvaccepted.annotationsFilepathtoannotationvalues.Rowsaresamplenames,headerisdifferentannotations.e.g.mutationstatus.

optionalarguments:-h,--helpshowthishelpmessageandexit--output_prefixOUTPUT_PREFIXOutputprefixforwritingfiles.Defaultoutliers.--iqrsIQRSNumberofinter-quartileranges(IQRs)aboveorbelowthemediantoconsideravalueanoutlier.Defaultis1.5.--up_or_down{up,down}Whethertolookforupordownoutliers.Choicesareupordown.Defaultup.--do_not_aggregateUseflagifyoudonotwanttosumoutliersbasedonsiteprefixes.--write_outlier_tableUseflagtowriteatableofoutliercounts.--write_frac_tableUseflagifyouwanttowriteatablewithfractionofvaluespersitepersamplethatareoutliers.Usefulforcustomvisualization.--ind_sepIND_SEPIfsitelabelshaveaparentmolecule(e.g.agenenamesuchasATM)andasiteidentifier(e.g.S365)thisisthedelimiterbetweenthetwoelements.Defaultis---frac_filterFRAC_FILTERTheminimumfractionofsamplespergroupthatmusthaveanoutlierinagenetoconsiderthatgeneintheanalysis.Thisisusedtopreventahighnumberofoutliervaluesin1samplefromdrivingalowqvalue.Default0.3--write_comparison_summariesUseflagtowriteaseparatefileforeachcolumnintheannotationstable,withoutliercountsineachgroup,p-valuesandq-valuesineachgroup.--fdrFDRFDRthresholdtousetoselectgenestovisualize.Default0.05--write_gene_listUseflagtowritealistofsignificantlyenrichedgenesforeachvalueineachcomparison.--make_heatmapsUseflagtodrawaheatmapofsignificantlyenrichedgenesforeachvalueineachcomparison.Ifused,needanfdrthresholdaswell.--red_or_blue{red,blue}Colorofvaluestodrawonheatmap.Defaultred.--annotation_colorsANNOTATION_COLORSFilewithcolormaptouseforannotationheader.Musthavealinewith'value color'formatforeachvalueinannotations.Anyvaluenotrepresentedwillbeassignedanewcolor.For finding the value differences that cannot be explained by a different data level. For example
, this can be used to find out variation due to differential phosphorylation (phospho as
target_values) not due to protein abundance variation (protein as normalizer_values).Warning: Row IDs between the two tables must matchusage:blacksheepnormalize[-h][--ind_sepIND_SEP][--output_prefixOUTPUT_PREFIX]target_valuesnormalizer_values

Takesatargettableandanormalizertable,andreturnsanormalizedtarget
table.Buildsaregularizedlinearmodelforeachlineinthetargettable
usingthematchingrowIDinthenormalizertable,andfindstheresidualsof
thatmodelforeachvalue.forexample,thiscouldbeusedtonormalize
phospho-peptidedatabyproteinabundancedata;resultingvalueswillreflect
onlyabundancedifferencesduetophosphorylationchanges,notpeptide
abundances.AnotherusecouldbenormalizingRNAbyCNA.

positionalarguments:target_valuesTableofvaluestobenormalized.Sites/genesasrows,samplesascolumns.Rowidentifiersmustbeunique.normalizer_valuesTableofvaluestousefornormalization.Sites/genesasrows,samplesascolumns.Rowidentifiersmustbeunique,andmustmatchthepre-ind_seppartofthetargetvaluesidentifiers.

optionalarguments:-h,--helpshowthishelpmessageandexit--ind_sepIND_SEPSeparatorusedinindexiftargetissitespecific.RowIDsbeforeind_sepinthetargetmustmatchtherowIDsinnormalizer_values.IfrowIDsalreadymatch,leaveblank.--output_prefixOUTPUT_PREFIXPrefixforoutputfile.Suffixwillbe'.normalized.tsv'For a more thorough vignette, refer to oursupplementary notebooks