Package: roux
rouxConvenience functions in Python.Examples·Explore the APIExamplesInstallationpip install roux              # with basic dependencies  
pip install roux[all]         # with all the additional dependencies (recommended).With additional dependencies as required:pip install roux[viz]         # for visualizations e.g. seaborn etc.
pip install roux[data]        # for data operations e.g. reading excel files etc.
pip install roux[stat]        # for statistics e.g. statsmodels etc.
pip install roux[fast]        # for faster processing e.g. parallelization etc.
pip install roux[workflow]    # for workflow operations e.g. omegaconf etc.
pip install roux[interactive] # for interactive operations in jupyter notebook e.g. watermark, icecream etc.Command-line usage🗺️ Read configuration.roux read-config path/to/file🗺️ Read metadata.roux read-metadata path/to/file📁 Find the latest and the oldest file in a list.roux read-ps list_of_paths💾 Backup a directory with a timestamp (ISO).roux backup path/to/directory⭐ Remove *'s from a jupyter notebook'.roux removestar path/to/notebookℹ️ Available command line tools and their usage.roux --helpHow to cite?Using BibTeX:@software{Dandage_roux,
  title   = {roux: Streamlined and Versatile Data Processing Toolkit},
  author  = {Dandage, Rohan},
  year    = {2023},
  url     = {https://zenodo.org/doi/10.5281/zenodo.2682670},
  version = {v0.1.0},
  note    = {The URL is a DOI link to the permanent archive of the software.},
}DOI link:, orUsing citation information fromCITATION.CFF file.Future directions, for which contributions are welcomeAddition of visualization function as attributes torddataframes.Refactoring of the workflow functions.Similar projectshttps://github.com/v-popov/helper_funcshttps://github.com/nficano/yakutilsAPImoduleroux.global_importsFor importing commonly used functions at the development phase.Usage: in interactive sessions (e.g. in jupyter notebooks) to facilitate faster code development.Note: Post-development, to remove *s from the code, use removestar (pip install removestar).removestar fileGlobal VariablesFONTSIZEPADmoduleroux.lib.dfFor processing individual pandas DataFrames/Seriesfunctionget_nameget_name(df1:DataFrame,cols:list=None,coff:float=2,out=None)Gets the name of the dataframe.Especially useful withingroupby+pandarellelcontext.Parameters:df1(DataFrame):  input dataframe.cols(list):  list groupby columns.coff(int):  cutoff of unique values to infer the name.out(str):  format of the output (list|not).Returns:name(tuple|str|list):  name of the dataframe.functionget_groupby_columnsget_groupby_columns(df_)Get the columns supplied togroupby.Parameters:df_(DataFrame):  input dataframe.Returns:columns(list):  list of columns.functionget_constantsget_constants(df1)Get the columns with a single unique value.Parameters:df1(DataFrame):  input dataframe.Returns:columns(list):  list of columns.functiondrop_unnamedcoldrop_unnamedcol(df)Deletes the columns with "Unnamed" prefix.Parameters:df(DataFrame):  input dataframe.Returns:df(DataFrame):  output dataframe.functiondrop_unnamedcoldrop_unnamedcol(df)Deletes the columns with "Unnamed" prefix.Parameters:df(DataFrame):  input dataframe.Returns:df(DataFrame):  output dataframe.functiondrop_levelcoldrop_levelcol(df)Deletes the potentially temporary columns names with "level" prefix.Parameters:df(DataFrame):  input dataframe.Returns:df(DataFrame):  output dataframe.functiondrop_constantsdrop_constants(df)Deletes columns with a single unique value.Parameters:df(DataFrame):  input dataframe.Returns:df(DataFrame):  output dataframe.functiondropby_patternsdropby_patterns(df1,patterns=None,strict=False,test=False,verbose=True)Deletes columns containing substrings i.e. patterns.Parameters:df1(DataFrame):  input dataframe.patterns(list):  list of substrings.test(bool):  verbose.Returns:df1(DataFrame):  output dataframe.functionflatten_columnsflatten_columns(df:DataFrame,sep:str=' ',**kws)→DataFrameMulti-index columns to single-level.Parameters:df(DataFrame):  input dataframe.sep(str):  separator within the joined tuples (' ').Returns:df(DataFrame):  output dataframe.Keyword Arguments:kws(dict):  parameters provided tocoltuples2strfunction.functionlower_columnslower_columns(df)Column names of the dataframe to lower-case letters.Parameters:df(DataFrame):  input dataframe.Returns:df(DataFrame):  output dataframe.functionrenameby_replacerenameby_replace(df:DataFrame,replaces:dict,ignore:bool=True,**kws)→DataFrameRename columns by replacing sub-strings.Parameters:df(DataFrame):  input dataframe.replaces(dict|list):  from->to format or list containing substrings to remove.ignore(bool):  if True, not validate the successful replacements.Returns:df(DataFrame):  output dataframe.Keyword Arguments:kws(dict):  parameters provided toreplacemanyfunction.functionclean_columnsclean_columns(df:DataFrame)→DataFrameStandardise columns.Steps:  1. Strip flanking white-spaces.  2. Lower-case letters.Parameters:df(DataFrame):  input dataframe.Returns:df(DataFrame):  output dataframe.functioncleanclean(df:DataFrame,cols:list=[],drop_constants:bool=False,drop_unnamed:bool=True,verb:bool=False)→DataFrameDeletes potentially temporary columns.Steps:  1. Strip flanking white-spaces.  2. Lower-case letters.Parameters:df(DataFrame):  input dataframe.drop_constants(bool):  whether to delete the columns with a single unique value.drop_unnamed(bool):  whether to delete the columns with 'Unnamed' prefix.verb(bool):  verbose.Returns:df(DataFrame):  output dataframe.functioncompresscompress(df1,coff_categories=20,test=False)Compress the dataframe by converting columns containing strings/objects to categorical.Parameters:df1(DataFrame):  input dataframe.coff_categories(int):  if the number of unique values are less than cutoff the it will be converted to categories.test(bool):  verbose.Returns:df1(DataFrame):  output dataframe.functionclean_compressclean_compress(df,kws_compress={},**kws_clean)cleanandcompressthe dataframe.Parameters:df(DataFrame):  input dataframe.kws_compress(int):  keyword arguments for thecompressfunction.test(bool):  verbose.Keyword Arguments:kws_clean(dict):  parameters provided tocleanfunction.Returns:df1(DataFrame):  output dataframe.See Also:cleancompressfunctioncheck_nacheck_na(df,subset=None,out=True,perc=False,log=True)Number of missing values in columns.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.out(bool):  output, else not which can be applicable in chained operations.Returns:ds(Series):  output stats.functionvalidate_no_navalidate_no_na(df,subset=None)Validate no missing values in columns.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.perc(bool):  output percentages.Returns:ds(Series):  output stats.functionassert_no_naassert_no_na(df,subset=None)Assert that no missing values in columns.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.perc(bool):  output percentages.Returns:ds(Series):  output stats.functionto_strto_str(data,log=False)functioncheck_nuniquecheck_nunique(df:DataFrame,subset:list=None,groupby:str=None,perc:bool=False,auto=False,out=True,log=True)→SeriesNumber/percentage of unique values in columns.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.perc(bool):  output percentages.Returns:ds(Series):  output stats.functioncheck_inflationcheck_inflation(df1,subset=None)Occurances of values in columns.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.Returns:ds(Series):  output stats.functioncheck_dupscheck_dups(df,subset=None,perc=False,out=True)Check duplicates.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.perc(bool):  output percentages.Returns:ds(Series):  output stats.functioncheck_duplicatedcheck_duplicated(df,**kws)Check duplicates (alias ofcheck_dups)functionvalidate_no_dupsvalidate_no_dups(df,subset=None)Validate that no duplicates.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.functionvalidate_no_duplicatesvalidate_no_duplicates(df,subset=None)Validate that no duplicates (alias ofvalidate_no_dups)functionassert_no_dupsassert_no_dups(df,subset=None)Assert that no duplicatesfunctionvalidate_densevalidate_dense(df01:DataFrame,subset:list=None,duplicates:bool=True,na:bool=True,message=None)→DataFrameValidate no missing values and no duplicates in the dataframe.Parameters:df01(DataFrame):  input dataframe.subset(list):  list of columns.duplicates(bool):  whether to check duplicates.na(bool):  whether to check na.message(str):  error messagefunctionassert_denseassert_dense(df01:DataFrame,subset:list=None,duplicates:bool=True,na:bool=True,message=None)→DataFrameAlias ofvalidate_dense.Notes:to be deprecated in future releases.functionclassify_mappingsclassify_mappings(df1:DataFrame,subset,clean:bool=False)→DataFrameClassify mappings between items in two columns.Parameters:df1(DataFrame):  input dataframe.col1(str):  column #1.col2(str):  column #2.clean(str):  drop columns with the counts.Returns:(pd.DataFrame):  output.functioncheck_mappingscheck_mappings(df:DataFrame,subset:list=None,out=True)→DataFrameMapping between items in two columns.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.out(str):  format of the output.Returns:ds(Series):  output stats.functionassert_1_1_mappingsassert_1_1_mappings(df:DataFrame,subset:list=None)→DataFrameValidate that the papping between items in two columns is 1:1.Parameters:df(DataFrame):  input dataframe.subset(list):  list of columns.out(str):  format of the output.functionget_mappingsget_mappings(df1:DataFrame,subset=None,keep='all',clean=False,cols=None)→DataFrameClassify the mapapping between items in two columns.Parameters:df1(DataFrame):  input dataframe.subset(list):  list of columns.keep(str):  type of mapping (1:1|1:m|m:1).clean(bool):  whether remove temporary columns.cols(list):  alias ofsubset.Returns:df(DataFrame):  output dataframe.functionto_map_binaryto_map_binary(df:DataFrame,colgroupby=None,colvalue=None)→DataFrameConvert linear mappings to a binary mapParameters:df(DataFrame):  input dataframe.colgroupby(str):  name of the column for groupby.colvalue(str):  name of the column containing values.Returns:df1(DataFrame):  output dataframe.functioncheck_intersectionscheck_intersections(df:DataFrame,colindex=None,colgroupby=None,plot=False,**kws_plot)→DataFrameCheck intersections. Linear dataframe to is converted to a binary map and then to a series usinggroupby.Parameters:df(DataFrame):  input dataframe.colindex(str):  name of the index column.colgroupby(str):  name of the groupby column.plot(bool):  plot or not.Returns:ds1(Series):  output Series.Keyword Arguments:kws_plot(dict):  parameters provided to the plotting function.functionget_totalsget_totals(ds1)Get totals from the output ofcheck_intersections.Parameters:ds1(Series):  input Series.Returns:d(dict):  output dictionary.functionfilter_rowsfilter_rows(df,d,sign='==',logic='and',drop_constants=False,test=False,verbose=True)Filter rows using a dictionary.Parameters:df(DataFrame):  input dataframe.d(dict):  dictionary.sign(str):  condition within mappings ('==').logic(str):  condition between mappings ('and').drop_constants(bool):  to drop the columns with single unique value (False).test(bool):  testing (False).verbose(bool):  more verbose (True).Returns:df(DataFrame):  output dataframe.functionget_boolsget_bools(df,cols,drop=False)Columns to bools. One-hot-encoder (get_dummies).Parameters:df(DataFrame):  input dataframe.cols(list):  columns to encode.drop(bool):  drop thecols(False).Returns:df(DataFrame):  output dataframe.functionagg_boolsagg_bools(df1,cols)Bools to columns. Reverse of one-hot encoder (get_dummies).Parameters:df1(DataFrame):  input dataframe.cols(list):  columns.Returns:ds(Series):  output series.functionmelt_pairedmelt_paired(df:DataFrame,cols_index:list=None,suffixes:list=None,cols_value:list=None,clean:bool=False)→DataFrameMelt a paired dataframe.Parameters:df(DataFrame):  input dataframe.cols_index(list):  paired index columns (None).suffixes(list):  paired suffixes (None).cols_value(list):  names of the columns containing the values (None).Notes:Partial melt melts selected columnscols_value.Examples:Paired parameters:  cols_value=['value1','value2'],  suffixes=['gene1','gene2'],functionget_chunksget_chunks(df1:DataFrame,colindex:str,colvalue:str,bins:int=None,value:str='right')→DataFrameGet chunks of a dataframe.Parameters:df1(DataFrame):  input dataframe.colindex(str):  name of the index column.colvalue(str):  name of the column containing values [0-100]bins(int):  number of bins.value(str):  value to use as the name of the chunk ('right').Returns:ds(Series):  output series.functionsample_near_quantilessample_near_quantiles(data:DataFrame,col:str,n:int,clean:bool=False)Get rows with values closest to the quantiles.functionget_groupget_group(groups,i:int=None,verbose:bool=True)→DataFrameGet a dataframe for a group out of thegroupbyobject.Parameters:groups(object):  groupby object.i(int):  index of the group. default None returns the largest group.verbose(bool):  verbose (True).Returns:df(DataFrame):  output dataframe.Notes:Useful for testinggroupby.functiongroupby_samplegroupby_sample(df:DataFrame,groupby:list,i:int=None,**kws_get_group)→DataFrameSamples a group (similar to .sample)Parameters:df(pd.DataFrame):  input dataframe.groupby(list):  columns to group by.i(int):  index of the group. default None returns the largest group.Keyword arguments: keyword parameters provided to theget_groupfunctionReturns:pd.DataFramefunctiongroupby_agg_nestedgroupby_agg_nested(df1:DataFrame,groupby:list,subset:list,func:dict=None,cols_value:list=None,verbose:bool=False,**kws_agg)→DataFrameAggregate serially from the lower level subsets to upper level ones.Parameters:df1(pd.DataFrame):  input dataframe.groupby(list):  groupby columns i.e. list of columns to be used as ids in the output.subset(list):  nested groups i.e. subsets.func(dict):  map betweek columns with value to aggregate and the function for aggregation.cols_value(list):  columns with value to aggregate, (optional).verbose(bool):  verbose.Keyword arguments:kws_agg:  keyword arguments provided to pandas's.aggfunction.Returns:output dataframe with the aggregated values.functiongroupby_filter_fastgroupby_filter_fast(df1:DataFrame,col_groupby,fun_agg,expr,col_agg:str='temporary',**kws_query)→DataFrameGroupby and filter fast.Parameters:df1(DataFrame):  input dataframe.by(str|list):  column name/s to groupby with.fun(object):  function to filter with.how(str):  greater or less thancoff(>|<).coff(float):  cut-off.Returns:df1(DataFrame):  output dataframe.Todo:Deprecation ifpandas.core.groupby.DataFrameGroupBy.filteris faster.functioninfer_indexinfer_index(data:DataFrame,cols_drop=[],include=<class'object'>,exclude=None)→listInfer the index (id) of the table.functionto_multiindex_columnsto_multiindex_columns(df,suffixes,test=False)Single level columns to multiindex.Parameters:df(DataFrame):  input dataframe.suffixes(list):  list of suffixes.test(bool):  verbose (False).Returns:df(DataFrame):  output dataframe.functionto_rangesto_ranges(df1,colindex,colbool,sort=True)Ranges from boolean columns.Parameters:df1(DataFrame):  input dataframe.colindex(str):  column containing index items.colbool(str):  column containing boolean values.sort(bool):  sort the dataframe (True).Returns:df1(DataFrame):  output dataframe.TODO:compare with io_sets.bools2intervals.functionto_booleanto_boolean(df1)Boolean from ranges.Parameters:df1(DataFrame):  input dataframe.Returns:ds(Series):  output series.TODO:compare with io_sets.bools2intervals.functionto_catto_cat(ds1,cats,ordered=True)To series containing categories.Parameters:ds1(Series):  input series.cats(list):  categories.ordered(bool):  if the categories are ordered (True).Returns:ds1(Series):  output series.functionastype_catastype_cat(df1:DataFrame,col:str,cats:list)functionsort_valuesby_listsort_valuesby_list(df1:DataFrame,by:str,cats:list,by_more:list=[],**kws)Sort dataframe by custom order of items in a column.Parameters:df1(DataFrame):  input dataframe.by(str):  column.cats(list):  ordered list of items.Keyword parameters:kws(dict):  parameters provided tosort_values.Returns:df(DataFrame):  output dataframe.functionagg_by_orderagg_by_order(x,order)Get first item in the order.Parameters:x(list):  list.order(list):  desired order of the items.Returns:k:  first item.Notes:Used for sorting strings. e.g.damaging > other non-conserving > other conservingTODO:Convert categories to numbers and take minfunctionagg_by_order_countsagg_by_order_counts(x,order)Get the aggregated counts by order*.Parameters:x(list):  list.order(list):  desired order of the items.Returns:df(DataFrame):  output dataframe.Examples:df=pd.DataFrame({'a1':['a','b','c','a','b','c','d'], 'b1':['a1','a1','a1','b1','b1','b1','b1'],}) df.groupby('b1').apply(lambda df : agg_by_order_counts(x=df['a1'],  order=['b','c','a'],  ))functiongroupby_sort_valuesgroupby_sort_values(df:DataFrame,col_groupby:list,col_sortby:list,subset:list=None,col_subset:list=None,func:str='mean',ascending:bool=True)Sort groups.Parameters:df(DataFrame):  input dataframe.col_groupby(str|list):  column/s to groupby with.col_sortby(str|list):  column/s to sort values with.subset(list):  columns (None).col_subset(str):  column containing the subset (None).func(str):  aggregate function, provided to numpy ('mean').ascending(bool):  sort values ascending (True).Returns:df(DataFrame):  output dataframe.functiongroupby_sort_valuesgroupby_sort_values(df:DataFrame,col_groupby:list,col_sortby:list,subset:list=None,col_subset:list=None,func:str='mean',ascending:bool=True)Sort groups.Parameters:df(DataFrame):  input dataframe.col_groupby(str|list):  column/s to groupby with.col_sortby(str|list):  column/s to sort values with.subset(list):  columns (None).col_subset(str):  column containing the subset (None).func(str):  aggregate function, provided to numpy ('mean').ascending(bool):  sort values ascending (True).Returns:df(DataFrame):  output dataframe.functionswap_paired_colsswap_paired_cols(df_,suffixes=['gene1','gene2'])Swap suffixes of paired columns.Parameters:df_(DataFrame):  input dataframe.suffixes(list):  suffixes.Returns:df(DataFrame):  output dataframe.functionsort_columns_by_valuessort_columns_by_values(df:DataFrame,subset:list,suffixes:list=None,order:list=None,clean=False)→DataFrameSort the values in columns in ascending order.Parameters:df(DataFrame):  input dataframe.subset(list):  columns.suffixes(list):  suffixes.order(list):  ordered list.Returns:df(DataFrame):  output dataframe.Notes:In the output dataframe,sortedmeans values are sorted because gene1>gene2.functionmake_idsmake_ids(df:DataFrame,cols:list,ids_have_equal_length:bool,sep:str='--',sort:bool=False)→SeriesMake ids by joining string ids in more than one columns.Parameters:df(DataFrame):  input dataframe.cols(list):  columns.ids_have_equal_length(bool):  ids have equal length, if True faster processing.sep(str):  separator between the ids ('--').sort(bool):  sort the ids before joining (False).Returns:ds(Series):  output series.functionmake_ids_sortedmake_ids_sorted(df:DataFrame,cols:list,ids_have_equal_length:bool,sep:str='--',sort:bool=False)→SeriesMake sorted ids by joining string ids in more than one columns.Parameters:df(DataFrame):  input dataframe.cols(list):  columns.ids_have_equal_length(bool):  ids have equal length, if True faster processing.sep(str):  separator between the ids ('--').Returns:ds(Series):  output series.functionget_alt_idget_alt_id(s1:str,s2:str,sep:str='--')Get alternate/partner id from a paired id.Parameters:s1(str):  joined id.s2(str):  query id.Returns:s(str):  partner id.functionsplit_idssplit_ids(df1,col,sep='--',prefix=None)Split joined ids to individual ones.Parameters:df1(DataFrame):  input dataframe.col(str):  column containing the joined ids.sep(str):  separator within the joined ids ('--').prefix(str):  prefix of the individual ids (None).Return:df1(DataFrame):  output dataframe.functiondict2dfdict2df(d,colkey='key',colvalue='value')Dictionary to DataFrame.Parameters:d(dict):  dictionary.colkey(str):  name of column containing the keys.colvalue(str):  name of column containing the values.Returns:df(DataFrame):  output dataframe.functionlog_shape_changelog_shape_change(d1,fun='')Report the changes in the shapes of a DataFrame.Parameters:d1(dic):  dictionary containing the shapes.fun(str):  name of the function.functionlog_applylog_apply(df,fun,validate_equal_length=False,validate_equal_width=False,validate_equal_shape=False,validate_no_decrease_length=False,validate_no_decrease_width=False,validate_no_increase_length=False,validate_no_increase_width=False,*args,**kwargs)Report (log) the changes in the shapes of the dataframe before and after an operation/s.Parameters:df(DataFrame):  input dataframe.fun(object):  function to apply on the dataframe.validate_equal_length(bool):  Validate that the number of rows i.e. length of the dataframe remains the same before and after the operation.validate_equal_width(bool):  Validate that the number of columns i.e. width of the dataframe remains the same before and after the operation.validate_equal_shape(bool):  Validate that the number of rows and columns i.e. shape of the dataframe remains the same before and after the operation.Keyword parameters:args(tuple):  provided tofun.kwargs(dict):  provided tofun.Returns:df(DataFrame):  output dataframe.classlogReport (log) the changes in the shapes of the dataframe before and after an operation/s.TODO:Create the attribures (attr) using strings e.g. setattr.  import inspect  fun=inspect.currentframe().f_code.co_namemethod__init____init__(pandas_obj)methodcheck_dupscheck_dups(**kws)methodcheck_nacheck_na(**kws)methodcleanclean(**kws)methoddropdrop(**kws)methoddrop_duplicatesdrop_duplicates(**kws)methoddropnadropna(**kws)methodexplodeexplode(**kws)methodfilter_filter_(**kws)methodfilter_rowsfilter_rows(**kws)methodgroupbygroupby(**kws)methodjoinjoin(**kws)methodmeltmelt(**kws)methodmelt_pairedmelt_paired(**kws)methodmergemerge(**kws)methodpivotpivot(**kws)methodpivot_tablepivot_table(**kws)methodqueryquery(**kws)methodstackstack(**kws)methodunstackunstack(**kws)moduleroux.lib.dfsFor processing multiple pandas DataFrames/Seriesfunctionfilter_dfsfilter_dfs(dfs:list,cols:list,how:str='inner')→DataFrameFilter dataframes based items in the common columns.Parameters:dfs(list):  list of dataframes.cols(list):  list of columns.how(str):  how to filter ('inner')Returnsdfs(list):  list of dataframes.functionmerge_with_many_columnsmerge_with_many_columns(df1:DataFrame,right:str,left_on:str,right_ons:list,right_id:str,how:str='inner',validate:str='1:1',test:bool=False,verbose:bool=False,**kws_merge)→DataFrameMerge with many columns. For example, if ids in the left table can map to ids located in multiple columns of the right table.Parameters:df1(pd.DataFrame):  left table.right(pd.DataFrame):  right table.left_on(str):  column in the left table to merge on.right_ons(list):  columns in the right table to merge on.right_id(str):  column in the right dataframe with for example the ids to be merged.Keyword parameters:kws_merge:  to be supplied topandas.DataFrame.merge.Returns:Merged table.functionmerge_pairedmerge_paired(df1:DataFrame,df2:DataFrame,left_ons:list,right_on:list,common:list=[],right_ons_common:list=[],how:str='inner',validates:list=['1:1','1:1'],suffixes:list=None,test:bool=False,verb:bool=True,**kws)→DataFrameMerge uppaired dataframes to a paired dataframe.Parameters:df1(DataFrame):  paired dataframe.df2(DataFrame):  unpaired dataframe.left_ons(list):  columns of thedf1(suffixed).right_on(str|list):  column/s of thedf2(to be suffixed).common(str|list):  common column/s betweendf1anddf2(not suffixed).right_ons_common(str|list):  common column/s betweendf2to be used for merging (not to be suffixed).how(str):  method of merging ('inner').validates(list):  validate mappings for the 1st mapping betweendf1anddf2and 2nd one betweendf1+df2anddf2(['1:1','1:1']).suffixes(list):  suffixes to be used (None).test(bool):  testing (False).verb(bool):  verbose (True).Keyword Parameters:kws(dict):  parameters provided tomerge.Returns:df(DataFrame):  output dataframe.Examples:Parameters:how='inner',  left_ons=['gene id gene1','gene id gene2'], # suffixed  common='sample id', # not suffixed  right_on='gene id', # to be suffixed  right_ons_common=[], # not to be suffixedfunctionmerge_dfsmerge_dfs(dfs:list,**kws)→DataFrameMerge dataframes from left to right.Parameters:dfs(list):  list of dataframes.Keyword Parameters:kws(dict):  parameters provided tomerge.Returns:df(DataFrame):  output dataframe.Notes:For example, reduce(lambda x, y: x.merge(y), [1, 2, 3, 4, 5]) merges ((((1.merge(2)).merge(3)).merge(4)).merge(5)).functioncompare_rowscompare_rows(df1,df2,test=False,**kws)moduleroux.lib.dictFor processing dictionaries.functionhead_dicthead_dict(d,lines=5)functionsort_dictsort_dict(d1,by=1,ascending=True)Sort dictionary by values.Parameters:d1(dict):  input dictionary.by(int):  index of the value among the values.ascending(bool):  ascending order.Returns:d1(dict):  output dictionary.functionmerge_dictsmerge_dicts(l:list)→dictMerge dictionaries.Parameters:l(list):  list containing the dictionaries.Returns:d(dict):  output dictionary.TODOs: 1. In python>=3.9,merged = d1 | d2?functionmerge_dicts_deepmerge_dicts_deep(left:dict,right:dict)→dictMerge nested dictionaries. Overwrites left with right.Parameters:left(dict):  dictionary #1right(dict):  dictionary #2TODOs: 1. In python>=3.9,merged = d1 | d2?functionmerge_dict_valuesmerge_dict_values(l,test=False)Merge dictionary values.Parameters:l(list):  list containing the dictionaries.test(bool):  verbose.Returns:d(dict):  output dictionary.functionflip_dictflip_dict(d)switch values with keys and vice versa.Parameters:d(dict):  input dictionary.Returns:d(dict):  output dictionary.moduleroux.lib.googleProcessing files form google-cloud services.functionget_serviceget_service(service_name='drive',access_limit=True,client_config=None)Creates a google service object.:param service_name: name of the service e.g. drive :param access_limit: True is access limited else False :param client_config: custom client config ... :return: google service objectRef:https://developers.google.com/drive/api/v3/about-authfunctionget_serviceget_service(service_name='drive',access_limit=True,client_config=None)Creates a google service object.:param service_name: name of the service e.g. drive :param access_limit: True is access limited else False :param client_config: custom client config ... :return: google service objectRef:https://developers.google.com/drive/api/v3/about-authfunctionlist_files_in_folderlist_files_in_folder(service,folderid,filetype=None,fileext=None,test=False)Lists files in a google drive folder.:param service: service object e.g. drive :param folderid: folder id from google drive :param filetype: specify file type :param fileext: specify file extension :param test: True if verbose else False ... :return: list of files in the folderfunctionget_file_idget_file_id(p)functiondownload_filedownload_file(p=None,file_id=None,service=None,outd=None,outp=None,convert=False,force=False,test=False)Downloads a specified file.:param service: google service object :param file_id: file id as on google drive :param filetypes: specify file type :param outp: path to the ouput file :param test: True if verbose else FalseRef:https://developers.google.com/drive/api/v3/ref-export-formatsfunctionupload_fileupload_file(service,filep,folder_id,test=False)Uploads a local file onto google drive.:param service: google service object :param filep: path of the file :param folder_id: id of the folder on google drive where the file will be uploaded  :param test: True is verbose else False ... :return: id of the uploaded filefunctionupload_filesupload_files(service,ps,folder_id,**kws)functiondownload_drawingsdownload_drawings(folderid,outd,service=None,test=False)Download specific files: drawingsTODOs: 1. use download_filefunctionget_commentsget_comments(fileid,fields='comments/quotedFileContent/value,comments/content,comments/id',service=None)Get comments.fields: comments/  kind:  id:  createdTime:  modifiedTime:  author:  kind:  displayName:  photoLink:  me:  True  htmlContent:  content:  deleted:  quotedFileContent:  mimeType:  value:  anchor:  replies:  []functionsearchsearch(query,results=1,service=None,**kws_search)Google search.:param query: exact terms ... :return: dictfunctionget_search_stringsget_search_strings(text,num=5,test=False)Google search.:param text: string :param num: number of results :param test: True if verbose else False ... :return lines: listfunctionget_metadata_of_paperget_metadata_of_paper(file_id,service_drive,service_search,metadata=None,force=False,test=False)Get the metadata of a pdf document.functionshareshare(drive_service,content_id,share=False,unshare=False,user_permission=None,permissionId='anyoneWithLink'):params user_permission:      user_permission = {  'type': 'anyone',  'role': 'reader',  'email':'@' } Ref:https://developers.google.com/drive/api/v3/manage-sharingclassslidesmethodcreate_imagecreate_image(service,presentation_id,page_id,image_id)image less than 1.5 Mbmethodget_page_idsget_page_ids(service,presentation_id)moduleroux.lib.ioFor input/output of data files.functionread_zipread_zip(p:str,file_open:str=None,fun_read=None,test:bool=False)Read the contents of a zip file.Parameters:p(str):  path of the file.file_open(str):  path of file within the zip file to open.fun_read(object):  function to read the file.Examples:Settingfun_readparameter for reading tab-separated table from a zip file.from io import StringIO  ...  fun_read=lambda x: pd.read_csv(io.StringIO(x.decode('utf-8')),sep=' ',header=None),orfrom io import BytesIO  ...  fun_read=lambda x: pd.read_table(BytesIO(x)),functionto_zip_dirto_zip_dir(source,destination=None,fmt='zip')Zip a folder. Ref:https://stackoverflow.com/a/50381250/3521099functionto_zipto_zip(p:str,outp:str=None,func_rename=None,fmt:str='zip',test:bool=False)Compress a file/directory.Parameters:p(str):  path to the file/directory.outp(str):  path to the output compressed file.fmt(str):  format of the compressed file.Returns:outp(str):  path of the compressed file.functionto_dirto_dir(paths:dict,output_dir_path:str,rename_basename=None,force=False,test=False)functionget_versionget_version(suffix:str='')→strGet the time-based version string.Parameters:suffix(string):  suffix.Returns:version(string):  version.functionto_versionto_version(p:str,outd:str=None,test:bool=False,name:str=None,**kws:dict)→strRename a file/directory to a version.Parameters:p(str):  path.outd(str):  output directory.Keyword parameters:kws(dict):  provided toget_version.Returns:version(string):  version.TODOs: 1. Useto_dir.functionbackupbackup(p:str,outd:str=None,versioned:bool=False,suffix:str='',zipped:bool=False,move_only:bool=False,test:bool=True,verbose:bool=False,no_test:bool=False)Backup a directorySteps:  0. create version dir in outd  1. move ps to version (time) dir with common parents till the level of the version dir  2. zip or notParameters:p(str):  input path.outd(str):  output directory path.versioned(bool):  custom version for the backup (False).suffix(str):  custom suffix for the backup ('').zipped(bool):  whether to zip the backup (False).test(bool):  testing (True).no_test(bool):  no testing. Usage in command line (False).TODOs: 1. Useto_dir. 2. Option to remove dirs  find and move/zip  "find -regex ./_."  "find -regex ./test."functionread_urlread_url(url)Read text from an URL.Parameters:url(str):  URL link.Returns:s(string):  text content of the URL.functiondownloaddownload(url:str,path:str=None,outd:str=None,force:bool=False,verbose:bool=True)→strDownload a file.Parameters:url(str):  URL.path(str):  custom output path (None)outd(str):  output directory ('data/database').force(bool):  overwrite output (False).verbose(bool):  verbose (True).Returns:path(str):  output path (None)functionread_textread_text(p)Read a file.  To be called by other functionsArgs:p(str):  path.Returns:s(str):  contents.functionto_listto_list(l1,p)Save list.Parameters:l1(list):  input list.p(str):  path.Returns:p(str):  path.functionread_listread_list(p)Read the lines in the file.Args:p(str):  path.Returns:l(list):  list.functionread_listread_list(p)Read the lines in the file.Args:p(str):  path.Returns:l(list):  list.functionis_dictis_dict(p)functionread_dictread_dict(p,fmt:str='',apply_on_keys=None,**kws)→dictRead dictionary file.Parameters:p(str):  path.fmt(str):  format of the file.Keyword Arguments:kws(d):  parameters provided to reader function.Returns:d(dict):  output dictionary.functionto_dictto_dict(d,p,**kws)Save dictionary file.Parameters:d(dict):  input dictionary.p(str):  path.Keyword Arguments:kws(d):  parameters provided to export function.Returns:p(str):  path.functionpost_read_tablepost_read_table(df1:DataFrame,clean:bool,tables:list,verbose:bool=True,**kws_clean:dict)Post-reading a table.Parameters:df1(DataFrame):  input dataframe.clean(bool):  whether to applycleanfunction.  tables ()verbose(bool):  verbose.Keyword parameters:kws_clean(dict):  paramters provided to thecleanfunction.Returns:df(DataFrame):  output dataframe.functionread_tableread_table(p:str,ext:str=None,clean:bool=True,filterby_time=None,params:dict={},kws_clean:dict={},kws_cloud:dict={},check_paths:bool=True,tables:int=1,test:bool=False,verbose:bool=True,engine:str='fastparquet',**kws_read_tables:dict)Table/s reader.Parameters:- <b>`p`</b> (str):  path of the file. It could be an input for `read_ps`, which would include strings with wildcards, list etc.  
 - <b>`ext`</b> (str):  extension of the file (default: None meaning infered from the path). 
 - <b>`clean=(default`</b>: True). filterby_time=None). 
 - <b>`check_paths`</b> (bool):  read files in the path column (default:True).  
 - <b>`test`</b> (bool):  testing (default:False). 
 - <b>`params`</b>:  parameters provided to the 'pd.read_csv' (default:{}). For example 
 - <b>`params['columns']`</b>:  columns to read. 
 - <b>`kws_clean`</b>:  parameters provided to 'rd.clean' (default:{}). 
 - <b>`kws_cloud`</b>:  parameters for reading files from google-drive (default:{}). 
 - <b>`tables`</b>:  how many tables to be read (default:1). 
 - <b>`verbose`</b>:  verbose (default:True).Keyword parameters:
-kws_read_tables(dict):  parameters provided toread_tablesfunction. For example:
-to_col={colindex:  replaces_index}Returns:- <b>`df`</b> (DataFrame):  output dataframe.Examples:For reading specific columns only setparams=dict(columns=list).For reading many files, convert paths to a column with corresponding values:to_col={colindex: replaces_index}Reading a vcf file.  p='*.vcf|vcf.gz'  read_table(p,  params_read_csv=dict(  #compression='gzip',  sep='        ',comment='#',header=None,  names=replace_many(get_header(path,comment='#',lineno=-1),['#',' '],'').split('  '))  )functionget_logpget_logp(ps:list)→strInfer the path of the log file.Parameters:ps(list):  list of paths.Returns:p(str):  path of the output file.functionapply_on_pathsapply_on_paths(ps:list,func,replaces_outp:str=None,to_col:dict=None,replaces_index=None,drop_index:bool=True,colindex:str='path',filter_rows:dict=None,fast:bool=False,progress_bar:bool=True,params:dict={},dbug:bool=False,test1:bool=False,verbose:bool=True,kws_read_table:dict={},**kws:dict)Apply a function on list of files.Parameters:ps(str|list):  paths or string to infer paths usingread_ps.to_col(dict):  convert the paths to a column e.g. {colindex: replaces_index}func(function):  function to be applied on each of the paths.replaces_outp(dict|function):  infer the output path (outp) by replacing substrings in the input paths (p).filter_rows(dict):  filter the rows based on dict, usingrd.filter_rows.fast(bool):  parallel processing (default:False).progress_bar(bool):  show progress bar(default:True).params(dict):  parameters provided to thepd.read_csvfunction.dbug(bool):  debug mode on (default:False).test1(bool):  test on one path (default:False).kws_read_table(dict):  parameters provided to theread_tablefunction (default:{}).replaces_index(object|dict|list|str):  for example, 'basenamenoext' if path to basename.drop_index(bool):  whether to drop the index column e.g.path(default: True).colindex(str):  the name of the column containing the paths (default: 'path')Keyword parameters:kws(dict):  parameters provided to the function.Example:Function:   def apply_(p,outd='data/data_analysed',force=False):  outp=f"{outd}/{basenamenoext(p)}.pqt'  if exists(outp) and not force:  return  df01=read_table(p)  apply_on_paths(  ps=glob("data/data_analysed/*"),  func=apply_,  outd="data/data_analysed/",  force=True,  fast=False,  read_path=True,  )TODOs: Move out of io.functionread_tablesread_tables(ps:list,fast:bool=False,filterby_time=None,to_dict:bool=False,params:dict={},tables:int=None,**kws_apply_on_paths:dict)Read multiple tables.Parameters:ps(list):  list of paths.fast(bool):  parallel processing (default:False)filterby_time(str):  filter by time (default:None)drop_index(bool):  drop index (default:True)to_dict(bool):  output dictionary (default:False)params(dict):  parameters provided to thepd.read_csvfunction (default:{})tables:  number of tables (default:None).Keyword parameters:kws_apply_on_paths(dict):  parameters provided toapply_on_paths.Returns:df(DataFrame):  output dataframe.TODOs: Parameter to report the creation dates of the newest and the oldest files.functionto_tableto_table(df:DataFrame,p:str,colgroupby:str=None,test:bool=False,**kws)Save table.Parameters:df(DataFrame):  the input dataframe.p(str):  output path.colgroupby(str|list):  columns to groupby with to save the subsets of the data as separate files.test(bool):  testing on (default:False).Keyword parameters:kws(dict):  parameters provided to theto_manytablesfunction.Returns:p(str):  path of the output.functionto_manytablesto_manytables(df:DataFrame,p:str,colgroupby:str,fmt:str='',ignore:bool=False,kws_get_chunks={},**kws_to_table)Save many table.Parameters:df(DataFrame):  the input dataframe.p(str):  output path.colgroupby(str|list):  columns to groupby with to save the subsets of the data as separate files.fmt(str):  if '=' column names in the folder name e.g. col1=True.ignore(bool):  ignore the warnings (default:False).Keyword parameters:kws_get_chunks(dict):  parameters provided to theget_chunksfunction.Returns:p(str):  path of the output.TODOs:1. Change in default parameter:fmt='='.functionto_table_pqtto_table_pqt(df:DataFrame,p:str,engine:str='fastparquet',compression:str='gzip',**kws_pqt:dict)→strSave a parquet file.Parameters:df(pd.DataFrame):  table.p(str):  path.Keyword parameters: Parameters provided topd.DataFrame.to_parquet.Returns:functiontsv2pqttsv2pqt(p:str)→strConvert tab-separated file to Apache parquet.Parameters:p(str):  path of the input.Returns:p(str):  path of the output.functionpqt2tsvpqt2tsv(p:str)→strConvert Apache parquet file to tab-separated.Parameters:p(str):  path of the input.Returns:p(str):  path of the output.functionread_excelread_excel(p:str,sheet_name:str=None,kws_cloud:dict={},test:bool=False,**kws)Read excel fileParameters:p(str):  path of the file.sheet_name(str|None):  read 1st sheet if None (default:None)kws_cloud(dict):  parameters provided to read the file from the google drive (default:{})test(bool):  if False and sheet_name not provided, return all sheets as a dictionary, else if True, print list of sheets.Keyword parameters:kws:  parameters provided to the excel reader.functionto_excel_commentedto_excel_commented(p:str,comments:dict,outp:str=None,author:str=None)Add comments to the columns of excel file and save.Args:p(str):  input path of excel file.comments(dict):  map between column names and comment e.g. description of the column.outp(str):  output path of excel file. Defaults to None.author(str):  author of the comments. Defaults to 'Author'.TODOs: 1. Increase the limit on comments can be added to number of columns. Currently it is 26 i.e. upto Z1.functionto_excelto_excel(sheetname2df:dict,outp:str,comments:dict=None,save_input:bool=False,author:str=None,append:bool=False,adjust_column_width:bool=True,**kws)Save excel file.Parameters:sheetname2df(dict):  dictionary mapping the sheetname to the dataframe.outp(str):  output path.append(bool):  append the dataframes (default:False).comments(dict):  map between column names and comment e.g. description of the column.save_input(bool):  additionally save the input tables in text format.Keyword parameters:kws:  parameters provided to the excel writer.functioncheck_chunkscheck_chunks(outd,col,plot=True)Create chunks of the tables.Parameters:outd(str):  output directory.col(str):  the column with values that are used for getting the chunks.plot(bool):  plot the chunk sizes (default:True).Returns:df3(DataFrame):  output dataframe.moduleroux.libGlobal Variablesdfsetstrsysdfstextiodictfunctionto_classto_class(cls)Get the decorator to attach functions.Parameters:cls(class):  class object.Returns:decorator(decorator):  decorator object.References:https: //gist.github.com/mgarod/09aa9c3d8a52a980bd4d738e52e5b97afunctiondecoratordecorator(func)classrdroux-dataframe(.rd) extension.method__init____init__(pandas_obj)moduleroux.lib.setFor processing list-like sets.functionunionunion(l)Union of lists.Parameters:l(list):  list of lists.Returns:l(list):  list.functionunionunion(l)Union of lists.Parameters:l(list):  list of lists.Returns:l(list):  list.functionintersectionintersection(l)Intersections of lists.Parameters:l(list):  list of lists.Returns:l(list):  list.functionintersectionintersection(l)Intersections of lists.Parameters:l(list):  list of lists.Returns:l(list):  list.functionnunionnunion(l)Count the items in union.Parameters:l(list):  list of lists.Returns:i(int):  count.functionnintersectionnintersection(l)Count the items in intersetion.Parameters:l(list):  list of lists.Returns:i(int):  count.functioncheck_non_overlaps_withcheck_non_overlaps_with(l1:list,l2:list,out_count:bool=False,log=False)functionvalidate_overlaps_withvalidate_overlaps_with(l1,l2)functionassert_overlaps_withassert_overlaps_with(l1,l2,out_count=False)functionjaccard_indexjaccard_index(l1,l2)functiondropnadropna(x)Dropnp.nanitems from a list.Parameters:x(list):  list.Returns:x(list):  list.functionuniqueunique(l)Unique items in a list.Parameters:l(list):  input list.Returns:l(list):  list.Notes:The function can return list of lists if used inpandas.core.groupby.DataFrameGroupBy.aggcontext.functionlist2strlist2str(x,ignore=False)Returns string if single item in a list.Parameters:x(list):  listReturns:s(str):  string.functionunique_strunique_str(l,**kws)Unique single item from a list.Parameters:l(list):  input list.Returns:l(list):  list.functionnuniquenunique(l,**kws)Count unique items in a listParameters:l(list):  listReturns:i(int):  count.functionflattenflatten(l)List of lists to list.Parameters:l(list):  input list.Returns:l(list):  output list.functionget_altget_alt(l1,s)Get alternate item between two.Parameters:l1(list):  list.s(str):  item.Returns:s(str):  alternate item.functionintersectionsintersections(dn2list,jaccard=False,count=True,fast=False,test=False)Get intersections between lists.Parameters:dn2list(dist):  dictionary mapping to lists.jaccard(bool):  return jaccard indices.count(bool):  return counts.fast(bool):  fast.test(bool):  verbose.Returns:df(DataFrame):  output dataframe.TODOs:  1. feed as an estimator todf.corr(). 2. faster processing by filling up the symetric half of the adjacency matrix.functionrange_overlaprange_overlap(l1,l2)Overlap between ranges.Parameters:l1(list):  start and end integers of one range.l2(list):  start and end integers of other range.Returns:l(list):  overlapped range.functionget_windowsget_windows(a,size=None,overlap=None,windows=None,overlap_fraction=None,stretch_last=False,out_ranges=True)Windows/segments from a range.Parameters:a(list):  range.size(int):  size of the windows.windows(int):  number of windows.overlap_fraction(float):  overlap fraction.overlap(int):  overlap length.stretch_last(bool):  stretch last window.out_ranges(bool):  whether to output ranges.Returns:df1(DataFrame):  output dataframe.Notes:For development, use ofintprovidesnp.floor.functionbools2intervalsbools2intervals(v)Convert bools to intervals.Parameters:v(list):  list of bools.Returns:l(list):  intervals.functionlist2rangeslist2ranges(l)functionget_pairsget_pairs(items:list,items_with:list=None,size:int=2,with_self:bool=False)→DataFrameCreates a dataframe with the paired items.Parameters:items:  the list of items to pair. items_with: list of items to pair with. size: size of the combinations. with_self: pair with self or not.Returns:table with pairs of items.Notes:the ids of the items are sorted e.g. 'a'-'b' not 'b'-'a'. 2. itertools.combinations does not pair self.moduleroux.lib.strFor processing strings.functionsubstitutionsubstitution(s,i,replaceby)Substitute character in a string.Parameters:s(string):  string.i(int):  location.replaceby(string):  character to substitute with.Returns:s(string):  output string.functionsubstitutionsubstitution(s,i,replaceby)Substitute character in a string.Parameters:s(string):  string.i(int):  location.replaceby(string):  character to substitute with.Returns:s(string):  output string.functionreplace_manyreplace_many(s:str,replaces:dict,replacewith:str='',ignore:bool=False)Rename by replacing sub-strings.Parameters:s(str):  input string.replaces(dict|list):  from->to format or list containing substrings to remove.replacewith(str):  replace to in casereplacesis a list.ignore(bool):  if True, not validate the successful replacements.Returns:s(DataFrame):  output dataframe.functionreplace_manyreplace_many(s:str,replaces:dict,replacewith:str='',ignore:bool=False)Rename by replacing sub-strings.Parameters:s(str):  input string.replaces(dict|list):  from->to format or list containing substrings to remove.replacewith(str):  replace to in casereplacesis a list.ignore(bool):  if True, not validate the successful replacements.Returns:s(DataFrame):  output dataframe.functionfilter_listfilter_list(l:list,patterns:list,kind='out')→listFilter a list of strings.Args:l(list):  list of strings.patterns(list):  list of regex patterns. patterns are applied after stripping the whitespaces.Returns:(list) list of filtered strings.functiontuple2strtuple2str(tup,sep=' ')Join tuple items.Parameters:tup(tuple|list):  input tuple/list.sep(str):  separator between the items.Returns:s(str):  output string.functionlinebreakerlinebreaker(text,width=None,break_pt=None,sep='\n',**kws)Insertnewlines within a string.Parameters:text(str):  string.width(int):  insertnewlineat this interval.sep(string):  separator to split the sub-strings.Returns:s(string):  output string.References:1.textwrap``:https://docs.python.org/3/library/textwrap.htmlfunctionfindallfindall(s,ss,outends=False,outstrs=False,suffixlen=0)Find the substrings or their locations in a string.Parameters:s(string):  input string.ss(string):  substring.outends(bool):  output end positions.outstrs(bool):  output strings.suffixlen(int):  length of the suffix.Returns:l(list):  output list.functionget_marked_substringsget_marked_substrings(s,leftmarker='{',rightmarker='}',leftoff=0,rightoff=0)→listGet the substrings flanked with markers from a string.Parameters:s(str):  input string.leftmarker(str):  marker on the left.rightmarker(str):  marker on the right.leftoff(int):  offset on the left.rightoff(int):  offset on the right.Returns:l(list):  list of substrings.functionget_marked_substringsget_marked_substrings(s,leftmarker='{',rightmarker='}',leftoff=0,rightoff=0)→listGet the substrings flanked with markers from a string.Parameters:s(str):  input string.leftmarker(str):  marker on the left.rightmarker(str):  marker on the right.leftoff(int):  offset on the left.rightoff(int):  offset on the right.Returns:l(list):  list of substrings.functionmark_substringsmark_substrings(s,ss,leftmarker='(',rightmarker=')')→strMark sub-string/s in a string.Parameters:s(str):  input string.ss(str):  substring.leftmarker(str):  marker on the left.rightmarker(str):  marker on the right.Returns:s(str):  string.functionget_bracketget_bracket(s,leftmarker='(',righttmarker=')')→strGet bracketed substrings.Parameters:s(string):  string.leftmarker(str):  marker on the left.rightmarker(str):  marker on the right.Returns:s(str):  string.TODOs: 1. Useget_marked_substrings.functionalignalign(s1:str,s2:str,prefix:bool=False,suffix:bool=False,common:bool=True)→listAlign strings.Parameters:s1(str):  string #1.s2(str):  string #2.prefix(str):  prefix.suffix(str):  suffix.common(str):  common substring.Returns:l(list):  output list.Notes:Code to test: [ get_prefix(source,target,common=False), get_prefix(source,target,common=True), get_suffix(source,target,common=False), get_suffix(source,target,common=True),]functionget_prefixget_prefix(s1,s2:str=None,common:bool=True,clean:bool=True)→strGet the prefix of the stringsParameters:s1(str|list):  1st string.s2(str):  2nd string (default:None).common(bool):  get the common prefix (default:True).clean(bool):  clean the leading and trailing whitespaces (default:True).Returns:s(str):  prefix.functionget_suffixget_suffix(s1,s2:str=None,common:bool=True,clean:bool=True)→strGet the suffix of the stringsParameters:s1(str|list):  1st string.s2(str):  2nd string (default:None).common(bool):  get the common prefix (default:True).clean(bool):  clean the leading and trailing whitespaces (default:True).Returns:s(str):  prefix.functionget_fixget_fix(s1:str,s2:str,**kws:dict)→strInfer common prefix or suffix.Parameters:s1(str):  1st string.s2(str):  2nd string.Keyword parameters:kws:  parameters provided to theget_prefixandget_suffixfunctions.Returns:s(str):  prefix or suffix.functionremovesuffixremovesuffix(s1:str,suffix:str)→strRemove suffix.Paramters:  s1 (str): input string.  suffix (str): suffix.Returns:s1(str):  string without the suffix.TODOs:  1. Deprecate in py>39 use .removesuffix() instead.functionstr2dictstr2dict(s:str,reversible:bool=True,sep:str=';',sep_equal:str='=')→dictString to dictionary.Parameters:s(str):  string.sep(str):  separator between entries (default:';').sep_equal(str):  separator between the keys and the values (default:'=').Returns:d(dict):  dictionary.References:1. https: //stackoverflow.com/a/186873/3521099functiondict2strdict2str(d1:dict,reversible:bool=True,sep:str=';',sep_equal:str='=')→strDictionary to string.Parameters:d(dict):  dictionary.sep(str):  separator between entries (default:';').sep_equal(str):  separator between the keys and the values (default:'=').reversible(str):  use jsonReturns:s(str):  string.functionstr2numstr2num(s:str)→floatString to number.Parameters:s(str):  string.Returns:i(int):  number.functionnum2strnum2str(num:float,magnitude:bool=False,coff:float=10000,decimals:int=0)→strNumber to string.Parameters:num(int):  number.magnitude(bool):  use magnitudes (default:False).coff(int):  cutoff (default:10000).decimals(int):  decimal points (default:0).Returns:s(str):  string.TODOs 1. ~ if magnitude else notfunctionencodeencode(data,short:bool=False,method_short:str='sha256',**kws)→strEncode the data as a string.Parameters:data(str|dict|Series):  input data.short(bool):  Outputs short string, compatible with paths but non-reversible. Defaults to False.method_short(str):  method used for encoding when short=True.Keyword parameters:kws:  parameters provided to encoding function.Returns:s(string):  output string.functiondecodedecode(s,out=None,**kws_out)Decode data from a string.Parameters:s(string):  encoded string.out(str):  output format (dict|df).Keyword parameters:kws_out:  parameters provided todict2df.Returns:d(dict|DataFrame):  output data.functionto_formulato_formula(replaces={' ':'SPACE','(':'LEFTBRACKET',')':'RIGHTTBRACKET','.':'DOT',',':'COMMA','%':'PERCENT',"'":'INVCOMMA','+':'PLUS','-':'MINUS'},reverse=False)→dictConverts strings to the formula format, compatible withpatsyfor example.moduleroux.lib.sysFor processing file paths for example.functionbasenamenoextbasenamenoext(p)Basename without the extension.Args:p(str):  path.Returns:s(str):  output.functionremove_extsremove_exts(p:str,exts:tuple=None)Filename without the extension.Args:p(str):  path.exts(tuple):  extensions.Returns:s(str):  output.functionread_psread_ps(ps,test:bool=True,verbose:bool=True)→listRead a list of paths.Parameters:ps(list|str):  list of paths or a string with wildcard/s.test(bool):  testing.verbose(bool):  verbose.Returns:ps(list):  list of paths.functionto_pathto_path(s,replacewith='_',verbose=False,coff_len_escape_replacement=100)Normalise a string to be used as a path of file.Parameters:s(string):  input string.replacewith(str):  replace the whitespaces or incompatible characters with.Returns:s(string):  output string.functionto_pathto_path(s,replacewith='_',verbose=False,coff_len_escape_replacement=100)Normalise a string to be used as a path of file.Parameters:s(string):  input string.replacewith(str):  replace the whitespaces or incompatible characters with.Returns:s(string):  output string.functionmakedirsmakedirs(p:str,exist_ok=True,**kws)Make directories recursively.Args:p(str):  path.exist_ok(bool, optional):  no error if the directory exists. Defaults to True.Returns:p_(str):  the path of the directory.functionto_output_pathto_output_path(ps,outd=None,outp=None,suffix='')Infer a single output path for a list of paths.Parameters:ps(list):  list of paths.outd(str):  path of the output directory.outp(str):  path of the output file.suffix(str):  suffix of the filename.Returns:outp(str):  path of the output file.functionto_output_pathsto_output_paths(input_paths:list=None,inputs:list=None,output_path_base:str=None,encode_short:bool=True,replaces_output_path=None,key_output_path:str=None,force:bool=False,verbose:bool=False)→dictInfer a output path for each of the paths or inputs.Parameters:input_paths (list):  list of input paths. Defaults to None.inputs (list):  list of inputs e.g. dictionaries. Defaults to None.output_path_base (str):  output path with a placeholder '{KEY}' to be replaced. Defaults to None.encode_short:  (bool) : short encoded string, else long encoded string (reversible) is used. Defaults to True.replaces_output_path:  list, dictionary or function to replace the input paths. Defaults to None.key_output_path (str):  key to be used to incorporate output_path variable among the inputs. Defaults to None.force(bool):  overwrite the outputs. Defaults to False.verbose (bool):  show verbose. Defaults to False.Returns:dictionary with the output path mapped to input paths or inputs.TODOs: 1. Placeholders other than {KEY}.functionget_encodingget_encoding(p)Get encoding of a file.Parameters:p(str):  file pathReturns:s(string):  encoding.functionget_all_subpathsget_all_subpaths(d='.',include_directories=False)Get all the subpaths.Args:d(str, optional):description. Defaults to '.'.include_directories(bool, optional):  to include the directories. Defaults to False.Returns:paths(list):  sub-paths.functionget_envget_env(env_name:str,return_path:bool=False)Get the virtual environment as a dictionary.Args:env_name(str):  name of the environment.Returns:d(dict):  parameters of the virtual environment.functionrunbashrunbash(s1,env=None,test=False,**kws)Run a bash command.Args:s1(str):  command.env(str):  environment name.test(bool, optional):  testing. Defaults to False.Returns:output:  output of thesubprocess.callfunction.TODOs: 1. logp 2. error ignoringfunctionrunbash_tmprunbash_tmp(s1:str,env:str,df1=None,inp='INPUT',input_type='df',output_type='path',tmp_infn='in.txt',tmp_outfn='out.txt',outp=None,force=False,test=False,**kws)Run a bash command in/tmpdirectory.Args:s1(str):  command.env(str):  environment name.df1(DataFrame, optional):  input dataframe. Defaults to None.inp(str, optional):  input path. Defaults to 'INPUT'.input_type(str, optional):  input type. Defaults to 'df'.output_type(str, optional):  output type. Defaults to 'path'.tmp_infn(str, optional):  temporary input file. Defaults to 'in.txt'.tmp_outfn(str, optional):  temporary output file.. Defaults to 'out.txt'.outp(type, optional):  output path. Defaults to None.force(bool, optional):  force. Defaults to False.test(bool, optional):  test. Defaults to False.Returns:output:  output of thesubprocess.callfunction.functioncreate_symlinkcreate_symlink(p:str,outp:str,test=False,force=False)Create symbolic links.Args:p(str):  input path.outp(str):  output path.test(bool, optional):  test. Defaults to False.Returns:outp(str):  output path.TODOs:Usepathlib``:Path(p).symlink_to(Path(outp))functioninput_binaryinput_binary(q:str)Get input in binary format.Args:q(str):  question.Returns:b(bool):  response.functionis_interactiveis_interactive()Check if the UI is interactive e.g. jupyter or command line.functionis_interactive_notebookis_interactive_notebook()Check if the UI is interactive e.g. jupyter or command line.Notes:Reference:functionget_excecution_locationget_excecution_location(depth=1)Get the location of the function being executed.Args:depth(int, optional):  Depth of the location. Defaults to 1.Returns:tuple(tuple):  filename and line number.functionget_datetimeget_datetime(outstr:bool=True,fmt='%G%m%dT%H%M%S')Get the date and time.Args:outstr(bool, optional):  string output. Defaults to True.fmt(str):  format of the string.Returns:s:  date and time.functionp2timep2time(filename:str,time_type='m')Get the creation/modification dates of files.Args:filename(str):  filename.time_type(str, optional):description. Defaults to 'm'.Returns:time(str):  time.functionps2timeps2time(ps:list,**kws_p2time)Get the times for a list of files.Args:ps(list):  list of paths.Returns:ds(Series):  paths mapped to corresponding times.functionget_loggerget_logger(program='program',argv=None,level=None,dp=None)Get the logging object.Args:program(str, optional):  name of the program. Defaults to 'program'.argv(type, optional):  arguments. Defaults to None.level(type, optional):  level of logging. Defaults to None.dp(type, optional):description. Defaults to None.functiontreetree(folder_path:str,log=True)moduleroux.lib.textFor processing text files.functionget_headerget_header(path:str,comment='#',lineno=None)Get the header of a file.Args:path(str):  path.comment(str):  comment identifier.lineno(int):  line numbers upto.Returns:lines(list):  header.functioncatcat(ps,outp)Concatenate text files.Args:ps(list):  list of paths.outp(str):  output path.Returns:outp(str):  output path.moduleroux.stat.binaryFor processing binary data.functioncompare_bools_jaccardcompare_bools_jaccard(x,y)Compare bools in terms of the jaccard index.Args:x(list):  list of bools.y(list):  list of bools.Returns:float:  jaccard index.functioncompare_bools_jaccard_dfcompare_bools_jaccard_df(df:DataFrame)→DataFramePairwise compare bools in terms of the jaccard index.Args:df(DataFrame):  dataframe with boolean columns.Returns:DataFrame:  matrix with comparisons between the columns.functionclassify_boolsclassify_bools(l:list)→strClassify bools.Args:l(list):  list of boolsReturns:str:  classification.functionfracfrac(x:list)→floatFraction.Args:x(list):  list of bools.Returns:float:  fraction of True values.functionpercperc(x:list)→floatPercentage.Args:x(list):  list of bools.Returns:float:  Percentage of the True valuesfunctionget_stats_confusion_matrixget_stats_confusion_matrix(df_:DataFrame)→DataFrameGet stats confusion matrix.Args:df_(DataFrame):  Confusion matrix.Returns:DataFrame:  stats.functionget_cutoffget_cutoff(y_true,y_score,method,show_diagonal=True,show_area=True,kws_area:dict={},show_cutoff=True,plot_pr=True,color='k',returns=['ax'],ax=None)Obtain threshold based on ROC or PR curve.Returns:Table:columns:  valuesmethod:  ROC, PRvariable:  threshold (index), TPR, FPR, TP counts, precision, recall values:  Plots: AUC ROC, TPR vs TP counts PR Specificity vs TP counts Dictionary: Thresholds from AUC, PRTODOs:  1. Separate the plotting functions.moduleroux.stat.clusterFor clustering data.functioncheck_clusterscheck_clusters(df:DataFrame)Check clusters.Args:df(DataFrame):  dataframe.functionget_clustersget_clusters(X:<built-infunctionarray>,n_clusters:int,random_state=88,params={},test=False)→dictGet clusters.Args:X(np.array):  vectorn_clusters(int):  intrandom_state(int, optional):  random state. Defaults to 88.params(dict, optional):  parameters for theMiniBatchKMeansfunction. Defaults to {}.test(bool, optional):  test. Defaults to False.Returns:dict:functionget_n_clusters_optimumget_n_clusters_optimum(df5:DataFrame,test=False)→intGet n clusters optimum.Args:df5(DataFrame):  input dataframetest(bool, optional):  test. Defaults to False.Returns:int:  knee point.functionplot_silhouetteplot_silhouette(df:DataFrame,n_clusters_optimum=None,ax=None)Plot silhouetteArgs:df(DataFrame):  input dataframe.n_clusters_optimum(int, optional):  number of clusters. Defaults to None:int.ax(axes, optional):  axes object. Defaults to None:axes.Returns:ax(axes, optional):  axes object. Defaults to None:axes.functionget_clusters_optimumget_clusters_optimum(X:<built-infunctionarray>,n_clusters=range(2,11),params_clustering={},test=False)→dictGet optimum clusters.Args:X(np.array):  samples to cluster in indexed format.n_clusters(int, optional):description. Defaults to range(2,11).params_clustering(dict, optional):  parameters provided toget_clusters. Defaults to {}.test(bool, optional):  test. Defaults to False.Returns:dict:descriptionfunctionget_gmm_paramsget_gmm_params(g,x,n_clusters=2,test=False)Intersection point of the two peak Gaussian mixture Models (GMMs).Args:out(str):coffonly orparamsfor all the parameters.functionget_gmm_intersectionget_gmm_intersection(x,two_pdfs,means,weights,test=False)functioncluster_1dcluster_1d(ds:Series,n_clusters:int,clf_type='gmm',random_state=1,test=False,returns=['coff'],**kws_clf)→dictCluster 1D data.Args:ds(Series):  series.n_clusters(int):  number of clusters.clf_type(str, optional):  type of classification. Defaults to 'gmm'.random_state(int, optional):  random state. Defaults to 88.test(bool, optional):  test. Defaults to False.returns(list, optional):  return format. Defaults to ['df','coff','ax','model'].ax(axes, optional):  axes object. Defaults to None.Raises:ValueError:  clf_typeReturns:dict:descriptionfunctionget_pos_umapget_pos_umap(df1,spread=100,test=False,k='',**kws)→DataFrameGet positions of the umap points.Args:df1(DataFrame):  input dataframespread(int, optional):  spead extent. Defaults to 100.test(bool, optional):  test. Defaults to False.k(str, optional):  number of clusters. Defaults to ''.Returns:DataFrame:  output dataframe.moduleroux.stat.compareFor comparison related stats.functionget_comparisonget_comparison(df1:DataFrame,d1:dict=None,coff_p:float=0.05,between_ys:bool=False,verbose:bool=False,**kws)Compare the x and y columns.Parameters:df1(pd.DataFrame):  input table.d1(dict):  columns dict, output ofget_cols_x_for_comparison.between_ys(bool):  compare y'sNotes:Column information: d1={'cols_index': ['id'], 'cols_x': {'cont': [], 'desc': []}, 'cols_y': {'cont': [], 'desc': []}}
Comparison types: 1. continuous vs continuous -> correlation 2. decrete vs continuous -> difference 3. decrete vs decrete -> FE or chi squarefunctioncompare_stringscompare_strings(l0:list,l1:list,cutoff:float=0.5)→DataFrameCompare two lists of strings.Parameters:l0(list):  list of strings.l1(list):  list of strings to compare with.cutoff(float):  threshold to filter the comparisons.Returns:table with the similarity scores.TODOs:  1. Add option for semantic similarity.moduleroux.stat.corrFor correlation stats.functionresampledresampled(x:<built-infunctionarray>,y:<built-infunctionarray>,method_fun:object,method_kws:dict={},ci_type:str='max',cv:int=5,random_state:int=1,verbose:bool=False)→tupleGet correlations after resampling.Args:x(np.array):  x vector.y(np.array):  y vector.method_fun(str, optional):  method function.ci_type(str, optional):  confidence interval type. Defaults to 'max'.cv(int, optional):  number of resamples. Defaults to 5.random_state(int, optional):  random state. Defaults to 1.verbose(bool):  verbose.Returns:dict:  results containing mean correlation coefficient, CI and CI type.functionget_corrget_corr(x:str,y:str,method:str,df:DataFrame=None,method_kws:dict={},pval:bool=True,preprocess:bool=True,n_min=10,preprocess_kws:dict={},resample:bool=False,cv=5,resample_kws:dict={},verbose:bool=False,test:bool=False)→dictCorrelation between vectors.  A unifying wrapper aroundscipy's functions to calculate correlations and distances. Allows application of resampling on those functions.Usage:  1. Linear table with paired values. For a matrix, usepd.DataFrame.corrinstead.Args:x(str):  x column name or a vector.y(str):  y column name or a vector.method(str):  method name.df(pd.DataFrame):  input table.pval(bool):  calculate p-value.resample(bool, optional):  resampling. Defaults to False.preprocess(bool):  preprocess the inputpreprocess_kws (dict):  parameters provided to the pre-processing function i.e._pre.resample(bool):  resampling.resample_kws(dict):  parameters provided to the resampling function i.e.resample.verbose(bool):  verbose.Returns:res(dict):  a dictionary containing results.Notes:resdirectory contains following values:
method : method name r : correlation coefficient or distance p : pvalue of the correlation. n : sample size rr: resampled average 'r' ci: CI ci_type: CI typefunctionget_corrsget_corrs(data:DataFrame,method:str,cols:list=None,cols_with:list=None,coff_inflation_min:float=None,get_pairs_kws={},fast:bool=False,test:bool=False,verbose:bool=False,**kws_get_corr)→DataFrameCorrelate many columns of a dataframes.Parameters:df1(DataFrame):  input dataframe.method(str):  method of correlationspearmanorpearson.cols(str):  columns.cols_with(str):  columns to correlate with i.e. variable2.fast(bool):  use parallel-processing if True.Keyword arguments:kws_get_corr:  parameters provided toget_corrfunction.Returns:DataFrame:  output dataframe.Notes:In the fast mode (fast=True), to set the number of processes, before executing theget_corrscommand, run
from pandarallel import pandarallel pandarallel.initialize(nb_workers={},progress_bar=True,use_memory_fs=False)functioncheck_collinearitycheck_collinearity(df1:DataFrame,threshold:float=0.7,colvalue:str='r',cols_variable:list=['variable1','variable2'],coff_pval:float=0.05,method:str='spearman',coff_inflation_min:int=50)→SeriesCheck collinearity.Args:df1(DataFrame):  input dataframe.threshold(float):  minimum threshold for the colinearity.Returns:DataFrame:  output dataframe with minimum correlation among correlated subnetwork of columns.functionpairwise_chi2pairwise_chi2(df1:DataFrame,cols_values:list)→DataFramePairwise chi2 test.Args:df1(DataFrame):  pd.DataFramecols_values(list):  list of columns.Returns:DataFrame:  output dataframe.TODOs: 0. uselib.set.get_pairsto get the combinations.moduleroux.stat.diffFor difference related stats.functioncompare_classescompare_classes(x,y,method=None)Compare classesfunctioncompare_classes_manycompare_classes_many(df1:DataFrame,cols_y:list,cols_x:list)→DataFramefunctionget_pvalget_pval(df:DataFrame,colvalue='value',colsubset='subset',colvalue_bool=False,colindex=None,subsets=None,test=False,fun=None)→tupleGet p-value.Args:df(DataFrame):  input dataframe.colvalue(str, optional):  column with values. Defaults to 'value'.colsubset(str, optional):  column with subsets. Defaults to 'subset'.colvalue_bool(bool, optional):  column with boolean values. Defaults to False.colindex(str, optional):  column with the index. Defaults to None.subsets(list, optional):  subset types. Defaults to None.test(bool, optional):  test. Defaults to False.fun(function, optional):  function. Defaults to None.Raises:ArgumentError:  colvalue or colsubset not found in df.ValueError:  need only 2 subsets.Returns:tuple:  stat,p-valuefunctionget_statget_stat(df1:DataFrame,colsubset:str,colvalue:str,colindex:str,subsets=None,cols_subsets=['subset1','subset2'],df2=None,stats=[<functionmeanat0x7fd396dafb00>,<functionmedianat0x7fd396c73b00>,<functionvarat0x7fd396daff80>,<built-infunctionlen>],coff_samples_min=None,verb=False,**kws)→DataFrameGet statistics.Args:df1(DataFrame):  input dataframe.colvalue(str, optional):  column with values. Defaults to 'value'.colsubset(str, optional):  column with subsets. Defaults to 'subset'.colindex(str, optional):  column with the index. Defaults to None.subsets(list, optional):  subset types. Defaults to None.cols_subsets(list, optional):  columns with subsets. Defaults to ['subset1', 'subset2'].df2(DataFrame, optional):  second dataframe. Defaults to None.stats(list, optional):  summary statistics. Defaults to [np.mean,np.median,np.var]+[len].coff_samples_min(int, optional):  minimum sample size required. Defaults to None.verb(bool, optional):  verbose. Defaults to False.Keyword Arguments:kws:  parameters provided toget_pvalfunction.Raises:ArgumentError:  colvalue or colsubset not found in df.ValueError:  len(subsets)<2Returns:DataFrame:  output dataframe.TODOs: 1. Rename to more specificget_diff, also otherget_stat*/get_pval*functions.functionget_statsget_stats(df1:DataFrame,colsubset:str,cols_value:list,colindex:str,subsets=None,df2=None,cols_subsets=['subset1','subset2'],stats=[<functionmeanat0x7fd396dafb00>,<functionmedianat0x7fd396c73b00>,<functionvarat0x7fd396daff80>,<built-infunctionlen>],axis=0,test=False,**kws)→DataFrameGet statistics by iterating over columns wuth values.Args:df1(DataFrame):  input dataframe.colsubset(str, optional):  column with subsets.cols_value(list):  list of columns with values.colindex(str, optional):  column with the index.subsets(list, optional):  subset types. Defaults to None.df2(DataFrame, optional):  second dataframe, e.g.pd.DataFrame({"subset1":['test'],"subset2":['reference']}). Defaults to None.cols_subsets(list, optional):  columns with subsets. Defaults to ['subset1', 'subset2'].stats(list, optional):  summary statistics. Defaults to [np.mean,np.median,np.var]+[len].axis(int, optional):  1 if different tests else use 0. Defaults to 0.Keyword Arguments:kws:  parameters provided toget_pvalfunction.Raises:ArgumentError:  colvalue or colsubset not found in df.ValueError:  len(subsets)<2Returns:DataFrame:  output dataframe.TODOs: 1. No column prefix iflen(cols_value)==1.functionget_significant_changesget_significant_changes(df1:DataFrame,coff_p=0.025,coff_q=0.1,alpha=None,changeby='mean',value_aggs=['mean','median'])→DataFrameGet significant changes.Args:df1(DataFrame):  input dataframe.coff_p(float, optional):  cutoff on p-value. Defaults to 0.025.coff_q(float, optional):  cutoff on q-value. Defaults to 0.1.alpha(float, optional):  alias forcoff_p. Defaults to None.changeby(str, optional):  "" if check for change by both mean and median. Defaults to "".value_aggs(list, optional):  values to aggregate. Defaults to ['mean','median'].Returns:DataFrame:  output dataframe.functionapply_get_significant_changesapply_get_significant_changes(df1:DataFrame,cols_value:list,cols_groupby:list,cols_grouped:list,fast=False,**kws)→DataFrameApply on dataframe to get significant changes.Args:df1(DataFrame):  input dataframe.cols_value(list):  columns with values.cols_groupby(list):  columns with groups.Returns:DataFrame:  output dataframe.functionget_stats_groupbyget_stats_groupby(df1:DataFrame,cols_group:list,coff_p:float=0.05,coff_q:float=0.1,alpha=None,fast=False,**kws)→DataFrameIterate over groups, to get the differences.Args:df1(DataFrame):  input dataframe.cols_group(list):  columns to interate over.coff_p(float, optional):  cutoff on p-value. Defaults to 0.025.coff_q(float, optional):  cutoff on q-value. Defaults to 0.1.alpha(float, optional):  alias forcoff_p. Defaults to None.fast(bool, optional):  parallel processing. Defaults to False.Returns:DataFrame:  output dataframe.functionget_diffget_diff(df1:DataFrame,cols_x:list,cols_y:list,cols_index:list,cols_group:list,coff_p:float=None,test:bool=False,**kws)→DataFrameWrapper around theget_stats_groupbyKeyword parameters:  cols=['variable x','variable y'],  coff_p=0.05,  coff_q=0.01,  colindex=['id'],functionbinby_pvalue_coffsbinby_pvalue_coffs(df1:DataFrame,coffs=[0.01,0.05,0.1],color=False,testn='MWU test, FDR corrected',colindex='genes id',colgroup='tissue',preffix='',colns=None,palette=None)→tupleBin data by pvalue cutoffs.Args:df1(DataFrame):  input dataframe.coffs(list, optional):  cut-offs. Defaults to [0.01,0.05,0.25].color(bool, optional):  color asignment. Defaults to False.testn(str, optional):  test number. Defaults to 'MWU test, FDR corrected'.colindex(str, optional):  column with index. Defaults to 'genes id'.colgroup(str, optional):  column with the groups. Defaults to 'tissue'.preffix(str, optional):  prefix. Defaults to ''.colns(type, optional):  columns number. Defaults to None.notcountedpalette(type, optional):description. Defaults to None.Returns:tuple:  output.Notes:To be deprecated in the favor of the functions used for enrichment analysis for example.moduleroux.stat.ioFor input/output of stats.functionperc_labelperc_label(a,b=None,bracket=True)functionpval2annotpval2annot(pval:float,alternative:str=None,alpha:float=0.05,fmt:str='*',power:bool=True,linebreak:bool=False,replace_prefix:str=None)P/Q-value to annotation.Parameters:fmt(str):  *|<|'num'moduleroux.statGlobal Variablesbinaryiomoduleroux.stat.networkFor network related stats.functionget_subgraphsget_subgraphs(df1:DataFrame,source:str,target:str)→DataFrameSubgraphs from the the edge list.Args:df1(pd.DataFrame):  input dataframe containing edge-list.source(str):  source node.target(str):  taget node.Returns:pd.DataFrame:  output.moduleroux.stat.normFor normalisation.functionnorm_by_quantilenorm_by_quantile(X:<built-infunctionarray>)→<built-infunctionarray>Quantile normalize the columns of X.Parameters:X:  2D array of float, shape (M, N). The input data, with M rows (genes/features) and N columns (samples).Returns:Xn:  2D array of float, shape (M, N). The normalized data.Notes:Faster processing (~5 times compared to other function tested) because of the use of numpy arrays.
TODOs: Usefrom sklearn.preprocessing import QuantileTransformerwithoutput_distributionparameter allowing rescaling back to the same distribution kind.functionnorm_by_gaussian_kdenorm_by_gaussian_kde(values:<built-infunctionarray>)→<built-infunctionarray>Normalise matrix by gaussian KDE.Args:values(np.array):  input matrix.Returns:np.array:  output matrix.References:https: //github.com/saezlab/protein_attenuation/blob/6c1e81af37d72ef09835ee287f63b000c7c6663c/src/protein_attenuation/utils.pyfunctionzscorezscore(df:DataFrame,cols:list=None)→DataFrameZ-score.Args:df(pd.DataFrame):  input table.Returns:pd.DataFrame:  output table.TODOs: 1. Use scipy or sklearn's zscore because of it's additional options  from scipy.stats import zscore  df.apply(zscore)functionzscore_robustzscore_robust(a:<built-infunctionarray>)→<built-infunctionarray>Robust Z-score.Args:a(np.array):  input data.Returns:np.array:  output.Example:t = sc.stats.norm.rvs(size=100, scale=1, random_state=123456) plt.hist(t,bins=40) plt.hist(apply_zscore_robust(t),bins=40) print(np.median(t),np.median(apply_zscore_robust(t)))functionnorm_covariance_PCAnorm_covariance_PCA(X:<built-infunctionarray>,use_svd:bool=True,use_sklearn:bool=True,rescale_centered:bool=True,random_state:int=0,test:bool=False,verbose:bool=False)→<built-infunctionarray>Covariance normalization by PCA whitening.Args:X(np.array):  input arrayuse_svd(bool, optional):  use SVD method. Defaults to True.use_sklearn(bool, optional):  useskelearnfor SVD method. Defaults to True.rescale_centered(bool, optional):  rescale to centered input. Defaults to True.random_state(int, optional):  random state. Defaults to 0.test(bool, optional):  test mode. Defaults to False.verbose(bool, optional):  verbose. Defaults to False.Returns:np.array:  transformed data.moduleroux.stat.pairedFor paired stats.functionget_ratio_sortedget_ratio_sorted(a:float,b:float,increase=True)→floatGet ratio sorted.Args:a(float):  value #1.b(float):  value #2.increase(bool, optional):  check for increase. Defaults to True.Returns:float:  output.functiondiffdiff(a:float,b:float,absolute=True)→floatGet differenceArgs:a(float):  value #1.b(float):  value #2.absolute(bool, optional):  get absolute difference. Defaults to True.Returns:float:  output.functionget_diff_sortedget_diff_sorted(a:float,b:float)→floatDifference sorted/absolute.Args:a(float):  value #1.b(float):  value #2.Returns:float:  output.functionbalancebalance(a:float,b:float,absolute=True)→floatBalance.Args:a(float):  value #1.b(float):  value #2.absolute(bool, optional):  absolute difference. Defaults to True.Returns:float:  output.functionget_paired_sets_statsget_paired_sets_stats(l1:list,l2:list,test:bool=False)→listPaired stats comparing two sets.Args:l1(list):  set #1.l2(list):  set #2.test(bool):  test mode. Defaults to False.Returns:list:  tuple (overlap, intersection, union, ratio).functionget_stats_pairedget_stats_paired(df1:DataFrame,cols:list,input_logscale:bool,prefix:str=None,drop_cols:bool=False,unidirectional_stats:list=['min','max'],fast:bool=False)→DataFramePaired stats, row-wise.Args:df1(pd.DataFrame):  input data.cols(list):  columns.input_logscale(bool):  if the input data is log-scaled.prefix(str, optional):  prefix of the output column/s. Defaults to None.drop_cols(bool, optional):  drop these columns. Defaults to False.unidirectional_stats(list, optional):  column-wise status. Defaults to ['min','max'].fast(bool, optional):  parallel processing. Defaults to False.Returns:pd.DataFrame:  output dataframe.functionget_stats_paired_aggget_stats_paired_agg(x:<built-infunctionarray>,y:<built-infunctionarray>,ignore:bool=False,verb:bool=True)→SeriesPaired stats aggregated, for example, to classify 2D distributions.Args:x(np.array):  x vector.y(np.array):  y vector.ignore(bool, optional):  suppress warnings. Defaults to False.verb(bool, optional):  verbose. Defaults to True.Returns:pd.Series:  output.functionclassify_sharingclassify_sharing(df1:DataFrame,column_value:str,bins:list=[0,25,75,100],labels:list=['low','medium','high'],prefix:str='',verbose:bool=False)→DataFrameClassify sharing % calculated from Jaccard index.Parameters:df1(pd.DataFrame):  input table.column_value(str):  column with values.bins(list):  bins. Defaults to [0,25,75,100].labels(list):  bin labels. Defaults to ['low','medium','high'],prefix(str):  prefix of the columns.verbose(bool):  verbose. Defaults to False.moduleroux.stat.preprocessFor classification.functiondropna_matrixdropna_matrix(df1,coff_cols_min_perc_na=5,coff_rows_min_perc_na=5,test=False,verbose=False)functiondrop_low_complexitydrop_low_complexity(df1:DataFrame,min_nunique:int,max_inflation:int,max_nunique:int=None,cols:list=None,cols_keep:list=[],test:bool=False,verbose:bool=False)→DataFrameRemove low-complexity columns from the data.Args:df1(pd.DataFrame):  input data.min_nunique(int):  minimum unique values.max_inflation(int):  maximum over-representation of the values.cols(list, optional):  columns. Defaults to None.cols_keep(list, optional):  columns to keep. Defaults to [].test(bool, optional):  test mode. Defaults to False.Returns:pd.DataFrame:  output data.functionget_cols_x_for_comparisonget_cols_x_for_comparison(df1:DataFrame,cols_y:list,cols_index:list,cols_drop:list=[],cols_dropby_patterns:list=[],dropby_low_complexity:bool=True,min_nunique:int=5,max_inflation:int=50,dropby_collinearity:bool=True,coff_rs:float=0.7,dropby_variance_inflation:bool=True,verbose:bool=False,test:bool=False)→dictIdentify X columns.Parameters:df1(pd.DataFrame):  input table.cols_y(list):  y columns.functionto_preprocessed_datato_preprocessed_data(df1:DataFrame,columns:dict,fill_missing_desc_value:bool=False,fill_missing_cont_value:bool=False,normby_zscore:bool=False,verbose:bool=False,test:bool=False)→DataFramePreprocess data.functionto_filteredby_samplesto_filteredby_samples(df1:DataFrame,colindex:str,colsample:str,coff_samples_min:int,colsubset:str,coff_subsets_min:int=2)→DataFrameFilter table before calculating differences. (1) Retain minimum number of samples per item representing a subset and (2) Retain minimum number of subsets per item.Parameters:          df1 (pd.DataFrame): input table.  colindex (str): column containing items.  colsample (str): column containing samples.  coff_samples_min (int): minimum number of samples.  colsubset (str): column containing subsets.  coff_subsets_min (int): minimum number of subsets. Defaults to 2.Returns:pd.DataFrameExamples:Parameters:colindex='genes id',  colsample='sample id',  coff_samples_min=3,  colsubset= 'pLOF or WT'   coff_subsets_min=2,functionget_cvsplitsget_cvsplits(X:<built-infunctionarray>,y:<built-infunctionarray>,cv:int=5,random_state:int=None,outtest:bool=True)→dictGet cross-validation splits. A friendly wrapper aroundsklearn.model_selection.KFold.Args:X(np.array):  X matrix.y(np.array):  y vector.cv(int, optional):  cross validations. Defaults to 5.random_state(int, optional):  random state. Defaults to None.outtest(bool, optional):  output test data. Defaults to True.Returns:dict:  output.moduleroux.stat.setsFor set related stats.functionget_overlapget_overlap(items_set:list,items_test:list,output_format:str='list')→listGet overlapping items as a string.Args:items_set(list):  items in the reference setitems_test(list):  items to testoutput_format(str, optional):  format of the output. Defaults to 'list'.Raises:ValueError:  output_format can be list or strfunctionget_overlap_sizeget_overlap_size(items_set:list,items_test:list,fraction:bool=False,perc:bool=False,by:str=None)→floatPercentage Jaccard index.Args:items_set(list):  items in the reference setitems_test(list):  items to testfraction(bool, optional):  output fraction. Defaults to False.perc(bool, optional):  output percentage. Defaults to False.by(str, optional):  fraction by. Defaults to None.Returns:float:  overlap size.functionget_item_set_size_by_backgroundget_item_set_size_by_background(items_set:list,background:int)→floatItem set size by backgroundArgs:items_set(list):  items in the reference setbackground(int):  background sizeReturns:float:  Item set size by backgroundNotes:Denominator of the fold change.functionget_fold_changeget_fold_change(items_set:list,items_test:list,background:int)→floatGet fold change.Args:items_set(list):  items in the reference setitems_test(list):  items to testbackground(int):  background sizeReturns:float:  fold changeNotes:fc = (intersection/(test items))/((items in the item set)/background)functionget_hypergeom_pvalget_hypergeom_pval(items_set:list,items_test:list,background:int)→floatCalculate hypergeometric P-value.Args:items_set(list):  items in the reference setitems_test(list):  items to testbackground(int):  background sizeReturns:float:  hypergeometric P-valuefunctionget_contigency_tableget_contigency_table(items_set:list,items_test:list,background:int)→listGet a contingency table required for the Fisher's test.Args:items_set(list):  items in the reference setitems_test(list):  items to testbackground(int):  background sizeReturns:list:  contingency tableNotes:within item (/referenece) set: True            False within test item: True  intersection    True False False   False False     total-size of unionfunctionget_odds_ratioget_odds_ratio(items_set:list,items_test:list,background:int)→floatCalculate Odds ratio and P-values using Fisher's exact test.Args:items_set(list):  items in the reference setitems_test(list):  items to testbackground(int):  background sizeReturns:float:  Odds ratiofunctionget_enrichmentget_enrichment(df1:DataFrame,df2:DataFrame,colid:str,colset:str,background:int,coltest:str=None,verbose:bool=False)→DataFrameCalculate the enrichments.Args:df1(pd.DataFrame):  table containing items to testdf2(pd.DataFrame):  table containing refence sets and itemscolid(str):  column with IDs of itemscolset(str):  column setscoltest(str):  column testsbackground(int):  background size.verbose(bool):  verboseReturns:pd.DataFrame:  output tablemoduleroux.stat.solveFor solving equations.functionget_intersection_locationsget_intersection_locations(y1:<built-infunctionarray>,y2:<built-infunctionarray>,test:bool=False,x:<built-infunctionarray>=None)→listGet co-ordinates of the intersection (x[idx]).Args:y1(np.array):  vector.y2(np.array):  vector.test(bool, optional):  test mode. Defaults to False.x(np.array, optional):  vector. Defaults to None.Returns:list:  output.moduleroux.stat.transformFor transformations.functionplogplog(x,p:float,base:int)Psudo-log.Args:x(float|np.array):  input.p(float):  pseudo-count.base(int):  base of the log.Returns:output.functionanti_ploganti_plog(x,p:float,base:int)Anti-psudo-log.Args:x(float|np.array):  input.p(float):  pseudo-count.base(int):  base of the log.Returns:output.functionlog_pvallog_pval(x,errors:str='raise',replace_zero_with:float=None,p_min:float=None)Transform p-values to Log10.Paramters:   x: input.  errors (str): Defaults to 'raise' else replace (in case of visualization only).  p_min (float): Replace zeros with this value. Note: to be used for visualization only.Returns:output.functionget_qget_q(ds1:Series,col:str=None,verb:bool=True,test_coff:float=0.1)To FDR corrected P-value.functionglogglog(x:float,l=2)Generalised logarithm.Args:x(float):  input.l(int, optional):  psudo-count. Defaults to 2.Returns:float:  output.functionrescalerescale(a:<built-infunctionarray>,range1:tuple=None,range2:tuple=[0,1])→<built-infunctionarray>Rescale within a new range.Args:a(np.array):  input vector.range1(tuple, optional):  existing range. Defaults to None.range2(tuple, optional):  new range. Defaults to [0,1].Returns:np.array:  output.functionrescale_divergentrescale_divergent(df1:DataFrame,col:str)→DataFrameRescale divergently i.e. two-sided.Args:df1(pd.DataFrame):descriptioncol(str):  column.Returns:pd.DataFrame:  column.Notes:Under development.moduleroux.stat.varianceFor variance related stats.functionconfidence_interval_95confidence_interval_95(x:<built-infunctionarray>)→float95% confidence interval.Args:x(np.array):  input vector.Returns:float:  output.functionget_ciget_ci(rs,ci_type,outstr=False)functionget_variance_inflationget_variance_inflation(data,coly:str,cols_x:list=None)Variance Inflation Factor (VIF). A wrapper aroundstatsmodels's 'variance_inflation_factorfunction.Parameters:data(pd.DataFrame):  input data.coly(str):  dependent variable.cols_x(list):  independent variables.Returns:pd.Seriesmoduleroux.viz.annotFor annotations.functionannot_sideannot_side(ax:Axes,df1:DataFrame,colx:str,coly:str,cols:str=None,hue:str=None,loc:str='right',scatter=False,scatter_marker='|',scatter_alpha=0.75,lines=True,offx3:float=0.15,offymin:float=0.1,offymax:float=0.9,length_axhline:float=3,text=True,text_offx:float=0,text_offy:float=0,invert_xaxis:bool=False,break_pt:int=25,va:str='bottom',zorder:int=2,color:str='gray',kws_line:dict={},kws_scatter:dict={},**kws_text)→AxesAnnot elements of the plots on the of the side plot.Args:df1(pd.DataFrame):  input datacolx(str):  column with x values.coly(str):  column with y values.cols(str):  column with labels.hue(str):  column with colors of the labels.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.loc(str, optional):  location. Defaults to 'right'.invert_xaxis(bool, optional):  invert xaxis. Defaults to False.offx3(float, optional):  x-offset for bend position of the arrow. Defaults to 0.15.offymin(float, optional):  x-offset minimum. Defaults to 0.1.offymax(float, optional):  x-offset maximum. Defaults to 0.9.break_pt(int, optional):  break point of the labels. Defaults to 25.length_axhline(float, optional):  length of the horizontal line i.e. the "underline". Defaults to 3.zorder(int, optional):  z-order. Defaults to 1.color(str, optional):  color of the line. Defaults to 'gray'.kws_line(dict, optional):  parameters for formatting the line. Defaults to {}.Keyword Args:kws:  parameters provided to theax.textfunction.Returns:plt.Axes:plt.Axesobject.functionshow_outlinesshow_outlines(data:DataFrame,colx:str,coly:str,column_outlines:str,outline_colors:dict,style=None,legend:bool=True,kws_legend:dict={},zorder:int=3,ax:Axes=None,**kws_scatter)→AxesOutline points on the scatter plot by categories.functionshow_confidence_ellipseshow_confidence_ellipse(x,y,ax,n_std=3.0,facecolor='none',**kwargs)Create a plot of the covariance confidence ellipse ofxandy.Parameters:---------- x, y : array-like, shape (n, )  Input data.ax : matplotlib.axes.Axes  The axes object to draw the ellipse into.n_std : float  The number of standard deviations to determine the ellipse's radiuses.**kwargs  Forwarded to~matplotlib.patches.EllipseReturns
------- matplotlib.patches.EllipseReferences
----------https://matplotlib.org/3.5.0/gallery/statistics/confidence_ellipse.htmlfunctionshow_boxshow_box(ax:Axes,xy:tuple,width:float,height:float,fill:str=None,alpha:float=1,lw:float=1.1,edgecolor:str='k',clip_on:bool=False,scale_width:float=1,scale_height:float=1,xoff:float=0,yoff:float=0,**kws)→AxesHighlight sections of a plot e.g. heatmap by drawing boxes.Args:xy(tuple):  position of left, bottom corner of the box.width(float):  width.height(float):  height.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.fill(str, optional):  fill the box with color. Defaults to None.alpha(float, optional):  alpha of color. Defaults to 1.lw(float, optional):  line width. Defaults to 1.1.edgecolor(str, optional):  edge color. Defaults to 'k'.clip_on(bool, optional):  clip the boxes by the axis limit. Defaults to False.scale_width(float, optional):  scale width. Defaults to 1.scale_height(float, optional):  scale height. Defaults to 1.xoff(float, optional):  x-offset. Defaults to 0.yoff(float, optional):  y-offset. Defaults to 0.Keyword Args:kws:  parameters provided to theRectanglefunction.Returns:plt.Axes:plt.Axesobject.functioncolor_axcolor_ax(ax:Axes,c:str,linewidth:float=None)→AxesColor border ofplt.Axes.Args:ax(plt.Axes):plt.Axesobject.c(str):  color.linewidth(float, optional):  line width. Defaults to None.Returns:plt.Axes:plt.Axesobject.functionshow_n_legendshow_n_legend(ax,df1:DataFrame,colid:str,colgroup:str,**kws)functionshow_scatter_statsshow_scatter_stats(ax:Axes,data:DataFrame,x,y,z,method:str,resample:bool=False,show_n:bool=True,show_n_prefix:str='',prefix:str='',loc=None,zorder:int=5,verbose:bool=True,**kws_set_label)resample (bool, optional): resample data. Defaults to False.functionshow_crosstab_statsshow_crosstab_stats(data:DataFrame,cols:list,method:str=None,alpha:float=0.05,loc:str=None,xoff:float=0,yoff:float=0,linebreak:bool=False,ax:Axes=None,**kws_set_label)→AxesAnnotate a confusion matrix.Args:data(pd.DataFrame):  input data.cols(list):  list of columns with the categories.method(str, optional):  method used to calculate the statistical significance.alpha(float, optional):  alpha for the stats. Defaults to 0.05.loc(str, optional):  location. Over-rides kws_set_label. Defaults to None.xoff(float, optional):  x offset. Defaults to 0.yoff(float, optional):  y offset. Defaults to 0.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws_set_label:  keyword parameters provided toset_label.Returns:plt.Axes:plt.Axesobject.functionshow_confusion_matrix_statsshow_confusion_matrix_stats(df_:DataFrame,ax:Axes=None,off:float=0.5)→AxesAnnotate a confusion matrix.Args:df_(pd.DataFrame):  input data.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.off(float, optional):  offset. Defaults to 0.5.Returns:plt.Axes:plt.Axesobject.functionget_logo_axget_logo_ax(ax:Axes,size:float=0.5,bbox_to_anchor:list=None,loc:str=1,axes_kwargs:dict={'zorder':-1})→AxesGetplt.Axesfor placing the logo.Args:ax(plt.Axes):plt.Axesobject.size(float, optional):  size of the subplot. Defaults to 0.5.bbox_to_anchor(list, optional):  location. Defaults to None.loc(str, optional):  location. Defaults to 1.axes_kwargs(type, optional):  parameters provided toinset_axes. Defaults to {'zorder':-1}.Returns:plt.Axes:plt.Axesobject.functionset_logoset_logo(imp:str,ax:Axes,size:float=0.5,bbox_to_anchor:list=None,loc:str=1,axes_kwargs:dict={'zorder':-1},params_imshow:dict={'aspect':'auto','alpha':1,'interpolation':'catrom'},test:bool=False,force:bool=False)→AxesSet logo.Args:imp(str):  path to the logo file.ax(plt.Axes):plt.Axesobject.size(float, optional):  size of the subplot. Defaults to 0.5.bbox_to_anchor(list, optional):  location. Defaults to None.loc(str, optional):  location. Defaults to 1.axes_kwargs(type, optional):  parameters provided toinset_axes. Defaults to {'zorder':-1}.params_imshow(type, optional):  parameters provided to theimshowfunction. Defaults to {'aspect':'auto','alpha':1, 'interpolation':'catrom'}.test(bool, optional):  test mode. Defaults to False.force(bool, optional):  overwrite file. Defaults to False.Returns:plt.Axes:plt.Axesobject.functionset_suptitleset_suptitle(axs,title,offy=0,**kws_text)Combined title for a list of subplots.moduleroux.viz.ax_For setting up subplots.functionset_axes_minimalset_axes_minimal(ax,xlabel=None,ylabel=None,off_axes_pad=0)→AxesSet minimal axes labels, at the lower left corner.functionset_labelset_label(s:str,ax:Axes,x:float=0,y:float=0,ha:str='left',va:str='top',loc=None,off_loc=0.01,title:bool=False,**kws)→AxesSet label on a plot.Args:x(float):  x position.y(float):  y position.s(str):  label.ax(plt.Axes):plt.Axesobject.ha(str, optional):  horizontal alignment. Defaults to 'left'.va(str, optional):  vertical alignment. Defaults to 'top'.loc(int, optional):  location of the label. 1:'upper right', 2:'upper left', 3:'lower left':3, 4:'lower right'offs_loc(tuple,optional):  x and y location offsets.title(bool, optional):  set as title. Defaults to False.Returns:plt.Axes:plt.Axesobject.functionset_ylabelset_ylabel(ax:Axes,s:str=None,x:float=-0.1,y:float=1.02,xoff:float=0,yoff:float=0)→AxesSet ylabel horizontal.Args:ax(plt.Axes):plt.Axesobject.s(str, optional):  ylabel. Defaults to None.x(float, optional):  x position. Defaults to -0.1.y(float, optional):  y position. Defaults to 1.02.xoff(float, optional):  x offset. Defaults to 0.yoff(float, optional):  y offset. Defaults to 0.Returns:plt.Axes:plt.Axesobject.functionget_ax_labelsget_ax_labels(ax:Axes)functionformat_labelsformat_labels(ax,fmt='cap1',title_fontsize=15,rename_labels=None,test=False)functionrename_ticklabelsrename_ticklabels(ax:Axes,axis:str,rename:dict=None,replace:dict=None,ignore:bool=False)→AxesRename the ticklabels.Args:ax(plt.Axes, optional):plt.Axesobject. Defaults to None.axis(str):  axis (x|y).rename(dict, optional):  replace strings. Defaults to None.replace(dict, optional):  replace sub-strings. Defaults to None.ignore(bool, optional):  ignore warnings. Defaults to False.Raises:ValueError:  eitherrenameorreplaceshould be provided.Returns:plt.Axes:plt.Axesobject.functionget_ticklabel_positionget_ticklabel_position(ax:Axes,axis:str)→AxesGet positions of the ticklabels.Args:ax(plt.Axes):plt.Axesobject.axis(str):  axis (x|y).Returns:plt.Axes:plt.Axesobject.functionset_ticklabels_colorset_ticklabels_color(ax:Axes,ticklabel2color:dict,axis:str='y')→AxesSet colors to ticklabels.Args:ax(plt.Axes):plt.Axesobject.ticklabel2color(dict):  colors of the ticklabels.axis(str):  axis (x|y).Returns:plt.Axes:plt.Axesobject.functionformat_ticklabelsformat_ticklabels(ax:Axes,axes:tuple=['x','y'],interval:float=None,n:int=None,fmt:str=None,font:str=None)→Axesformat_ticklabelsArgs:ax(plt.Axes):plt.Axesobject.axes(tuple, optional):  axes. Defaults to ['x','y'].n(int, optional):  number of ticks. Defaults to None.fmt(str, optional):  format e.g. '.0f'. Defaults to None.font(str, optional):  font. Defaults to 'DejaVu Sans Mono'.Returns:plt.Axes:plt.Axesobject.TODOs:  1. include color_ticklabelsfunctionsplit_ticklabelssplit_ticklabels(ax:Axes,fmt:str,axis='x',group_x=-0.45,group_y=-0.25,group_prefix=None,group_suffix=False,group_loc='center',group_colors=None,group_alpha=0.2,show_group_line=True,group_line_off_x=0.15,group_line_off_y=0.1,show_group_span=False,group_span_kws={},sep:str='-',pad_major=6,off:float=0.2,test:bool=False,**kws)→AxesSplit ticklabels into major and minor. Two minor ticks are created per major tick.Args:ax(plt.Axes):plt.Axesobject.fmt(str):  'group'-wise or 'pair'-wise splitting of the ticklabels.axis(str):  name of the axis: x or y.sep(str, optional):  separator within the tick labels. Defaults to ' '.test(bool, optional):  test-mode. Defaults to False.Returns:plt.Axes:plt.Axesobject.functionget_axlimsby_dataget_axlimsby_data(X:Series,Y:Series,off:float=0.2,equal:bool=False)→AxesInfer axis limits from data.Args:X(pd.Series):  x values.Y(pd.Series):  y values.off(float, optional):  offsets. Defaults to 0.2.equal(bool, optional):  equal limits. Defaults to False.Returns:plt.Axes:plt.Axesobject.functionget_axlimsget_axlims(ax:Axes)→AxesGet axis limits.Args:ax(plt.Axes):plt.Axesobject.Returns:plt.Axes:plt.Axesobject.functionset_equallimset_equallim(ax:Axes,diagonal:bool=False,difference:float=None,format_ticks:bool=True,**kws_format_ticklabels)→AxesSet equal axis limits.Args:ax(plt.Axes):plt.Axesobject.diagonal(bool, optional):  show diagonal. Defaults to False.difference(float, optional):  difference from . Defaults to None.Returns:plt.Axes:plt.Axesobject.functionset_axlimsset_axlims(ax:Axes,off:float,axes:list=['x','y'],equal=False,**kws_set_equallim)→AxesSet axis limits.Args:ax(plt.Axes):plt.Axesobject.off(float):  offset.axes(list, optional):  axis name/s. Defaults to ['x','y'].Returns:plt.Axes:plt.Axesobject.functionset_gridsset_grids(ax:Axes,axis:str=None)→AxesShow grids based on the shape (aspect ratio) of the plot.Args:ax(plt.Axes):plt.Axesobject.axis(str, optional):  axis name. Defaults to None.Returns:plt.Axes:plt.Axesobject.functionrename_legendsrename_legends(ax:Axes,replaces:dict,**kws_legend)→AxesRename legends.Args:ax(plt.Axes):plt.Axesobject.replaces(dict):descriptionReturns:plt.Axes:plt.Axesobject.functionappend_legendsappend_legends(ax:Axes,labels:list,handles:list,**kws)→AxesAppend to legends.Args:ax(plt.Axes):plt.Axesobject.labels(list):  labels.handles(list):  handles.Returns:plt.Axes:plt.Axesobject.functionsort_legendssort_legends(ax:Axes,sort_order:list=None,**kws)→AxesSort or filter legends.Args:ax(plt.Axes):plt.Axesobject.sort_order(list, optional):  order of legends. Defaults to None.Returns:plt.Axes:plt.Axesobject.Notes:Filter the legends by providing the indices of the legends to keep.functiondrop_duplicate_legenddrop_duplicate_legend(ax,**kws)functionreset_legend_colorsreset_legend_colors(ax)Reset legend colors.Args:ax(plt.Axes):plt.Axesobject.Returns:plt.Axes:plt.Axesobject.functionset_legends_mergedset_legends_merged(axs)Reset legend colors.Args:axs(list):  list ofplt.Axesobjects.Returns:plt.Axes:  firstplt.Axesobject in the list.functionset_legend_customset_legend_custom(ax:Axes,legend2param:dict,param:str='color',lw:float=1,marker:str='o',markerfacecolor:bool=True,size:float=10,color:str='k',linestyle:str='',title_ha:str='center',frameon:bool=True,**kws)→AxesSet custom legends.Args:ax(plt.Axes):plt.Axesobject.legend2param(dict):  legend name to parameter to change e.g. name of the color.param(str, optional):  parameter to change. Defaults to 'color'.lw(float, optional):  line width. Defaults to 1.marker(str, optional):  marker type. Defaults to 'o'.markerfacecolor(bool, optional):  marker face color. Defaults to True.size(float, optional):  size of the markers. Defaults to 10.color(str, optional):  color of the markers. Defaults to 'k'.linestyle(str, optional):  line style. Defaults to ''.title_ha(str, optional):  title horizontal alignment. Defaults to 'center'.frameon(bool, optional):  show frame. Defaults to True.Returns:plt.Axes:plt.Axesobject.TODOs: 1. differnet number of points for eachh entryfrom matplotlib.legend_handler import HandlerTuple  l1, = plt.plot(-1, -1, lw=0, marker="o",  markerfacecolor='k', markeredgecolor='k')  l2, = plt.plot(-0.5, -1, lw=0, marker="o",  markerfacecolor="none", markeredgecolor='k')  plt.legend([(l1,), (l1, l2)], ["test 1", "test 2"],handler_map={tuple:  HandlerTuple(2)} )References:https: //matplotlib.org/stable/api/markers_api.htmlhttp: //www.cis.jhu.edu/~shanest/mpt/js/mathjax/mathjax-dev/fonts/Tables/STIX/STIX/All/All.htmlfunctionget_line_cap_lengthget_line_cap_length(ax:Axes,linewidth:float)→AxesGet the line cap length.Args:ax(plt.Axes):plt.Axesobjectlinewidth(float):  width of the line.Returns:plt.Axes:plt.Axesobjectfunctionset_colorbarset_colorbar(fig:object,ax:Axes,ax_pc:Axes,label:str,bbox_to_anchor:tuple=(0.05,0.5,1,0.45),orientation:str='vertical')Set colorbar.Args:fig(object):  figure object.ax(plt.Axes):plt.Axesobject.ax_pc(plt.Axes):plt.Axesobject for the colorbar.label(str):  labelbbox_to_anchor(tuple, optional):  location. Defaults to (0.05, 0.5, 1, 0.45).orientation(str, optional):  orientation. Defaults to "vertical".Returns:figure object.functionset_colorbar_labelset_colorbar_label(ax:Axes,label:str)→AxesFind colorbar and set label for it.Args:ax(plt.Axes):plt.Axesobject.label(str):  label.Returns:plt.Axes:plt.Axesobject.moduleroux.viz.barFor bar plots.functionplot_barhplot_barh(df1:DataFrame,colx:str,coly:str,colannnotside:str=None,x1:float=None,offx:float=0,ax:Axes=None,**kws)→AxesPlot horizontal bar plot with text on them.Args:df1(pd.DataFrame):  input data.colx(str):  x column.coly(str):  y column.colannnotside(str):  column with annotations to show on the right side of the plot.x1(float):  x position of the text.offx(float):  x-offset of x1, multiplier.color(str):  color of the bars.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to thebarhfunction.Returns:plt.Axes:plt.Axesobject.functionplot_value_countsplot_value_counts(df:DataFrame,col:str,logx:bool=False,kws_hist:dict={'bins':10},kws_bar:dict={},grid:bool=False,axes:list=None,fig:object=None,hist:bool=True)Plot pandas'svalue_counts.Args:df(pd.DataFrame):  input datavalue_counts.col(str):  column with counts.logx(bool, optional):  x-axis on log-scale. Defaults to False.kws_hist(type, optional):  parameters provided to thehistfunction. Defaults to {'bins':10}.kws_bar(dict, optional):  parameters provided to thebarfunction. Defaults to {}.grid(bool, optional):  show grids or not. Defaults to False.axes(list, optional):  list ofplt.axes. Defaults to None.fig(object, optional):  figure object. Defaults to None.hist(bool, optional):  show histgram. Defaults to True.functionplot_barh_stacked_percentageplot_barh_stacked_percentage(df1:DataFrame,coly:str,colannot:str,color:str=None,yoff:float=0,ax:Axes=None)→AxesPlot horizontal stacked bar plot with percentages.Args:df1(pd.DataFrame):  input data. values in rows sum to 100%.coly(str):  y column. yticklabels, e.g. retained and dropped.colannot(str):  column with annotations.color(str, optional):  color. Defaults to None.yoff(float, optional):  y-offset. Defaults to 0.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Returns:plt.Axes:plt.Axesobject.functionplot_bar_serialplot_bar_serial(d1:dict,polygon:bool=False,polygon_x2i:float=0,labelis:list=[],y:float=0,ylabel:str=None,off_arrowy:float=0.15,kws_rectangle={'height':0.5,'linewidth':1},ax:Axes=None)→AxesBarplots with serial increase in resolution.Args:d1(dict):  dictionary with the data.polygon(bool, optional):  show polygon. Defaults to False.polygon_x2i(float, optional):  connect polygon to this subset. Defaults to 0.labelis(list, optional):  label these subsets. Defaults to [].y(float, optional):  y position. Defaults to 0.ylabel(str, optional):  y label. Defaults to None.off_arrowy(float, optional):  offset for the arrow. Defaults to 0.15.kws_rectangle(type, optional):  parameters provided to therectanglefunction. Defaults to dict(height=0.5,linewidth=1).ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Returns:plt.Axes:plt.Axesobject.functionplot_barh_stacked_percentage_intersectionsplot_barh_stacked_percentage_intersections(df0:DataFrame,colxbool:str,colybool:str,colvalue:str,colid:str,colalt:str,colgroupby:str,coffgroup:float=0.95,ax:Axes=None)→AxesPlot horizontal stacked bar plot with percentages and intesections.Args:df0(pd.DataFrame):  input data.colxbool(str):  x column.colybool(str):  y column.colvalue(str):  column with the values.colid(str):  column with ids.colalt(str):  column with the alternative subset.colgroupby(str):  column with groups.coffgroup(float, optional):  cut-off between the groups. Defaults to 0.95.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Returns:plt.Axes:plt.Axesobject.Examples:Parameters:colxbool='paralog',  colybool='essential',  colvalue='value',  colid='gene id',  colalt='singleton',  coffgroup=0.95,  colgroupby='tissue',functionto_input_data_sankeyto_input_data_sankey(df0,colid,cols_groupby=None,colall='all',remove_all=False)functionplot_sankeyplot_sankey(df1,cols_groupby=None,hues=None,node_color=None,link_color=None,info=None,x=None,y=None,colors=None,hovertemplate=None,text_width=20,convert=True,width=400,height=400,outp=None,validate=True,test=False,**kws)moduleroux.viz.colorsFor setting up colors.functionrgbfloat2intrgbfloat2int(rgb_float)functionget_colors_defaultget_colors_default()→listget default colors.Returns:list:  colors.functionget_ncolorsget_ncolors(n:int,cmap:str='Spectral',ceil:bool=False,test:bool=False,N:int=20,out:str='hex',**kws_get_cmap_section)→listGet colors.Args:n(int):  number of colors to get.cmap(str, optional):  colormap. Defaults to 'Spectral'.ceil(bool, optional):  ceil. Defaults to False.test(bool, optional):  test mode. Defaults to False.N(int, optional):  number of colors in the colormap. Defaults to 20.out(str, optional):  output. Defaults to 'hex'.Returns:list:  colors.functionget_val2colorget_val2color(ds:Series,vmin:float=None,vmax:float=None,cmap:str='Reds')→dictGet color for a value.Args:ds(pd.Series):  values.vmin(float, optional):  minimum value. Defaults to None.vmax(float, optional):  maximum value. Defaults to None.cmap(str, optional):  colormap. Defaults to 'Reds'.Returns:dict:  output.functionsaturate_colorsaturate_color(color,alpha:float)→objectSaturate a color.Args:color (type):alpha(float):  alpha level.Returns:object:  output.References:https: //stackoverflow.com/a/60562502/3521099functionmix_colorsmix_colors(d:dict)→strMix colors.Args:d(dict):  colors to alpha map.Returns:str:  hex color.References:https: //stackoverflow.com/a/61488997/3521099functionmake_cmapmake_cmap(cs:list,N:int=20,**kws)Create a colormap.Args:cs(list):  colorsN(int, optional):  resolution i.e. number of colors. Defaults to 20.Returns:cmap.functionget_cmap_sectionget_cmap_section(cmap,vmin:float=0.0,vmax:float=1.0,n:int=100)→objectGet section of a colormap.Args:cmap(object| str):  colormap.vmin(float, optional):  minimum value. Defaults to 0.0.vmax(float, optional):  maximum value. Defaults to 1.0.n(int, optional):  resolution i.e. number of colors. Defaults to 100.Returns:object:  cmap.functionappend_cmapappend_cmap(cmap:str='Reds',color:str='#D3DDDC',cmap_min:float=0.2,cmap_max:float=0.8,ncolors:int=100,ncolors_min:int=1,ncolors_max:int=0)Append a color to colormap.Args:cmap(str, optional):  colormap. Defaults to 'Reds'.color(str, optional):  color. Defaults to '#D3DDDC'.cmap_min(float, optional):  cmap_min. Defaults to 0.2.cmap_max(float, optional):  cmap_max. Defaults to 0.8.ncolors(int, optional):  number of colors. Defaults to 100.ncolors_min(int, optional):  number of colors minimum. Defaults to 1.ncolors_max(int, optional):  number of colors maximum. Defaults to 0.Returns:cmap.References:https: //matplotlib.org/stable/tutorials/colors/colormap-manipulation.htmlmoduleroux.viz.compareFor comparative plots.functionplot_comparisonsplot_comparisons(plot_data,x,ax=None,output_dir_path=None,force=False,return_path=False)Parameters:plot_data:  output of.stat.compare.get_comparisonNotes:sample type: different sample of the same data.moduleroux.viz.diagramFor diagrams e.g. flowchartsfunctiondiagram_nbdiagram_nb(graph:str,out:bool=False)Show a diagram in jupyter notebook using mermaid.js.Parameters:graph(str):  markdown-formatted graph. Please seehttps://mermaid.js.org/intro/n00b-syntaxReference.htmlout(bool):  Output the URL. Defaults to False.References:1. https: //mermaid.js.org/config/Tutorials.html#jupyter-integration-with-mermaid-jsExamples:graph LR;  i1(["input1"]) & d1[("data1")]
-->   p1[["process1"]]--> o1(["output1"])  p1--> o2["output2"]:::ends classDef ends fill:#fff,stroke:#fffmoduleroux.viz.distFor distribution plots.functionhist_annothist_annot(dplot:DataFrame,colx:str,colssubsets:list=[],bins:int=100,subset_unclassified:bool=True,cmap:str='hsv',ymin=None,ymax=None,ylimoff:float=1,ywithinoff:float=1.2,annotaslegend:bool=True,annotn:bool=True,params_scatter:dict={'zorder':2,'alpha':0.1,'marker':'|'},xlim:tuple=None,ax:Axes=None,**kws)→AxesAnnoted histogram.Args:dplot(pd.DataFrame):  input dataframe.colx(str):  x column.colssubsets(list, optional):  columns indicating subsets. Defaults to [].bins(int, optional):  bins. Defaults to 100.subset_unclassified(bool, optional):  call non-annotated subset as 'unclassified'. Defaults to True.cmap(str, optional):  colormap. Defaults to 'Reds_r'.ylimoff(float, optional):  y-offset for y-axis limit . Defaults to 1.2.ywithinoff(float, optional):  y-offset for the distance within labels. Defaults to 1.2.annotaslegend(bool, optional):  convert labels to legends. Defaults to True.annotn(bool, optional):  annotate sample sizes. Defaults to True.params_scatter(type, optional):  parameters of the scatter plot. Defaults to {'zorder':2,'alpha':0.1,'marker':'|'}.xlim(tuple, optional):  x-axis limits. Defaults to None.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to thehistfunction.Returns:plt.Axes:plt.Axesobject.TODOs: For scatter, useannot_sidewithloc='top'.functionplot_gmmplot_gmm(x:Series,coff:float=None,mix_pdf:object=None,two_pdfs:tuple=None,weights:tuple=None,n_clusters:int=2,bins:int=20,show_cutoff:bool=True,show_cutoff_line:bool=True,colors:list=['gray','gray','lightgray'],out_coff:bool=False,hist:bool=True,test:bool=False,ax:Axes=None,kws_axvline={'color':'k'},**kws)→AxesPlot Gaussian mixture Models (GMMs).Args:x(pd.Series):  input vector.coff(float, optional):  intersection between two fitted distributions. Defaults to None.mix_pdf(object, optional):  Probability density function of the mixed distribution. Defaults to None.two_pdfs(tuple, optional):  Probability density functions of the separate distributions. Defaults to None.weights(tuple, optional):  weights of the individual distributions. Defaults to None.n_clusters(int, optional):  number of distributions. Defaults to 2.bins(int, optional):  bins. Defaults to 50.colors(list, optional):  colors of the invividual distributions and of the mixed one. Defaults to ['gray','gray','lightgray']. 'gray'out_coff(bool,False):  return the cutoff. Defaults to False.hist(bool, optional):  show histogram. Defaults to True.test(bool, optional):  test mode. Defaults to False.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to thehistfunction.kws_axvline:  parameters provided to theaxvlinefunction.Returns:plt.Axes:plt.Axesobject.functionplot_normalplot_normal(x:Series,ax:Axes=None)→AxesPlot normal distribution.Args:x(pd.Series):  input vector.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Returns:plt.Axes:plt.Axesobject.functionget_jitter_positionsget_jitter_positions(ax,df1,order,column_category,column_position)functionplot_distsplot_dists(df1:DataFrame,x:str,y:str,colindex:str,hue:str=None,order:list=None,hue_order:list=None,kind:str='box',show_p:bool=True,show_n:bool=True,show_n_prefix:str='',show_n_ha=None,show_n_ticklabels:bool=True,show_outlines:bool=False,kws_outlines:dict={},alternative:str='two-sided',offx_n:float=0,axis_cont_lim:tuple=None,axis_cont_scale:str='linear',offs_pval:dict=None,alpha:float=0.5,ax:Axes=None,test:bool=False,kws_stats:dict={},**kws)→AxesPlot distributions.Args:df1(pd.DataFrame):  input data.x(str):  x column.y(str):  y column.colindex(str):  index column.hue(str, optional):  column with values to be encoded as hues. Defaults to None.order(list, optional):  order of categorical values. Defaults to None.hue_order(list, optional):  order of values to be encoded as hues. Defaults to None.kind(str, optional):  kind of distribution. Defaults to 'box'.show_p(bool, optional):  show p-values. Defaults to True.show_n(bool, optional):  show sample sizes. Defaults to True.show_n_prefix(str, optional):  show prefix of sample size label i.e.n=. Defaults to ''.offx_n(float, optional):  x-offset for the sample size label. Defaults to 0.axis_cont_lim(tuple, optional):  x-axis limits. Defaults to None.offs_pval(float, optional):  x and y offsets for the p-value labels.# saturate_color_alpha (float, optional):  saturation of the color. Defaults to 1.5.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.test(bool, optional):  test mode. Defaults to False.kws_stats(dict, optional):  parameters provided to the stat function. Defaults to {}.Keyword Args:kws:  parameters provided to theseabornfunction.Returns:plt.Axes:plt.Axesobject.TODOs: 1. Sort categories. 2. Change alpha of the boxplot rather than changing saturation of the swarmplot.functionpointplot_groupbyedgecolorpointplot_groupbyedgecolor(data:DataFrame,ax:Axes=None,**kws)→AxesPlot seaborn'spointplotgrouped by edgecolor of points.Args:data(pd.DataFrame):  input data.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to theseaborn'spointplotfunction.Returns:plt.Axes:plt.Axesobject.moduleroux.viz.figureFor setting up figures.functionget_childrenget_children(fig)Get all the individual objects included in the figure.functionget_child_textget_child_text(search_name,all_children=None,fig=None)Get text object.functionalign_textsalign_texts(fig,texts:list,align:str,test=False)Align text objects.functionlabelplotslabelplots(axes:list=None,fig=None,labels:list=None,xoff:float=0,yoff:float=0,auto:bool=False,xoffs:dict={},yoffs:dict={},va:str='center',ha:str='left',verbose:bool=True,test:bool=False,**kws_text)Label (sub)plots.Args:fig:plt.figureobject.axes(type):  list ofplt.Axesobjects.xoff(int, optional):  x offset. Defaults to 0.yoff(int, optional):  y offset. Defaults to 0.params_alignment(dict, optional):  alignment parameters. Defaults to {}.params_text(dict, optional):  parameters provided toplt.text. Defaults to {'size':20,'va':'bottom', 'ha':'right' }.test(bool, optional):  test mode. Defaults to False.Todos: 1. Get the x coordinate of the ylabel.moduleroux.viz.heatmapFor heatmaps.functionplot_tableplot_table(df1:DataFrame,xlabel:str=None,ylabel:str=None,annot:bool=True,cbar:bool=False,linecolor:str='k',linewidths:float=1,cmap:str=None,sorty:bool=False,linebreaky:bool=False,scales:tuple=[1,1],ax:Axes=None,**kws)→AxesPlot to show a table.Args:df1(pd.DataFrame):  input data.xlabel(str, optional):  x label. Defaults to None.ylabel(str, optional):  y label. Defaults to None.annot(bool, optional):  show numbers. Defaults to True.cbar(bool, optional):  show colorbar. Defaults to False.linecolor(str, optional):  line color. Defaults to 'k'.linewidths(float, optional):  line widths. Defaults to 1.cmap(str, optional):  color map. Defaults to None.sorty(bool, optional):  sort rows. Defaults to False.linebreaky(bool, optional):  linebreak for y labels. Defaults to False.scales(tuple, optional):  scale of the table. Defaults to [1,1].ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to thesns.heatmapfunction.Returns:plt.Axes:plt.Axesobject.moduleroux.viz.imageFor visualization of images.functionplot_imageplot_image(imp:str,ax:Axes=None,force=False,margin=0,axes=False,test=False,**kwarg)→AxesPlot image e.g. schematic.Args:imp(str):  path of the image.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.force(bool, optional):  overwrite output. Defaults to False.margin(int, optional):  margins. Defaults to 0.test(bool, optional):  test mode. Defaults to False.Returns:plt.Axes:plt.Axesobject.:param kwarg: cairosvg: {'dpi':500,'scale':2}; imagemagick: {'trim':False,'alpha':False}moduleroux.viz.ioFor input/output of plots.functionto_plotpto_plotp(ax:Axes=None,prefix:str='plot/plot_',suffix:str='',fmts:list=['png'])→strInfer output path for a plot.Args:ax(plt.Axes):plt.Axesobject.prefix(str, optional):  prefix with directory path for the plot. Defaults to 'plot/plot_'.suffix(str, optional):  suffix of the filename. Defaults to ''.fmts(list, optional):  formats of the images. Defaults to ['png'].Returns:str:  output path for the plot.functionsavefigsavefig(plotp:str,tight_layout:bool=True,bbox_inches:list=None,fmts:list=['png'],savepdf:bool=False,normalise_path:bool=True,replaces_plotp:dict=None,dpi:int=500,force:bool=True,kws_replace_many:dict={},kws_savefig:dict={},**kws)→strWrapper aroundplt.savefig.Args:plotp(str):  output path orplt.Axesobject.tight_layout(bool, optional):  tight_layout. Defaults to True.bbox_inches(list, optional):  bbox_inches. Defaults to None.savepdf(bool, optional):  savepdf. Defaults to False.normalise_path(bool, optional):  normalise_path. Defaults to True.replaces_plotp(dict, optional):  replaces_plotp. Defaults to None.dpi(int, optional):  dpi. Defaults to 500.force(bool, optional):  overwrite output. Defaults to True.kws_replace_many(dict, optional):  parameters provided to thereplace_manyfunction. Defaults to {}.Keyword Args:kws:  parameters provided toto_plotpfunction.kws_savefig:  parameters provided toto_savefigfunction.kws_replace_many:  parameters provided toreplace_manyfunction.Returns:str:  output path.functionsavelegendsavelegend(plotp:str,legend:object,expand:list=[-5,-5,5,5],**kws_savefig)→strSave only the legend of the plot/figure.Args:plotp(str):  output path.legend(object):  legend object.expand(list, optional):  expand. Defaults to [-5,-5,5,5].Returns:str:  output path.References:1. https: //stackoverflow.com/a/47749903/3521099functionupdate_kws_plotupdate_kws_plot(kws_plot:dict,kws_plotp:dict,test:bool=False)→dictUpdate the input parameters.Args:kws_plot(dict):  input parameters.kws_plotp(dict):  saved parameters.test(bool, optional):description. Defaults to False.Returns:dict:  updated parameters.functionget_plot_inputsget_plot_inputs(plotp:str,df1:DataFrame=None,kws_plot:dict={},outd:str=None)→tupleGet plot inputs.Args:plotp(str):  path of the plot.df1(pd.DataFrame):  data for the plot.kws_plot(dict):  parameters of the plot.outd(str):  output directory.Returns:tuple:  (path,dataframe,dict)functionlog_codelog_code()Log the code.functionlog_codelog_code()Log the code.functionget_linesget_lines(logp:str='log_notebook.log',sep:str='begin_plot()',test:bool=False)→listGet lines from the log.Args:logp(str, optional):  path to the log file. Defaults to 'log_notebook.log'.sep(str, optional):  label marking the start of code of the plot. Defaults to 'begin_plot()'.test(bool, optional):  test mode. Defaults to False.Returns:list:  lines of code.functionto_scriptto_script(srcp:str,plotp:str,defn:str='plot_',s4:str='    ',test:bool=False,**kws)→strSave the script with the code for the plot.Args:srcp(str):  path of the script.plotp(str):  path of the plot.defn(str, optional):  prefix of the function. Defaults to "plot_".s4(str, optional):  a tab. Defaults to '    '.test(bool, optional):  test mode. Defaults to False.Returns:str:  path of the script.TODOs: 1. Compatible with names of the input dataframes other thatdf1.  1. Get the variable name of the dataframedef get_df_name(df):  name =[x for x in globals() if globals()[x] is df and not x.startswith('-')][0]  return nameReplacedf1with the variable name of the dataframe.functionto_plotto_plot(plotp:str,data:DataFrame=None,df1:DataFrame=None,kws_plot:dict={},logp:str='log_notebook.log',sep:str='begin_plot()',validate:bool=False,show_path:bool=False,show_path_offy:float=-0.2,force:bool=True,test:bool=False,quiet:bool=True,**kws)→strSave a plot.Args:plotp(str):  output path.df1(pd.DataFrame, optional):  dataframe with plotting data. Defaults to None.data(pd.DataFrame, optional):  dataframe with plotting data. Defaults to None.kws_plot(dict, optional):  parameters for plotting. Defaults to dict().logp(str, optional):  path to the log. Defaults to 'log_notebook.log'.sep(str, optional):  separator marking the start of the plotting code in jupyter notebook. Defaults to 'begin_plot()'.validate(bool, optional):  validate the "readability" usingread_plotfunction. Defaults to False.show_path(bool, optional):  show path on the plot. Defaults to False.show_path_offy(float, optional):  y-offset for the path label. Defaults to 0.force(bool, optional):  overwrite output. Defaults to True.test(bool, optional):  test mode. Defaults to False.quiet(bool, optional):  quiet mode. Defaults to False.Returns:str:  output path.Notes:Requirement: 1. Start logging in the jupyter notebook.
from IPython import get_ipython log_notebookp=f'log_notebook.log';open(log_notebookp, 'w').close();get_ipython().run_line_magic('logstart','{log_notebookp} over')functionread_plotread_plot(p:str,safe:bool=False,test:bool=False,**kws)→AxesGenerate the plot from data, parameters and a script.Args:p(str):  path of the plot saved usingto_plotfunction.safe(bool, optional):  read as an image. Defaults to False.test(bool, optional):  test mode. Defaults to False.Returns:plt.Axes:plt.Axesobject.functionto_concatto_concat(ps:list,how:str='h',use_imagemagick:bool=False,use_conda_env:bool=False,test:bool=False,**kws_outp)→strConcat images.Args:ps(list):  list of paths.how(str, optional):  horizontal (h) or verticalv. Defaults to 'h'.test(bool, optional):  test mode. Defaults to False.Returns:str:  path of the output.functionto_montageto_montage(ps:list,layout:str,source_path:str=None,env_name:str=None,hspace:float=0,vspace:float=0,output_path:str=None,test:bool=False,**kws_outp)→strTo montage.Args:ps(type):  list of paths.layout(type):  layout of the images.hspace(int, optional):  horizontal space. Defaults to 0.vspace(int, optional):  vertical space. Defaults to 0.test(bool, optional):  test mode. Defaults to False.Returns:str:  path of the output.functionto_gifto_gif(ps:list,outp:str,duration:int=200,loop:int=0,optimize:bool=True)→strConvert to GIF.Args:ps(list):  list of paths.outp(str):  output path.duration(int, optional):  duration. Defaults to 200.loop(int, optional):  loop or not. Defaults to 0.optimize(bool, optional):  optimize the size. Defaults to True.Returns:str:  output path.References:1. https: //pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#gif2. https: //stackoverflow.com/a/57751793/3521099functionto_datato_data(path:str)→strConvert to base64 string.Args:path(str):  path of the input.Returns:base64 string.functionto_convertto_convert(filep:str,outd:str=None,fmt:str='JPEG')→strConvert format of image usingPIL.Args:filep(str):  input path.outd(str, optional):  output directory. Defaults to None.fmt(str, optional):  format of the output. Defaults to "JPEG".Returns:str:  output path.functionto_rasterto_raster(plotp:str,dpi:int=500,alpha:bool=False,trim:bool=False,force:bool=False,test:bool=False)→strto_rastersummaryArgs:plotp(str):  input path.dpi(int, optional):  DPI. Defaults to 500.alpha(bool, optional):  transparency. Defaults to False.trim(bool, optional):  trim margins. Defaults to False.force(bool, optional):  overwrite output. Defaults to False.test(bool, optional):  test mode. Defaults to False.Returns:str:descriptionNotes:Runs a bash command:convert -density 300 -trim.functionto_rastersto_rasters(plotd,ext='svg')Convert many images to raster. Uses inkscape.Args:plotd(str):  directory.ext(str, optional):  extension of the output. Defaults to 'svg'.moduleroux.viz.lineFor line plots.functionplot_rangeplot_range(df00:DataFrame,colvalue:str,colindex:str,k:str,headsize:int=15,headcolor:str='lightgray',ax:Axes=None,**kws_area)→AxesPlot range/intervals e.g. genome coordinates as lines.Args:df00(pd.DataFrame):  input data.colvalue(str):  column with values.colindex(str):  column with ids.k(str):  subset name.headsize(int, optional):  margin at top. Defaults to 15.headcolor(str, optional):  color of the margin. Defaults to 'lightgray'.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword args:kws:  keyword parameters provided toareafunction.Returns:plt.Axes:plt.Axesobject.functionplot_connectionsplot_connections(dplot:DataFrame,label2xy:dict,colval:str='$r_{s}$',line_scale:int=40,legend_title:str='similarity',label2rename:dict=None,element2color:dict=None,xoff:float=0,yoff:float=0,rectangle:dict={'width':0.2,'height':0.32},params_text:dict={'ha':'center','va':'center'},params_legend:dict={'bbox_to_anchor':(1.1,0.5),'ncol':1,'frameon':False},legend_elements:list=[],params_line:dict={'alpha':1},ax:Axes=None,test:bool=False)→AxesPlot connections between points with annotations.Args:dplot(pd.DataFrame):  input data.label2xy(dict):  label to position.colval(str, optional):  column with values. Defaults to '{s}$'.line_scale(int, optional):  line_scale. Defaults to 40.legend_title(str, optional):  legend_title. Defaults to 'similarity'.label2rename(dict, optional):  label2rename. Defaults to None.element2color(dict, optional):  element2color. Defaults to None.xoff(float, optional):  xoff. Defaults to 0.yoff(float, optional):  yoff. Defaults to 0.rectangle(type, optional):  rectangle. Defaults to {'width':0.2,'height':0.32}.params_text(type, optional):  params_text. Defaults to {'ha':'center','va':'center'}.params_legend(type, optional):  params_legend. Defaults to {'bbox_to_anchor':(1.1, 0.5), 'ncol':1, 'frameon':False}.legend_elements(list, optional):  legend_elements. Defaults to [].params_line(type, optional):  params_line. Defaults to {'alpha':1}.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.test(bool, optional):  test mode. Defaults to False.Returns:plt.Axes:plt.Axesobject.functionplot_kineticsplot_kinetics(df1:DataFrame,x:str,y:str,hue:str,cmap:str='Reds_r',ax:Axes=None,test:bool=False,kws_legend:dict={},**kws_set)→AxesPlot time-dependent kinetic data.Args:df1(pd.DataFrame):  input data.x(str):  x column.y(str):  y column.hue(str):  hue column.cmap(str, optional):  colormap. Defaults to 'Reds_r'.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.test(bool, optional):  test mode. Defaults to False.kws_legend(dict, optional):  legend parameters. Defaults to {}.Returns:plt.Axes:plt.Axesobject.functionplot_stepsplot_steps(df1:DataFrame,col_step_name:str,col_step_size:str,ax:Axes=None,test:bool=False)→AxesPlot step-wise changes in numbers, e.g. for a filtering process.Args:df1(pd.DataFrame):  input data.col_step_size(str):  column containing the numbers.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.test(bool, optional):  test mode. Defaults to False.Returns:plt.Axes:plt.Axesobject.moduleroux.vizGlobal Variablesiocolorsdiagrammoduleroux.viz.scatterFor scatter plots.functionplot_scatter_aggplot_scatter_agg(dplot:DataFrame,x:str=None,y:str=None,z:str=None,kws_legend={'bbox_to_anchor':[1,1],'loc':'upper left'})UNDER DEV.functionplot_scatterplot_scatter(data:DataFrame,x:str=None,y:str=None,z:str=None,kind:str='scatter',scatter_kws={},line_kws={},stat_method:str='spearman',stat_kws={},hollow:bool=False,ax:Axes=None,verbose:bool=True,**kws)→AxesPlot scatter with multiple layers and stats.Args:data(pd.DataFrame):  input dataframe.x(str):  x column.y(str):  y column.z(str, optional):  z column. Defaults to None.kind(str, optional):  kind of scatter. Defaults to 'hexbin'.trendline_method(str, optional):  trendline method ['poly','lowess']. Defaults to 'poly'.stat_method(str, optional):  method of annoted stats ['mlr',"spearman"]. Defaults to "spearman".cmap(str, optional):  colormap. Defaults to 'Reds'.label_colorbar(str, optional):  label of the colorbar. Defaults to None.gridsize(int, optional):  number of grids in the hexbin. Defaults to 25.bbox_to_anchor(list, optional):  location of the legend. Defaults to [1,1].loc(str, optional):  location of the legend. Defaults to 'upper left'.title(str, optional):  title of the plot. Defaults to None.#params_plot (dict, optional):  parameters provided to theplotfunction. Defaults to {}.line_kws(dict, optional):  parameters provided to theplot_trendlinefunction. Defaults to {}.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to theplotfunction.Returns:plt.Axes:plt.Axesobject.Notes:For a rasterized scatter plot setscatter_kws={'rasterized': True}2. This function does not apply multiple colors, similar tosns.regplot.functionplot_qqplot_qq(x:Series)→Axesplot QQ.Args:x(pd.Series):  input vector.Returns:plt.Axes:plt.Axesobject.functionplot_ranksplot_ranks(df1:DataFrame,colid:str,colx:str,coly:str='rank',ascending:bool=True,ax=None,**kws)→AxesPlot rankings.Args:dplot(pd.DataFrame):  input data.colx(str):  x column.coly(str):  y column.colid(str):  column with unique ids.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Keyword Args:kws:  parameters provided to theseaborn.scatterplotfunction.Returns:plt.Axes:plt.Axesobject.functionplot_volcanoplot_volcano(data:DataFrame,colx:str,coly:str,colindex:str,hue:str='x',style:str='P=0',style_order:list=['o','^'],markers:list=['o','^'],show_labels:int=None,show_outlines:int=None,outline_colors:list=['k'],collabel:str=None,show_line=True,line_pvalue=0.1,line_x:float=0.0,line_x_min:float=None,show_text:bool=True,text_increase:str=None,text_decrease:str=None,text_diff:str=None,legend:bool=False,verbose:bool=False,p_min:float=None,ax:Axes=None,outmore:bool=False,kws_legend:dict={},**kws_scatterplot)→AxesVolcano plot.Parameters:Keyword parameters:Returns:plt.Axesmoduleroux.viz.setsFor plotting sets.functionplot_vennplot_venn(ds1:Series,ax:Axes=None,figsize:tuple=[2.5,2.5],show_n:bool=True,outmore=False,**kws)→AxesPlot Venn diagram.Args:ds1(pd.Series):  input pandas.Series or dictionary. Subsets in the index levels, mapped to counts.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.figsize(tuple, optional):  figure size. Defaults to [2.5,2.5].show_n(bool, optional):  show sample sizes. Defaults to True.Returns:plt.Axes:plt.Axesobject.functionplot_intersection_countsplot_intersection_counts(df1:DataFrame,cols:list=None,kind:str='table',method:str=None,show_pval:bool=True,confusion:bool=False,rename_cols:bool=False,sort_cols:tuple=[True,True],order_x:list=None,order_y:list=None,cmap:str='Reds',ax:Axes=None,kws_show_stats:dict={},**kws_plot)→AxesPlot counts for the intersection between two sets.Args:df1(pd.DataFrame):  input datacols(list, optional):  columns. Defaults to None.kind(str, optional):  kind of plot: table or barplot. Detaults to table.method(str, optional):  method to check the association ['chi2','FE']. Defaults to None.rename_cols(bool, optional):  rename the columns. Defaults to True.show_pval(bool, optional):  annotate p-values. Defaults to True.cmap(str, optional):  colormap. Defaults to 'Reds'.kws_show_stats(dict, optional):  arguments provided to stats function. Defaults to {}.ax(plt.Axes, optional):plt.Axesobject. Defaults to None.Raises:ValueError:show_pvalposition should be the allowed one.Keyword Args:kws_plot:  keyword arguments provided to the plotting function.Returns:plt.Axes:plt.Axesobject.TODOs: 1. Usecompare_classesto get the stats.functionplot_intersectionsplot_intersections(ds1:Series,item_name:str=None,figsize:tuple=[4,4],text_width:float=2,yorder:list=None,sort_by:str='cardinality',sort_categories_by:str=None,element_size:int=40,facecolor:str='gray',bari_annot:int=None,totals_bar:bool=False,totals_text:bool=True,intersections_ylabel:float=None,intersections_min:float=None,test:bool=False,annot_text:bool=False,set_ylabelx:float=-0.25,set_ylabely:float=0.5,**kws)→AxesPlot upset plot.Args:ds1(pd.Series):  input vector.item_name(str, optional):  name of items. Defaults to None.figsize(tuple, optional):  figure size. Defaults to [4,4].text_width(float, optional):  max. width of the text. Defaults to 2.yorder(list, optional):  order of y elements. Defaults to None.sort_by(str, optional):  sorting method. Defaults to 'cardinality'.sort_categories_by(str, optional):  sorting method. Defaults to None.element_size(int, optional):  size of elements. Defaults to 40.facecolor(str, optional):  facecolor. Defaults to 'gray'.bari_annot(int, optional):  annotate nth bar. Defaults to None.totals_text(bool, optional):  show totals. Defaults to True.intersections_ylabel(float, optional):  y-label of the intersections. Defaults to None.intersections_min(float, optional):  intersection minimum to show. Defaults to None.test(bool, optional):  test mode. Defaults to False.annot_text(bool, optional):  annotate text. Defaults to False.set_ylabelx(float, optional):  x position of the ylabel. Defaults to -0.25.set_ylabely(float, optional):  y position of the ylabel. Defaults to 0.5.Keyword Args:kws:  parameters provided to theupset.plotfunction.Returns:plt.Axes:plt.Axesobject.Notes:sort_by:{‘cardinality’, ‘degree’} If ‘cardinality’, subset are listed from largest to smallest. If ‘degree’, they are listed in order of the number of categories intersected. sort_categories_by:{‘cardinality’, None} Whether to sort the categories by total cardinality, or leave them in the provided order.
References:https://upsetplot.readthedocs.io/en/stable/api.htmlfunctionplot_enrichmentplot_enrichment(data:DataFrame,x:str,y:str,s:str,hue='Q',xlabel=None,ylabel='significance\n(-log10(Q))',size:int=None,color:str=None,annots_side:int=5,annots_side_labels=None,coff_fdr:float=None,xlim:tuple=None,xlim_off:float=0.2,ylim:tuple=None,ax:Axes=None,break_pt:int=25,annot_coff_fdr:bool=False,kws_annot:dict={'loc':'right','offx3':0.15},returns='ax',**kwargs)→AxesPlot enrichment stats.Args:- <b>`data`</b> (pd.DataFrame):  input data. 
 - <b>`x`</b> (str):  x column. 
 - <b>`y`</b> (str):  y column. 
 - <b>`s`</b> (str):  size column. 
 - <b>`size`</b> (int, optional):  size of the points. Defaults to None. 
 - <b>`color`</b> (str, optional):  color of the points. Defaults to None. 
 - <b>`annots_side`</b> (int, optional):  how many labels to show on side. Defaults to 5. 
 - <b>`coff_fdr`</b> (float, optional):  FDR cutoff. Defaults to None. 
 - <b>`xlim`</b> (tuple, optional):  x-axis limits. Defaults to None. 
 - <b>`xlim_off`</b> (float, optional):  x-offset on limits. Defaults to 0.2. 
 - <b>`ylim`</b> (tuple, optional):  y-axis limits. Defaults to None. 
 - <b>`ax`</b> (plt.Axes, optional):  `plt.Axes` object. Defaults to None. 
 - <b>`break_pt`</b> (int, optional):  break point (' ') for the labels. Defaults to 25. 
 - <b>`annot_coff_fdr`</b> (bool, optional):  show FDR cutoff. Defaults to False. 
 - <b>`kws_annot`</b> (dict, optional):  parameters provided to the `annot_side` function. Defaults to dict( loc='right', annot_count_max=5, offx3=0.15, ).Keyword Args:
-kwargs:  parameters provided to thesns.scatterplotfunction.Returns:- <b>`plt.Axes`</b>:  `plt.Axes` object.functionplot_pieplot_pie(counts:list,labels:list,scales_line_xy:tuple=(1.1,1.1),remove_wedges:list=None,remove_wedges_index:list=[],line_color:str='k',annot_side:bool=False,kws_annot_side:dict={},ax:Axes=None,**kws_pie)→AxesPie plot.Args:counts(list):  counts.labels(list):  labels.scales_line_xy(tuple, optional):  scales for the lines. Defaults to (1.1,1.1).remove_wedges(list, optional):  remove wedge/s. Defaults to None.remove_wedges_index(list, optional):  remove wedge/s by index. Defaults to [].line_color(str, optional):  line color. Defaults to 'k'.annot_side(bool, optional):  annotations on side using theannot_sidefunction. Defaults to False.kws_annot_side(dict, optional):  keyword arguments provided to theannot_sidefunction. Defaults to {}.ax(plt.Axes, optional):  subplot. Defaults to None.Keyword Args:kws_pie:  keyword arguments provided to thepiechart function.Returns:plt.Axes:  subplotReferences:https: //matplotlib.org/stable/gallery/pie_and_polar_charts/pie_and_donut_labels.htmlmoduleroux.vizimoduleroux.workflow.checksFor workflow checks.functiongrepgrep(p,checks,exclude=[],exclude_str=[],verbose=True)Get the output of grep as a list of strings.moduleroux.workflow.dfFor management of tables.functionexclude_itemsexclude_items(df1:DataFrame,metadata:dict)→DataFrameExclude items from the table with the workflow info.Args:df1(pd.DataFrame):  input table.metadata(dict):  metadata of the repository.Returns:pd.DataFrame:  output.moduleroux.workflow.functionFor function management.functionget_quoted_pathget_quoted_path(s1:str)→strQuoted paths.Args:s1(str):  path.Returns:str:  quoted path.functionget_pathget_path(s:str,validate:bool,prefixes=['data/','metadata/','plot/'],test=False)→strExtract pathsfrom a line of code.Args:s(str):  line of code.validate(bool):  validate the output.prefixes(list, optional):  allowed prefixes. Defaults to ['data/','metadata/','plot/'].test(bool, optional):  test mode. Defaults to False.Returns:str:  path.TODOs: 1. Use wildcards i.e. *'s.functionremove_dirs_from_outputsremove_dirs_from_outputs(outputs:list,test:bool=False)→listRemove directories from the output paths.Args:outputs(list):  output paths.test(bool, optional):  test mode. Defaults to False.Returns:list:  paths.functionget_iosget_ios(l:list,test=False)→tupleGet input and output (IO) paths.Args:l(list):  list of lines of code.test(bool, optional):  test mode. Defaults to False.Returns:tuple:  paths of inputs and outputs.functionget_nameget_name(s:str,i:int,sep_step:str='## step')→strGet name of the function.Args:s(str):  lines in markdown format.sep_step(str, optional):  separator marking the start of a step. Defaults to "## step".i(int):  index of the step.Returns:str:  name of the function.functionget_stepget_step(l:list,name:str,sep_step:str='## step',sep_step_end:str='## tests',test=False,tab='    ')→dictGet code for a step.Args:l(list):  list of lines of codename(str):  name of the function.test(bool, optional):  test mode. Defaults to False.tab(str, optional):  tab format. Defaults to '    '.Returns:dict:  step name to code map.functionto_taskto_task(notebookp,task=None,sep_step:str='## step',sep_step_end:str='## tests',notebook_suffix:str='_v',force=False,validate=False,path_prefix=None,verbose=True,test=False)→strGet the lines of code for a task (script to be saved as an individual.pyfile).Args:notebookp(type):  path of the notebook.sep_step(str, optional):  separator marking the start of a step. Defaults to "## step".sep_step_end(str, optional):  separator marking the end of a step. Defaults to "## tests".notebook_suffix(str, optional):  suffix of the notebook file to be considered as a "task".force(bool, optional):  overwrite output. Defaults to False.validate(bool, optional):  validate output. Defaults to False.path_prefix(type, optional):  prefix to the path. Defaults to None.verbose(bool, optional):  show verbose. Defaults to True.test(bool, optional):  test mode. Defaults to False.Returns:str:  lines of the code.functionget_global_importsget_global_imports()→DataFrameGet the metadata of the functions imported fromfrom roux import global_imports.moduleroux.workflow.ioFor input/output of workflow.functionclear_variablesclear_variables(dtype=None,variables=None)Clear dataframes from the workspace.functionclear_dataframesclear_dataframes()functionto_pyto_py(notebookp:str,pyp:str=None,force:bool=False,**kws_get_lines)→strTo python script (.py).Args:notebookp(str):  path to the notebook path.pyp(str, optional):  path to the python file. Defaults to None.force(bool, optional):  overwrite output. Defaults to False.Returns:str:  path of the output.functionto_nb_cellsto_nb_cells(notebook,outp,new_cells,validate_diff=None)Replace notebook cells.functionimport_from_fileimport_from_file(pyp:str)Import functions from python (.py) file.Args:pyp(str):  python file (.py).functioninfer_parametersinfer_parameters(input_value,default_value)Infer the input values and post warning messages.Parameters:input_value:  the primary value.default_value:  the default/alternative/inferred value.Returns:Inferred value.functionto_parametersto_parameters(f:object,test:bool=False)→dictGet function to parameters map.Args:f(object):  function.test(bool, optional):  test mode. Defaults to False.Returns:dict:  output.functionread_configread_config(p:str,config_base=None,inputs=None,append_to_key=None,convert_dtype:bool=True,verbose:bool=True)Read configuration.Parameters:p(str):  input path.config_base:  base config with the inputs for the interpolationsfunctionread_metadataread_metadata(p:str,ind:str=None,max_paths:int=30,config_path_key:str='config_path',config_paths:list=[],config_paths_auto=False,verbose:bool=False,**kws_read_config)→dictRead metadata.Args:p(str, optional):  file containing metadata. Defaults to './metadata.yaml'.ind(str, optional):  directory containing specific setings and other data to be incorporated into metadata. Defaults to './metadata/'.Returns:dict:  output.functionto_workflowto_workflow(df2:DataFrame,workflowp:str,tab:str='    ')→strSave workflow file.Args:df2(pd.DataFrame):  input table.workflowp(str):  path of the workflow file.tab(str, optional):  tab format. Defaults to '    '.Returns:str:  path of the workflow file.functioncreate_workflow_reportcreate_workflow_report(workflowp:str,env:str)→intCreate report for the workflow run.Parameters:workflowp(str):  path of the workflow file (snakemake).env(str):  name of the conda virtual environment where required the workflow dependency is available i.e.snakemake.functionreplacestarreplacestar(input_path,output_path=None,replace_from='from roux.global_imports import *',in_place:bool=False,attributes={'pandarallel':['parallel_apply'],'rd':['.rd.','.log.']},verbose:bool=False,test:bool=False,**kws_fix_code)Post-development, replace wildcard (global) import from roux i.e. 'from roux.global_imports import *' with individual imports with accompanying documentation.Parameters  input_path (str): path to the .py or .ipynb file.  output_path (str): path to the output.  py_path (str): path to the intermediate .py file.  in_place (bool): whether to carry out the modification in place.  return_replacements (bool): return dict with strings to be replaced.  attributes (dict): attribute names mapped to their keywords for searching.  verbose (bool): verbose toggle.  test (bool): test-mode if output file not provided and in-place modification not allowed.Returns:output_path(str):  path to the modified notebook.moduleroux.workflow.knitFor workflow set up.functionnb_to_pynb_to_py(notebookp:str,test:bool=False,validate:bool=True,sep_step:str='## step',notebook_suffix:str='_v')notebook to script.Args:notebookp(str):  path to the notebook.sep_step(str, optional):  separator marking the start of a step. Defaults to "## step".notebook_suffix(str, optional):  suffix of the notebook file to be considered as a "task".test(bool, optional):  test mode. Defaults to False.validate(bool, optional):  validate. Defaults to True.TODOs:  1. Addcheck_outputsparameter to only filter out non-executable code (i.e. tests) if False else edit the code.functionsort_stepnssort_stepns(l:list)→listSort steps (functions) of a task (script).Args:l(list):  list of steps.Returns:list:  sorted list of steps.moduleroux.workflow.logfunctionprint_parametersprint_parameters(d:dict)Print a directory with parameters as lines of codeParameters:d(dict):  directory with parametersmoduleroux.workflowGlobal Variablesiologmoduleroux.workflow.monitorFor workflow monitors.functionplot_workflow_logplot_workflow_log(dplot:DataFrame)→AxesPlot workflow log.Args:dplot(pd.DataFrame):  input data (dparam).Returns:plt.Axes:  output.TODOs: 1. use the statistics tagged as## stats.moduleroux.workflow.nbFor operations on jupyter notebooks.functionget_linesget_lines(p:str,keep_comments:bool=True)→listGet lines of code from notebook.Args:p(str):  path to notebook.keep_comments(bool, optional):  keep comments. Defaults to True.Returns:list:  lines.functionread_nb_mdread_nb_md(p:str,n:int=None)→listRead notebook's documentation in the markdown cells.Args:p(str):  path of the notebook.n(int):  number of the markdown cells to extract.Returns:list:  lines of the strings.functionto_infoto_info(p:str,outp:str,linkd:str='')→strSave README.md file.Args:p(str, optional):  path of the notebook files that would be converted to "tasks".outp(str, optional):  path of the output file, e.g. 'README.md'.Returns:str:  path of the output file.functionto_replaced_nbto_replaced_nb(nb_path,output_path,replaces:dict={},cell_type:str='code',drop_lines_with_substrings:list=None,test=False)Replace text in a jupyter notebook.Parameters  nb: notebook object obtained fromnbformat.reads.  replaces (dict): mapping of text to 'replace from' to the one to 'replace with'.  cell_type (str): the type of the cell.Returns:new_nb:  notebook object.functionto_filtered_nbto_filtered_nb(p:str,outp:str,header:str,kind:str='include',validate_diff:int=None)Filter sections in a notebook based on markdown headings.Args:header(str):  exact first line of a markdown cell marking a section in a notebook. validate_difffunctionto_filter_nbby_patternsto_filter_nbby_patterns(p,outp,patterns=None,**kws)Filter out notebook cells if the pattern string is found.Args:patterns(list):  list of string patterns.functionto_clear_unused_cellsto_clear_unused_cells(notebook_path,new_notebook_path,validate_diff:int=None)Remove code cells with all lines commented.functionto_clear_outputsto_clear_outputs(notebook_path,new_notebook_path)functionto_filtered_outputsto_filtered_outputs(input_path,output_path,warnings=True,strings=True)functionto_diff_notebooksto_diff_notebooks(notebook_paths,url_prefix='https://localhost:8888/nbdime/difftool?',remove_prefix='file://',verbose=True)→list"Diff" notebooks usingnbdiff(https://nbdime.readthedocs.io/en/latest/)Start the nb-diff session by running:nbdiff-webTodos:  1. Deprecate if functionality added tonbdiff-web.moduleroux.workflow.taskFor task management.functionrun_taskrun_task(parameters:dict,input_notebook_path:str,kernel:str=None,output_notebook_path:str=None,test=False,verbose=False,force=False,**kws_papermill)→strRun a single task.Prameters:  parameters (dict): parameters includingoutput_paths.  input_notebook_path (dict): path to the input notebook which is parameterized.  kernel (str): kernel to be used.  output_notebook_path: path to the output notebook which is used as a report.   test (bool): test-mode.  verbose (bool): verbose.Keyword parameters:  kws_papermill: parameters provided to thepm.execute_notebookfunction.Returns:Output path.functionrun_tasksrun_tasks(input_notebook_path:str,kernel:str=None,inputs:list=None,output_path_base:str=None,parameters_list:list=None,fast:bool=False,fast_workers:int=6,to_filter_nbby_patterns_kws=None,input_notebook_temp_path=None,out_paths:bool=False,test1:bool=False,force:bool=False,test:bool=False,verbose:bool=False,**kws_papermill)→listRun a list of tasks.Prameters:  input_notebook_path (dict): path to the input notebook which is parameterized.  kernel (str): kernel to be used.  inputs (list): list of parameters without the output paths, which would be inferred by encoding.    output_path_base (str): output path with a placeholder e.g. 'path/to/{KEY}/file'.    parameters_list (list): list of parameters including the output paths.    fast (bool): enable parallel-processing.  fast_workers (bool): number of parallel-processes.  force (bool): overwrite the outputs.  test (bool): test-mode.  verbose (bool): verbose.Keyword parameters:  kws_papermill: parameters provided to thepm.execute_notebookfunction.  to_filter_nbby_patterns_kws (list): dictionary containing parameters to be provided toto_filter_nbby_patternsfunction (Defaults to None).Returns:parameters_list(list):  list of parameters including the output paths, inferred if not provided.TODOs:  0. Ignore temporary parameters e.g test, verbose etc while encoding inputs.  1. Integrate with apply_on_paths for parallel processing etc.Notes:To resolveRuntimeError: This event loop is already running in pythonfrommultiprocessing, execute import nest_asyncio nest_asyncio.apply()moduleroux.workflow.versionFor version control.functiongit_commitgit_commit(repop:str,suffix_message:str='',force=False)Version control.Args:repop(str):  path to the repository.suffix_message(str, optional):  add suffix to the version (commit) message. Defaults to ''.moduleroux.workflow.workflowFor workflow management.functionget_scriptsget_scripts(ps:list,notebook_prefix:str='\\d{2}',notebook_suffix:str='_v\\d{2}',test:bool=False,fast:bool=True,cores:int=6,force:bool=False,tab:str='    ',**kws)→DataFrameGet scripts.Args:ps(list):  paths.notebook_prefix(str, optional):  prefix of the notebook file to be considered as a "task".notebook_suffix(str, optional):  suffix of the notebook file to be considered as a "task".test(bool, optional):  test mode. Defaults to False.fast(bool, optional):  parallel processing. Defaults to True.cores(int, optional):  cores to use. Defaults to 6.force(bool, optional):  overwrite the outputs. Defaults to False.tab(str, optional):  tab in spaces. Defaults to '    '.Returns:pd.DataFrame:  output table.functionto_scriptsto_scripts(packagep:str,notebooksdp:str,validate:bool=False,ps:list=None,notebook_prefix:str='\\d{2}',notebook_suffix:str='_v\\d{2}',scripts:bool=True,workflow:bool=True,sep_step:str='## step',todos:bool=False,git:bool=True,clean:bool=False,test:bool=False,force:bool=True,tab:str='    ',**kws)To scripts.Args:# packagen (str):  package name.packagep(str):  path to the package.notebooksdp(str, optional):  path to the notebooks. Defaults to None.validate(bool, optional):  validate if functions are formatted correctly. Defaults to False.ps(list, optional):  paths. Defaults to None.notebook_prefix(str, optional):  prefix of the notebook file to be considered as a "task".notebook_suffix(str, optional):  suffix of the notebook file to be considered as a "task".scripts(bool, optional):  make scripts. Defaults to True.workflow(bool, optional):  make workflow file. Defaults to True.sep_step(str, optional):  separator marking the start of a step. Defaults to "## step".todos(bool, optional):  show todos. Defaults to False.git(bool, optional):  save version. Defaults to True.clean(bool, optional):  clean temporary files. Defaults to False.test(bool, optional):  test mode. Defaults to False.force(bool, optional):  overwrite outputs. Defaults to True.tab(str, optional):  tab size. Defaults to '    '.Keyword parameters:kws:  parameters provided to theget_scriptfunction,   includingsep_stepandsep_step_endTODOs:1. For version control, use https: //github.com/jupyterlab/jupyterlab-git.