Package: odus
Table of Contents1Introduction2Installation3Getting some resources4Poking around4.1df_store4.2Viewing trajectories4.2.1Viewing a single trajectory4.2.2Flip over all (or some) trajectories4.3Making a pdf of trajectories4.4Demo s and v4.5cstore4.6pstore4.7Scrap place5Potential Calculus Experimentations6Extra scrap# %load_ext autoreload# %autoreload 2IntroductionODUS (for Older Drug User Study) contains data and tools to study the drug use of older drug users.Essentially, there are these are tools:To get prepared data on the 119 "trajectories" describing 31 variables (drug use, social, etc.) over time of 119 different respondents.To vizualize these trajectories in various waysTo create pdfs of any selection of these trajectories and variablesTo make count tables for any combinations of the variables: Essential step of any Markovian or Bayesian analysis.To make probability (joint or conditional) tables from any combination of the variablesTo operate on these count and probability tables, thus enabling inference operationsInstallationYou need to have python 3.7+ to run this notebook.And you'll need to haveodus, which you get by doingpip install odus(And if you don't have pip then, well... how to put it... ha ha ha!)But if you're the type, you can also just get the source fromhttps://github.com/thorwhalen/odus.Oh, and pull requests etc. are welcome!Stars, likes, references, and coffee also welcome.And if you want to donate: Donate to a charity that will help the people understand and make policies surrounding the use of substances.A simple flowchart about the architecture:Getting some resourcesfrommatplotlib.pylabimport*fromnumpyimport*importseabornassnsimportosfrompy2store.stores.local_storeimportRelativePathFormatStorefrompy2store.mixinsimportReadOnlyMixinfrompy2store.baseimportStorefromioimportBytesIOfromspyn.ppi.potimportPot,ProbPotfromcollectionsimportUserDict,Counterimportnumpyasnpimportpandasaspdfromut.ml.feature_extraction.sequential_var_setsimportPVar,VarSet,DfData,VarSetFactoryfromIPython.displayimportImagefromodus.analysis_utilsimport*fromodus.daccimportDfStore,counts_of_kps,Dacc,VarSetCountsStore,\mk_pvar_struct,PotStore,_commun_columns_of_dfs,Struct,mk_pvar_str_struct,VarStrfromodus.plot_utilsimportplot_life_coursefromodusimportdata_dir,data_path_ofsurvey_dir=data_dirdata_dir'/D/Dropbox/dev/p3/proj/odus/odus/data'df_store=DfStore(data_dir+'/{}.xlsx')len(df_store)cstore=VarSetCountsStore(df_store)v=mk_pvar_struct(df_store,only_for_cols_in_all_dfs=True)s=mk_pvar_str_struct(v)f,df=cstore.df_store.head()pstore=PotStore(df_store)Poking arounddf_storeA df_store is a key-value store where the key is the xls file and the value is the prepared dataframelen(df_store)119it=iter(df_store.values())foriinrange(5):# skip five first_=next(it)df=next(it)# get the one I wantdf.head(3)<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>categoryRURALSUBURBANURBAN/CITYHOMELESSINCARCERATIONWORKSON/DAUGHTERSIBLINGFATHER/MOTHERSPOUSE...METHAMPHETAMINEAS PRESCRIBED OPIOIDNOT AS PRESCRIBED OPIOIDHEROINOTHER OPIOIDINJECTEDIN TREATMENTSelects States belowGeorgiaPennsylvaniaage110100001100...0000000110120100001100...0100000110130100001100...00000001103 rows × 31 columnsprint(df.columns.values)['RURAL' 'SUBURBAN' 'URBAN/CITY' 'HOMELESS' 'INCARCERATION' 'WORK'
 'SON/DAUGHTER' 'SIBLING' 'FATHER/MOTHER' 'SPOUSE'
 'OTHER (WHO?, FILL IN BRACKETS HERE)' 'FRIEND USER' 'FRIEND NON USER'
 'MENTAL ILLNESS' 'PHYSICAL ILLNESS' 'LOSS OF LOVED ONE' 'TOBACCO'
 'MARIJUANA' 'ALCOHOL' 'HAL/LSD/XTC/CLUBDRUG' 'COCAINE/CRACK'
 'METHAMPHETAMINE' 'AS PRESCRIBED OPIOID' 'NOT AS PRESCRIBED OPIOID'
 'HEROIN' 'OTHER OPIOID' 'INJECTED' 'IN TREATMENT' 'Selects States below'
 'Georgia' 'Pennsylvania']t=df[['ALCOHOL','TOBACCO']]t.head(3)<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>categoryALCOHOLTOBACCOage110012001300c=Counter()fori,rint.iterrows():c.update([tuple(r.to_list())])cCounter({(0, 0): 6, (1, 0): 4, (1, 1): 9, (0, 1): 2})defcount_tuples(dataframe):c=Counter()fori,rindataframe.iterrows():c.update([tuple(r.to_list())])returncfields=['ALCOHOL','TOBACCO']# do it for every onec=Counter()fordfindf_store.values():c.update(count_tuples(df[fields]))cCounter({(0, 1): 903, (1, 1): 1343, (0, 0): 240, (1, 0): 179})pd.Series(c)0  1     903
1  1    1343
0  0     240
1  0     179
dtype: int64# Powerful! You can use that with several pairs and get some nice probabilities. Look up Naive Bayes.Viewing trajectoriesimportitertoolsfromfunctoolsimportpartialfromodus.utilimportwrite_imagesfromodus.plot_utilsimportplot_life,life_plots,write_trajectories_to_fileihead=lambdait:itertools.islice(it,0,5)Viewing a single trajectoryk=next(iter(df_store))# get the first keyprint(f"k:{k}")# print itplot_life(df_store[k])# plot the trajectoryk: surveys/B24.xlsxplot_life(df_store[k],fields=[s.in_treatment,s.injected])# only want two fieldsFlip over all (or some) trajectoriesgen=life_plots(df_store)next(gen)# launch to get the next trajectory<matplotlib.axes._subplots.AxesSubplot at 0x12b21f070>Get three trajectories, but only over two fields.# fields = [s.in_treatment, s.injected]fields=[s.physical_illness,s.as_prescribed_opioid,s.heroin,s.other_opioid]keys=list(df_store)[:10]# print(f"keys={keys}")axs=[xforxinlife_plots(df_store,fields,keys=keys)];Making a pdf of trajectorieswrite_trajectories_to_file(df_store,fields,keys,fp='three_respondents_two_fields.pdf');write_trajectories_to_file(df_store,fp='all_respondents_all_fields.pdf');Demo s and vprint(list(filter(lambdax:notx.startswith('__'),dir(s))))['alcohol', 'as_prescribed_opioid', 'cocaine_crack', 'father_mother', 'hal_lsd_xtc_clubdrug', 'heroin', 'homeless', 'in_treatment', 'incarceration', 'injected', 'loss_of_loved_one', 'marijuana', 'mental_illness', 'methamphetamine', 'not_as_prescribed_opioid', 'other_opioid', 'physical_illness', 'rural', 'sibling', 'son_daughter', 'suburban', 'tobacco', 'urban_city', 'work']s.heroin'HEROIN'v.heroinPVar('HEROIN', 0)v.heroin-1PVar('HEROIN', -1)cstore# cstore[v.alcohol, v.tobacco]cstore[v.as_prescribed_opioid-1,v.heroin]Counter({(0, 0): 1026, (1, 0): 264, (0, 1): 1108, (1, 1): 148})pd.Series(cstore[v.as_prescribed_opioid-1,v.heroin])0  0    1026
1  0     264
0  1    1108
1  1     148
dtype: int64cstore[v.alcohol,v.tobacco,v.heroin]Counter({(0, 0, 1): 427,
         (1, 0, 1): 656,
         (1, 1, 1): 687,
         (0, 0, 0): 189,
         (0, 1, 1): 476,
         (0, 1, 0): 51,
         (1, 0, 0): 133,
         (1, 1, 0): 46})cstore[v.alcohol-1,v.alcohol]Counter({(0, 0): 994, (1, 1): 1375, (1, 0): 90, (0, 1): 87})cstore[v.alcohol-1,v.alcohol,v.tobacco]Counter({(0, 0, 1): 807,
         (1, 1, 1): 1220,
         (1, 0, 0): 26,
         (0, 1, 1): 76,
         (0, 0, 0): 187,
         (1, 1, 0): 155,
         (0, 1, 0): 11,
         (1, 0, 1): 64})t=pd.Series(cstore[v.alcohol-1,v.alcohol,v.tobacco])t.loc[t.index]<pandas.core.indexing._LocIndexer at 0x130955db0>pstoret=pstore[s.alcohol-1,s.alcohol]tpval
ALCOHOL-1 ALCOHOL      
0         0         994
          1          87
1         0          90
          1        1375t.tb<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>ALCOHOL-1ALCOHOLpval0099401871090111375t/[]pval
ALCOHOL-1 ALCOHOL          
0         0        0.390416
          1        0.034171
1         0        0.035350
          1        0.540063t[s.alcohol-1]pval
ALCOHOL-1      
0          1081
1          1465t/t[s.alcohol-1]# cond prob!pval
ALCOHOL-1 ALCOHOL          
0         0        0.919519
          1        0.080481
1         0        0.061433
          1        0.938567tt=pstore[s.alcohol,s.tobacco]ttpval
ALCOHOL TOBACCO      
0       0         240
        1         903
1       0         179
        1        1343tt/tt[s.alcohol]pval
ALCOHOL TOBACCO          
0       0        0.209974
        1        0.790026
1       0        0.117608
        1        0.882392tt/tt[s.tobacco]pval
ALCOHOL TOBACCO          
0       0        0.572792
1       0        0.427208
0       1        0.402048
1       1        0.597952Scrap placet=pstore[s.as_prescribed_opioid-1,s.heroin-1,s.heroin]tpval
AS PRESCRIBED OPIOID-1 HEROIN-1 HEROIN      
0                      0        0        927
                                1        172
                       1        0         99
                                1        936
1                      0        0        249
                                1         33
                       1        0         15
                                1        115tt=t/t[s.as_prescribed_opioid-1,s.heroin-1]# cond prob!ttpval
AS PRESCRIBED OPIOID-1 HEROIN-1 HEROIN          
0                      0        0       0.843494
                                1       0.156506
                       1        0       0.095652
                                1       0.904348
1                      0        0       0.882979
                                1       0.117021
                       1        0       0.115385
                                1       0.884615tt.tb<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>AS PRESCRIBED OPIOID-1HEROIN-1HEROINpval0000.8434940010.1565060100.0956520110.9043481000.8829791010.1170211100.1153851110.884615AS PRESCRIBED OPIOID-1	HEROIN-1	HEROIN	
0	0	0	0.843494
0	0	1	0.156506
1	0	0	0.882979
1	0	1	0.1170210.117021/0.1565060.7477093529960512prob_of_heroin_given_presc_op=0.359223prob_of_heroin_given_not_presc_op=0.519213prob_of_heroin_given_presc_op/prob_of_heroin_given_not_presc_op0.6918605658949217prob_of_heroin_given_not_presc_op/prob_of_heroin_given_presc_op1.4453779407220584Potential Calculus Experimentations# survey_dir = '/D/Dropbox/others/Miriam/python/ProcessedSurveys'df_store=DfStore(survey_dir+'/{}.xlsx')len(df_store)119cstore=VarSetCountsStore(df_store)v=mk_pvar_struct(df_store,only_for_cols_in_all_dfs=True)s=mk_pvar_str_struct(v)f,df=cstore.df_store.head()df.head(3)<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>categoryRURALSUBURBANURBAN/CITYHOMELESSINCARCERATIONWORKSON/DAUGHTERSIBLINGFATHER/MOTHERSPOUSE...HAL/LSD/XTC/CLUBDRUGCOCAINE/CRACKMETHAMPHETAMINEAS PRESCRIBED OPIOIDNOT AS PRESCRIBED OPIOIDHEROINOTHER OPIOIDINJECTEDIN TREATMENTMassachusettsage160100101110...0000000001170100011110...0000100001180100011110...00001000013 rows × 29 columnscstore=VarSetCountsStore(df_store)cstore.mk_pvar_attrs()fromodus.daccimportDfStore,counts_of_kps,Dacc,plot_life_course,VarSetCountsStore,mk_pvar_struct,PotStorepstore=PotStore(df_store)pstore.mk_pvar_attrs()p=pstore[v.homeless-1,v.incarceration]ppval
HOMELESS-1 INCARCERATION      
0          0              1690
           1               577
1          0               192
           1                87p/[]pval
HOMELESS-1 INCARCERATION          
0          0              0.663786
           1              0.226630
1          0              0.075412
           1              0.034171pstore[v.incarceration]pval
INCARCERATION      
0              1989
1               676pstore[v.alcohol-1,v.loss_of_loved_one]pval
ALCOHOL-1 LOSS OF LOVED ONE      
0         0                   990
          1                    91
1         0                  1321
          1                   144tw=pstore[v.tobacco,v.work]mw=pstore[v.marijuana,v.work]aw=pstore[v.alcohol,v.work]w=pstore[v.work]evid_t=Pot.from_hard_evidence(**{s.tobacco:1})evid_m=Pot.from_hard_evidence(**{s.marijuana:1})evid_a=Pot.from_hard_evidence(**{s.alcohol:1})evid_apval
ALCOHOL      
1           1awpval
ALCOHOL WORK      
0       0      431
        1      712
1       0      448
        1     1074w/[]pval
WORK          
0     0.329831
1     0.670169(evid_m*mw)/[]pval
MARIJUANA WORK          
1         0     0.350603
          1     0.649397(evid_t*tw)/[]pval
TOBACCO WORK          
1       0     0.313001
        1     0.686999(evid_a*aw)/[]pval
ALCOHOL WORK         
1       0     0.29435
        1     0.70565Extra scrap# from graphviz import Digraph# Digraph(body="""# raw -> data -> count -> prob# raw [label="excel files (one per respondent)" shape=folder]# data [label="dataframes" shape=folder]# count [label="counts for any combinations of the variables in the data" shape=box3d]# prob [label="probabilities for any combinations of the variables in the data" shape=box3d]# """.split('\n'))