Package: sichuanMojo
sichuanMojoBuilt on the top ofpandas, sichuanMojo is a grammar of data manipulation with data frame, providing a consistent a series of utility functions that help you solve the most common data manipulation challenges:unite_col: unite cols into one columnseparate_col: separate column by a pattern into individual columngroupby_col: provide a summary table of a selected column which grouped by desired column(s)groupby_across: provide a summary table of a selected column with basic statistical infomation (min, max, sum, median) which grouped by desired column(s)pivot_tb: provide a pivot (count/frequency) table of a selected column which grouped by desired column(s)pivot_rate: provide a pivot (rate/percentage) table of a selected column which grouped by desired column(s)simplify_network_df: simplify the network dataframe from directed to undirecteInstallation from githubUse the package managerpipto install foobar.pip install sichuanMojoBackgroudThe Sichuan province is pandas' hometown which explains the name of this library!UsageimportsichuanMojoassmimportpandasaspd# test_dict as an example inputdata={'name':['Tom','nick','krish','jack','Mike','Jan'],'age':[20,21,19,18,21,33],'group':['A','A','A','B','B',"B"],'major':['biology','english','biology','english','biology','biology'],'response':['good','good','ok','bad','bad','bad']}# Create DataFrametest_df=pd.DataFrame(data)nameagegroupmajorresponse0Tom20Abiologygood1nick21Aenglishgood2krish19Abiologyok3jack18Benglishbad4Mike21Bbiologybadunite cols into one columnNote: if arg output_json = True, output will be json format. Otherwise, output will be a data frame.sm.unite_col(test_df,unite_by=["group","major","response"],pattern="; ",united_col_name="New_col",output_json=False)nameagegroupmajorresponseNew_col0Tom20AbiologygoodA; biology; good1nick21AenglishgoodA; english; good2krish19AbiologyokA; biology; ok3jack18BenglishbadB; english; bad4Mike21BbiologybadB; biology; badseparate column by a pattern into individual columnNote: if arg output_json = True, output will be json format. Otherwise, output will be a data frame.# Test datadata2={'name':['Tom','nick','krish','jack','Mike'],'age':[20,21,19,18,21],'group':['A','A','A','B','B'],'major':['biology','english','biology','english','biology'],'date':['2022-01-13','2022-06-23','2022-03-12','2022-08-23','2022-09-13']}# Create DataFrametest_df2=pd.DataFrame(data2)nameagegroupmajordate0Tom20Abiology2022-01-131nick21Aenglish2022-06-232krish19Abiology2022-03-123jack18Benglish2022-08-234Mike21Bbiology2022-09-13sm.separate_col(test_df2,sep_by="date",pattern="-",sep_to_names=["year","month","date"],output_json=False)nameagegroupmajordateyearmonth0Tom20Abiology132022011nick21Aenglish232022062krish19Abiology122022033jack18Benglish232022084Mike21Bbiology13202209provide a summary table of a selected column which grouped by desired column(s)Note: if argoutput_json = True, output will be json format. Otherwise, output will be a data frame.operator should be one of mean, sum, min, max, median, unique, count, nuniquesm.groupby_col(test_df,group_by=["group","major"],summmarize_at="name",operator="nunique",output_json=True)#returns [{'group': 'A', 'major': 'biology', 'name': 2},# {'group': 'A', 'major': 'english', 'name': 1},# {'group': 'B', 'major': 'biology', 'name': 2},# {'group': 'B', 'major': 'english', 'name': 1}]sm.groupby_col(test_df,group_by=["group","major"],summmarize_at="name",operator="nunique")groupmajorname0Abiology21Aenglish12Bbiology23Benglish1provide a summary table of a selected column with basic statistical infomation (min, max, sum, median) which grouped by desired column(s)Note: if arg output_json = True, output will be json format. Otherwise, output will be a data frame.sm.groupby_across(test_df,["group","major"],summmarize_at="age",operation=None,output_json=False)groupmajorage_minage_maxage_sumage_median0Abiology19203919.51Aenglish212121212Bbiology213354273Benglish18181818provide a pivot (count/frequency) table of a selected column which grouped by desired column(s)Note: if arg output_json = True, output will be json format. Otherwise, output will be a data frame.sm.pivot_tb(test_df,group_by=["group","major"],summmarize_at="response",operation="count",output_json=False,na_fill=0)groupmajorbadgoodok0Abiology0111Aenglish0102Bbiology2003Benglish100provide a pivot (rate/percentage) table of a selected column which grouped by desired column(s)Note: if arg output_json = True, output will be json format. Otherwise, output will be a data frame.sm.pivot_rate(test_df,group_by=["major","group"],summmarize_at="response",output_json=False,na_fill=0)majorgroupresponse_count_badresponse_count_goodresponse_count_okresponse_perc_badresponse_perc_goodresponse_perc_ok0biologyA011050501biologyB200100002englishA010010003englishB10010000provide an easy way to simplify network data frame from directed to undirectedNote: if argkeep = "last", output dataframe will keep the last row of duplicated rows. if argkeep = "first", output dataframe will keep the first row of duplicated rows.data={'from':['Tom','Jack','Jen','Sam'],'overlap':[20,21,19,18],'to':['Jack','Tom','Emily','John']}test_df=pd.DataFrame(data)fromoverlapto0Tom20Jack1Jack21Tom2Jen19Emily3Sam18JohnIn the test_df, we have duplicated Tom <--> Jack pairs.sm.simplify_network_df(test_df,from_col="from",to_col="to",keep="first")fromoverlapto0Tom20Jack2Jen19Emily3Sam18JohnWe can see output only keeps one Tom <--> Jack pair.