Package: a-pandas-ex-columns-and-index
Some useful Pandas methods for df.index and df.columnspipinstalla-pandas-ex-columns-and-indexfroma_pandas_ex_columns_and_indeximportpd_add_index_and_columnspd_add_index_and_columns()importpandasaspddf=pd.read_csv("https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_long.csv")The code above will add some new methods to your dfdf.d_swap_2_columnsdf.ds_sort_by_str_lengthdf.d_insert_column_before_anotherdf.ds_reversedf.d_add_prefix_to_column_when_regex_matchdf.d_add_prefix_to_index_when_regex_matchdf.d_filter_df_by_regex_in_indexdf.d_filter_df_by_regex_in_columnsdf.d_columns_upperdf.d_index_upperdf.d_index_lowerdf.d_columns_lowerdf.d_make_columns_dot_compatibledf.d_natsort_indexdf.d_natort_columnsdf.d_natsort_df_by_columnd_rename_indexd_rename_columnsAll methods added to pandas have one of this prefixes:ds_(for DataFrames and Series)s_(only for Series)d_(only for DataFrames)df.d_swap_2_columnsdf.columnsOut[3]:Index(['city','country','date.utc','location','parameter','value','unit'],dtype='object')print(df)citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df2=df.d_swap_2_columns('city','country')print(df2.columns)print(df2)Index(['country','city','date.utc','location','parameter','value','unit'],dtype='object')countrycitydate.utc...parametervalueunit0BEAntwerpen2019-06-1806:00:00+00:00...pm2518.0µg/m³1BEAntwerpen2019-06-1708:00:00+00:00...pm256.5µg/m³2BEAntwerpen2019-06-1707:00:00+00:00...pm2518.5µg/m³3BEAntwerpen2019-06-1706:00:00+00:00...pm2516.0µg/m³4BEAntwerpen2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267GBLondon2019-04-0906:00:00+00:00...no241.0µg/m³5268GBLondon2019-04-0905:00:00+00:00...no241.0µg/m³5269GBLondon2019-04-0904:00:00+00:00...no241.0µg/m³5270GBLondon2019-04-0903:00:00+00:00...no267.0µg/m³5271GBLondon2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.ds_sort_by_str_lengthdfOut[3]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df2=df.ds_sort_by_str_length('city')print(df2)citycountrydate.utc...parametervalueunit2635ParisFR2019-05-1505:00:00+00:00...no246.5µg/m³2182ParisFR2019-06-0309:00:00+00:00...no246.0µg/m³2183ParisFR2019-06-0308:00:00+00:00...no243.9µg/m³2184ParisFR2019-06-0307:00:00+00:00...no250.0µg/m³2185ParisFR2019-06-0306:00:00+00:00...no244.1µg/m³.....................3554AntwerpenBE2019-05-1915:00:00+00:00...no233.0µg/m³3555AntwerpenBE2019-05-1914:00:00+00:00...no223.0µg/m³3556AntwerpenBE2019-05-1913:00:00+00:00...no214.5µg/m³3548AntwerpenBE2019-05-1921:00:00+00:00...no212.5µg/m³0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³[5272rowsx7columns]d_insert_column_before_anotherdfOut[6]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_insert_column_before_another(df.city+df.country,'city_country','value')Out[7]:citycountrydate.utc...city_countryvalueunit0AntwerpenBE2019-06-1806:00:00+00:00...AntwerpenBE18.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...AntwerpenBE6.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...AntwerpenBE18.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...AntwerpenBE16.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...AntwerpenBE7.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...LondonGB41.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...LondonGB41.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...LondonGB41.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...LondonGB67.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...LondonGB67.0µg/m³[5272rowsx8columns]df.ds_reversedfOut[3]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.ds_reverse()Out[4]:citycountrydate.utc...parametervalueunit5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³.....................4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³[5272rowsx7columns]df.d_add_prefix_to_column_when_regex_matchdfOut[8]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_add_prefix_to_column_when_regex_match(prefix='aa_',regular_expression='^c')Out[9]:aa_cityaa_countrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_add_prefix_to_index_when_regex_matchOut[12]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_add_prefix_to_index_when_regex_match('five_',regular_expression='^5')Out[13]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................five_5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³five_5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³five_5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³five_5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³five_5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_filter_df_by_regex_in_columnsdfOut[14]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_filter_df_by_regex_in_columns('^[cu]')Out[15]:citycountryunit0AntwerpenBEµg/m³1AntwerpenBEµg/m³2AntwerpenBEµg/m³3AntwerpenBEµg/m³4AntwerpenBEµg/m³.........5267LondonGBµg/m³5268LondonGBµg/m³5269LondonGBµg/m³5270LondonGBµg/m³5271LondonGBµg/m³[5272rowsx3columns]df.d_filter_df_by_regex_in_indexdfOut[16]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_filter_df_by_regex_in_index(r'^5\d\d0$')Out[17]:citycountrydate.utc...parametervalueunit5000LondonGB2019-04-2016:00:00+00:00...no248.0µg/m³5010LondonGB2019-04-2006:00:00+00:00...no233.0µg/m³5020LondonGB2019-04-1920:00:00+00:00...no258.0µg/m³5030LondonGB2019-04-1910:00:00+00:00...no244.0µg/m³5040LondonGB2019-04-1823:00:00+00:00...no261.0µg/m³5050LondonGB2019-04-1813:00:00+00:00...no249.0µg/m³5060LondonGB2019-04-1803:00:00+00:00...no250.0µg/m³5070LondonGB2019-04-1717:00:00+00:00...no254.0µg/m³5080LondonGB2019-04-1707:00:00+00:00...no251.0µg/m³5090LondonGB2019-04-1620:00:00+00:00...no283.0µg/m³5100LondonGB2019-04-1609:00:00+00:00...no266.0µg/m³5110LondonGB2019-04-1522:00:00+00:00...no247.0µg/m³5120LondonGB2019-04-1512:00:00+00:00...no227.0µg/m³5130LondonGB2019-04-1502:00:00+00:00...no232.0µg/m³5140LondonGB2019-04-1416:00:00+00:00...no223.0µg/m³5150LondonGB2019-04-1406:00:00+00:00...no235.0µg/m³5160LondonGB2019-04-1320:00:00+00:00...no229.0µg/m³5170LondonGB2019-04-1310:00:00+00:00...no245.0µg/m³5180LondonGB2019-04-1300:00:00+00:00...no229.0µg/m³5190LondonGB2019-04-1214:00:00+00:00...no239.0µg/m³5200LondonGB2019-04-1204:00:00+00:00...no233.0µg/m³5210LondonGB2019-04-1116:00:00+00:00...no234.0µg/m³5220LondonGB2019-04-1106:00:00+00:00...no246.0µg/m³5230LondonGB2019-04-1019:00:00+00:00...no235.0µg/m³5240LondonGB2019-04-1009:00:00+00:00...no235.0µg/m³5250LondonGB2019-04-0923:00:00+00:00...no238.0µg/m³5260LondonGB2019-04-0913:00:00+00:00...no256.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³[28rowsx7columns]df.d_columns_upper, df.d_columns_lower,df.d_make_columns_dot_compatible,df.d_index_upper,df.d_index_lowerprint(df.columns)print(df.d_make_columns_dot_compatible().columns)print(df.d_columns_upper().columns)print(df.d_columns_lower().columns)df2=df.copy()df2.index=df2.parameterprint(df2.index)print(df2.d_index_upper().index)print(df2.d_index_lower().index)Index(['city','country','date.utc','location','parameter','value','unit'],dtype='object')Index(['city','country','date_utc','location','parameter','value','unit'],dtype='object')Index(['CITY','COUNTRY','DATE_UTC','LOCATION','PARAMETER','VALUE','UNIT'],dtype='object')Index(['city','country','date_utc','location','parameter','value','unit'],dtype='object')Index(['pm25','pm25','pm25','pm25','pm25','pm25','pm25','pm25','pm25','pm25',...'no2','no2','no2','no2','no2','no2','no2','no2','no2','no2'],dtype='object',name='parameter',length=5272)Index(['PM25','PM25','PM25','PM25','PM25','PM25','PM25','PM25','PM25','PM25',...'NO2','NO2','NO2','NO2','NO2','NO2','NO2','NO2','NO2','NO2'],dtype='object',length=5272)Index(['pm25','pm25','pm25','pm25','pm25','pm25','pm25','pm25','pm25','pm25',...'no2','no2','no2','no2','no2','no2','no2','no2','no2','no2'],dtype='object',length=5272)df.d_natsort_indexdf2=df.sample(len(df)).copy()dftempindex=df2.index[:2500].to_list()tempvalue=df2.loc[dftempindex].parameter.copy()tempvalue=tempvalue.apply(lambdax:str(x).upper())df2.loc[dftempindex,'parameter']=tempvalueprint(df2)df2.index=df2.parameterprint(df2.d_natsort_index())fora,b,c,dinzip(df2.d_natsort_index(sort_numbers_after_non_numbers=True).index.to_list(),df2.d_natsort_index(lowercase_first=True).index.to_list(),df2.d_natsort_index(group_lower_and_uppercase=True).index.to_list(),df2.d_natsort_index(uppercase_first=True).index.to_list()):print(a,b,c,d)no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2no2no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2no2NO2PM25no2df.d_natsort_df_by_columndfOut[3]:citycountrydate.utc...parametervalueunit0AntwerpenBE2019-06-1806:00:00+00:00...pm2518.0µg/m³1AntwerpenBE2019-06-1708:00:00+00:00...pm256.5µg/m³2AntwerpenBE2019-06-1707:00:00+00:00...pm2518.5µg/m³3AntwerpenBE2019-06-1706:00:00+00:00...pm2516.0µg/m³4AntwerpenBE2019-06-1705:00:00+00:00...pm257.5µg/m³.....................5267LondonGB2019-04-0906:00:00+00:00...no241.0µg/m³5268LondonGB2019-04-0905:00:00+00:00...no241.0µg/m³5269LondonGB2019-04-0904:00:00+00:00...no241.0µg/m³5270LondonGB2019-04-0903:00:00+00:00...no267.0µg/m³5271LondonGB2019-04-0902:00:00+00:00...no267.0µg/m³[5272rowsx7columns]df.d_natsort_df_by_column('date.utc')Out[4]:citycountrydate.utc...parametervalueunit176AntwerpenBE2019-04-0901:00:00+00:00...pm2576.0µg/m³3500ParisFR2019-04-0901:00:00+00:00...no224.4µg/m³3663AntwerpenBE2019-04-0901:00:00+00:00...no222.5µg/m³175AntwerpenBE2019-04-0902:00:00+00:00...pm2591.5µg/m³1824LondonGB2019-04-0902:00:00+00:00...pm2542.0µg/m³.....................1827ParisFR2019-06-2022:00:00+00:00...no226.5µg/m³178LondonGB2019-06-2023:00:00+00:00...pm257.0µg/m³1826ParisFR2019-06-2023:00:00+00:00...no221.8µg/m³177LondonGB2019-06-2100:00:00+00:00...pm257.0µg/m³1825ParisFR2019-06-2100:00:00+00:00...no220.0µg/m³[5272rowsx7columns]df.d_rename_columns / df.d_rename_indexdf=pd.read_csv("https://github.com/pandas-dev/pandas/raw/main/doc/data/titanic.csv")print(df)PassengerIdSurvivedPclass...FareCabinEmbarked0103...7.2500NaNS1211...71.2833C85C2313...7.9250NaNS3411...53.1000C123S4503...8.0500NaNS.......................88688702...13.0000NaNS88788811...30.0000B42S88888903...23.4500NaNS88989011...30.0000C148C89089103...7.7500NaNQdf.d_rename_columns(Fare='Embarked',Embarked='Fare',Cabin='cabin2')df.d_rename_index({1:1000000,2:50022})print(df)PassengerIdSurvivedPclass...Embarkedcabin2Fare0103...7.2500NaNS1000000211...71.2833C85C50022313...7.9250NaNS3411...53.1000C123S4503...8.0500NaNS.....................88688702...13.0000NaNS88788811...30.0000B42S88888903...23.4500NaNS88989011...30.0000C148C89089103...7.7500NaNQ[891rowsx12columns]df.d_rename_columns({'Embarked':'Fare','Fare':'Embarked','cabin2':'Cabin'})df.index=df.index.astype('string')df.index='a'+df.indexdf.d_rename_index(a1000000=1,a50022=2)print(df)PassengerIdSurvivedPclass...FareCabinEmbarkeda0103...7.2500NaNS1211...71.2833C85C2313...7.9250NaNSa3411...53.1000C123Sa4503...8.0500NaNS.....................a88688702...13.0000NaNSa88788811...30.0000B42Sa88888903...23.4500NaNSa88989011...30.0000C148Ca89089103...7.7500NaNQ