Saravanan666/pypi-org huggingface dataset:
Package: a-pandas-ex-bs4df-lite
Light version ofhttps://github.com/hansalemaos/a_pandas_ex_bs4dfLess details, but much fasterpip install a_pandas_ex_bs4df_litefroma_pandas_ex_bs4df_liteimportpd_add_bs4_to_df_liteimportpandasaspdpd_add_bs4_to_df_lite()df=pd.Q_bs4_to_df_lite(r'https://github.com/search?l=Python&q=python&type=Repositories',parser="lxml",fake_header=True)aa_nameaa_contentsaa_parentaa_previous_elementaa_next_elementaa_next_siblingaa_previous_siblingaa_hiddenaa_element_indexaa_valueaa_key990div[\nJumpto...[\n,[\n,[\n,<...\n\nJumpto\...\n\nFalse159d-noneclass991div[\nJumpto...[\n,[\n,[\n,<...\n\nJumpto\...\n\nFalse159d-on-nav-focusclass992div[\nJumpto...[\n,[\n,[\n,<...\n\nJumpto\...\n\nFalse159js-jump-to-badge...class993span[↵][\nJumpto...\nJumpto\...↵\n\nJumpto\...False160d-inline-blockclass994span[↵][\nJumpto...\nJumpto\...↵\n\nJumpto\...False160ml-1class995span[↵][\nJumpto...\nJumpto\...↵\n\nJumpto\...False160v-align-middleclass996ul[\n,[\n,[Nosu...[\n,[\n,[\n,<...\n\n\n\nFalse161d-noneclass997ul[\n,[\n,[Nosu...[\n,[\n,[\n,<...\n\n\n\nFalse161js-jump-to-no-re...class998li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162d-flexclass999li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162flex-justify-centerclass1000li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162flex-items-centerclass1001li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162f5class1002li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162d-noneclass1003li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162js-jump-to-sugge...class1004li[\n,[Nosuggest...[\n,[\n,[Nosu...\n\n\n\nFalse162p-2class
Package: a-pandas-ex-bstcfg2df
Edit bluestacks.conf with pandaspip install a-pandas-ex-bstcfg2dfTested against BlueStacks 5.10froma_pandas_ex_bstcfg2dfimportget_bst_config_df,write_df_to_bstcfgdf=get_bst_config_df(conffile=r"C:\ProgramData\BlueStacks_nxt\bluestacks.conf")print(df)"""aa_key_1                     aa_key_2  ... aa_values aa_values_stripped0        bst        bluestacks_account_id  ...        ""1        bst                campaign_hash  ...        ""2        bst                campaign_name  ...        ""3        bst           custom_resolutions  ...        ""4        bst  feedback_popup_ignored_apps  ...        ""..       ...                          ...  ...       ...                ...687      bst                     instance  ...   "52383"              52383688      bst                     instance  ...   "52383"              52383689      bst                     instance  ...   "52383"              52383690      bst                     instance  ...   "52383"              52383691      bst                     instance  ...   "54468"              54468[692 rows x 7 columns]"""write_df_to_bstcfg(df,conffile=r"C:\ProgramData\BlueStacks_nxt\bluestacks.conf",value_column="aa_values",)write_df_to_bstcfg(df,conffile=r"C:\ProgramData\BlueStacks_nxt\bluestacks.conf",value_column="aa_values_stripped",)
Package: a-pandas-ex-closest-color
Calculates the closest colors from 2 listspipinstalla-pandas-ex-closest-colorfroma_pandas_ex_closest_colorimportpd_add_closest_colorimportpandasaspdpd_add_closest_color()colorlist=[(0,0,0),# black(230,25,75),# red(60,180,75),# green(255,225,25),# yellow(0,130,200),# blue(245,130,48),# orange(145,30,180),# purple(70,240,240),# cyan(240,50,230),# magenta(210,245,60),# lime(250,190,190),# pink(0,128,128),# teal(230,190,255),# lavender(170,110,40),# brown(255,250,200),# beige(128,0,0),# maroon(170,255,195),# mint(128,128,0),# olive(255,215,180),# coral(0,0,128),# navy(128,128,128),# grey(255,255,255),# white(115,12,37),# dark red(30,90,37),# dark green(127,112,12),# dark yellow(0,65,100),# dark blue(122,65,24),# dark orange(72,15,90),# dark purple(35,120,120),# dark cyan(120,25,115),# dark magenta(105,122,30),# dark lime(125,95,95),# dark pink(0,64,64),# dark teal(115,95,127),# dark lavender(85,55,20),# dark brown(127,125,100),# dark beige(64,0,0),# dark maroon(85,127,97),# dark mint(64,64,0),# dark olive(127,107,90),# dark coral(0,0,64),# dark navy(64,64,64),# dark grey]wanted_colors=[(255,0,0),(255,255,0),(0,0,0)]df=pd.Q_find_closest_color(wanted_colors=wanted_colors,colorlist=colorlist)print(df)rgbratingrgb0230257582.9375(255,0,0)112800127.0000(255,0,0)224513048139.0000(255,0,0)317011040144.6250(255,0,0)41151237145.2500(255,0,0).................121250190190367.0000(0,0,0)122255215180379.0000(0,0,0)123230190255392.5000(0,0,0)124255250200409.2500(0,0,0)125255255255441.7500(0,0,0)[126rowsx5columns]
Package: a-pandas-ex-closest-neighbours
Find the lower and upper neighbours in a pandas.Seriespipinstalla-pandas-ex-closest-neighboursUsage:froma_pandas_ex_closest_neighboursimportpd_add_closest_neighboursimportpandasaspdfromrandomimportchoice# random dataframepd_add_closest_neighbours()sizes=list(range(1,100))df=pd.DataFrame([choice(sizes)forxinrange(1000)])df.columns=['num']dfOut[3]:num017132217390476.....9956999682997659988499962[1000rowsx1columns]min_neighbours2,max_neighbours2=df.num.s_find_closest_neighbours(value=72,convertdtypes=True,accept_exact_match=False)min_neighbours2Out[8]:lower_indexlower_value0957111847122657132917144867155697166447176527187427198047110830711196471max_neighbours2Out[9]:upper_indexupper_value026731447321197331617342197353977364157374927385937396107310612731180273min_neighbours2,max_neighbours2=df.num.s_find_closest_neighbours(value=72,convertdtypes=True,accept_exact_match=True)max_neighbours1Out[4]:upper_indexupper_value010572114772221072328172431772536172637772738672848572952172106757211956721295772min_neighbours1Out[5]:lower_indexlower_value010572114772221072328172431772536172637772738672848572952172106757211956721295772Parameters:series:pd.SeriesOnlypandas.Seriesvalue:Union[float,int]thevalueyouwanttohavetheclosestneighboursfromconvertdtypes:boolconvertsstringnumberstonumbers(default=False)accept_exact_match:boolexample:True:ifyouarepassing72,and72isintheSeries,theminimumandmaximumvalueswillbeallthesame,thatmeans:72False:Allcellswith72areexcluded,andyouwon't see 72 in the results(default=False)Returns:tuple
Package: a-pandas-ex-combinatoric-iterators-to-df
Creates DataFrames from product, permutations, combinations, combinations_with_replacement with best dtype# Tested with:# Python 3.9.13# Windows 10pipinstalla-pandas-ex-combinatoric-iterators-to-dffroma_pandas_ex_combinatoric_iterators_to_dfimportpd_add_combinatoric_iterators_to_dfimportpandasaspdpd_add_combinatoric_iterators_to_df()df1=pd.Q_product_to_df(iterable=list(range(256)),n=3)df1Out[3]:01200001001200230034004.........1677721125525525116777212255255252167772132552552531677721425525525416777215255255255[16777216rowsx3columns]df1.dtypesOut[4]:0uint81uint82uint8dtype:objectdf2=pd.Q_permutations_to_df(iterable=list(range(256)),n=3)df2Out[5]:01200121013201430154016.........1658111525525424916581116255254250165811172552542511658111825525425216581119255254253[16581120rowsx3columns]df3=pd.Q_combinations_to_df(iterable=list(range(256)),n=3)df3Out[6]:01200121013201430154016.........27635152512542552763516252253254276351725225325527635182522542552763519253254255[2763520rowsx3columns]df4=pd.Q_combinations_with_replacement_to_df(iterable=list(range(256)),n=3)df4Out[7]:01200001001200230034004.........28290512532552552829052254254254282905325425425528290542542552552829055255255255[2829056rowsx3columns]"""
Package: a-pandas-ex-console-plot
Retro Pandas - Plotting back in 1985$pipinstalla-pandas-ex-console-plotfroma_pandas_ex_console_plotimportpd_add_console_plotimportpandasaspdimportnumpyasnppd_add_console_plot()#https://pandas.pydata.org/pandas-docs/stable/user_guide/visualization.htmldf2=pd.DataFrame(np.random.rand(10,4),columns=["a","b","c","d"])df2.ds_console_plot(sizepic=70,kind="bar",subplots=False)Isn't that beautiful? hahahaha
Package: a-pandas-ex-crosstab-dict
Pandas DataFrame to dict with crosstab DataFrames$pipinstalla-pandas-ex-crosstab-dictfroma_pandas_ex_crosstab_dictimportpd_add_crosstab_dictimportpandasaspdpd_add_crosstab_dict()df=pd.read_csv("https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv")df.ds_get_crosstab_dict(maincolumn="Cabin",columns=None)df.ds_get_crosstab_dict(maincolumn="Cabin",columns=["Survived","Sex",'Fare'])Out[5]:{'Survived':CabinA10A14A16A19A20A23A24...FG73F2F33F38F4G6TSurvived...01101001...210102110010110...0230220[2rowsx147columns],'Sex':CabinA10A14A16A19A20A23A24...FG73F2F33F38F4G6TSex...female0010000...0030140male1101111...2301101[2rowsx147columns],'Fare':CabinA10A14A16A19A20A23A24...FG73F2F33F38F4G6TFare...0.00000000000...00000005.00000000000...00000007.65000000000...20000007.75000000000...0001000
Package: a-pandas-ex-df2htmlstring
exports pandas DataFrames/Series as HTML (formatted string that looks like a table!)pip install a-pandas-ex-df2htmlstringTested against Windows 10 / Python 3.10 / AnacondaHow to use itfroma_pandas_ex_df2htmlstringimportpd_add_df2htmlstringpd_add_df2htmlstring()importpandasaspdfromrandomimportchoicecsvtests=["https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_long.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_no2.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_no2_long.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_parameters.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_pm25_long.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_stations.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/baseball.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/titanic.csv",]csvfile=choice(csvtests)df=pd.read_csv(csvfile)outfile="e:\\outputhtml.html"df.ds_2htmlstring(outputfile=outfile,fontsize=12,fontcolor="black",repeat_columns_n_rows=70)os.startfile(outfile)df.Name.ds_2htmlstring(outputfile=outfile,fontsize=12,fontcolor="black",repeat_columns_n_rows=70)ConvertsapandasDataFrametoanHTMLstringwithcustomizablefontsize,fontcolor,andrepeatedcolumnheaders.Args:df(pandas.DataFrame/pandas.Series):TheDataFrametobeconvertedtoanHTMLstring.(automaticallypassed)outputfile(str,optional):ThenameoftheoutputfiletosavetheHTMLstringto.DefaultstoNone(won't be saved on HDD).fontsize(int,optional):ThefontsizeoftheHTMLstring.Defaultsto12.fontcolor(str,optional):ThefontcoloroftheHTMLstring.Defaultsto"black".repeat_columns_n_rows(int,optional):Thenumberofrowsafterwhichtorepeatthecolumnheaders.Defaultsto70.Returns:str:TheHTMLstringrepresentationoftheDataFrame.
Package: a-pandas-ex-df-to-string
Installationpipinstalla-pandas-ex-df-to-stringUsagefroma_pandas_ex_df_to_stringimportpd_add_to_stringpd_add_to_string()importpandasaspdfromrandomimportchoicecsvtests=["https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_long.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_no2.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_no2_long.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_parameters.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_pm25_long.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/air_quality_stations.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/baseball.csv","https://github.com/pandas-dev/pandas/raw/main/doc/data/titanic.csv",]csvfile=choice(csvtests)df=pd.read_csv(csvfile)print(df)print(df.dtypes)df2=df.ds_to_string()print(df2)print(df2.dtypes)iddescriptionname0bcBlackCarbonBC1coCarbonMonoxideCO2no2NitrogenDioxideNO23o3OzoneO34pm10Particulatematterlessthan10micrometersin...PM105pm25Particulatematterlessthan2.5micrometersi...PM2.56so2SulfurDioxideSO2idobjectdescriptionobjectnameobjectdtype:objectiddescriptionname0bcBlackCarbonBC1coCarbonMonoxideCO2no2NitrogenDioxideNO23o3OzoneO34pm10Particulatematterlessthan10micrometersin...PM105pm25Particulatematterlessthan2.5micrometersi...PM2.56so2SulfurDioxideSO2idstringdescriptionstringnamestringdtype:object
Package: a-pandas-ex-df-updater
pipinstalla-pandas-ex-df-updater#    Update a DataFrame/Series with the values of another.froma_pandas_ex_df_updaterimportpd_add_df_updaterpd_add_df_updater()df1=pd.DataFrame(np.array([[1,2,3],[4,5,6],[7,8,9]]),columns=['a','b','c'])df2=pd.DataFrame(np.array([[11,2,3],[4,55,6],[7,8,9],[17,2,93]]),columns=['a','x','y'])df3=df1.ds_update(forupdate=df2,update_existing_values=True,add_new_columns=False,add_new_rows=False)df1abc012314562789df2axy01123145562789317293df3abc0112314562789df3=df1.ds_update(forupdate=df2,update_existing_values=True,add_new_columns=True,add_new_rows=False)df1abc012314562789df2axy01123145562789317293df3abcxy01123231456556278989df3=df1.ds_update(forupdate=df2,update_existing_values=True,add_new_columns=True,add_new_rows=True)df1abc012314562789df2axy01123145562789317293df3abcxy0112.03.023145.06.0556278.09.089317NaNNaN293df1.ds_update(forupdate=df2,update_existing_values=False,add_new_columns=True,add_new_rows=True)df1abc012314562789df2axy01123145562789317293df3abcxy012.03.023145.06.0556278.09.089317NaNNaN293Parameters:df:Union[pd.Series,pd.DataFrame]forupdate:Union[pd.Series,pd.DataFrame]update_existing_values:bool(default=True)add_new_columns=True(default=True)add_new_rows=True(default=True)Returns:Union[pd.Series,pd.DataFrame]
Package: a-pandas-ex-dillpickle
Pickle pandas DataFrames without "AttributeError: Can't pickle local object ... "pip install a-pandas-ex-dillpickleimportpandasaspdfroma_pandas_ex_dillpickleimportpd_add_dillpickle# Good example, because dirdf includes lambda functions: https://github.com/hansalemaos/DirDFfromdirdfimportpd_add_dfdirpd_add_dfdir()pd_add_dillpickle()df2=pd.Q_folder_to_df_with_functions(folder=r"F:\30000",ls_path="ls",last_access_time=True,exit_keys="ctrl+x",timeout=None,strings_path="strings",fzf_path="fzf",rip_grep_path="rg.exe",add_flatcopy_sorted=True,add_flatcopy=True,add_extract_strings=True,add_fuzzy_extract=True,add_ripgrep=True,add_open_file=True,add_move_file=True,)print(df2)aa_date...ff_flatcopy02021-03-0803:41:41.239919600-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...12021-03-0803:41:44.653175900-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...22021-03-0803:41:47.120014100-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...32021-03-0804:01:53.086047300-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...42021-03-0805:11:38.713301-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...52021-03-0805:12:22.607746500-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...62021-03-0805:32:22.251188300-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...[7rowsx21columns]# regular pandas versiondf2.to_pickle('f:\\testpi.pkl')Traceback(mostrecentcalllast):File"C:\Users\Gamer\anaconda3\envs\dfdir\lib\site-packages\IPython\core\interactiveshell.py",line3398,inrun_codeexec(code_obj,self.user_global_ns,self.user_ns)File"<ipython-input-3-b78c152ff8d4>",line27,in<cellline:27>df2.to_pickle('f:\\testpi.pkl')File"C:\Users\Gamer\anaconda3\envs\dfdir\lib\site-packages\pandas\core\generic.py",line3064,into_pickleto_pickle(File"C:\Users\Gamer\anaconda3\envs\dfdir\lib\site-packages\pandas\io\pickle.py",line112,into_picklepickle.dump(obj,handles.handle,protocol=protocol)AttributeError:Can't pickle local object 'copy_func.<locals>.<lambda>'# dillpickle - uses https://github.com/uqfoundation/dilldf2.to_dillpickle('f:\\dillpi.pkl')# The file can be loaded using dill or pickledf3=pd.read_dillpickle('f:\\dillpi.pkl')df3=pd.read_pickle('f:\\dillpi.pkl')print(df3)aa_date...ff_flatcopy02021-03-0803:41:41.239919600-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...12021-03-0803:41:44.653175900-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...22021-03-0803:41:47.120014100-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...32021-03-0804:01:53.086047300-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...42021-03-0805:11:38.713301-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...52021-03-0805:12:22.607746500-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...62021-03-0805:32:22.251188300-03:00...dest_folder:str,foldersep:str='ǀ',symlink:bo...[7rowsx21columns]
Package: a-pandas-ex-duplicates-to-df
Creates a DataFrame/Series from duplicatespipinstalla-pandas-ex-duplicates-to-dffroma_pandas_ex_duplicates_to_dfimportpd_add_duplicates_to_dfimportpandasaspdpd_add_duplicates_to_df()df=pd.read_csv("https://github.com/pandas-dev/pandas/raw/main/doc/data/titanic.csv")df2=pd.read_csv("https://github.com/pandas-dev/pandas/raw/main/doc/data/titanic.csv")[:50]df=pd.concat([df,df2],ignore_index=True)dupl=df.ds_get_duplicates()duplOut[5]:PassengerIdSurvivedPclass...CabinEmbarkedDUPLICATEINDEX0103...NaNS(0,891)1103...NaNS(0,891)21012...NaNC(9,900)31012...NaNC(9,900)41113...G6S(10,901).......................95701...E46S(6,897)96803...NaNS(7,898)97803...NaNS(7,898)98913...NaNS(8,899)99913...NaNS(8,899)[100rowsx13columns]dupl2=df.ds_get_duplicates(subset=['Survived'])dupl2Out[7]:PassengerId...DUPLICATEINDEX01...(0,4,5,6,7,12,13,14,16,18,20,24,26...15...(0,4,5,6,7,12,13,14,16,18,20,24,26...26...(0,4,5,6,7,12,13,14,16,18,20,24,26...37...(0,4,5,6,7,12,13,14,16,18,20,24,26...48...(0,4,5,6,7,12,13,14,16,18,20,24,26..............93637...(1,2,3,8,9,10,11,15,17,19,21,22,23...93740...(1,2,3,8,9,10,11,15,17,19,21,22,23...93844...(1,2,3,8,9,10,11,15,17,19,21,22,23...93945...(1,2,3,8,9,10,11,15,17,19,21,22,23...94048...(1,2,3,8,9,10,11,15,17,19,21,22,23...[941rowsx13columns]df.Embarked.ds_get_duplicates()EmbarkedDUPLICATEINDEX0NaN(61,829)1NaN(61,829)2C(1,9,19,26,30,31,34,36,39,42,43,48,...3C(1,9,19,26,30,31,34,36,39,42,43,48,...4C(1,9,19,26,30,31,34,36,39,42,43,48,...........936S(0,2,3,4,6,7,8,10,11,12,13,14,15,...937S(0,2,3,4,6,7,8,10,11,12,13,14,15,...938S(0,2,3,4,6,7,8,10,11,12,13,14,15,...939S(0,2,3,4,6,7,8,10,11,12,13,14,15,...940S(0,2,3,4,6,7,8,10,11,12,13,14,15,...[941rowsx2columns]
Package: a-pandas-ex-excel-edit
Quickly edit Pandas DataFrames and Series in ExcelUse this methods to quickly edit your DataFrame with MS Excel.Of course, Pandas is a lot better than Excel, but if you have to change a lot of  arbitrary values which don't have a clear pattern,  a GUI is imho the best choice.pipinstalla-pandas-ex-excel-edit#Here is an example:importpandasaspdfroma_pandas_ex_excel_editimportpd_add_excel_editor#pd_add_excel_editor will add 2 new methods:#pandas.Series.s_edit_in_excel#pandas.DataFrame.d_edit_in_excelpd_add_excel_editor()dframe=pd.read_csv("https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv")#Let's add a row with lists, a tough data type to handledframe['list_in_columns']=[[[1]*10]]*len(dframe)PassengerIdSurvived...Embarkedlist_in_columns010...S[[1,1,1,1,1,1,1,1,1,1]]121...C[[1,1,1,1,1,1,1,1,1,1]]231...S[[1,1,1,1,1,1,1,1,1,1]]341...S[[1,1,1,1,1,1,1,1,1,1]]450...S[[1,1,1,1,1,1,1,1,1,1]].................8868870...S[[1,1,1,1,1,1,1,1,1,1]]8878881...S[[1,1,1,1,1,1,1,1,1,1]]8888890...S[[1,1,1,1,1,1,1,1,1,1]]8898901...C[[1,1,1,1,1,1,1,1,1,1]]8908910...Q[[1,1,1,1,1,1,1,1,1,1]][891rowsx13columns]dframe.dtypesOut[6]:PassengerIdint64Survivedint64Pclassint64NameobjectSexobjectAgefloat64SibSpint64Parchint64TicketobjectFarefloat64CabinobjectEmbarkedobjectlist_in_columnsobjectdtype:objectdf=dframe.d_edit_in_excel()#DataFramesOut[7]:PassengerIdSurvived...Embarkedlist_in_columns0100019999...NOTYET[[1,99999,1,1,1,1,1,1,1,1]]1100001...C[[1,1,1,1,1,1,1,1,1,1]]299991...NOTYET[[1,1,1,1,1,1,1,1,1,1]]399981...NOTYET[[1,1,1,1,1,1,1,1,1,1]]499970...S[[1,1,1,1,1,1,1,1,1,1]].................8868870...S[[1,1,1,1,1,1,1,1,1,1]]8878881...S[[1,1,1,1,1,1,1,1,1,1]]8888890...S[[1,1,1,1,1,1,1,1,1,1]]8898901...C[[1,1,1,1,1,1,1,1,1,1]]8908910...Q[[1,1,1,1,1,1,1,1,1,1]][891rowsx13columns]df.dtypesOut[9]:PassengerIduint16Surviveduint16Pclassuint8NamestringSexcategoryAgeobjectSibSpuint8Parchuint8TicketobjectFarefloat64CabincategoryEmbarkedcategorylist_in_columnsobject#you can even edit lists, dicts and tuples with Excel!dtype:objectdf2=dframe.Name.s_edit_in_excel()#Seriesdf2Out[8]:0HANNIBALLECTOR1Cumings,Mrs.JohnBradley(FlorenceBriggsTh...2Heikkinen,Miss.Laina3Futrelle,Mrs.JacquesHeath(LilyMayPeel)4Allen,Mr.WilliamHenry...886Montvila,Rev.Juozas887Graham,Miss.MargaretEdith888Johnston,Miss.CatherineHelen"Carrie"889Behr,Mr.KarlHowell890Dooley,Mr.PatrickName:Name,Length:891,dtype:string
Package: a-pandas-ex-fastsort
Speedup up to 40 percent when sorting Pandas index/SeriesMSVC C++ x64/x86 build tools must be installed.This module useshttps://pypi.org/project/npfastsortcpp/There you can get all instructionsImportant: Only for float/intTested against Windows 10 / Python 3.9.13importpandasaspdfroma_pandas_ex_fastsortimportpd_add_fastsortpd_add_fastsort()dafra="https://github.com/pandas-dev/pandas/raw/main/doc/data/titanic.csv"df5=pd.read_csv(dafra)# Speed gain even for small DataFramesdf=pd.concat([df5.copy()forxinrange(10)],ignore_index=True)df=df.sample(len(df))%timeitdf.d_fast_reindex()# Values must be unique846µs±37.4µsperloop(mean±std.dev.of7runs,1,000loopseach)%timeitdf.sort_index()933µs±25.7µsperloop(mean±std.dev.of7runs,1,000loopseach)# The bigger, the betterdf=pd.concat([df5.copy()forxinrange(100)],ignore_index=True)df=df.sample(len(df))%timeitdf.d_fast_reindex()# Values must be unique11.1ms±131µsperloop(mean±std.dev.of7runs,100loopseach)%timeitdf.sort_index()15ms±220µsperloop(mean±std.dev.of7runs,100loopseach)df=pd.concat([df5.copy()forxinrange(100)],ignore_index=True)df=df.sample(len(df))%timeitdf.Pclass.sort_values()2.08ms±66µsperloop(mean±std.dev.of7runs,100loopseach)%timeitdf.Pclass.s_fastsort_copy()# Be careful: original index will be dropped!583µs±5.85µsperloop(mean±std.dev.of7runs,1,000loopseach)# Be careful:df.Pclass.s_fastsort_inplace()# sorts only one Series in place,# values in other columns are not being sorted!df# starting with:Out[19]:PassengerIdSurvivedPclass...FareCabinEmbarked3410224503...7.2250NaNC2832970911...151.5500NaNS5001812302...30.0708NaNC5125847203...8.6625NaNS5181313602...15.0458NaNC.....................3635771812...10.5000E101S7860820103...9.5000NaNS6498983803...8.0500NaNS2082433201...28.5000C124S2110861612...65.0000NaNS[89100rowsx12columns]df.Pclass.s_fastsort_inplace()df# Result - Only Pclass has been sortedOut[21]:PassengerIdSurvivedPclass...FareCabinEmbarked3410224501...7.2250NaNC2832970911...151.5500NaNS5001812301...30.0708NaNC5125847201...8.6625NaNS5181313601...15.0458NaNC.....................3635771813...10.5000E101S7860820103...9.5000NaNS6498983803...8.0500NaNS2082433203...28.5000C124S2110861613...65.0000NaNS[89100rowsx12columns]
Package: a-pandas-ex-fuzzymerge
Merges two DataFrames using fuzzy matching on specified columnsTested against Windows / Python 3.11 / Anacondapip install a-pandas-ex-fuzzymergeThisfunctionperformsafuzzymatchingbetweentwoDataFrames`df1`and`df2`basedonthecolumnsspecifiedin`right_on`and`left_on`.Fuzzymatchingallowsyoutofindsimilarvaluesbetweenthesecolumns,makingitusefulformatchingdatawithsmallvariations,suchastyposorabbreviations.Parameters:df1(DataFrame):ThefirstDataFrametobemerged.df2(DataFrame):ThesecondDataFrametobemerged.right_on(str):Thecolumnnamein`df2`tobeusedformatching.left_on(str):Thecolumnnamein`df1`tobeusedformatching.usedtype(numpy.dtype,optional):Thedatatypetouseforthedistancematrix.Defaultsto`np.uint8`.scorer(function,optional):Thescoringfunctiontouseforfuzzymatching.Defaultsto`fuzz.WRatio`.concat_value(bool,optional):Whethertoadda'concat_value'columnintheresultDataFrame,containingthesimilarityscores.Defaultsto`True`.**kwargs:Additionalkeywordargumentstopasstothe`pandas.merge`function.Returns:DataFrame:AmergedDataFramewithrowsthatmatchedbasedonthespecifiedfuzzycriteria.Example:froma_pandas_ex_fuzzymergeimportpd_add_fuzzymergeimportpandasaspdimportnumpyasnpfromrapidfuzzimportfuzzpd_add_fuzzymerge()df1=pd.read_csv("https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv")df2=df1.copy()df2=pd.concat([df2forxinrange(3)],ignore_index=True)df2.Name=(df2.Name+np.random.uniform(1,2000,len(df2)).astype("U"))df1=pd.concat([df1forxinrange(3)],ignore_index=True)df1.Name=(df1.Name+np.random.uniform(1,2000,len(df1)).astype("U"))df3=df1.d_fuzzy_merge(df2,right_on='Name',left_on='Name',usedtype=np.uint8,scorer=fuzz.partial_ratio,concat_value=True)print(df3)