Package: a-pandas-ex-fastloc
Extremely fast by combining np.select and numexpr.evaluate (works with utf-8!)Here is a simple query:https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv%timeit(df.PassengerId>800)|(df.PassengerId<60)194µs±586nsperloop(mean±std.dev.of7runs,10,000loopseach)When you're working with pandas, speed matters!.Well, there are 2 tricks to tremendously increase the speed of Pandas(Here are 2 examples: in case you have never heard about them)1) np.selectb=df.PassengerId.__array__()%timeitnp.select([b>800,b<60],[True,True],False)21.5µs±44.5nsperloop(mean±std.dev.of7runs,10,000loopseach)2) numexpr.evaluateb=df.PassengerId.__array__()%timeit[numexpr.evaluate('(b > 800)|(b < 60)')]10.5µs±25nsperloop(mean±std.dev.of7runs,100,000loopseach)Combining bothWell, why not combining those two?  numexpr.evaluate to do the queries, and np.select to filter the queries (True/False)The problem is that numexpr has many limitations:https://numexpr.readthedocs.io/projects/NumExpr3/en/latest/user_guide.html#supported-operatorsIf you are working with numbers, you probably won't miss a lot,but with strings, it is different: numexpr nor supports the utf-8 format neither regex.I am a German teacher and work most of the time with DataFrames containing strings (mainly bilingual dictionaries)anywhere from 500 to 10000000 rows. Unfortunately, it is not that easy converting German wordsto ASCII strings due to the special characters (ö, ä ...)I found a way to simplify the conversion either way, without losing the special characters:df.Name.s_str().s_to_ascii()df.Nameuni.s_str().s_to_utf8()But converting them all the time back and forth (to useSeries.str [only UTF-8]) doesn't make any sense,because we are loosing all the speed gained from numexpr.evaluate/np.select.Unfortunately,Series.strdoesn't accept binary data.But since I use all methods of the Series.str class frequently andI also want the benefits of numexpr.evaluate/np.select, I changedalmost all methods of Series.str to make them work with binary data.Don’t be afraid:It won’t overwriteSeries.str, it just addss_str(parenthesis when you want to access the methods!)Some of the adapted methods are faster, some are slower (still faster than converting)and some are really slow, and should not be used (casefold/wrap/normalize)and few of them are not working yet [(encode/decode) → adaption probably senseless].And of course, the new insanely fast methods:s_uniascii_contains# only for strings, pass strings as utf-8s_uniascii_equal# only for strings, pass strings as utf-8d_floc# replacement for df.loc / combine it with numexpr.evaluateget_compiled_regex_from_wordlist# instead of searching [Maria, Carlos, ...], use this function to generate a binary Trie regexScroll down to see the execution speed of every method compared toSeries.strHow to usefroma_pandas_ex_fastlocimportpd_add_fastloc,get_byte_regex,get_compiled_regex_from_wordlist,convert_utf8_to_asciipd_add_fastloc()importpandasaspddf=pd.read_csv("https://raw.githubusercontent.com/pandas-dev/pandas/main/doc/data/titanic.csv")df['Nameuni']=df.Name+'ö'# get some utf-8 data with special charactersdf['Nameuni2']=df['Nameuni'].s_str().s_to_ascii()dfbig=pd.concat([df.copy()forxinrange(100)],ignore_index=True)dfbigbig=pd.concat([df.copy()forxinrange(1000)],ignore_index=True)# The highlightss_uniascii_contains%timeitdfbigbig.loc[dfbigbig.Nameuni.str.contains('Harrisö')]239ms±1.62msperloop(mean±std.dev.of7runs,1loopeach)%timeitdfbigbig.loc[dfbigbig.Nameuni2.s_str().s_uniascii_contains('Harrisö')]14.7ms±111µsperloop(mean±std.dev.of7runs,100loopseach)s_uniascii_equal%timeitdfbigbig.loc[dfbigbig.Nameuni=='Braund, Mr. Owen Harrisö']29.3ms±179µsperloop(mean±std.dev.of7runs,10loopseach)%timeitdfbigbig.loc[dfbigbig.Nameuni2.s_str().s_uniascii_equal('Braund, Mr. Owen Harrisö')]3.89ms±5.78µsperloop(mean±std.dev.of7runs,100loopseach)s_to_utf8# Data can be converted easily to utf-8:dfbigbig.loc[dfbigbig.Nameuni2.s_str().s_uniascii_contains('Harrisö')].Nameuni2.s_str().s_to_utf8()Out[65]:0Braund,Mr.OwenHarrisö891Braund,Mr.OwenHarrisö1782Braund,Mr.OwenHarrisö2673Braund,Mr.OwenHarrisö3564Braund,Mr.OwenHarrisö...Name:Nameuni2,Length:1000,dtype:objectd_floc# You can also use numbers, but you won't get a speedup for simple queries%timeit-n4dfbigbig.d_floc([('(Age > 10)',True)])%timeit-n4dfbigbig.loc[dfbigbig.Age>10]139ms±562µsperloop(mean±std.dev.of7runs,4loopseach)140ms±486µsperloop(mean±std.dev.of7runs,4loopseach)# Once the query gets more sophisticated, things are different:%timeit-n4dfbigbig.d_floc([('((Age*3221/3220 > (10*.432/.431)) & (PassengerId%10 ==0))',True)])%timeit-n4dfbigbig.loc[(dfbigbig.Age*3221/3220>10*.432/.431)&(dfbigbig.PassengerId%10==0)]23.4ms±583µsperloop(mean±std.dev.of7runs,4loopseach)35.2ms±342µsperloop(mean±std.dev.of7runs,4loopseach)%timeit-n4dfbigbig.d_floc([('((Age*3221/3220 > (10*.432/.431)) & (PassengerId%10 ==0))',True),(f'contains(Nameuni2,{repr(convert_utf8_to_ascii("Harrisö"))})',True)],convertstr=False)%timeit-n4dfbigbig.loc[((dfbigbig.Age*3221/3220>10*.432/.431)&(dfbigbig.PassengerId%10==0))|dfbigbig.Nameuni.str.contains("Harrisö")]39.8ms±1.01msperloop(mean±std.dev.of7runs,4loopseach)272ms±1.12msperloop(mean±std.dev.of7runs,4loopseach)# Different ways of doing the query, check out the numexpr documentation%timeit-n4dfbigbig.d_floc([('((Age*3221/3220 > (10*.432/.431)) & (PassengerId%10 ==0))',True),(f'contains(Nameuni2,{repr(convert_utf8_to_ascii("Harrisö"))})',True)],convertstr=False)%timeit-n4dfbigbig.d_floc([(f'((Age*3221/3220 > (10*.432/.431)) & (PassengerId%10 ==0)) | contains(Nameuni2,{repr(convert_utf8_to_ascii("Harrisö"))})',True)],convertstr=False)# How to negate results (operator ~)%timeit-n4dfbigbig.d_floc([(f'((Age*3221/3220 > (10*.432/.431)) & (PassengerId%10 ==0)) | contains(Nameuni2,{repr(convert_utf8_to_ascii("Harrisö"))})',False)],convertstr=False,default=True)%timeit-n4dfbigbig.loc[~(((dfbigbig.Age*3221/3220>10*.432/.431)&(dfbigbig.PassengerId%10==0))|dfbigbig.Nameuni.str.contains("Harrisö"))]194ms±1.55msperloop(mean±std.dev.of7runs,4loopseach)436ms±3.34msperloop(mean±std.dev.of7runs,4loopseach)# without converting the Series with utf-8 strings to ASCIIqueries=[('contains(Name, "Maria")',True)]# convertstr=True -> Converts utf-8 to bytes, must be True if the column is not ASCII, and False if ASCII%timeitdf.d_floc(queries,default=False,convertstr=True)%timeitdf.loc[df.Name.str.contains('Maria')]# Even though, the Series has been converted to bytes in every loop, fastloc is still much faster:329µs±4.25µsperloop(mean±std.dev.of7runs,1,000loopseach)491µs±2.14µsperloop(mean±std.dev.of7runs,1,000loopseach)# How to convertdf['Name2']=df.Name.s_str().s_to_ascii()# works also with special characters (ö, ï ...)df['Nameuni2']=df.Nameuni.s_str().s_to_ascii()# Faster when converted (convertstr=False)queries=[('contains(Name2, "Maria")',True)]%timeitdf.d_floc(queries,default=False,convertstr=False)%timeitdf.loc[df.Name.str.contains('Maria')]270µs±1.29µsperloop(mean±std.dev.of7runs,1,000loopseach)510µs±1.15µsperloop(mean±std.dev.of7runs,1,000loopseach)# You can also use:%timeitdf.loc[df.Name2.s_str().s_uniascii_contains('Maria')]# The bigger the dataframe, the better:dfbig['Nameuni2']=dfbig.Nameuni.s_str().s_to_ascii()dfbig['Name2']=dfbig.Name.s_str().s_to_ascii()queries=[('contains(Name2, "Maria")',True)]%timeitdfbig.d_floc(queries,default=False,convertstr=False)%timeitdfbig.loc[dfbig.Name.str.contains('Maria')]2.04ms±206µsperloop(mean±std.dev.of7runs,1,000loopseach)25.6ms±90.1µsperloop(mean±std.dev.of7runs,10loopseach)dfbigbig['Nameuni2']=dfbigbig.Nameuni.s_str().s_to_ascii()dfbigbig['Name2']=dfbigbig.Name.s_str().s_to_ascii()queries=[('contains(Name2, "Maria")',True)]%timeitdfbigbig.d_floc(queries,default=False,convertstr=False)%timeitdfbigbig.loc[dfbigbig.Name.str.contains('Maria')]18.3ms±347µsperloop(mean±std.dev.of7runs,100loopseach)251ms±392µsperloop(mean±std.dev.of7runs,1loopeach)# Converting the results to unicode:dfbigbig.d_floc(queries,default=False,convertstr=False).Nameuni2.s_str().s_to_utf8()Out[42]:18VanderPlanke,Mrs.Julius(EmeliaMariaVande...38VanderPlanke,Miss.AugustaMariaö85Backstrom,Mrs.KarlAlfred(MariaMathildaGu...119Andersson,Miss.EllisAnnaMariaö307PenascoyCastellana,Mrs.VictordeSatode(M......890535Clarke,Mrs.CharlesV(AdaMariaWinfield)ö890687Caram,Mrs.Joseph(MariaElias)ö890690Thayer,Mrs.JohnBorland(MarianLongstrethM...890747Panula,Mrs.Juha(MariaEmiliaOjala)ö890925Heininen,Miss.WendlaMariaöName:Nameuni2,Length:12000,dtype:objectconvert_utf8_to_ascii# Searching for Unicode using numexpr directly# Use this to convert the string to bin: {repr(convert_utf8_to_ascii("Harrisö"))}queriesuni=[(f'contains(Nameuni2,{repr(convert_utf8_to_ascii("Harrisö"))})',True)]%timeitdf.d_floc(queriesuni,default=False,convertstr=False)293µs±974nsperloop(mean±std.dev.of7runs,1,000loopseach)Out[11]:PassengerId...Nameuni201...b'Braund, Mr. Owen Harris\\u00f6'[1rowsx15columns]# Convert to utf-8df.d_floc(queriesuni,default=False,convertstr=False).Nameuni2.s_str().s_to_utf8()Out[17]:0Braund,Mr.OwenHarrisöName:Nameuni2,dtype:object#################################################################%timeitdf.loc[df.Nameuni.str.contains('Harrisö')]514µs±2.82µsperloop(mean±std.dev.of7runs,1,000loopseach)Out[13]:PassengerId...Nameuni201...b'Braund, Mr. Owen Harris\\u00f6'[1rowsx15columns]get_byte_regex# Regex works too, but be careful with special characters# Avoid this because you might get other results than you want:comp0=get_byte_regex('^.*?[öä]$')print(comp0)re.compile(b'^.*?[\\\\u00f6\\\\u00e4]$')# Bettercomp1=get_byte_regex('^.*?(?:(?:ö)|(?:ä))$')print(comp1)re.compile(b'^.*?(?:(?:\\\\u00f6)|(?:\\\\u00e4))$')%timeitdf.Nameuni2.s_str().findall(comp1)%timeitdf.Nameuni.str.findall('^.*?(?:(?:ö)|(?:ä))$')# Not much faster, but at least a little633µs±1.35µsperloop(mean±std.dev.of7runs,1,000loopseach)641µs±906nsperloop(mean±std.dev.of7runs,1,000loopseach)Out[21]:0[b'Braund, Mr. Owen Harris\\u00f6']1[b'Cumings, Mrs. John Bradley (Florence Briggs...2[b'Heikkinen, Miss. Laina\\u00f6']3[b'Futrelle, Mrs. Jacques Heath (Lily May Peel...4[b'Allen, Mr. William Henry\\u00f6']...886[b'Montvila, Rev. Juozas\\u00f6']887[b'Graham, Miss. Margaret Edith\\u00f6']888[b'Johnston, Miss. Catherine Helen\\"Carrie\\...889[b'Behr, Mr. Karl Howell\\u00f6']890[b'Dooley, Mr. Patrick\\u00f6']Name:Nameuni2,Length:891,dtype:objectOut[22]:0[Braund,Mr.OwenHarrisö]1[Cumings,Mrs.JohnBradley(FlorenceBriggsT...2[Heikkinen,Miss.Lainaö]3[Futrelle,Mrs.JacquesHeath(LilyMayPeel)ö]4[Allen,Mr.WilliamHenryö]...886[Montvila,Rev.Juozasö]887[Graham,Miss.MargaretEdithö]888[Johnston,Miss.CatherineHelen"Carrie"ö]889[Behr,Mr.KarlHowellö]890[Dooley,Mr.Patrickö]Name:Nameuni,Length:891,dtype:object# With big DataFrames%timeitdfbigbig.Nameuni2.s_str().findall(comp1)%timeitdfbigbig.Nameuni.str.findall('^.*?(?:(?:ö)|(?:ä))$')556ms±2.61msperloop(mean±std.dev.of7runs,1loopeach)571ms±2.52msperloop(mean±std.dev.of7runs,1loopeach)################################################################## You can use all regex methods from a Series.str.# But usually no speedup when using regex (sometimes a little faster, sometimes a little slower)regexpextractall='^.*?((?:ö)|(?:ä))$'comp1=get_byte_regex(regexpextractall)%timeitdfbig.Nameuni2.s_str().extractall(comp1)%timeitdfbig.Nameuni.str.extractall(regexpextractall)209ms±2.65msperloop(mean±std.dev.of7runs,1loopeach)206ms±3.98msperloop(mean±std.dev.of7runs,10loopseach)regexpextractall='^.*?((?:ö)|(?:ä))$'comp1=get_byte_regex(regexpextractall)%timeitdfbig.Nameuni2.s_str().replace(comp1,b'XX',regex=True)%timeitdfbig.Nameuni.str.replace(regexpextractall,'XX',regex=True)69.9ms±747µsperloop(mean±std.dev.of7runs,10loopseach)71.5ms±293µsperloop(mean±std.dev.of7runs,10loopseach)#################################################################get_compiled_regex_from_wordlist# If you have a list of words, create a Trie regex:# e.g. all names with starting with Mlotsofwords=([oforoinset([re.sub(r'\W+','',x)forxin"\n".join(df.Name.unique().tolist()).split()])ifo.startswith('M')])lotsofwordsre=get_compiled_regex_from_wordlist(wordlist=lotsofwords,boundary_right=True,boundary_left=True,capture=False,match_whole_line=False,flags=re.I)%timeitdf.loc[df.Name.apply(lambdax:Trueif[yforyinlotsofwordsifyinx]elseFalse)]%timeitdf.loc[df.Name2.s_str().contains(lotsofwordsre,regex=True)]7.02ms±47.1µsperloop(mean±std.dev.of7runs,100loopseach)914µs±1.45µsperloop(mean±std.dev.of7runs,1,000loopseach)#################################################################Methods faster than Series.str# To avoid converting back and forth all the time (ascii - utf-8)# I adapted most of the methods from the Series.str class# to work with bytes# Some methods are faster, some are slower and some are really slow# (casefold/wrap/normalize)# But surely better than converting all the time# Lets create a copy of the DFdf5=dfbig.copy()df5['Name1']=df5.Nameuni2.copy()#################################################################print(df5.Name2.s_str().capitalize())print(df5.Name1.s_str().capitalize())print(df5.Name.str.capitalize())%timeit-n5df5.Name2.s_str().capitalize()%timeit-n5df5.Name1.s_str().capitalize()%timeit-n5df5.Name.str.capitalize()13.8ms±313µsperloop(mean±std.dev.of7runs,5loopseach)13.9ms±153µsperloop(mean±std.dev.of7runs,5loopseach)20.5ms±169µsperloop(mean±std.dev.of7runs,5loopseach)################################################################## Not part of Series.str, but similar to Series.str.containsprint(df5.Name2.s_str().s_uniascii_equal('Svensson, Mr. Johan'))print(df5.Name1.s_str().s_uniascii_equal('Svensson, Mr. Johan'))print(df5.loc[df5.Name=='Svensson, Mr. Johan'])%timeit-n5df5.Name2.s_str().s_uniascii_equal('Svensson, Mr. Johan')%timeit-n5df5.Name1.s_str().s_uniascii_equal('Svensson, Mr. Johan')%timeit-n5df5.loc[df5.Name==('Svensson, Mr. Johan')]350µs±126µsperloop(mean±std.dev.of7runs,5loopseach)317µs±27.4µsperloop(mean±std.dev.of7runs,5loopseach)3.13ms±34.7µsperloop(mean±std.dev.of7runs,5loopseach)################################################################## Not part of Series.str, but similar to Series.str.containsprint(df5.Name2.s_str().s_uniascii_contains('a'))print(df5.Name1.s_str().s_uniascii_contains('a'))print(df5.Name.str.contains('a'))%timeit-n5df5.Name2.s_str().s_uniascii_contains('a')%timeit-n5df5.Name1.s_str().s_uniascii_contains('a')%timeit-n5df5.Name.str.contains('a')1.12ms±73.1µsperloop(mean±std.dev.of7runs,5loopseach)1.07ms±16.4µsperloop(mean±std.dev.of7runs,5loopseach)24.6ms±506µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().isalnum())print(df5.Name1.s_str().isalnum())print(df5.Name.str.isalnum())%timeit-n5df5.Name2.s_str().isalnum()%timeit-n5df5.Name1.s_str().isalnum()%timeit-n5df5.Name.str.isalnum()9.67ms±125µsperloop(mean±std.dev.of7runs,5loopseach)9.44ms±74.2µsperloop(mean±std.dev.of7runs,5loopseach)10ms±122µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().isalpha())print(df5.Name1.s_str().isalpha())print(df5.Name.str.isalpha())%timeit-n5df5.Name2.s_str().isalpha()%timeit-n5df5.Name1.s_str().isalpha()%timeit-n5df5.Name.str.isalpha()9.74ms±164µsperloop(mean±std.dev.of7runs,5loopseach)9.45ms±91.4µsperloop(mean±std.dev.of7runs,5loopseach)9.77ms±44.4µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().istitle())print(df5.Name1.s_str().istitle())print(df5.Name.str.istitle())%timeit-n5df5.Name2.s_str().istitle()%timeit-n5df5.Name1.s_str().istitle()%timeit-n5df5.Name.str.istitle()14ms±563µsperloop(mean±std.dev.of7runs,5loopseach)13.8ms±83.5µsperloop(mean±std.dev.of7runs,5loopseach)17.5ms±188µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().partition())print(df5.Name1.s_str().partition())print(df5.Name.str.partition())%timeit-n5df5.Name2.s_str().partition()%timeit-n5df5.Name1.s_str().partition()%timeit-n5df5.Name.str.partition()86.8ms±817µsperloop(mean±std.dev.of7runs,5loopseach)86.8ms±297µsperloop(mean±std.dev.of7runs,5loopseach)89.5ms±412µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().rpartition())print(df5.Name1.s_str().rpartition())print(df5.Name.str.rpartition())%timeit-n5df5.Name2.s_str().rpartition()%timeit-n5df5.Name1.s_str().rpartition()%timeit-n5df5.Name.str.rpartition()89.1ms±773µsperloop(mean±std.dev.of7runs,5loopseach)90.2ms±490µsperloop(mean±std.dev.of7runs,5loopseach)90.5ms±383µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().repeat(100))print(df5.Name1.s_str().repeat(100))print(df5.Name.str.repeat(100))%timeit-n5df5.Name2.s_str().repeat(100)%timeit-n5df5.Name1.s_str().repeat(100)%timeit-n5df5.Name.str.repeat(100)75.7ms±4.08msperloop(mean±std.dev.of7runs,5loopseach)92.6ms±5.07msperloop(mean±std.dev.of7runs,5loopseach)113ms±387µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().slice_replace(start=1,stop=10,repl=b'2'))print(df5.Name1.s_str().slice_replace(start=1,stop=10,repl=b'2'))print(df5.Name.str.slice_replace(start=1,stop=10,repl='2'))%timeit-n5df5.Name2.s_str().slice_replace(start=1,stop=10,repl=b'2')%timeit-n5df5.Name1.s_str().slice_replace(start=1,stop=10,repl=b'2')%timeit-n5df5.Name.str.slice_replace(start=1,stop=10,repl='2')30.8ms±544µsperloop(mean±std.dev.of7runs,5loopseach)31ms±285µsperloop(mean±std.dev.of7runs,5loopseach)44.5ms±281µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().split())print(df5.Name1.s_str().split())print(df5.Name.str.split())%timeit-n5df5.Name2.s_str().split()%timeit-n5df5.Name1.s_str().split()%timeit-n5df5.Name.str.split()39.1ms±558µsperloop(mean±std.dev.of7runs,5loopseach)39.8ms±280µsperloop(mean±std.dev.of7runs,5loopseach)40.1ms±244µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().swapcase())print(df5.Name1.s_str().swapcase())print(df5.Name.str.swapcase())%timeit-n5df5.Name2.s_str().swapcase()%timeit-n5df5.Name1.s_str().swapcase()%timeit-n5df5.Name.str.swapcase()17.7ms±773µsperloop(mean±std.dev.of7runs,5loopseach)18.9ms±334µsperloop(mean±std.dev.of7runs,5loopseach)24.2ms±140µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().title())print(df5.Name1.s_str().title())print(df5.Name.str.title())%timeit-n5df5.Name2.s_str().title()%timeit-n5df5.Name1.s_str().title()%timeit-n5df5.Name.str.title()17.9ms±475µsperloop(mean±std.dev.of7runs,5loopseach)19.6ms±384µsperloop(mean±std.dev.of7runs,5loopseach)24.4ms±162µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().strip(b'Svensson, Mr. Johan'))print(df5.Name1.s_str().strip(b'Svensson, Mr. Johan'))print(df5.Name.str.strip('Svensson, Mr. Johan'))%timeit-n5df5.Name2.s_str().strip(b'Svensson, Mr. Johan')%timeit-n5df5.Name1.s_str().strip(b'Svensson, Mr. Johan')%timeit-n5df5.Name.str.strip('Svensson, Mr. Johan')22.1ms±595µsperloop(mean±std.dev.of7runs,5loopseach)19.3ms±336µsperloop(mean±std.dev.of7runs,5loopseach)19.4ms±156µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().translate(bytes.maketrans(b'\x47',b'X')))print(df5.Name1.s_str().translate(bytes.maketrans(b'\x47',b'X')))print(df5.Name.str.translate(str.maketrans('G','X')))%timeit-n5df5.Name2.s_str().translate(bytes.maketrans(b'\x47',b'X'))%timeit-n5df5.Name1.s_str().translate(bytes.maketrans(b'\x47',b'X'))%timeit-n5df5.Name.str.translate(str.maketrans('G','X'))20ms±486µsperloop(mean±std.dev.of7runs,5loopseach)20.8ms±183µsperloop(mean±std.dev.of7runs,5loopseach)86.3ms±173µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().upper())print(df5.Name1.s_str().upper())print(df5.Name.str.upper())%timeit-n5df5.Name2.s_str().upper()%timeit-n5df5.Name1.s_str().upper()%timeit-n5df5.Name.str.upper()13.4ms±324µsperloop(mean±std.dev.of7runs,5loopseach)13.7ms±172µsperloop(mean±std.dev.of7runs,5loopseach)15.4ms±180µsperloop(mean±std.dev.of7runs,5loopseach)###################################################################################################################################################################################################Methods slower than Series.strprint(df5.Name2.s_str().replace(rb'a\w{3,7}',b'xxx',regex=True))print(df5.Name1.s_str().replace(rb'a\w{3,7}',b'xxx',regex=True))print(df5.Name.str.replace(r'a\w{3,7}','xxx',regex=True))%timeit-n5df5.Name2.s_str().replace(rb'a\w{3,7}',b'xxx',regex=True)%timeit-n5df5.Name1.s_str().replace(rb'a\w{3,7}',b'xxx',regex=True)%timeit-n5df5.Name.str.replace(r'a\w{3,7}','xxx',regex=True)63ms±504µsperloop(mean±std.dev.of7runs,5loopseach)63.4ms±325µsperloop(mean±std.dev.of7runs,5loopseach)60ms±326µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().removeprefix(b'A'))print(df5.Name1.s_str().removeprefix(b'A'))print(df5.Name.str.removeprefix('A'))%timeit-n5df5.Name2.s_str().removeprefix(b'A')%timeit-n5df5.Name1.s_str().removeprefix(b'A')%timeit-n5df5.Name.str.removeprefix('A')23.3ms±625µsperloop(mean±std.dev.of7runs,5loopseach)23ms±185µsperloop(mean±std.dev.of7runs,5loopseach)17.4ms±163µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().isupper())print(df5.Name1.s_str().isupper())print(df5.Name.str.isupper())%timeit-n5df5.Name2.s_str().isupper()%timeit-n5df5.Name1.s_str().isupper()%timeit-n5df5.Name.str.isupper()8.98ms±197µsperloop(mean±std.dev.of7runs,5loopseach)8.57ms±48.5µsperloop(mean±std.dev.of7runs,5loopseach)8.49ms±53.9µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().isspace())print(df5.Name1.s_str().isspace())print(df5.Name.str.isspace())%timeit-n5df5.Name2.s_str().isspace()%timeit-n5df5.Name1.s_str().isspace()%timeit-n5df5.Name.str.isspace()8.82ms±226µsperloop(mean±std.dev.of7runs,5loopseach)8.61ms±122µsperloop(mean±std.dev.of7runs,5loopseach)8.29ms±73.4µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().islower())print(df5.Name1.s_str().islower())print(df5.Name.str.islower())%timeit-n5df5.Name2.s_str().islower()%timeit-n5df5.Name1.s_str().islower()%timeit-n5df5.Name.str.islower()9ms±269µsperloop(mean±std.dev.of7runs,5loopseach)8.59ms±81.4µsperloop(mean±std.dev.of7runs,5loopseach)8.37ms±129µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().slice(start=1,stop=10,step=2))print(df5.Name1.s_str().slice(start=1,stop=10,step=2))print(df5.Name.str.slice(start=1,stop=10,step=2))%timeit-n5df5.Name2.s_str().slice(start=1,stop=10,step=2)%timeit-n5df5.Name1.s_str().slice(start=1,stop=10,step=2)%timeit-n5df5.Name.str.slice(start=1,stop=10,step=2)18.9ms±304µsperloop(mean±std.dev.of7runs,5loopseach)19.1ms±235µsperloop(mean±std.dev.of7runs,5loopseach)15.5ms±160µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().rfind(b'A'))print(df5.Name1.s_str().rfind(b'A'))print(df5.Name.str.rfind('A'))%timeit-n5df5.Name2.s_str().rfind(b'A')%timeit-n5df5.Name1.s_str().rfind(b'A')%timeit-n5df5.Name.str.rfind('A')41ms±618µsperloop(mean±std.dev.of7runs,5loopseach)41.2ms±301µsperloop(mean±std.dev.of7runs,5loopseach)35.2ms±216µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().startswith(b'Svensson, Mr. Johan'))print(df5.Name1.s_str().startswith(b'Svensson, Mr. Johan'))print(df5.Name.str.startswith('Svensson, Mr. Johan'))%timeit-n5df5.Name2.s_str().startswith(b'Svensson, Mr. Johan')%timeit-n5df5.Name1.s_str().startswith(b'Svensson, Mr. Johan')%timeit-n5df5.Name.str.startswith('Svensson, Mr. Johan')24.2ms±511µsperloop(mean±std.dev.of7runs,5loopseach)24.4ms±327µsperloop(mean±std.dev.of7runs,5loopseach)18.6ms±296µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().rjust(100))print(df5.Name1.s_str().rjust(100))print(df5.Name.str.rjust(100))%timeit-n5df5.Name2.s_str().rjust(100)%timeit-n5df5.Name1.s_str().rjust(100)%timeit-n5df5.Name.str.rjust(100)23.6ms±513µsperloop(mean±std.dev.of7runs,5loopseach)23.6ms±160µsperloop(mean±std.dev.of7runs,5loopseach)18.7ms±129µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().isdigit())print(df5.Name1.s_str().isdigit())print(df5.Name.str.isdigit())%timeit-n5df5.Name2.s_str().isdigit()%timeit-n5df5.Name1.s_str().isdigit()%timeit-n5df5.Name.str.isdigit()8.81ms±209µsperloop(mean±std.dev.of7runs,5loopseach)8.45ms±94µsperloop(mean±std.dev.of7runs,5loopseach)8.34ms±77.4µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().rstrip(b'A'))print(df5.Name1.s_str().rstrip(b'A'))print(df5.Name.str.rstrip('A'))%timeit-n5df5.Name2.s_str().rstrip(b'A')%timeit-n5df5.Name1.s_str().rstrip(b'A')%timeit-n5df5.Name.str.rstrip('A')17.5ms±598µsperloop(mean±std.dev.of7runs,5loopseach)17.2ms±208µsperloop(mean±std.dev.of7runs,5loopseach)11.9ms±159µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().rsplit(b'A'))print(df5.Name1.s_str().rsplit(b'A'))print(df5.Name.str.rsplit('A'))%timeit-n5df5.Name2.s_str().rsplit(b'A')%timeit-n5df5.Name1.s_str().rsplit(b'A')%timeit-n5df5.Name.str.rsplit('A')29ms±700µsperloop(mean±std.dev.of7runs,5loopseach)29.2ms±338µsperloop(mean±std.dev.of7runs,5loopseach)22.9ms±215µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().rstrip(b'A'))print(df5.Name1.s_str().rstrip(b'A'))print(df5.Name.str.rstrip('A'))%timeit-n5df5.Name2.s_str().rstrip(b'A')%timeit-n5df5.Name1.s_str().rstrip(b'A')%timeit-n5df5.Name.str.rstrip('A')17.3ms±441µsperloop(mean±std.dev.of7runs,5loopseach)17.3ms±217µsperloop(mean±std.dev.of7runs,5loopseach)12.1ms±132µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().match(b'zz'))print(df5.Name1.s_str().match(b'zz'))print(df5.Name.str.match('zz'))%timeit-n5df5.Name2.s_str().match(b'zz')%timeit-n5df5.Name1.s_str().match(b'zz')%timeit-n5df5.Name.str.match('zz')33.6ms±718µsperloop(mean±std.dev.of7runs,5loopseach)32.6ms±187µsperloop(mean±std.dev.of7runs,5loopseach)26.9ms±286µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().lstrip(b'zz'))print(df5.Name1.s_str().lstrip(b'zz'))print(df5.Name.str.lstrip('zz'))%timeit-n5df5.Name2.s_str().lstrip(b'zz')%timeit-n5df5.Name1.s_str().lstrip(b'zz')%timeit-n5df5.Name.str.lstrip('zz')17.5ms±447µsperloop(mean±std.dev.of7runs,5loopseach)17ms±178µsperloop(mean±std.dev.of7runs,5loopseach)11.8ms±98.1µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().lower())print(df5.Name1.s_str().lower())print(df5.Name.str.lower())%timeit-n5df5.Name2.s_str().lower()%timeit-n5df5.Name1.s_str().lower()%timeit-n5df5.Name.str.lower()13.5ms±224µsperloop(mean±std.dev.of7runs,5loopseach)13.3ms±168µsperloop(mean±std.dev.of7runs,5loopseach)11.9ms±582µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().ljust(100))print(df5.Name1.s_str().ljust(100))print(df5.Name.str.ljust(100))%timeit-n5df5.Name2.s_str().ljust(100)%timeit-n5df5.Name1.s_str().ljust(100)%timeit-n5df5.Name.str.ljust(100)24.1ms±540µsperloop(mean±std.dev.of7runs,5loopseach)23.8ms±144µsperloop(mean±std.dev.of7runs,5loopseach)19.7ms±146µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().removesuffix(b'A'))print(df5.Name1.s_str().removesuffix(b'A'))print(df5.Name.str.removesuffix('A'))%timeit-n5df5.Name2.s_str().removesuffix(b'A')%timeit-n5df5.Name1.s_str().removesuffix(b'A')%timeit-n5df5.Name.str.removesuffix('A')22.2ms±672µsperloop(mean±std.dev.of7runs,5loopseach)22.2ms±322µsperloop(mean±std.dev.of7runs,5loopseach)16.4ms±86.2µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().len())print(df5.Name1.s_str().len())print(df5.Name.str.len())%timeit-n5df5.Name2.s_str().len()%timeit-n5df5.Name1.s_str().len()%timeit-n5df5.Name.str.len()24ms±500µsperloop(mean±std.dev.of7runs,5loopseach)23.7ms±275µsperloop(mean±std.dev.of7runs,5loopseach)17.8ms±121µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().fullmatch(rb'(a\w{3})(\w+)'))print(df5.Name1.s_str().fullmatch(rb'(a\w{3})(\w+)'))print(df5.Name.str.fullmatch(r'(a\w{3})(\w+)'))%timeit-n5df5.Name2.s_str().fullmatch(rb'(a\w{3})(\w+)')%timeit-n5df5.Name1.s_str().fullmatch(rb'(a\w{3})(\w+)')%timeit-n5df5.Name.str.fullmatch(r'(a\w{3})(\w+)')29.3ms±470µsperloop(mean±std.dev.of7runs,5loopseach)29.3ms±330µsperloop(mean±std.dev.of7runs,5loopseach)23.9ms±385µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().findall(rb'(a\w{3})(\w+)'))print(df5.Name1.s_str().findall(rb'(a\w{3})(\w+)'))print(df5.Name.str.findall(r'(a\w{3})(\w+)'))%timeit-n5df5.Name2.s_str().findall(rb'(a\w{3})(\w+)')%timeit-n5df5.Name1.s_str().findall(rb'(a\w{3})(\w+)')%timeit-n5df5.Name.str.findall(r'(a\w{3})(\w+)')42.8ms±411µsperloop(mean±std.dev.of7runs,5loopseach)43.2ms±506µsperloop(mean±std.dev.of7runs,5loopseach)39.5ms±314µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().find(rb'(a\w{3})(\w+)'))print(df5.Name1.s_str().find(rb'(a\w{3})(\w+)'))print(df5.Name.str.find(r'(a\w{3})(\w+)'))%timeit-n5df5.Name2.s_str().find(rb'(a\w{3})(\w+)')%timeit-n5df5.Name1.s_str().find(rb'(a\w{3})(\w+)')%timeit-n5df5.Name.str.find(r'(a\w{3})(\w+)')40.5ms±641µsperloop(mean±std.dev.of7runs,5loopseach)40.4ms±175µsperloop(mean±std.dev.of7runs,5loopseach)34.5ms±231µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().extractall(rb'(a\w{3})(\w+)'))print(df5.Name1.s_str().extractall(rb'(a\w{3})(\w+)'))print(df5.Name.str.extractall(r'(a\w{3})(\w+)'))%timeit-n5df5.Name2.s_str().extractall(rb'(a\w{3})(\w+)')%timeit-n5df5.Name1.s_str().extractall(rb'(a\w{3})(\w+)')%timeit-n5df5.Name.str.extractall(r'(a\w{3})(\w+)')165ms±439µsperloop(mean±std.dev.of7runs,5loopseach)167ms±346µsperloop(mean±std.dev.of7runs,5loopseach)156ms±308µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().extract(rb'(a\w{3})'))print(df5.Name1.s_str().extract(rb'(a\w{3})'))print(df5.Name.str.extract(r'(a\w{3})'))%timeit-n5df5.Name2.s_str().extract(rb'(a\w{3})')%timeit-n5df5.Name1.s_str().extract(rb'(a\w{3})')%timeit-n5df5.Name.str.extract(r'(a\w{3})')64.7ms±699µsperloop(mean±std.dev.of7runs,5loopseach)63.7ms±343µsperloop(mean±std.dev.of7runs,5loopseach)55.8ms±352µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().endswith(b'a'))print(df5.Name1.s_str().endswith(b'a'))print(df5.Name.str.endswith('a'))%timeit-n5df5.Name2.s_str().endswith(b'a')%timeit-n5df5.Name1.s_str().endswith(b'a')%timeit-n5df5.Name.str.endswith('a')24.4ms±800µsperloop(mean±std.dev.of7runs,5loopseach)23.9ms±97.7µsperloop(mean±std.dev.of7runs,5loopseach)18.2ms±160µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().count(re.compile(b'a+')))print(df5.Name1.s_str().count(re.compile(b'a+')))print(df5.Name.str.count(re.compile('a+')))%timeit-n5df5.Name2.s_str().count(re.compile(b'a+'))%timeit-n5df5.Name1.s_str().count(re.compile(b'a+'))%timeit-n5df5.Name.str.count(re.compile('a+'))88.5ms±407µsperloop(mean±std.dev.of7runs,5loopseach)94.7ms±280µsperloop(mean±std.dev.of7runs,5loopseach)83.4ms±238µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().zfill(100))print(df5.Name1.s_str().zfill(100))print(df5.Name.str.zfill(100))%timeit-n5df5.Name2.s_str().zfill(100)%timeit-n5df5.Name1.s_str().zfill(100)%timeit-n5df5.Name.str.zfill(100)22.9ms±369µsperloop(mean±std.dev.of7runs,5loopseach)22.5ms±216µsperloop(mean±std.dev.of7runs,5loopseach)17.9ms±80.3µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().contains(b'a'))print(df5.Name1.s_str().contains(b'a'))print(df5.Name.str.contains('a'))%timeit-n5df5.Name2.s_str().contains(b'a')%timeit-n5df5.Name1.s_str().contains(b'a')%timeit-n5df5.Name.str.contains('a')30.5ms±551µsperloop(mean±std.dev.of7runs,5loopseach)30.5ms±287µsperloop(mean±std.dev.of7runs,5loopseach)24.4ms±249µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().pad(width=100,side='both',fillchar=b'-'))print(df5.Name1.s_str().pad(width=100,side='both',fillchar=b'-'))print(df5.Name.str.pad(width=100,side='both',fillchar='-'))%timeit-n5df5.Name2.s_str().pad(width=100,side='both',fillchar=b'-')%timeit-n5df5.Name1.s_str().pad(width=100,side='both',fillchar=b'-')%timeit-n5df5.Name.str.pad(width=100,side='both',fillchar='-')23.9ms±579µsperloop(mean±std.dev.of7runs,5loopseach)24.2ms±411µsperloop(mean±std.dev.of7runs,5loopseach)19.5ms±77.7µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().center(width=5))print(df5.Name1.s_str().center(width=5))print(df5.Name.str.center(width=5))%timeit-n5df5.Name2.s_str().center(width=5)%timeit-n5df5.Name1.s_str().center(width=5)%timeit-n5df5.Name.str.center(width=5)17.7ms±368µsperloop(mean±std.dev.of7runs,5loopseach)18ms±327µsperloop(mean±std.dev.of7runs,5loopseach)12.5ms±75.7µsperloop(mean±std.dev.of7runs,5loopseach)Methods much slower than Series.strprint(df5.Name2.s_str().cat())print(df5.Name1.s_str().cat())print(df5.Name.str.cat())%timeit-n5df5.Name2.s_str().cat()%timeit-n5df5.Name1.s_str().cat()%timeit-n5df5.Name.str.cat()14.1ms±706µsperloop(mean±std.dev.of7runs,5loopseach)13.9ms±877µsperloop(mean±std.dev.of7runs,5loopseach)3.74ms±17.6µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().casefold())print(df5.Name1.s_str().casefold())print(df5.Name.str.casefold())%timeit-n5df5.Name2.s_str().casefold()%timeit-n5df5.Name1.s_str().casefold()%timeit-n5df5.Name.str.casefold()168ms±556µsperloop(mean±std.dev.of7runs,5loopseach)200ms±466µsperloop(mean±std.dev.of7runs,5loopseach)11.5ms±115µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().join(b'zz'))print(df5.Name1.s_str().join(b'zz'))print(df5.Name.str.join('zz'))%timeit-n5df5.Name2.s_str().join(b'zz')%timeit-n5df5.Name1.s_str().join(b'zz')%timeit-n5df5.Name.str.join('zz')360ms±2.22msperloop(mean±std.dev.of7runs,5loopseach)366ms±514µsperloop(mean±std.dev.of7runs,5loopseach)54ms±271µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().normalize(form='NFC'))print(df5.Name1.s_str().normalize(form='NFC'))print(df5.Name.str.normalize(form='NFC'))%timeit-n5df5.Name2.s_str().normalize(form='NFC')%timeit-n5df5.Name1.s_str().normalize(form='NFC')%timeit-n5df5.Name.str.normalize(form='NFC')173ms±892µsperloop(mean±std.dev.of7runs,5loopseach)189ms±688µsperloop(mean±std.dev.of7runs,5loopseach)17.7ms±127µsperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().wrap(10))print(df5.Name1.s_str().wrap(10))print(df5.Name.str.wrap(10))%timeit-n5df5.Name2.s_str().wrap(10)%timeit-n5df5.Name1.s_str().wrap(10)%timeit-n5df5.Name.str.wrap(10)1.48s±67.7msperloop(mean±std.dev.of7runs,5loopseach)1.55s±25.5msperloop(mean±std.dev.of7runs,5loopseach)899ms±1.11msperloop(mean±std.dev.of7runs,5loopseach)#################################################################print(df5.Name2.s_str().get(1))print(df5.Name1.s_str().get(1))print(df5.Name.str.get(1))%timeit-n5df5.Name2.s_str().get(1)%timeit-n5df5.Name1.s_str().get(1)%timeit-n5df5.Name.str.get(1)45ms±604µsperloop(mean±std.dev.of7runs,5loopseach)44.8ms±294µsperloop(mean±std.dev.of7runs,5loopseach)24.5ms±644µsperloop(mean±std.dev.of7runs,5loopseach)