Package: fastnumbers
Super-fast and clean conversions to numbers.Source Code:https://github.com/SethMMorton/fastnumbersDownloads:https://pypi.org/project/fastnumbers/Documentation:https://fastnumbers.readthedocs.io/Quick StartTimingHigh-level AlgorithmHow To Run TestsHistoryfastnumbersis a module with the following three objectives (in order
of decreasing importance as to why the module was created):Provide a set of convenience functions that wrap calls tointandfloatand provides easy, concise, powerful, fast
and flexible error handling.Provide a set of functions that can be used to rapidly identify if
an inputcouldbe converted tointorfloat.Provide drop-in replacements for the Python built-inintandfloatthat are on par or faster with the Python equivalents
(see theTimingsection for details). These functions
should behaveidenticallyto the Python built-ins except for a few
specific corner-cases as mentioned in theAPI documentation for those functions.PLEASEread the quick start for these functions to fully
understand the caveats before using them.What kind of speedups can you expect?Here are some highlights, but please
see theTimingsection for the raw data if you want details.Up to 2x faster conversion of strings to integers than the built-inint()functionUp to 5x faster conversion of strings to floats than the built-infloat()function (possibly greater for very long strings)Up to 10x faster handling of errors during conversion than using
user-side error handlingOn top of the above, operations to convert a list of strings
(with themapoption ortry_arrayfunction) is 2x faster
than the equivalent list comprehension.NOTICE: As offastnumbersversion 4.0.0, only Python >= 3.7 is
supported.NOTICE: As offastnumbersversion 4.0.0, the functionsfast_real,fast_float,fast_int,fast_forceint,isreal,isfloat,isint, andisintlikehave been deprecated and are replaced withtry_real,try_float,try_int,try_forceint,check_real,check_float,check_int, andcheck_intlike, respectively. These
new functions have more flexible APIs and have names that better reflect
the intent of the functions. The old functions can still be used (they willneverbe removed fromfastnumbers), but the new ones should be
preferred for new development.NOTICE: As offastnumbersversion 4.0.0,query_typenow setsallow_underscorestoFalseby default instead ofTrue.Quick StartError-handling FunctionsChecking FunctionsDrop-in Replacement FunctionsThere are three broad categories of functions exposed byfastnumbers.
The below quick start will demonstrate each of these categories. The
quick start is “by example”, and will show a sample interactive session
using thefastnumbersAPI.Error-Handling FunctionsError-handling function APIFast operations on lists and other iterablesAbout the on_fail optionAbout the denoise optiontry_floatwill be used to demonstrate the functionality of thetry_*functions.>>>fromfastnumbersimportRAISE,try_float>>># Convert string to a float>>>try_float('56.07')56.07>>># Integers are converted to floats>>>try_float(54)54.0>>>>>># Unconvertable string returned as-is by default>>>try_float('bad input')'bad input'>>># Unconvertable strings can trigger a default value>>>try_float('bad input',on_fail=0)0>>>>>># One can ask inf or nan to be substituted with another value>>>try_float('nan')nan>>>try_float('nan',nan=0.0)0.0>>>try_float(float('nan'),nan=0.0)0.0>>>try_float('56.07',nan=0.0)56.07>>>>>># The default built-in float behavior can be triggered with>>># RAISE given to "on_fail".>>>try_float('bad input',on_fail=RAISE)#doctest: +IGNORE_EXCEPTION_DETAILTraceback(mostrecentcalllast):...ValueError:invalidliteralforfloat():badinput>>>>>># A function can be used to return an alternate value for invalid input>>>try_float('bad input',on_fail=len)9>>>try_float(54,on_fail=len)54.0>>>>>># Single unicode characters can be converted.>>>try_float('\u2164')# Roman numeral 5 (V)5.0>>>try_float('\u2466')# 7 enclosed in a circle7.0try_intbehaves the same astry_float, but for integers.>>>fromfastnumbersimporttry_int>>>try_int('1234')1234>>>try_int('\u2466')7try_realis liketry_floatortry_intdepending
on if there is any fractional component of thi return value.>>>fromfastnumbersimporttry_real>>>try_real('56')56>>>try_real('56.0')56>>>try_real('56.0',coerce=False)56.0>>>try_real('56.07')56.07>>>try_real(56.07)56.07>>>try_real(56.0)56>>>try_real(56.0,coerce=False)56.0try_forceintalways returns an integer.>>>fromfastnumbersimporttry_forceint>>>try_forceint('56')56>>>try_forceint('56.0')56>>>try_forceint('56.07')56>>>try_forceint(56.07)56Fast operations on lists and other iterablesEach of thetry_*functions have amapoption causes the function
to accept an iterable of items to convert and returns a list. Usingtry_floatas an example, the following are all functionally equivalent.>>>fromfastnumbersimporttry_float>>>iterable=["5","4.5","34567.6","32"]>>>try_float(iterable,map=list)==list(map(try_float,iterable))True>>>try_float(iterable,map=list)==[try_float(x)forxiniterable]True>>>try_float(iterable,map=list)==list(try_float(iterable,map=True))TrueThe difference is that themapoption is 2x the speed of the list
comprehension method, and 1.5x the speed of themapmethod. The reason
is that it avoids Python function call overhead on each iteration. Note thatTruecauses the function to return an iterator, andlistcauses it to
return alist. In practice the performance of these are similar
(seeTimingfor raw data).If you need to store your output in anumpyarray, you can usetry_arrayto do this conversion directly. This function has some
additional handling for overflow that is not present in the otherfastnumbersfunctions that may come in handy when dealing withnumpyarrays.>>>fromfastnumbersimporttry_array>>>importnumpyasnp>>>iterable=["5","4.5","34567.6","32"]>>>np.array_equal(np.array(try_float(iterable,map=list),dtype=np.float64),try_array(iterable))TrueYou will see about a 2x speedup of doing this in one step over converting
to a list then converting that list to an array.About theon_failoptionTheon_failoption is a way for you to doanythingin the event that
the given input cannot be converted to a number. It canreturn given object as-is if set tofastnumbers.INPUT(this is the default)raise aValueErrorif set tofastnumbers.RAISEreturn a default value if given any non-callable objectcall a function with the given object if given a single-argument callableBelow are a couple of ideas to get you thinking.NOTE:: There is also anon_type_erroroption that behaves the same ason_failexcept that a) it is triggered when the given object is of an
invalid type and b) the default value isfastnumbers.RAISE, notfastnumbers.INPUT.>>>fromfastnumbersimportINPUT,RAISE,try_float>>># You want to convert strings that can be converted to numbers, but>>># leave the rest as strings. Use fastnumbers.INPUT (the default)>>>try_float('45.6')45.6>>>try_float('invalid input')'invalid input'>>>try_float('invalid input',on_fail=INPUT)'invalid input'>>>>>>>>>>>># You want to convert any invalid string to NaN>>>try_float('45.6',on_fail=float('nan'))45.6>>>try_float('invalid input',on_fail=float('nan'))nan>>>>>>>>>>>># Simple callable case, send the input through some function to generate a number.>>>try_float('invalid input',on_fail=lambdax:float(x.count('i')))# count the 'i's3.0>>>>>>>>>>>># Suppose we know that our input could either be a number, or if not>>># then we know we just have to strip off parens to get to the number>>># e.g. the input could be '45' or '(45)'. Also, suppose that if it>>># still cannot be converted to a number we want to raise an exception.>>>defstrip_parens_and_try_again(x):...returntry_float(x.strip('()'),on_fail=RAISE)...>>>try_float('45',on_fail=strip_parens_and_try_again)45.0>>>try_float('(45)',on_fail=strip_parens_and_try_again)45.0>>>try_float('invalid input',on_fail=strip_parens_and_try_again)#doctest: +IGNORE_EXCEPTION_DETAILTraceback(mostrecentcalllast):...ValueError:invalidliteralforfloat():invalidinput>>>>>>>>>>>># Suppose that whenever an invalid input is given, it needs to be>>># logged and then a default value is returned.>>>deflog_and_default(x,log_method=print,default=0.0):...log_method("The input{!r}is not valid!".format(x))...returndefault...>>>try_float('45',on_fail=log_and_default)45.0>>>try_float('invalid input',on_fail=log_and_default)Theinput'invalid input'isnotvalid!0.0>>>try_float('invalid input',on_fail=lambdax:log_and_default(x,default=float('nan')))Theinput'invalid input'isnotvalid!nanAbout thedenoiseoptionThedenoiseoption is available on thetry_realandtry_forceintoptions.
To best understand its usage, consider the following native Python behavior:>>>int(3.453e21)3452999999999999737856>>>int(float("3.453e21"))3452999999999999737856>>># Most users would likely expect this result from decimal.Decimal>>>importdecimal>>>int(decimal.Decimal("3.453e21"))3453000000000000000000>>># But watch out, even decimal.Decimal doesn't help for float input>>>importdecimal>>>int(decimal.Decimal(3.453e21))3452999999999999737856Because the conversion of a float to an int goes through the Cdoubledata type which
has inherent limitations on accuracy (Seethis Stack Overflow question for examples)
the resultingintresult has “noise” digits that are not part of the original float
representation.For functions where this makes sense,fastnumbersprovides thedenoiseoption to
give you the results thatdecimal.Decimalwould give for strings containing floats.>>>fromfastnumbersimporttry_real>>>try_real(3.453e21)3452999999999999737856>>>try_real("3.453e21")3452999999999999737856>>>try_real(3.453e21,denoise=True)3453000000000000000000>>>try_real("3.453e21",denoise=True)3453000000000000000000Two things to keep in mind:Thedenoiseoption adds additional overhead to the conversion calculation, so please consider
the trade-offs between speed and accuracy when determining whether or not to use it. It issignificantlyfaster than usingdecimal.Decimal, but much slower than not using it at all.For string input,denoisewill return results identical todecimal.Decimal. For float
input,denoisewill return results that are accurate to about 15 digits (Cdoublecan
only store 16 decimal digits, so this means that only the last possible digit may not be accurate).Checking FunctionsChecking function APIcheck_floatwill be used to demonstrate the functionality of thecheck_*functions. There is also thequery_typefunction.>>>fromfastnumbersimportcheck_float>>>fromfastnumbersimportALLOWED,DISALLOWED,NUMBER_ONLY,STRING_ONLY>>># Check that a string can be converted to a float>>>check_float('56')True>>>check_float('56',strict=True)False>>>check_float('56.07')True>>>check_float('56.07 lb')False>>>>>># Check if a given number is a float>>>check_float(56.07)True>>>check_float(56)False>>>>>># Specify if only strings or only numbers are allowed>>>check_float(56.07,consider=STRING_ONLY)False>>>check_float('56.07',consider=NUMBER_ONLY)False>>>>>># Customize handling for nan or inf (see API for more details)>>>check_float('nan')False>>>check_float('nan',nan=ALLOWED)True>>>check_float(float('nan'))True>>>check_float(float('nan'),nan=DISALLOWED)Falsecheck_intworks the same ascheck_float, but for integers.>>>fromfastnumbersimportcheck_int>>>check_int('56')True>>>check_int(56)True>>>check_int('56.0')False>>>check_int(56.0)Falsecheck_realis very permissive - any float or integer is accepted.>>>fromfastnumbersimportcheck_real>>>check_real('56.0')True>>>check_real('56')True>>>check_real(56.0)True>>>check_real(56)Truecheck_intlikechecks if a number is “int-like”, if it has no
fractional component.>>>fromfastnumbersimportcheck_intlike>>>check_intlike('56.0')True>>>check_intlike('56.7')False>>>check_intlike(56.0)True>>>check_intlike(56.7)FalseThequery_typefunction can be used if you need to determine if
a value is one of many types, rather than whether or not it is one specific
type.>>>fromfastnumbersimportquery_type>>>query_type('56.0')<class'float'>>>>query_type('56')<class'int'>>>>query_type(56.0)<class'float'>>>>query_type(56)<class'int'>>>>query_type(56.0,coerce=True)<class'int'>>>>query_type('56.0',allowed_types=(float,int))<class'float'>>>>query_type('hey')<class'str'>>>>query_type('hey',allowed_types=(float,int))# returns NoneDrop-in Replacement FunctionsDrop-in replacement function APIPLEASEdo not take it for granted that these functions will provide you
with a speedup - they may not. Every platform, compiler, and data-set is
different, and you should perform a timing test on your system with your data
to evaluate if you will see a benefit. As you can see from the data linked in
theTimingsection, the amount of speedup you will get is particularly
data-dependent.In generalyou will see a performance boost for floats (and
this boost increases as the size of the float increases), but for integers it
is largely dependent on the length of the integer. You will likelynotsee
a performance boost if the input are already numbers instead of strings.NOTE: in the below examples, we usefrom fastnumbers import intinstead
ofimport fastnumbers. This is because callingfastnumbers.int()is a
bit slower than justint()because Python has to first findfastnumbersin your namespace, then findintin thefastnumbersnamespace, instead
of just findingintin your namespace - this will slow down the function
call and defeat the purpose of usingfastnumbers. If you do not want to
actually shadow the built-inintfunction, you can dofrom fastnumbers import int as fn_intor something like that.>>># Use is identical to the built-in functions>>>fromfastnumbersimportfloat,int>>>float('10')10.0>>>int('10')10>>>float('bad input')#doctest: +IGNORE_EXCEPTION_DETAILTraceback(mostrecentcalllast):...ValueError:invalidliteralforfloat():badinputrealis provided to give a float or int depending
on the fractional component of the input.>>>fromfastnumbersimportreal>>>real('56.0')56>>>real('56.7')56.7>>>real('56.0',coerce=False)56.0TimingJust how much faster isfastnumbersthan a pure python implementation?
Please lookhttps://github.com/SethMMorton/fastnumbers/tree/main/profiling.High-Level AlgorithmFor integers, CPython goes to great lengths to ensure that your string input
is converted to a numbercorrectlyandlosslessly(you can prove this to
yourself by examining the source code forinteger conversions).
This extra effort is only needed for integers that cannot fit into a 64-bit
integer data type - for those that can, a naive algorithm of < 10 lines
of C code is sufficient and significantly faster.fastnumbersuses a
heuristic to determine if the input can be safely converted with the much
faster naive algorithm, and if so it does so, falling back on
the CPython implementation for longer input strings.
Most real-world numbers pass the heuristic and so you should generally see
improved performance withfastnumbersfor integers.For floats,fastnumbersutilizes the ultra-fastfast_float::from_charsfunction
to convert strings representing floats into a Cdoubleboth quicklyand
safely- the conversion provides the same accuracy as the CPythonfloat conversion functionbut instead of scaling linearly with length of the input string it seems
to have roughly constant performance. By completely bypassing the CPython
converter we get significant performance gains with no penalty, so you
should always see improved performance withfastnumbersfor floats.InstallationUsepip!$ pip install fastnumbersHow to Run TestsPlease note thatfastnumbersis NOT set-up to supportpython setup.py test.The recommended way to run tests is withtox.
Suppose you want to run tests for Python 3.8 - you can run tests by simply
executing the following:$toxrun-epy38toxwill create virtual a virtual environment for your tests and install
all the needed testing requirements for you.If you want to run testing on all supported Python versions you can simply execute$toxrunYou can change the how much “random” input your tests will try with# Run fewer tests with "random" input - much faster$toxrun----hypothesis-profilefast# Run more tests with "random" input - takes much longer but is more thorough$toxrun----hypothesis-profilethoroughIf you want to run the performce analysis yourself, you can execute# This assumes Python 3.9 - adjust for the version you want to profile$toxrun-epy39-profIf you do not wish to usetox, you can install the testing dependencies with thedev-requirements.txtfile and then run the tests manually usingpytest.$pipinstall-rdev/requirements.txt$pytestAuthorSeth M. MortonHistoryPlease visit the changelogon GitHub.