Package: fast-query-parsers
ProjectStatusCI/CDPackageCommunityMetaFast Query ParsersThis library includes ultra-fast Rust based query string and urlencoded parsers. These parsers are used
byLitestar, but are developed separately - and can of course be used separately.[!IMPORTANT]Starlite has been renamed to LitestarInstallationpipinstallfast-query-parsersUsageThe library exposes two functionparse_query_stringandparse_url_encoded_dict.parse_query_stringThis function is used to parse a query string into a list of key/value tuples.fromfast_query_parsersimportparse_query_stringresult=parse_query_string(b"value=1&value=2&type=dollar&country=US","&")# [("value", "1"), ("value", "2"), ("type", "dollar"), ("country", "US")]The first argument to this function is a byte string that includes the query string to be parsed, the second argument is
the separator used.BenchmarksQuery string parsing is more than x5 times faster than the standard library:stdlib parse_qsl parsing query string: Mean +- std dev: 2.86 us +- 0.03 us
.....................
parse_query_string parsing query string: Mean +- std dev: 916 ns +- 13 ns
.....................
stdlib parse_qsl parsing urlencoded query string: Mean +- std dev: 8.30 us +- 0.10 us
.....................
parse_query_string urlencoded query string: Mean +- std dev: 1.50 us +- 0.03 usparse_url_encoded_dictThis function is used to parse a url-encoded form data dictionary and parse it into the python equivalent of JSON types.fromurllib.parseimporturlencodefromfast_query_parsersimportparse_url_encoded_dictencoded=urlencode([("value","10"),("value","12"),("veggies",'["tomato", "potato", "aubergine"]'),("nested",'{"some_key": "some_value"}'),("calories","122.53"),("healthy","true"),("polluting","false"),("json","null"),]).encode()result=parse_url_encoded_dict(encoded,parse_numbers=True)# result == {#     "value": [10, 12],#     "veggies": ["tomato", "potato", "aubergine"],#     "nested": {"some_key": "some_value"},#     "calories": 122.53,#     "healthy": True,#     "polluting": False,#     "json": None,# }This function handles type conversions correctly - unlike the standard library functionparse_qs. Additionally, it
does not nest all values inside lists.Note: the second argument passed toparse_url_encoded_dictdictates whether numbers should be parsed. IfTrue,
the value will be parsed into an int or float as appropriate, otherwise it will be kept as a string.
By default the value of this arg isTrue.BenchmarksUrl Encoded parsing is more than x2 times faster than the standard library, without accounting for parsing of values:stdlib parse_qs parsing url-encoded values into dict: Mean +- std dev: 8.99 us +- 0.09 us
.....................
parse_url_encoded_dict parse url-encoded values into dict: Mean +- std dev: 3.77 us +- 0.08 usTo actually mimic the parsing done byparse_url_encoded_dictwe will need a utility along these lines:fromcollectionsimportdefaultdictfromcontextlibimportsuppressfromjsonimportloads,JSONDecodeErrorfromtypingimportAny,DefaultDict,Dict,Listfromurllib.parseimportparse_qsldefparse_url_encoded_form_data(encoded_data:bytes)->Dict[str,Any]:"""Parse an url encoded form data into dict of parsed values"""decoded_dict:DefaultDict[str,List[Any]]=defaultdict(list)fork,vinparse_qsl(encoded_data.decode(),keep_blank_values=True):withsuppress(JSONDecodeError):v=loads(v)ifisinstance(v,str)elsevdecoded_dict[k].append(v)return{k:viflen(v)>1elsev[0]fork,vindecoded_dict.items()}With the above, the benchmarks looks like so:python parse_url_encoded_form_data parsing url-encoded values into dict: Mean +- std dev: 19.7 us +- 0.1 us
.....................
parse_url_encoded_dict parsing url-encoded values into dict: Mean +- std dev: 3.69 us +- 0.03 usContributingAll contributions are of course welcome!Repository SetupRuncargo installto setup the rust dependencies andpoetry installto setup the python dependencies.Install the pre-commit hooks withpre-commit install(requirespre-commit).BuildingRunpoetry run maturin develop --release --stripto install a release wheel (without debugging info). This wheel can be
used in tests and benchmarks.BenchmarkingThere are basic benchmarks using pyperf in place. To run these executepoetry run python benchrmarks.py.