Package: lxmlrpc_monkey
lxmlrpc - XMLRPClib patchThis module monkeypatch python’sxmlrpclibto uselxmlbased parser
to reduce memory consumption on big xmlrpc requests / responses (100+ Mb)NOTEThis module is useful only for*python2.7*.Use itonlyif you suffer fromhigh memory consumptionofxmlrpclibInstallThis module is hosted onPyPIso it could bu easily installed viapip:pipinstalllxmlrpc_monkeyUsageTo use this module just do followingimportxmlrpclibfromlxmlrpc_monkeyimportpatch_xmlrpclib# This line will monkey-patch xmlrpclib to use lxml for parserpatch_xmlrpclib()Benchmarks (how to run)To run benchmarks:installmemory_profilerprepare data withpython generate_data.py--path<demo data path>--size50000000run benchmarks withpython benchmark.py--path<demo data path>Benchmark results (50 Mb (real 65 Mb) data file)look at`p.feed(data)`call in`loads`function ofxmlrpclib*Running unpatched loads*Filename: /usr/lib/python2.7/xmlrpclib.pyLine #Mem usageIncrementLine Contents1134104.7 MiB0.0 MiBdef loads(data, use_datetime=0):1135“””data -> unmarshalled data, method name11361137Convert an XML-RPC packet to unmarshalled data plus a method1138name (None if not present).11391140If the XML-RPC packet represents a fault condition, this function1141raises a Fault exception.1142“””1143104.7 MiB0.0 MiBp, u = getparser(use_datetime=use_datetime)>1144622.4 MiB517.7 MiBp.feed(data)1145558.0 MiB-64.4 MiBp.close()1146558.0 MiB0.0 MiBreturn u.close(), u.getmethodname()*Running patched loads*Filename: /usr/lib/python2.7/xmlrpclib.pyLine #Mem usageIncrementLine Contents1134106.9 MiB0.0 MiBdef loads(data, use_datetime=0):1135“””data -> unmarshalled data, method name11361137Convert an XML-RPC packet to unmarshalled data plus a method1138name (None if not present).11391140If the XML-RPC packet represents a fault condition, this function1141raises a Fault exception.1142“””1143106.9 MiB0.0 MiBp, u = getparser(use_datetime=use_datetime)>1144235.9 MiB129.0 MiBp.feed(data)1145171.5 MiB-64.4 MiBp.close()1146171.5 MiB0.0 MiBreturn u.close(), u.getmethodname()Filename: bechmark.pyLine #Mem usageIncrementLine Contents13104.7 MiB0.0 MiB@profile14def bench_load(xmldata):15104.7 MiB0.0 MiBprint (“Running unpatched loads”)16106.9 MiB2.2 MiBloads(xmldata)1718106.9 MiB0.0 MiBlxmlrpc.patch_xmlrpclib()1920106.9 MiB0.0 MiBprint (“Running patched loads”)21107.1 MiB0.2 MiBloads(xmldata)Benchmark results (100 Mb (real 129 Mb) data file)*Running unpatched loads*—Filename: /usr/lib/python2.7/xmlrpclib.pyLine #Mem usageIncrementLine Contents1134169.2 MiB0.0 MiBdef loads(data, use_datetime=0):1135“””data -> unmarshalled data, method name11361137Convert an XML-RPC packet to unmarshalled data plus a method1138name (None if not present).11391140If the XML-RPC packet represents a fault condition, this function1141raises a Fault exception.1142“””1143169.2 MiB0.0 MiBp, u = getparser(use_datetime=use_datetime)>11441203.0 MiB1033.8 MiBp.feed(data)11451074.2 MiB-128.8 MiBp.close()11461074.2 MiB0.0 MiBreturn u.close(), u.getmethodname()*Running patched loads*—Filename: /usr/lib/python2.7/xmlrpclib.pyLine #Mem usageIncrementLine Contents1134171.6 MiB0.0 MiBdef loads(data, use_datetime=0):1135“””data -> unmarshalled data, method name11361137Convert an XML-RPC packet to unmarshalled data plus a method1138name (None if not present).11391140If the XML-RPC packet represents a fault condition, this function1141raises a Fault exception.1142“””1143171.6 MiB0.0 MiBp, u = getparser(use_datetime=use_datetime)>1144429.4 MiB257.8 MiBp.feed(data)1145300.6 MiB-128.8 MiBp.close()1146300.6 MiB0.0 MiBreturn u.close(), u.getmethodname()Filename: bechmark.pyLine #Mem usageIncrementLine Contents13169.2 MiB0.0 MiB@profile14def bench_load(xmldata):15169.2 MiB0.0 MiBprint (“Running unpatched loads”)16171.6 MiB2.4 MiBloads(xmldata)1718171.6 MiB0.0 MiBlxmlrpc.patch_xmlrpclib()1920171.6 MiB0.0 MiBprint (“Running patched loads”)21171.8 MiB0.2 MiBloads(xmldata)