Package: dike
dikePython asyncio tools for web service resilienceDocumentation:https://chr1st1ank.github.io/dike/FeaturesRetry decorator for asynchronous functionsA very common task especially for network calls is an automatic retry with proper exception
logging. There are good implementations like theretrypackage for classic functions. But dike provides a similar implementation for coroutine functions.
This is available with the@retrydecorator.Simplified example:importasyncioimportdatetimeimportloggingimportsysimportdike@dike.retry(attempts=2,delay=datetime.timedelta(milliseconds=10),exception_types=RuntimeError)asyncdefweb_request():raiseRuntimeError("Request failed!")asyncdefmain():response=awaitweb_request()print(response)logging.basicConfig(stream=sys.stdout)asyncio.run(main())The output shows first a warning log message including the exception info (that's configurable).
This is especially useful if you use structured logging.# Log for first attempt:
WARNING:dike:Caught exception RuntimeError('Request failed!'). Retrying in 0.01s ...
Traceback (most recent call last):
...
RuntimeError: Request failed!Then, the for the final failure the exception is propagated to the function caller:Traceback (most recent call last):
  ...
RuntimeError: Request failed!

Process finished with exit code 1Concurrency limiting for asynchronous functionsThe@limit_jobsdecorator allows to limit the number of concurrent excecutions of a coroutine
function. This can be useful for limiting queueing times or for limiting the load put
onto backend services.Example with an external web request using thehttpxlibrary:importasyncioimportdikeimporthttpx@dike.limit_jobs(limit=2)asyncdefweb_request():"""Sends a slow web request"""asyncwithhttpx.AsyncClient()asclient:response=awaitclient.get("https://httpstat.us/200?sleep=100")returnresponseasyncdefmain():# Send three requests at the same timecall1=web_request()call2=web_request()call3=web_request()responses=awaitasyncio.gather(call1,call2,call3,return_exceptions=True)# Print the responsesforrinresponses:ifisinstance(r,dike.TooManyCalls):print("too many calls")else:print(r)asyncio.run(main())The output shows that the first two requests succeed. The third one hits the concurrency limit and a TooManyCalls exception is returned:<Response [200 OK]>
<Response [200 OK]>
too many callsMini-batching for asynchronous function callsThe@batchdecorator groups function calls into batches and only calls the wrapped function
with the aggregated input.This is useful if the function scales well with the size of the input arguments but you're
getting the input data in smaller bits, e.g. as individual HTTP requests.The arguments can be batched together as a Python list or optionally also as numpy array.Example:importasyncioimportdike@dike.batch(target_batch_size=3,max_waiting_time=10)asyncdefadd_args(arg1,arg2):"""Elementwise sum of the values in arg1 and arg2"""print(f"arg1:{arg1}")print(f"arg2:{arg2}")return[a1+a2fora1,a2inzip(arg1,arg2)]asyncdefmain():result=awaitasyncio.gather(add_args([0],[1]),add_args([1],[1]),add_args([2,3],[1,1]),)print(f"Result:{result}")asyncio.run(main())Output:arg1: [0, 1, 2, 3]
arg2: [1, 1, 1, 1]
Result: [[1], [2], [3, 4]]InstallationSimply install from pypi. The library is pure Python without any dependencies other than the
standard library.pip install dike