Package: async-tamer
Async-TamerYou shouldn't have to split your codebase or jump through hoops just to useasync def. After all async functions are just functions ... with superpowers.
Async-Tamer helps you harness these superpowers by seeingasync deffor what
it is: A means to flag and reduce periods of busy waiting. (Details at the
end of this readme.)Features✅ 100% python✅ 100% free (BSD 3-clause)✅ 100% lean (no dependencies)Installationpip install async-tamerUsageIn a nutshell, you add@tamedto an asynchronous function to enable calling it
directly from either sync and async contexts. You may also assign a@tamedfunction to anAsyncScopeto get structured lifecycle management.importasynciofromtamerimporttamed,AsyncScope@tamed# <-- Notice the decoratorasyncdefslow_echo(msg:str,delay:int)->None:awaitasyncio.sleep(delay)print(msg)slow_echo("sync > DELAY(.5)",0.5)withAsyncScope()asscope:slow_echo("scope > DELAY(.2)",.2,_async_scope=scope)slow_echo("scope > DELAY(.1)",.1,_async_scope=scope)# implicit await :)# Output# ------## sync > DELAY(.5)# scope > DELAY(.1)# scope > DELAY(.2)The@tameddecoratorA@tamedasynchronous function changes its execution policy (how it behaves)
depending on the context it is called from. In synchronous contexts, it behaves
like an ordinary function (blocking). In async contexts, it behaves like an
ordinary coroutine (non-blocking), and when assigned to anAsyncScopeit
follows the scopes context manager (non-blocking).importasynciofromtamerimporttamed,AsyncScope@tamed# <-- notice the decoratorasyncdefslow_echo(msg:str,delay:int)->None:awaitasyncio.sleep(delay)print(msg)# ============================# Asynchronous Execution# ============================asyncdefmain():first=slow_echo("async > DELAY(1)",1)second=slow_echo("async > DELAY(.1)",0.1)third=slow_echo("async > DELAY(.5)",0.5)# Don't forget the await!awaitsecondawaitasyncio.gather(first,third)asyncio.run(main())# Output# ------## async > DELAY(.1)# async > DELAY(.5)# async > DELAY(1)# ============================# Synchronous Execution# ============================slow_echo("sync > DELAY(1)",1)slow_echo("sync > DELAY(.1)",0.1)slow_echo("sync > DELAY(.5)",0.5)# Output# ------## sync > DELAY(1)# sync > DELAY(.1)# sync > DELAY(.5)# ============================# AsyncScope Execution# ============================withAsyncScope()asscope:slow_echo("scope > DELAY(1)",1,_async_scope=scope)slow_echo("scope > DELAY(.1)",0.1,_async_scope=scope)slow_echo("scope > DELAY(.5)",0.5,_async_scope=scope)# Output# ------## scope > DELAY(.1)# scope > DELAY(.5)# scope > DELAY(1)Note: The_async_scopekwarg is injected by the@tameddecorator and
is used to add a@tamedfunction to anAsyncScope. The reason you may want
to do this is documented with examples in theAsyncScopesection.Returning Results@tamedfunctions know when (and how) you expect results to be returned.importasynciofromtamerimporttamed,AsyncScope@tamedasyncdefslow_io():awaitasyncio.sleep(0.1)return200,"Time to be awesome!"# ============================# Asynchronous Execution# ============================asyncdefmain():coro=slow_io()# <-- normal coroutinereturn_code,msg=awaitcoro# <-- await the resultprint(f"Status{return_code}: `{msg}`")asyncio.run(main())# Output# ------## Status 200: `Time to be awesome!`# ============================# Synchronous Execution# ============================return_code,msg=slow_io()# <-- immediate resultprint(f"Status{return_code}: `{msg}`")# Output# ------## Status 200: `Time to be awesome!`# ============================# AsyncScope Execution# ============================withAsyncScope()asscope:delayed_result=slow_io(_async_scope=scope)# <-- implicit await on exitreturn_code,msg=delayed_result.valueprint(f"Status{return_code}: `{msg}`")# Output# ------## Status 200: `Time to be awesome!`When called with anAsyncScopea@tamedfunction will return an instance ofDelayedResult. This object represents theresultof the@tamedfunction
and should not be confused with similar concepts like aFuture,asyncio.Task, orCoroutinewhich represent concurrently executing code.
While those are related objects, aDelayedResultis simpler. For example,
unlike code, results don't execute. As such you can't cancel them nor can you
chain callbacks. They (results) are simply values that a function outputs and in
the case of aDelayedResultit is a value that arrives late to the party.What you can do with aDelayedResultisawaitit in an async context or use
it to.block()a synchronous context until it becomes available. Further, you
can inspect it's.valuewhich will either return the result or raise anAttributeErrorif the result is unavailable.importasynciofromtamerimporttamed,AsyncScope@tamedasyncdefrequest(delay:int):awaitasyncio.sleep(delay)return200,"You are awesome!"@tamedasyncdefpost_process(raw_result):ret_code,msg=awaitraw_result# <-- awaitable in async contexta,b=msg.rsplit(" ",1)returnret_code," ".join((a,"very",b))withAsyncScope()asscope:raw_result=request(0.1,_async_scope=scope)result=post_process(raw_result,_async_scope=scope)# <-- pass it aroundtry:return_code,msg=result.valueexceptAttributeError:# <-- AttributeError, not attived yetprint(f"scope > result: Not yet available.")result.block()# <-- block in sync contextreturn_code,msg=result.valueprint(f"scope > result: Status{return_code}: `{msg}`")# Output# ------## scope > result: Not yet available.# scope > result: Status 200: `You are very awesome!`TheAsyncScopeAnAsyncScopemanages a set of@tamedfunctions and controls their
lifecycle. It's a structured way to add async sections to your program. Long
story short, you need to be aware of 3 keywords:Scheduling: Line of code that starts a@tamedfunction.Switching: Line of code that switches to another execution context.Cleaning: Line of code that deals with error handling.Youschedulea@tamedfunction by calling it with_async_scope=set to a
meaningful value and theAsyncScopehelps withswitchingandcleaning. To
that end it guarantees thatall functions within the scope have finished when
a scope exits. To achieve this, itswitchesbetween async contexts at the
end of the scope until all its functions have finished. Note here that finished
does not mean succeeded; functions may raise exceptions or get cancled. This is
where thecleaningpart comes in which we cover later during "Exception
Management".Additionally, you can nest scopes.@tamedfunctions assigned to anouter_scopeexecute independelty and alongside@tamedfunctions from aninner_scopeand there can be arbitraryswitchingbetween them. However,
since theinner_scopewaits for all its functions to complete beforeswitchingback to the synchronous context, theschedulingof new functions
below after theinner_scopewill wait forinner_scope's completion.importasynciofromtamerimporttamed,AsyncScope@tamedasyncdefslow_echo(msg:str,delay:int)->None:awaitasyncio.sleep(delay)print(msg)withAsyncScope()asouter_scope:slow_echo("Outer Scope > DELAY(1.5)",1.5,_async_scope=outer_scope)slow_echo("Outer Scope > DELAY(1)",1,_async_scope=outer_scope)withAsyncScope()asinner_scope:slow_echo("Outer Scope > Inner Scope > DELAY(2)",2,_async_scope=inner_scope)slow_echo("Outer Scope > Inner Scope > DELAY(1)",1,_async_scope=inner_scope)# await inner_scope functions# Note: scheduled after inner scope has finishedslow_echo("Outer Scope > DELAY(.5)",0.5,_async_scope=outer_scope)# Output# ------## Outer Scope > DELAY(1)# Outer Scope > Inner Scope > DELAY(1)# Outer Scope > DELAY(1.5)# Outer Scope > Inner Scope > DELAY(2)# Outer Scope > DELAY(.5)Just like@tamedandDelayedResult, this works not just in synchronous
contexts (with) but also in asynchronous ones (async with).importasynciofromtamerimporttamed,AsyncScope@tamedasyncdefslow_echo(msg:str,delay:int)->None:awaitasyncio.sleep(delay)print(msg)@tamedasyncdefslow_bulk_echo()->None:asyncwithAsyncScope()asouter_scope:# <-- `async with` in async contextsslow_echo("Outer Scope > DELAY(1.5)",1.5,_async_scope=outer_scope)slow_echo("Outer Scope > DELAY(1)",1,_async_scope=outer_scope)asyncwithAsyncScope()asinner_scope:slow_echo("Outer Scope > Inner Scope > DELAY(2)",2,_async_scope=inner_scope)slow_echo("Outer Scope > Inner Scope > DELAY(1)",1,_async_scope=inner_scope)# await inner_scope functions# Note: scheduled after inner scope has finishedslow_echo("Outer Scope > DELAY(.5)",0.5,_async_scope=outer_scope)slow_bulk_echo()# Output# ------## Outer Scope > DELAY(1)# Outer Scope > Inner Scope > DELAY(1)# Outer Scope > DELAY(1.5)# Outer Scope > Inner Scope > DELAY(2)# Outer Scope > DELAY(.5)The ability to nestAsyncScopesis especially useful when you combine it with
its kwargs:exit_modeanderror_mode. As the names suggest, theexit_modecontrols what happens when the scope exits and theerror_modecontrols what happens when an assigned function produces an exception.By default these are set toexit_mode="wait"anderror_mode="cancel". The
former will"wait"for unfinished functions at the end of the scope. The
latter will"cancel"other unfinished functions if one of them fails. This
behavior matches aasyncio.TaskGrouportrio.Nursery. It is useful when you
call functions in batches, e.g., when making several web API calls or reading a
batch of images from disk.Alternatively, you can useexit_mode="cancel"which will"cancel"unfinished
functions at the end of the scope. This is useful to shut down "infinity loops"
or to cancel ongoing requests for data that you thought you'd need, but didn't.importasynciofromdatetimeimportdatetimefromtamerimporttamed,AsyncScopeclassRateLimiter:def__init__(self):# allow an initial burstself.max_tokens=3self.tokens=self.max_tokens@tamedasyncdefgenerate_tokens(self,delay:int):whileTrue:# <-- generate new tokens foreverawaitasyncio.sleep(delay)self.tokens=min(self.tokens+1,self.max_tokens)@tamedasyncdefget_token(self):# Note: This would not work with threads, but is perfectly# fine in asynciowhileself.tokens==0:awaitasyncio.sleep(0)self.tokens-=1returnTrue@tamedasyncdeffake_request(rate_limiter):awaitrate_limiter.get_token()print(datetime.now().strftime("%H:%M:%S.%f"),"Requesting...")throttle=RateLimiter()withAsyncScope(exit_mode="cancel")asservice_layer:throttle.generate_tokens(1,_async_scope=service_layer)withAsyncScope()asbatch:for_inrange(6):fake_request(throttle,_async_scope=batch)# <-- wait for all requests to finish# <-- cancel the rate limiter# Output# ------# 00:22:28.348290 Requesting...# 00:22:28.348436 Requesting...# 00:22:28.348564 Requesting...# 00:22:29.347495 Requesting...# 00:22:30.347555 Requesting...# 00:22:31.347597 Requesting...Exception ManagementUnfortunately, shit happens. If it does, Python raises an exception and you, the
author of the program, have to decide how to respond.@tamedasync functions
follow suite and there is no difference between them and ordinary functions.fromtamerimporttamed,AsyncScope@tamedasyncdeffaulty_function()raiseRuntimeError("Oh no!")# ============================# Asynchronous Execution# ============================asyncdefmain():coro=faulty_function()try:awaitcoroexceptRuntimeError:print("Actually, I'm good.")asyncio.run(main())# Output# ------## Actually, I'm good.# ============================# Synchronous Execution# ============================try:faulty_function()exceptRuntimeError:print("Actually, I'm good.")# Output# ------## Actually, I'm good.# ============================# AsyncScope Execution# ============================withAsyncScope()asscope:delayed_result=faulty_function(_async_scope=scope)try:delayed_result.block()exceptRuntimeError:print("Actually, I'm good.")# Output# ------## Actually, I'm good.The one special case are functions in anAsyncScope. Here, you consume results
usingDelayedResult.valuebut handle exceptions when waiting for a result viaDelayedResult.block()or viaawait delayed_result. This is deliberate since
any code following the above statements can now assert that the result hassuccessfullyarrived.The implicit await at the end of anAsyncScopeacts as a catch-all that raises
any exceptions that you don't wait for explicitly. This ensures that no exception
is left behind and that your program doesn't produce unintended side-effects.fromtamerimporttamed,AsyncScope@tamedasyncdeffaulty_function()raiseRuntimeError("Oh no!")withAsyncScope()asscope:result=faulty_function(_async_scope=scope)# Output (excerpt)# ----------------## Traceback (most recent call last):#    [...]# RuntimeError: Oh no!Author's NoteNote: This section is quite philosophical and more about why this package
exists and less about how you use it. It's the "ramblings of an old man"
so I won't tell anyone if you choose to skip it :)If you read about python's asyncio library online you will find that people
generally approach the library with a performance mindset. However, only some
actually see its benefits materialize. This results in mixed, but often
negative sentiment about the library ranging between asyncio being "fake
parallelism", "too complicated to use", or "useful but very niche".In my mind, this sentiment is typically caused by a mix of three reasons that
lead users to believe that asyncio should be used like threads:Users think that asyncio uses thread-level parallelism under the hood and/or
is used to implement green threads.Asyncio uses thread-like semantics (see the table at the end).A lot of online documentation and tutorials point out thatasync/awaitgives massive performance boosts over thread-based webserver implementations.Writing asynchronous code while thinking about writing threads is not wrong ...
but it's incomplete and - in my opinion - missdirected. Why? Because they are
two different kinds of parallelism. Threadding takes a big block of work, slices
it into smaller chunks, and works on multiple of them in parallel across
multiple cores.Async/awaittakes a big block of work, flags periods of busy
waiting and reorders instructions to minimize idle time. In other words,
threadding uses thread-level parallelism (duh!) andasync/awaituses
instruction-level parallelism. Both are forms of parallelism but they areNOTthe same thing.ThreadsAsync/AwaitOrchestrationMain thread + inf-loopEvent LoopCreationtid = Thread(fn, ...)coro = fn(...)Synchronizationtid.join()await coroData exchangeshared memoryshared memoryOverhead~50 µs (OS level thread)None (function call)ConcurrencypreemtivecooperativeImplicit ParallelismInstead of thinking aboutasync/awaitas multi-threadded code, we should think
about it as code that splits loading and consuming of data. It is aware that
input data comes from external systems (DB, socket, filesystem, blob storage,
...), that these external systems are slow, and that programs spend most of
their time waiting for data.Asyncios (clever) trick is to realize that we can onlyexecuteinstructions
one by one (thank you GIL), but we canwaitfor any number of external systems
in parallel. Traditional (sync) code asks for a single piece of data, does
nothing while it is being prepared (busy waiting), consumes it, and then moves
on to ask for the next piece of data. Async code requestsallthe data first,busy waits for everything in parallelto arrive, and then consumes the data
one by one without waiting ever again. This reduces time spent idle fromsum(load_times)(sync) tomax(load_times)(async); a trick known as implicit
parallelism or - if you are into compilers and how CPUs work instruction-level
parallelism.Moreover, if our workload allows processing data out of order, we request all
data at the start, busy wait until the first piece arrives, process it, and then
either move to the next piece or resume busy waiting until data arrives again.
This way we spend betweenmin(load_times)andmax(load_times)busy waiting,
which can be ahugespeedup compared tosum(load_times)in the sync case.Why Create Async-TamerRealizing how the parallelism behindasync/awaitworks and moving away from a
thread-like design simplifies the code and its design. Functions that load data
becomeasync defand functions that consume the data remain as they were.
Unfortunately, it is not simple to express this with asyncio today. We can't useawaitoutside anasyncfunction and executing anasyncfunction requires
spinning up an event loop and orchestrating it.This is why I wrote async-tamer. It limits the extent to whichasyncproliferates through the codebase, i.e., it@tamedtheasynckeyword ^.^. We
need an event loop and its implicit parallelism while we are waiting for data.
We don't need an event loop when locally processing data and we get into a world
of pain if we attempt to do so. Both parts of a program should be clearly
separated andasync def(waiting for external stuff) vsdef(processing
local stuff) nicely fits that bill.The rest of the library really is syntax sugar and an attempt to remove as much
boilerplate as possible. I really like the idea of structured concurrency in
trio and it is exactly what we want when loading an "initial batch" of data.
Thus,AsyncScopeswork in a similar fashion. However, I also think thatNurseriesand (asyncio)TaskGroupslack fine-grained control and adding the
lifecycle kwargs felt like a natural extension of the idea.Thanks for reading all the way to the end and happy coding!