Package: wait-for2
wait_for2Alternate implementation ofasyncio.wait_for(). It handles several edge cases like simultaneous
cancellation of wait and completion of future differently and consistently across Python versions 3.7+.UpdatesThis library may become unnecessary for Python 3.12+, seehttps://github.com/python/cpython/pull/28149#issuecomment-1560278644A few more behaviour variances have been introduced that are not reflected in the details below. For example
Python 3.9.10 changes the behaviour of simultaneous timeout and completion compared to previous 3.9 releases.
PyPy 3 used to mirror the 3.7 behaviour, but the current release have changed the behaviour at some unspecified
release to 3.9.10+.DetailsThe tests in the repository are set up with TOX to cover and assert the following behaviours ofwait_forand the
alternate implementation for each Python version.Cancellation behaviour with simultaneous resultBuiltinasyncio.wait_for()behaviours:Python 3.7:
Cancellation ofwait_forcould lose the completed future's result.Python 3.8+ and PyPy3:
Cancellation ofwait_forcould lose the cancellation request.Whenever waiting for a future's result the user expects to either have the future completed or cancelled.
Unfortunately due to technical details there is a chance that both will happen simultaneously. The builtin version
ofasyncio.wait_for()shipped with Python either handles one or the other only. If losing the future's result or
ignoring the cancellation is critical to the application it may not be suitable for use.Using this implementation, in case both conditions occur at the same time a subclassedCancelledErrorwill be
raised which also contains the result of the future. The caller code must catch this exception and handle the
result if it is important. Otherwise, it can be used the same way as the builtinwait_for.If the caller prefers to handle the race-condition with a callback, therace_handlerargument may be provided.
It will be called with the result of the future when the waiter task is being cancelled. Even if this is provided,
the special error will be raised in the place of a normal CancelledError.NOTE:CancelledWithResultErroris limited to the coroutinewait_foris invoked from prior to Python 3.11!
If thewait_foris wrapped in tasks those will not propagate the special exception, but raise their ownCancelledErrorinstances. The callback-based solution may be preferred as that will always work in all Python versions.This table summarizes the behaviours in the race-condition cases.
The cross cells show what behaviour is observed:LR: looses result (returned result or raised exception)LC: looses cancellation requestRH: race-condition handling supported, cancellation is never ignored by the wait-forPython 3.7Python 3.8+ and PyPy3wait_for2explicit cancel & result (or exc.)LRLCRHTimeout handling behaviour with resultsWhen the timeout is reached, the inner future is cancelled. This can also cause race condition where the result is lost.The cross cells show what behaviour is observed:TE: prioritizes raisingTimeoutError, looses result or exceptionPR: prioritizes returning or raising the result exceptionPython 3.7, 3.8Python 3.9Python 3.10+ and PyPy3wait_for2result after cancel by timeoutTETEPRPRexception after cancel by timeoutTEPRPRPRCancellation behaviour with timeout handlingAdditionally, this implementation will inherit the behaviour of the inner future when it comes to ignoring
cancellation. The builtin version prefers to always be cancellable, even if that means the wrapped future may
not be terminated with it. (behaviour of builtin _cancel_and_wait) This behaviour is also improved in
timeout-cancel edge cases, where the builtin would not wait for the termination of the inner future if the
waiter was cancelled after timeout handling had already started. This is more consistent as the inner future
must always be stopped for it to return.The leftmost column describes the case where the behaviour is tested. It shows what timeout the wait-for is called with
and when the cancellation occurs relative to it.The cross-cells show the raised result and if the inner future is terminated before the wait-for implementation returns:C/T:CancelledError/TimeoutErrorwas raisedB/U:boundmeans the inner future terminated before the wait-for returned/unboundmeans the inner future was still
running when the wait-for returnedThe cells where the desired behaviour is observed (IMO) are formatted to be bold-italic.Python 3.7Python 3.8+ and PyPy3wait_for2no timeout, cancelC BC BC Bzero timeout, cancel beforeT UC UC Bzero timeout, cancel afterT UC UC Bzero timeout, no cancelT UT BT Bsome timeout, cancel beforeC UC BC Bsome timeout, cancel afterC UC UC Bsome timeout, no cancelT BT BT BInstall & usageA package is available on PyPI:$python-mpipinstallwait_for2importasyncioimportwait_for2task=asyncio.create_task(...)defprocess_result(r,is_exc=False):print("processed:",r,is_exc)...try:process_result(awaitwait_for2.wait_for(task,5.0))exceptwait_for2.CancelledWithResultErrorase:# NOTE: e.result could be an exception raised by the task; handling or ignoring it is up to the user code hereprocess_result(e.result,e.is_exception)raiseasyncio.CancelledError()# alternatively with a callback:process_result(awaitwait_for2.wait_for(task,5.0,race_handler=process_result))