Package: concurrent_tasks
asyncio-concurrent-tasksTooling to run asyncio tasks.Background taskPeriodic taskThread safe task poolThreaded task poolRestartable taskLoop exception handlerBackground taskTask that is running in the background until cancelled.
Can be used as a context manager.Example usage:importasynciofromtypingimportCallable,Awaitablefromconcurrent_tasksimportBackgroundTaskclassHeartBeat(BackgroundTask):def__init__(self,interval:float,func:Callable[[],Awaitable]):super().__init__(self._run,interval,func)asyncdef_run(self,interval:float,func:Callable[[],Awaitable])->None:whileTrue:awaitfunc()awaitasyncio.sleep(interval)Periodic taskTask that is running periodically in the background until cancelled.
Can be used as a context manager.
There is no guarantee that the time between calls is strictly the interval if the function takes more time than the interval to execute.Example usage:fromtypingimportCallable,Awaitablefromconcurrent_tasksimportPeriodicTaskclassHeartBeat(PeriodicTask):def__init__(self,interval:float,func:Callable[[],Awaitable]):super().__init__(interval,func)Thread safe task poolThe goal is to be able to safely run tasks from other threads.Parameters:sizecan be a positive integer to limit the number of tasks concurrently running.timeoutcan be set to define a maximum running time for each time after which it will be cancelled.
Note: this excludes time spent waiting to be started (time spent in the buffer).Example usage:fromconcurrent_tasksimportThreadSafeTaskPoolasyncdeffunc():...asyncwithThreadSafeTaskPool()aspool:# Create and run the task.result=awaitpool.run(func())# Create a task, the `concurrent.Future` will hold information about completion.future=pool.create_task(func())Threaded task poolRun async tasks in a dedicated thread. It will have its own event loop.
Under the hook,ThreadSafeTaskPoolis used.Parameters:namewill be used as the thread's name.sizeandtimeoutseeThreadSafeTaskPool.context_managercan be optional context managers that will be entered when the loop has started
and exited before the loop is stopped.💡 All tasks will be completed when the pool is stopped.💡 Blocking and async version are the same, prefer the async version if client code is async.Loop initialization⚠️ Asyncio primitives need to be instantiated with the proper event loop.To achieve that, use a context manager wrapping instantiation of objects:fromfunctoolsimportpartialfromconcurrent_tasksimportThreadedPoolContextManagerWrapper,AsyncThreadedTaskPoolpool=AsyncThreadedTaskPool(context_manager=ThreadedPoolContextManagerWrapper(partial(MyObj,...)))BlockingThis can be used to run async functions in a dedicated event loop, while keeping it running to handle background tasksExample usage:fromconcurrent_tasksimportBlockingThreadedTaskPoolasyncdeffunc():...withBlockingThreadedTaskPool()aspool:# Create and run the task.result=pool.run(func())# Create a task, the `concurrent.Future` will hold information about completion.future=pool.create_task(func())AsyncThreads can be useful in cooperation with asyncio to let the OS guarantee fair resource distribution between threads.
This is especially useful in case you cannot know if called code will properly cooperate with the event loop.Example usage:fromconcurrent_tasksimportAsyncThreadedTaskPoolasyncdeffunc():...asyncwithAsyncThreadedTaskPool()aspool:# Create and run the task.result=awaitpool.run(func())# Create a task, the asyncio.Future will hold information about completion.future=pool.create_task(func())Restartable taskTask that can be started and cancelled multiple times until it can finally be completed.
This is useful to handle pauses and retries when handling with a connection.💡 Usefunctools.partialto pass parameters to the function.Example usage:fromfunctoolsimportpartialfromconcurrent_tasksimportRestartableTaskasyncdefsend(data):...task:RestartableTask[int]=RestartableTask(partial(send,b"\x00"),timeout=1)task.start()assertawaittask==1# Running in other tasks:# On connection lost:task.cancel()# On connection resumed:task.start()# On response received:task.set_result(1)Loop exception handlerShut down process when an unhandled exception is caught or a signal is received.
To make this a graceful stop, pass astop_func.When creating multiple background tasks, exceptions raised within those will be forwarded directly to the event loop.
In order to act on those exceptions, we need to useloop.set_exception_handler.💡 When a signal is received and the process is already shutting down, it will be force killed.Example minimalistic implementation:importasynciofromconcurrent_tasksimportLoopExceptionHandlerasyncdefrun():event=asyncio.Event()tasks=[]asyncdef_stop():awaitasyncio.gather(*tasks)event.set()asyncwithLoopExceptionHandler(_stop):# Adding a bunch of tasks here...awaitevent.wait()