Package: scarletio
scarletioAsynchronous blackmagic & WitchcraftTable of contentsIntroductionCoroutinesRunning coroutinesTasksTask groupsTask cancellationTask suspensionTimeoutsRunning in threadsScheduling from other threadsLocksEventsIntroductionScarletio is a coroutine based concurrent Python library using modernasync / awaitsyntax.
Originally inspired by asyncio.One of the core concepts of the library is that the event loops should not intercept with synchronous code execution.
When an event loop is started it will not block the control flow, instead it provides you various synchronization
tools to start new asynchronous procedures and to retrieve their results cross environment.You can experiment with scarletio in the REPL:$ python3 -m scarletio
                     _      _   _
                    | |    | | (_)
  ___  ___ __ _ _ __| | ___| |_ _  ___
 / __|/ __/ _` | '__| |/ _ \ __| |/ _ \
 \__ \ (_| (_| | |  | |  __/ |_| | (_) |
 |___/\___\__,_|_|  |_|\___|\__|_|\___/

                                  1.0.56


Scarletio interactive console 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0] on linux.
Use "await" directly.
Type "help", "copyright", "credits" or "license" for more information.
In [0]:Notea great deal of Scarletio features only works on Linux[↑ Back to top ↑]CoroutinesCoroutinesare a special type of function that can be
suspended and resumed, allowing other code to run in between.
Coroutines enable developers to write asynchronous code in a more sequential and readable manner.In traditional programming functions are called, executed, and completed before control is returned to the caller.
However, coroutines differ in that they can be suspended in the middle of their execution, allowing the program to
switch to another task.
When a coroutine is suspended, it yields control back to the event loop, allowing other code to be executed.A coroutine function is declared by prefixing a function definition with theasynckeyword, like this:In[0]:asyncdefmain():...:print('hello')...:awaitsleep(1.0)...:print('world')...:This functions prints out "hello", waits 1 second and then prints "world".Inside a coroutine function, you can use theawaitkeyword to wait for the result of another coroutine or an
asynchronous operation.
When an await statement is encountered, the coroutine suspends its execution until the awaited task is complete,
allowing other coroutines to run in the meantime.Coroutines are scheduled and executed within an event loop, which is responsible for managing their execution and
switching between them.Note that simply calling a coroutine function will not schedule it to be executed:In[1]:main()<coroutineobjectmainat0x7fbb2615d340>To run the coroutine we have toawaitit:In[2]:awaitmain()helloworld[↑ Back to top ↑]Running coroutinesScarletio repl provides a native way of usingawait, but traditionallyawaitcan only be used inside of coroutine
functions.fromscarletioimportsleepasyncdefmain():print('hello')awaitsleep(1.0)print('world')awaitmain()# Produces:##  File "file.py", line 8#    await main()#    ^#SyntaxError: 'await' outside functionWe can usescarletio.runfunction to run our entry point, the "main" function.fromscarletioimportrun,sleepasyncdefmain():print('hello')awaitsleep(1.0)print('world')run(main())# Produces:## hello# worldFor more control over our application we want to access our event loop directly, since that is handling the scheduling
and the execution of our tasks. To get our event loop we will use thescarletio.get_or_create_event_loopfunction.fromscarletioimportget_or_create_event_loop,sleepasyncdefmain():print('hello')awaitsleep(1.0)print('world')LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# worldWhilescarletio.runhandles loop detection, creation and stopping as required,loop.runwill not stop the event
loop after our coroutine finishes.[↑ Back to top ↑]TasksTasks are used to schedule coroutines concurrently. To schedule up a task we wrap our coroutine with a function
just asloop.create_task. It will return our task and schedule up the coroutine.fromscarletioimportget_or_create_event_loop,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)asyncdefmain():# We schedule up `say_after`task=LOOP.create_task(say_after('hello',1))# Lets wait for our task's completion.awaittaskLOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## helloTo run coroutines concurrently we just have to create more tasks.fromscarletioimportget_or_create_event_loop,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)asyncdefmain():# We schedule up `say_after`. One finishes after 1 and the other after 2 seconds.task_0=LOOP.create_task(say_after('hello',1))task_1=LOOP.create_task(say_after('world',2))# Lets await for out tasks' completion.awaittask_0awaittask_1LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# world[↑ Back to top ↑]Task groupsTask groups allow you to manage and coordinate a collection of tasks.
They provide convenient way to work with multiple tasks concurrently and track their progress and results.Task groups provide methods to create and add individual tasks to the group. They are particularly useful in scenarios
where you have a set of related tasks that can be executed concurrently and need to be managed collectively.
They make it easier to handle complex workflows and improve the readability of asynchronous code.fromscarletioimportTaskGroup,get_or_create_event_loop,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)# We schedule up `say_after`.task_group.create_task(say_after('hello',1))task_group.create_task(say_after('world',2))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# world[↑ Back to top ↑]Task cancellationTasks can easily and safely be cancelled. When a task is cancelled ascarletio.CancelledErrorwill be raised into
task at the next opportunity.fromscarletioimportTaskGroup,get_or_create_event_loop,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)# We schedule up `say_after`. By cancelling `hello` it will not print.task_group.create_task(say_after('hello',1)).cancel()task_group.create_task(say_after('world',2))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## world[↑ Back to top ↑]Task suspensionTasks can be suspended using eithersleep,skip_poll_cycleorskip_ready_cycle.sleepsuspends the task for the given amount of seconds.fromtimeimportstrftimefromscarletioimportget_or_create_event_loop,sleepasyncdefmain():# Print out the current time every 4 seconds.whileTrue:print(strftime('%X'))awaitsleep(4)LOOP=get_or_create_event_loop()try:LOOP.run(main())finally:LOOP.stop()sleepreturns aFuture, so they can becancelledor simply used inside a task group too.fromscarletioimportTaskGroup,get_or_create_event_loop,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)task_group.create_task(say_after('hello',1))task_group.add_future(sleep(2))# Lets await for out tasks' completion.awaittask_group.wait_all()print('world')LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# worldskip_ready_cycleskips every scheduled and ready to run tasks. This can be used to synchronise between other tasks,
or to wait for other scheduled callbacks to finish before we continue our tasks' execution.This is particularly useful in event driven programming when we know our event handlers will be run, but we do not
know in what order.fromscarletioimportTaskGroup,get_or_create_event_loop,skip_ready_cycleasyncdefsay_first(to_say):print(to_say)asyncdefsay_second(to_say):awaitskip_ready_cycle()print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)# `say_second` will always print after `say_first` if scheduled concurrently.task_group.create_task(say_second('world'))task_group.create_task(say_first('hello'))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# worldskip_poll_cycleis a more extreme version ofskip_ready_cycle, because it schedules the task back after the next
io polling.fromscarletioimportTaskGroup,get_or_create_event_loop,skip_poll_cycle,skip_ready_cycleasyncdefskip_10_times_then_say(to_say):for_inrange(10):awaitskip_ready_cycle()print(to_say)asyncdefskip_io_poll_then_say(to_say):awaitskip_poll_cycle()print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)task_group.create_task(skip_io_poll_then_say('world'))task_group.create_task(skip_10_times_then_say('hello'))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# worldSince tasks are usually scheduled afteriooperations, Scarletio will always prefer to finish all already
scheduled and ready to run tasks before again polling from io.[↑ Back to top ↑]TimeoutsTimeouts can be applied toFuture-s andTask-s using theirapply_timeoutmethod. If timeout occurs theTaskis cancelled and aTimeoutErroris propagated.fromscarletioimportget_or_create_event_loop,sleepasyncdefwait_forever():awaitsleep(3600)asyncdefmain():task=LOOP.create_task(wait_forever())task.apply_timeout(1.0)try:awaittaskexceptTimeoutError:print('TIMEOUT!')LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## TIMEOUT!We can userepeat_timeoutwhen executing a loop where timeout should be applied on each cycle.fromscarletioimportget_or_create_event_loop,repeat_timeout,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)asyncdefmain():after=2try:withrepeat_timeout(5)asloop:for_inloop:# The timeout is reapplied with every iteration.awaitsay_after('hi',after)after+=2exceptTimeoutError:print('TIMEOUT!')LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hi# hi# TIMEOUT![↑ Back to top ↑]Running in threadsWhile tasks run inside an event loop, it is possible to move their execution into a separate thread or,
to be more accurate, into an executor.
It can be useful when running IO-bound functions that would otherwise block the event loop.fromthreadingimportcurrent_threadfromtimeimportsleepasblocking_sleepfromscarletioimportenter_executor,get_or_create_event_loopasyncdefmain():print(f'before entering:{current_thread().ident}')asyncwithenter_executor():print(f'after entering:{current_thread().ident}')blocking_sleep(2)print(f'before exiting:{current_thread().ident}')print(f'after exiting:{current_thread().ident}')LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces: (example)## before entering: 140664167724800# after entering: 140664159332096# before exiting: 140664159332096# after exiting: 140664167724800[↑ Back to top ↑]Scheduling from other threadsWe can create tasks from other threads by using thecreate_task_thread_safemethod.
If we want to retrieve their result we usetask.sync_wrap().wait().fromscarletioimportget_or_create_event_loop,sleepasyncdefsay_after(to_say,after):awaitsleep(after)print(to_say)LOOP=get_or_create_event_loop()# Create our tasks from a different threadtask_0=LOOP.create_task_thread_safe(say_after('hello',1))task_1=LOOP.create_task_thread_safe(say_after('world',2))# Wait for their execution to finish.task_0.sync_wrap().wait()task_1.sync_wrap().wait()LOOP.stop()It is also possible to wait for tasks' results from other event loop usingawait task.async_wrap(loop).[↑ Back to top ↑]LocksScarletioLockcan be used to guarantee exclusive access to a shared resource.
Should be used withasync withstatement.The example will:Printhelloafter 1 second.Printworldafter 2 seconds.Printhello worldafter 4 seconds.fromscarletioimportLock,TaskGroup,get_or_create_event_loop,sleepasyncdefsay_after(to_say,after,lock):asyncwithlock:awaitsleep(after)print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)lock=Lock(LOOP)task_group.create_task(say_after('hello',1,lock))task_group.create_task(say_after('world',1,lock))task_group.create_task(say_after('hello world',2,lock))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()A scarletioScarletLockcan be used to guarantee access to a shared resourcenamount of times.The example will:Printhelloandworldafter 1 second.Printhello worldafter 3 seconds.fromscarletioimportScarletLock,TaskGroup,get_or_create_event_loop,sleepasyncdefsay_after(to_say,after,lock):asyncwithlock:awaitsleep(after)print(to_say)asyncdefmain():task_group=TaskGroup(LOOP)lock=ScarletLock(LOOP,2)task_group.create_task(say_after('hello',1,lock))task_group.create_task(say_after('world',1,lock))task_group.create_task(say_after('hello world',2,lock))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()[↑ Back to top ↑]EventsA scarletioeventcan be used to notify multiple tasks that some event has happened.fromscarletioimportEvent,TaskGroup,get_or_create_event_loop,sleepasyncdefset_event_after(event,after):awaitsleep(after)event.set()asyncdefsay_when_set(to_say,event):awaiteventprint(to_say)asyncdefmain():task_group=TaskGroup(LOOP)event=Event(LOOP)task_group.create_task(set_event_after(event,2))task_group.create_task(say_when_set('hello',event))task_group.create_task(say_when_set('world',event))# Lets await for out tasks' completion.awaittask_group.wait_all()LOOP=get_or_create_event_loop()LOOP.run(main())LOOP.stop()# Produces:## hello# world[↑ Back to top ↑]