Package: tractor
tractor: next-gen Python parallelismtractoris astructured concurrent,multi-processingruntime
built ontrio.Fundamentallytractorgives you parallelism viatrio-”actors”:
ournurserieslet you spawn new Python processes which each run atrioscheduled runtime - a call totrio.run().We believe the system adheres to the3 axiomsof an “actor model”
but likelydoes notlook like whatyouprobably think an “actor
model” looks like, and that’sintentional.The first step to groktractoris to get the basics oftriodown.
A great place to start is thetrio docsand thisblog post.FeaturesIt’s justatrioAPIInfinitely nesteableprocess treesBuiltin IPC streaming APIs with task fan-out broadcastingA (first ever?) “native” multi-core debugger UX for Python usingpdb++Support for a swappable, OS specific, process spawning layerA modular transport stack, allowing for custom serialization (eg. withmsgspec), communications protocols, and environment specific IPC
primitivesSupport for spawning process-level-SC, inter-loop one-to-one-task orientedasyncioactors via “infectedasyncio” modestructured chadcurrencyfrom the ground upRun a func in a processUsetrio’s style of focussing ontasks as functions:"""
Run with a process monitor from a terminal using::

    $TERM -e watch -n 0.1  "pstree -a $$" \
        & python examples/parallelism/single_func.py \
        && kill $!

"""importosimporttractorimporttrioasyncdefburn_cpu():pid=os.getpid()# burn a core @ ~ 50kHzfor_inrange(50000):awaittrio.sleep(1/50000/50)returnos.getpid()asyncdefmain():asyncwithtractor.open_nursery()asn:portal=awaitn.run_in_actor(burn_cpu)#  burn rubber in the parent tooawaitburn_cpu()# wait on result from target functionpid=awaitportal.result()# end of nursery blockprint(f"Collected subproc{pid}")if__name__=='__main__':trio.run(main)This runsburn_cpu()in a new process and reaps it on completion
of the nursery block.If you only need to run a sync function and retreive a single result, you
might want to check outtrio-parallel.Zombie safe: self-destruct a process treetractortries to protect you from zombies, no matter what."""
Run with a process monitor from a terminal using::

    $TERM -e watch -n 0.1  "pstree -a $$" \
        & python examples/parallelism/we_are_processes.py \
        && kill $!

"""frommultiprocessingimportcpu_countimportosimporttractorimporttrioasyncdeftarget():print(f"Yo, i'm '{tractor.current_actor().name}' "f"running in pid{os.getpid()}")awaittrio.sleep_forever()asyncdefmain():asyncwithtractor.open_nursery()asn:foriinrange(cpu_count()):awaitn.run_in_actor(target,name=f'worker_{i}')print('This process tree will self-destruct in 1 sec...')awaittrio.sleep(1)# raise an error in root actor/process and trigger# reaping of all minionsraiseException('Self Destructed')if__name__=='__main__':try:trio.run(main)exceptException:print('Zombies Contained')If you can create zombie child processes (without using a system signal)
itis a bug.“Native” multi-process debuggingUsing the magic ofpdb++and our internal IPC, we’ve
been able to create a native feeling debugging experience for
any (sub-)process in yourtractortree.fromosimportgetpidimporttractorimporttrioasyncdefbreakpoint_forever():"Indefinitely re-enter debugger in child actor."whileTrue:yield'yo'awaittractor.breakpoint()asyncdefname_error():"Raise a ``NameError``"getattr(doggypants)asyncdefmain():"""Test breakpoint in a streaming actor.
    """asyncwithtractor.open_nursery(debug_mode=True,loglevel='error',)asn:p0=awaitn.start_actor('bp_forever',enable_modules=[__name__])p1=awaitn.start_actor('name_error',enable_modules=[__name__])# retreive resultsstream=awaitp0.run(breakpoint_forever)awaitp1.run(name_error)if__name__=='__main__':trio.run(main)You can run this with:>>> python examples/debugging/multi_daemon_subactors.pyAnd, yes, there’s a built-in crash handling mode B)We’re hoping to add a respawn-from-repl system soon!SC compatible bi-directional streamingYes, you saw it here first; we provide 2-way streams
with reliable, transitive setup/teardown semantics.Our nascent api is remniscent oftrio.Nursery.start()style invocation:importtrioimporttractor@tractor.contextasyncdefsimple_rpc(ctx:tractor.Context,data:int,)->None:'''Test a small ping-pong 2-way streaming server.

    '''# signal to parent that we're up much like# ``trio_typing.TaskStatus.started()``awaitctx.started(data+1)asyncwithctx.open_stream()asstream:count=0asyncformsginstream:assertmsg=='ping'awaitstream.send('pong')count+=1else:assertcount==10asyncdefmain()->None:asyncwithtractor.open_nursery()asn:portal=awaitn.start_actor('rpc_server',enable_modules=[__name__],)# XXX: this syntax requires py3.9asyncwith(portal.open_context(simple_rpc,data=10,)as(ctx,sent),ctx.open_stream()asstream,):assertsent==11count=0# receive msgs using async for styleawaitstream.send('ping')asyncformsginstream:assertmsg=='pong'awaitstream.send('ping')count+=1ifcount>=9:break# explicitly teardown the daemon-actorawaitportal.cancel_actor()if__name__=='__main__':trio.run(main)See original proposal and discussion in#53as well
as follow up improvements in#223that we’d love to
hear your thoughts on!Worker poolz are easy peasyThe initial ask from most new users is“how do I make a worker
pool thing?”.tractoris built to handle any SC (structured concurrent) process
tree you can imagine; a “worker pool” pattern is a trivial special
case.We have afull worker pool re-implementationof the std-lib’sconcurrent.futures.ProcessPoolExecutorexample for reference.You can run it like so (from this dir) to see the process tree in
real time:$TERM -e watch -n 0.1  "pstree -a $$" \
    & python examples/parallelism/concurrent_actors_primes.py \
    && kill $!This uses no extra threads, fancy semaphores or futures; all we need
istractor’s IPC!“Infectedasyncio” modeHave a bunch ofasynciocode you want to force to be SC at the process level?Check out our experimental system forguest-modecontrolledasyncioactors:importasynciofromstatisticsimportmeanimporttimeimporttrioimporttractorasyncdefaio_echo_server(to_trio:trio.MemorySendChannel,from_trio:asyncio.Queue,)->None:# a first message must be sent **from** this ``asyncio``# task or the ``trio`` side will never unblock from# ``tractor.to_asyncio.open_channel_from():``to_trio.send_nowait('start')# XXX: this uses an ``from_trio: asyncio.Queue`` currently but we# should probably offer something better.whileTrue:# echo the msg backto_trio.send_nowait(awaitfrom_trio.get())awaitasyncio.sleep(0)@tractor.contextasyncdeftrio_to_aio_echo_server(ctx:tractor.Context,):# this will block until the ``asyncio`` task sends a "first"# message.asyncwithtractor.to_asyncio.open_channel_from(aio_echo_server,)as(first,chan):assertfirst=='start'awaitctx.started(first)asyncwithctx.open_stream()asstream:asyncformsginstream:awaitchan.send(msg)out=awaitchan.receive()# echo back to parent actor-taskawaitstream.send(out)asyncdefmain():asyncwithtractor.open_nursery()asn:p=awaitn.start_actor('aio_server',enable_modules=[__name__],infect_asyncio=True,)asyncwithp.open_context(trio_to_aio_echo_server,)as(ctx,first):assertfirst=='start'count=0asyncwithctx.open_stream()asstream:delays=[]send=time.time()awaitstream.send(count)asyncformsginstream:recv=time.time()delays.append(recv-send)assertmsg==countcount+=1send=time.time()awaitstream.send(count)ifcount>=1e3:breakprint(f'mean round trip rate (Hz):{1/mean(delays)}')awaitp.cancel_actor()if__name__=='__main__':trio.run(main)Yes, we spawn a python process, runasyncio, starttrioon theasyncioloop, then send commands to thetrioscheduled tasks to
tellasynciotasks what to do XDWe need help refining theasyncio-side channel API to be moretrio-like. Feel free to sling your opinion in#273!Higher level “cluster” APIsTo be extra terse thetractordevs have started hacking some “higher
level” APIs for managing actor trees/clusters. These interfaces should
generally be condsidered provisional for now but we encourage you to try
them and provide feedback. Here’s a new API that let’s you quickly
spawn a flat cluster:importtrioimporttractorasyncdefsleepy_jane():uid=tractor.current_actor().uidprint(f'Yo i am actor{uid}')awaittrio.sleep_forever()asyncdefmain():'''
    Spawn a flat actor cluster, with one process per
    detected core.

    '''portal_map:dict[str,tractor.Portal]results:dict[str,str]# look at this hip new syntax!asyncwith(tractor.open_actor_cluster(modules=[__name__])asportal_map,trio.open_nursery()asn,):for(name,portal)inportal_map.items():n.start_soon(portal.run,sleepy_jane)awaittrio.sleep(0.5)# kill the cluster with a cancelraiseKeyboardInterruptif__name__=='__main__':try:trio.run(main)exceptKeyboardInterrupt:passInstallFrom PyPi:pip install tractorFrom git:pip install git+git://github.com/goodboy/tractor.gitUnder the hoodtractoris an attempt to pairtrionicstructured concurrencywith
distributed Python. You can think of it as atrio-across-processesor simply as an opinionated replacement for the
stdlib’smultiprocessingbut built on async programming primitives
from the ground up.Don’t be scared off by this description.tractoris justtriobut with nurseries for process management and cancel-able streaming IPC.
If you understand how to work withtrio,tractorwill give you
the parallelism you may have been needing.Wait, huh?! I thought “actors” have messages, and mailboxes and stuff?!Let’s stop and ask how many canon actor model papers have you actually read ;)From our experience many “actor systems” aren’t really “actor models”
since theydon’t adhereto the3 axiomsand pay even less
attention to the problem ofunbounded non-determinism(which was the
whole point for creation of the model in the first place).From the author’s mouth,the only thing requiredisadherance tothe3 axioms,and that’s it.tractoradheres to said base requirements of an “actor model”:In response to a message, an actor may:

- send a finite number of new messages
- create a finite number of new actors
- designate a new behavior to process subsequent messagesandrequiresno further api changesto accomplish this.If you want do debate this further please feel free to chime in on our
chat or discuss on one of the following issuesafter you’ve read
everything in them:https://github.com/goodboy/tractor/issues/210https://github.com/goodboy/tractor/issues/18Let’s clarify our parlanceWhether or nottractorhas “actors” underneath should be mostly
irrelevant to users other then for referring to the interactions of our
primary runtime primitives: each Python process +trio.run()+ surrounding IPC machinery. These are our high level, baseruntime-units-of-abstractionwhich bothare(as much as they can
be in Python) and will be referred to as our“actors”.The main goal oftractoris is to allow for highly distributed
software that, through the adherence tostructured concurrency,
results in systems which fail in predictable, recoverable and maybe even
understandable ways; being an “actor model” is just one way to describe
properties of the system.What’s on the TODO:Help us push toward the future of distributedPython.Erlang-style supervisors via composed context managers (see#22)Typed messaging protocols (ex. viamsgspec.Struct, see#36)Typed capability-based (dialog) protocols ( see#196with draft work
started in#311)Feel like saying hi?This project is very much coupled to the ongoing development oftrio(i.e.tractorgets most of its ideas from that brilliant
community). If you want to help, have suggestions or just want to
say hi, please feel free to reach us in ourmatrix channel.  If
matrix seems too hip, we’re also mostly all in the thetrio gitter
channel!