Package: scaled
ScaledThis project is aiming the target that provides very light, efficient, reliable way for distribute computing
framework, like dask and ray, it uses centralized scheduler (like dask, unlike ray) and language agnostic protocol
between client and scheduler, or between scheduler and worker.IntroductionThe goal for this project should be as simple as possibleIt built on top of zmqit has ready python version of Client, Scheduler, WorkerI will provide golang or Rust version of Scheduler, the goal for the Scheduler should be completely computer language
agnostic, which means they follow the same protocolScheduler might support function based computing tree in the futureInstallationpip install scaledif you want to use uvloop, please do:pip install uvloop, default we are using python builtin uvloopHow to use itThe use experience is very close to daskStart local scheduler and cluster at the same time in the codeimportrandomfromscaledimportClientfromscaledimportSchedulerClusterCombodefcalculate(sec:int):returnsec*1defmain():address="tcp://127.0.0.1:2345"cluster=SchedulerClusterCombo(address=address,n_workers=10,event_loop="uvloop")client=Client(address=address)tasks=[random.randint(0,100)for_inrange(100000)]futures=[client.submit(calculate,i)foriintasks]results=[future.result()forfutureinfutures]assertresults==tasksclient.disconnect()cluster.shutdown()if__name__=="__main__":main()Start scheduler and cluster independentlyusescaled_schedulerto start scheduler, for example:$scaled_schedulertcp://0.0.0.0:8516[INFO]2023-03-1912:16:10-0400:loggingto('/dev/stdout',)[INFO]2023-03-1912:16:10-0400:useeventloop:2[INFO]2023-03-1912:16:10-0400:Scheduler:monitoraddressisipc:///tmp/0.0.0.0_8516_monitor[INFO]2023-03-1912:16:10-0400:AsyncBinder:started[INFO]2023-03-1912:16:10-0400:VanillaTaskManager:started[INFO]2023-03-1912:16:10-0400:VanillaFunctionManager:started[INFO]2023-03-1912:16:10-0400:VanillaWorkerManager:started[INFO]2023-03-1912:16:10-0400:StatusReporter:startedusescaled_clusterto start 10 workers:$scaled_worker-n10tcp://127.0.0.1:8516[INFO]2023-03-1912:19:19-0400:loggingto('/dev/stdout',)[INFO]2023-03-1912:19:19-0400:ClusterProcess:starting23workers,heartbeat_interval_seconds=2,function_retention_seconds=3600[INFO]2023-03-1912:19:19-0400:Worker[0]started[INFO]2023-03-1912:19:19-0400:Worker[1]started[INFO]2023-03-1912:19:19-0400:Worker[2]started[INFO]2023-03-1912:19:19-0400:Worker[3]started[INFO]2023-03-1912:19:19-0400:Worker[4]started[INFO]2023-03-1912:19:19-0400:Worker[5]started[INFO]2023-03-1912:19:19-0400:Worker[6]started[INFO]2023-03-1912:19:19-0400:Worker[7]started[INFO]2023-03-1912:19:19-0400:Worker[8]started[INFO]2023-03-1912:19:19-0400:Worker[9]startedfor detail options of above 2 program, please use argument-hto check out all available optionsThen you can write simply write client code as:fromscaledimportClientdeffoobar(foo:int):returnfooclient=Client(address="tcp://127.0.0.1:2345")future=client.submit(foobar,1)print(future.result())SchedulerAs scaled_scheduler only need communicate with client and worker with protocol, so scaled_scheduler can be packaged and
distributed independently, unless protocol changes. It can be used for scheduling tasks that by other language
implementations of client/worker too.Assume there is implementation of scaled protocol written in C++, has client and worker side, it can use python version
of scaled_scheduler for task scheduling and balancingScheduler BenchmarkInitially I was thinking to implement C++ or Rust version of scheduler that follows the protocol, but seems PyPy works
pretty well, the scheduling overhead will be reduced by ~60%. Below is a simple benchmark chart, I used 100k noop tasks
to just test how much overhead does scheduler will introduceNote: as dask took 110 seconds to finish same amount of 100k noop tasks, I will not list it to below chartPythonVersionEvent LoopTimeCPython3.10asyncio19.131sCPython3.10uvloop17.809sCPython3.11.3asyncio19.475sCPython3.11.3uvloop18.406sPyPy3.9.16asyncio8.748sPyPy3.9.16uvloopN/ATest machine is AMD 5900X with 32gb RAM, started 10 workersPackage scheduler as one fileHere is how to use nuitka to compile package into one file (doesn't provide speedup):nuitka3--standalone--onefile--no-pyi-file--remove-output--include-package=uvloop--output-filename=scaled_scheduler./run_scheduler.py
nuitka3--standalone--onefile--no-pyi-file--remove-output--output-filename=scaled_top./run_top.pyHere is how to use pex to package into one file:pexscaled-cscaled_scheduler-oscaled_scheduler
pexscaled-cscaled_top-oscaled_topGraph TaskScaled also supports submit graph task, for example:fromscaledimportClientdefinc(i):returni+1defadd(a,b):returna+bdefminus(a,b):returna-bgraph={"a":2,"b":2,"c":(inc,"a"),# c = a + 1 = 2 + 1 = 3"d":(add,"a","b"),# d = a + b = 2 + 2 = 4"e":(minus,"d","c")# e = d - c = 4 - 3 = 1}client=Client(address="tcp://127.0.0.1:2345")print(client.get(graph,keys=["e"]))Scaled TopYou can usescaled_topto connect to scheduler monitor address to get some insides of the scaled_top$scaled_topipc:///tmp/0.0.0.0_8516_monitorWhich will something similar to top command, but it's for getting status of the scaled system:scheduler|task_manager|scheduler_sent|scheduler_receivedcpu0.0%|unassigned0|HeartbeatEcho1,593|Heartbeat1,593rss35.4m|running0|FunctionResponse12|FunctionRequest12|success10,000|TaskEcho10,000|Task10,000|failed0|Task10,000|TaskResult10,000|canceled0|TaskResult10,000|DisconnectRequest13|not_found0|DisconnectResponse13|----------------------------------------------------------------------------------------------
Shortcuts:worker[n]agt_cpu[C]agt_rss[M]cpu[c]rss[m]free[f]sent[w]queued[d]lag[l]Total10worker(s)workeragt_cpuagt_rss[cpu]rssfreesentqueuedlag|client_manager281379|desk-manjaro|e852+0.0%30.1m0.5%29.8m1000000.1ms|281386|desk-manjaro|f6fc+0.0%32.1m0.5%30.1m1000000.1ms|func_to_num_tasks281383|desk-manjaro|692a+0.0%30.1m0.0%29.9m1000000.2ms|281381|desk-manjaro|f415+0.0%30.0m0.0%30.0m1000000.1ms|281382|desk-manjaro|2d87+0.0%30.0m0.0%29.9m1000000.2ms|281387|desk-manjaro|9831+0.0%30.1m0.0%30.0m1000000.2ms|281380|desk-manjaro|9f8f+0.0%30.0m0.0%29.9m1000000.2ms|281378|desk-manjaro|1f49+0.0%30.0m0.0%29.8m1000000.1ms|281384|desk-manjaro|17b8+0.0%30.1m0.0%32.0m1000000.1ms|281385|desk-manjaro|5270+0.0%30.0m0.0%30.2m1000000.2ms|scheduler section is showing how much resources scheduler usedtask_manager section shows count for each task statusscheduler_sent section shows count for each type of messages scheduler sentscheduler_received section shows count for each type of messages scheduler receivedfunction_id_to_tasks section shows task count for each function usedworker section shows worker details, you can use shortcuts to sort by columns, the char * on column header show which
column is sorted right nowagt_cpu/agt_rss means cpu/memory usage of worker agentcpu/rss means cpu/memory usage of workerfree means number of free task slots for this workersent means how many tasks scheduler sent to the workerqueued means how many tasks worker received and queuedlag means the latency between worker and scheduler (round trip divided by 2), ms means microsecond