Package: test_run_multinode
What is Multinode?Multinodeallows you to rapidly deploy scalable applications withasynchronous tasks.Consider using multinode if your application runs tasks that:aretriggered on demandby the user of the application;take of the order ofminutes or hoursto complete;requireexpensive hardwarethat should be provisioned only when required.For example, multinode can be used within:a document/image/video processing appa data analytics appa scientific computing appThe main benefits of multinode are:Minimal boilerplate:Cloud API calls, cloud permissions,
task lifecycle management and task data storage are abstracted away.Responsive scaling:Compute resources are spun up as soon as a task is created,
and torn down as soon as the task is complete.Quick startDeploy the multinode control plane into your AWS account. TODO: Add link to instructions once merged in.Install the multinode Python package and authenticate with the Multinode control plane.pip install multinode
multinode loginDefine the task as a Python function.# File: tasks/main.pyfrommultinodeimportMultinodemn=Multinode()@mn.function(cpu=4.0,memory="16 GiB")defrun_expensive_task(x):out=# ... details of the task ...returnoutRegister the function with the multinode control plane.multinode deploy tasks/ --project-name=my_projectImplement the rest of the application, invoking the function when needed.# File: application/main.py# NB can be a different codebase from tasks/frommultinodeimportget_deployed_functionrun_expensive_task=get_deployed_function(project_name="my_project",function_name="run_expensive_task")# ... other code ...# Start a task invocation.# The computation runs on *remote* hardware, which is *provisioned on demand*.invocation_id=run_expensive_task.start(x=10000)# ... other code ...# Get the status of the task invocation, and the result (if available)invocation=sum_of_squares.get(invocation_id)print(invocation.status)# e.g. PENDING, RUNNING, SUCCEEDEDprint(invocation.result)# 333283335000 (if available), or None (if still running)Further functionalityIn addition to the above basic functionality, Multinode allows you to:Expose progress updates from an in-flight task.Cancel a task programmatically.Implement retries in case of code errors or hardware failures.Configure timeouts and concurrency limits.Inspect task logs.Add custom Python dependencies and environment variables.Manage the lifecycle of the deployed application.For further details, see the reference guide or the worked example. TODO: Add links.Approaches to scaling: When to use Multinode?Multinode's approach: Direct resource provisioning.Multinode makesdirect API callsto the cloud provider, to provision anew workerforeach new task.Alternative approach: Autoscaling a warm worker pool.Popular alternative frameworks for asynchronous tasks include Celery and Kafka consumers.
Applications written in these frameworks usually run on a warm pool of workers.
Each worker stays alive between task executions.
The number of workers isautoscaledaccording to some metric (e.g. the number of pending tasks).Advantage of Multinode's approach:Scales upimmediatelywhen new tasks are created; scales downimmediatelywhen a task finishes.No risk of interrupting a task execution when scaling down.Advantages of the alternative warm-pool-based approach:More suitable for processing ahigher volumeofshorter-livedtasks.Can maintain spare capacity to mitigate against cold starts.ArchitectureCurrently, Multinode runs onAWS.Asynchronous tasks are run asECS tasks, usingFargateas the serverless compute engine.The control plane is deployed as anECS service, again using Fargate.Task definitions and task outputs are stored in anAuroraserverless v2 database.Task logs are stored inCloudwatch Logs.With minimal API changes, the framework can be extended to
other AWS compute engines (e.g. EC2 with GPUs), to other cloud providers, and to Kubernetes.We may implement these extensions if there is demand.
We also welcome contributions from the open source community in this regard.Currently, you need to deploy Multinode inyour ownAWS account.
(Terraform is provided in this repository. TODO: Add link.)
We may offer Multinode as a managed service in the future.Programming language supportPython is the only supported language at the moment.If you need toinvokea deployed Python function from an application written in
another language such as Javascript, then you will need to use the REST API.
(Or you can contribute a Javascript client!)Let us know if you want todefineyour functions in other languages.