Package: backend.ai
Backend.AIBackend.AI is a streamlined, container-based computing cluster orchestrator
that hosts diverse programming languages and popular computing/ML frameworks,
with pluggable heterogeneous accelerator support including CUDA and ROCM.
It allocates and isolates the underlying computing resources for multi-tenant
computation sessions on-demand or in batches with customizable job schedulers.
All its functions are exposed as REST/GraphQL/WebSocket APIs.Server-side ComponentsIf you want to run a Backend.AI cluster on your own, you need to install and
configure the following server-side components.
All server-side components are licensed under LGPLv3 to promote non-proprietary open
innovation in the open-source community.There is no obligation to open your service/system codes if you just run the
server-side components as-is (e.g., just run as daemons or import the components
without modification in your codes).
Please contact us (contact-at-lablup-com) for commercial consulting and more
licensing details/options about individual use-cases.For details about server installation and configuration, please visitour
documentation.Manager with API GatewayIt routes external API requests from front-end services to individual agents.
It also monitors and scales the cluster of multiple agents (a few tens to hundreds).https://github.com/lablup/backend.ai-managerPackage namespace:ai.backend.gatewayandai.backend.managerPlugin interfacesbackendai_scheduler_v10backendai_hook_v10backendai_webapp_v10backendai_monitor_stats_v10backendai_monitor_error_v10AgentIt manages individual server instances and launches/destroys Docker containers where
REPL daemons (kernels) run.
Each agent on a new EC2 instance self-registers itself to the instance registry via
heartbeats.https://github.com/lablup/backend.ai-agentPackage namespace:ai.backend.agentPlugin interfacesbackendai_accelerator_v12backendai_monitor_stats_v10backendai_monitor_error_v10backendai_krunner_v10https://github.com/lablup/backend.ai-accelerator-cuda(CUDA accelerator plugin)Package namespace:ai.backend.acceelrator.cudahttps://github.com/lablup/backend.ai-accelerator-cuda-mock(CUDA mockup plugin)Package namespace:ai.backend.acceelrator.cudaThis emulates the presence of CUDA devices without actual CUDA devices,
so that developers can work on CUDA integration without real GPUs.https://github.com/lablup/backend.ai-accelerator-rocm(ROCM accelerator plugin)Package namespace:ai.backend.acceelrator.rocmServer-side common plugins (for both manager and agents)https://github.com/lablup/backend.ai-stats-monitorStatistics collector based on the Datadog APIPackage namespace:ai.backend.monitor.statshttps://github.com/lablup/backend.ai-error-monitorException collector based on the Sentry APIPackage namespace:ai.backend.monitor.errorKernelsA set of small ZeroMQ-based REPL daemons in various programming languages and
configurations.https://github.com/lablup/backend.ai-kernel-runnerPackage namespace:ai.backend.kernelA common interface for the agent to deal with various language runtimeshttps://github.com/lablup/backend.ai-kernelsRuntime-specific recipes to build the Docker images (Dockerfile)JailA programmable sandbox implemented using ptrace-based sytem call filtering written in
Go.https://github.com/lablup/backend.ai-jailHookA set of libc overrides for resource control and web-based interactive stdin (paired
with agents).https://github.com/lablup/backend.ai-hookCommonsA collection of utility modules commonly shared throughout Backend.AI projects.Package namespaces:ai.backend.commonhttps://github.com/lablup/backend.ai-commonClient-side ComponentsClient SDK LibrariesWe offer client SDKs in popular programming languages.
These SDKs are freely available with MIT License to ease integration with both
commercial and non-commercial software products and services.Python (provides the command-line interface)pip install backend.ai-clienthttps://github.com/lablup/backend.ai-client-pyJavaCurrently only available via GitHub releaseshttps://github.com/lablup/backend.ai-client-javaJavascriptnpm install backend.ai-clienthttps://github.com/lablup/backend.ai-client-jsPHP (under preparation)composer require lablup/backend.ai-clienthttps://github.com/lablup/backend.ai-client-phpMediaThe front-end support libraries to handle multi-media outputs (e.g., SVG plots,
animated vector graphics)The Python package (lablup) is installedinsidekernel containers.To interpret and display media generated by the Python package, you need to load
the Javascript part in the front-end.https://github.com/lablup/backend.ai-mediaInteracting with computation sessionsBackend.AI provides websocket tunneling into individual computation sessions (containers),
so that users can use their browsers and client CLI to access in-container applications directly
in a secure way.Jupyter Kernel: data scientists' favorite toolMost container sessions have intrinsic Jupyter and JupyterLab support.Web-based terminalAll container sessions have intrinsic ttyd support.SSHAll container sessions have intrinsic SSH/SFTP/SCP support with auto-generated per-user SSH keypair.
PyCharm and other IDEs can use on-demand sessions using SSH remote interpreters.VSCode (coming soon)Most container sessions have intrinsic web-based VSCode support.Integrations with IDEs and EditorsVisual Studio Code ExtensionSearch “Live Code Runner” among VSCode extensions.https://github.com/lablup/vscode-live-code-runnerAtom Editor pluginSearch “Live Code Runner” among Atom plugins.https://github.com/lablup/atom-live-code-runnerStorage managementBackend.AI provides an abstraction layer on top of existing network-based storages
(e.g., NFS/SMB), called vfolders (virtual folders).
Each vfolder works like a cloud storage that can be mounted into any computation
sessions and shared between users and user groups with differentiated privileges.LicenseRefer toLICENSE file.