The Live Migration of AI Computing Power
Published: 2024-01-05 08:59:11

In the time of AI, many companies have purchased a large number of computing devices, especially GPUs. But to utilize GPUs needs strong code compilation and tuning capabilities, which obviously not possessed by everyone. Thus, GPU pooling is the most suitable way for companies, all the GPU computing power in a pool, allocating it on demand, and adjusting it dynamically.

This is exactly what OrionX is designed for. OrionX is an AI computing power resource pooling software, developed by VirtAI Tech company. Many people believes that GPU is the minimizing computing unit, which functions to accelerate the computing speed. We often hear a project using hundreds of GPU a large model using thousands of GPUs to achieve acceleration and other news. It seems that GPU acceleration has been a commonplace. OrionX can create virtualized software-defined resource pools on the traditional IaaS hardware infrastructure architecture, so that GPU resources in the system can flow up to achieve flexible deployment to meet the needs of different times and different businesses.

In other words, OrionX enables multi-dimensional sharing of GPU acceleration resources, supporting the provision of GPUs from multiple servers to a single VM or container, as well as multiple GPU resource pools for the same VM or container. This maximizes GPU resource utilization, and also maximizes the training speed of large model applications such as AIGC.

The live migration of AI computing power is the biggest advantage of OrionX and one of the features needed by real business scenarios. Although GPU pooling with virtualization can achieve elastic scaling of cloud services, the success rate of elastic scaling is on the low side when cluster resources are limited or the fragmentation rate is high. The Al computing power live migration provided by OrionX can migrate the running computing tasks and data on the computing platform from one physical computing device to another, so that dynamic management and optimization of computing power can be achieved without affecting business continuity.

In high-performance computing, AI training and other scenarios, some computing resources may be overloaded, while some are in idle state due to workload imbalance. That’s a common problem encountered by many users. With live migration technology, overloaded AI computing resources can be migrated to idle resources, thus dynamic load balancing of resources can be achieved and overall system utilization and performance can be improved.

For the long-time running AI applications, the irregularity of the allocation and release of computing resources may lead to the fragmentation of computing resources and reduced utilization efficiency of computing power. With live migration technology, dynamic organization and optimization of arithmetic resources can be achieved to reintegrate fragmented arithmetic resources, improve arithmetic utilization efficiency, reduce the impact of computing fragmentation on performance, and thus improve the performance and stability of applications.

AI computing resource live migration technology can help to realize equipment operation and maintenance. In the process of server cluster operation, sometimes it is necessary to perform maintenance, upgrade or offline operation on some servers. At this time it is possible to migrate the workloads originally running on the servers that need maintenance to other normally running servers by using live migration. Thus, while realizing the offline maintenance operation of the target server, the stable operation and service quality of the overall system have been not affected.

With OrionX, the original idle GPU computing power can be integrated and reused maximally, and load balancing and on-demand distribution can be achieved. So, not only the operational efficiency is greatly improved, but also more projects can be carried on.