System and method for distributed virtualization of GPUs in desktop cloud

Embodiments are provided to enable graphic processing unit (GPU) virtualization for high bandwidth or rate demanding applications, such as 3D gaming, where a client communicates with a host via a virtual desktop infrastructure (VDI). The distributed GPU virtualization allows one or more VMs or comparable hosts or components without GPU access to communicate with a GPU at a different component or physical machine in a data center or a network using remote direct memory access (RDMA). A first physical machine that excludes a GPU starts a remote display driver function to handle a request to render graphics from a client via gateway. A second physical machine that comprises a GPU is instructed to start a render function for the client using the GPU. The render function communicates with the remote display driver function at the first physical machine. The rendered graphics is then sent to the client via the gateway.

TECHNICAL FIELD

The present invention relates to graphics processing, and, in particular embodiments, to a system and method for distributed virtualization of graphic processing units (GPUs) in desktop cloud environment.

BACKGROUND

In applications where graphics processing for a user or client is implemented remotely, such as on a virtual machine (VM) via a remote virtual desktop infrastructure (VDI), image/video data (e.g., for 3D graphics) can be rendered into a suitable format for display using a graphics processing unit (GPU) on a remote physical server. The rendered data is then remotely displayed at the client device. OpenGL is a graphics library that depends on GPU. Programmers use OpenGL application programming interface (API) to access the library to write graphics intensive programs such as Computer-Aided Design (CAD) or games. For some programs, such as games, the programs simulate real world objects with computer generated graphics.

OpenGL supports hardware based rendering for 3D graphics using a GPU, where hardware vendors can provide drivers for GPU rendering. However, some current physical machines with VMs may not comprise GPUs due to cost and possibly other resource limitations. In some scenarios, no or a limited number of VMs in a data center or network may directly access and use a GPU. In such cases, there is a need for a mechanism that enables the VMs without GPU access to access a GPU with sufficient performance to keep up with the fast data rate requirements of relatively high rate applications such as 3D gaming, or any other high data rate demanding remote desktop based applications that require 3D graphics rendering such as 3D CAD tools.

SUMMARY OF THE INVENTION

In accordance with an embodiment, a method for supporting distributed virtualization of graphic processing in a remote desktop environment includes detecting, at a management and control component, a request at a gateway from a client to render graphics for remote display at the client, and instructing a first physical machine that excludes a graphic processing unit (GPU) to start a remote display driver function to handle the request, wherein the first physical machine is assigned as a host for the client. A second physical machine that comprises a GPU is also instructed to start a render function for rendering graphics for the client using the GPU. The render function communicates with the remote display driver function at the first physical machine.

In accordance with another embodiment, a network component for supporting distributed virtualization of graphic processing in a remote desktop environment a processor and a computer readable storage medium storing programming for execution by the processor. The programming includes instructions to detect a request at a gateway from a client to render graphics for remote display at the client. The network component instructs a first physical machine that excludes a GPU to start a remote display driver function to handle the request, wherein the first physical machine is assigned as a host for the client. The network component further instructs a second physical machine that comprises a GPU to start a render function for rendering graphics for the client using the GPU, wherein the render function communicates with the remote display driver function at the first physical machine.

In accordance with another embodiment, a method for supporting distributed virtualization of graphic processing in a remote desktop environment includes receiving, at a gateway, a request from a client to render graphics for remote display at the client. The gateway then sends the request to a first physical machine that excludes a GPU to start a remote display driver function to handle the request, wherein the first physical machine is assigned as a host for the client. The gateway also receives from a second physical machine that comprises a GPU graphics data associated with the request for remote display at the client. The graphics data is rendered using the GPU at the second physical machine. The gateway then forwards the graphics data to the client.

In accordance with another embodiment, a network component for supporting distributed virtualization of graphic processing in a remote desktop environment a processor and a computer readable storage medium storing programming for execution by the processor. The programming includes instructions to receive a request from a client to render graphics for remote display at the client, and send the request to a first physical machine that excludes a GPU to start a remote display driver function to handle the request, wherein the first physical machine is assigned as a host for the client. The network component then receives, from a second physical machine that comprises a GPU, graphics data associated with the request for remote display at the client, wherein the graphics data is rendered using the GPU at the second physical machine. The received graphics data is then forwarded from the network component to the client.

In accordance with another embodiment, a method for supporting distributed virtualization of graphic processing in a remote desktop environment includes receiving, at a first physical machine that excludes a GPU, a request from a remote client via a gateway to render graphics for remote display at the client. The first physical machine is assigned as a host for the client. The method further includes starting at the first physical machine a remote display driver function to handle the request and sending graphics data associated with the request to a second physical machine that comprises a GPU to render the graphics data using the GPU.

In accordance with another embodiment, a network component for supporting distributed virtualization of graphic processing in a remote desktop environment includes a processor and a computer readable storage medium storing programming for execution by the processor. The programming includes instructions to receive a request from a remote client via a gateway to render graphics for remote display at the client. The network component is assigned as a host for the client, and the processor is a non-GPU processor unsuitable for rendering the graphics data. The network component hence starts, at the first physical machine, a remote display driver function to handle the request and sends graphics data associated with the request to a second network component that comprises GPU to render the graphics data using the GPU.

In accordance with another embodiment, a method for supporting distributed virtualization of graphic processing in a remote desktop environment includes receiving, at a physical machine comprising a GPU, instructions to start a render function for rendering graphics for a client using the GPU. The method further includes receiving, from a remote display driver at a host physical machine that excludes a GPU, graphics data for rendering, wherein the host physical machine is assigned to handle remote display of the graphics at the client. The graphics data is rendered using the render function and the GPU and then sent to a gateway that forwards the graphics data to the client.

In accordance with yet another embodiment, a network component for supporting distributed virtualization of graphic processing in a remote desktop environment includes at least one processor including a GPU and a computer readable storage medium storing programming for execution by the at least one processor. The programming including instructions to receive instructions to start a render function for rendering graphics for a client using the GPU, and receive, from a remote display driver at a host physical machine that excludes a GPU, graphics data for rendering. The host physical machine is assigned to handle remote display of the graphics at the client. The network component then renders the graphics data using the render function and the GPU. The rendered graphics data is sent to a gateway that forwards the graphics data to the client.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

A virtualized GPU allows desktop clouds to support high end graphics, such as for OpenGL and Direct3D (D3D) standards. Typical available GPU virtualization systems require using a GPU in the host (e.g., a server or hypervisor at a data center). The host may communicate with a user or a client via a desktop application at the client device and using the cloud, e.g., the Internet and/or any other suitable network(s). Some applications also require having a GPU at the host. For example, a 3D video/image processing virtual machine requires access to a GPU. Upgrading an existing host system to support a GPU can be challenging, such as in terms of cost and design. For example, in many host systems, hardware does not support GPU at all due to card size, energy consumption, or other practical issues.

Emerging technologies, such as remote direct memory access (RDMA) can be used to optimize communication performance inside a data center network. This may be suitable for GPU virtualization applications that use relatively high bandwidth. For instance, GPU virtualization systems for 3D gaming may need about 100 megabits/sec (mbts) of compressed data versus about 500 mbts of uncompressed data. This bandwidth is suitable for a distributed GPU system in a data center network. Embodiments are provided herein to enable a GPU virtualization solution for such high bandwidth/rate demanding applications, for instance for 3D gaming or other 3D imaging systems. The GPU virtualization solution uses distributed virtualization of GPUs in a desktop/cloud environment, e.g., where a client communicates with a host via a remote desktop or VDI and access via the cloud. The provided distributed GPU virtualization allows one or more VMs (or comparable hosts/components) without GPU access to communicate with a GPU at a different component or physical machine in a data center or a network using a RDMA based protocol and a gateway to the client, as described below. The embodiments herein can be implemented for virtualizing 3D gaming applications, other imaging applications, or any suitable application in a desktop/cloud based graphics rendering environment that may benefit from improved (fast) communication performance of the system.

FIG. 1shows an embodiment architecture100for distributed virtualization of GPUs in a desktop/cloud environment. The architecture100comprises one or more first hosts or physical machines110, one or more second hosts or physical machines120, a gateway130, a client140, and a management system150. These components may communicate or may be coupled to each other as shown inFIG. 1or in any other suitable arrangement that serves similar functions or purpose below. For example, the components may be located in the same data center or in one or more connected networks. The components may be separated into different physical components (as shown inFIG. 1). However, in other embodiments, at least some of the components may be combined into the same physical component, e.g., a server, a workstation, or any suitable network node (e.g., router or switch).

A first physical machine110may be any suitable physical component or unit (e.g., server, hypervisor, or network card) that does not comprise a GPU or has no direct GPU access. The first physical machine110comprises components or function blocks that may be implemented via software and/or hardware, including at least one virtual machine (VM)112for handling client requests from the client140(via the gateway130), and a remote display driver114for processing data for display at the client. The first physical machine110or its components are configured to implement and run OpenGL116and/or D3D118for desktop/cloud based services, or other similar resource demanding services and applications.

A second physical machine120may be any suitable physical component or unit (e.g., server, hypervisor, or network card) that comprises a GPU or has a direct GPU access. The second physical machine120comprises a plurality of component blocks that may be implemented via hardware, software, or both, including a dispatcher122for initiating one or more render servers124(e.g., an application via software). Each render server124comprises an image compression block123and a render125for handling rendering requests from the remote display driver114. The second physical machine120also includes a GPU128and a native display driver126that handles communications between the render125and the GPU128.

The dispatcher122at a second physical machine122, for instance when triggered by the management system150, may establish more than one render server124to handle more than one request from one or more clients140. At each render server124, the render125forwards the graphics data for the client140from the remote display driver114to the native display driver126for graphics rendering. The image compression block123then receives the rendered graphics from the GPU128via the native display driver126(with or without the render125), implements standard compression format (e.g., MPEG) with or without additional compression to meet the data rate or bandwidth requirement and hence high quality user experience in terms of speed, and forwards the compressed data to the gateway130.

The gateway130may be any suitable physical or logical (software) component or unit that mitigates and handles communications, including graphics rendering requests and returned compressed rendered graphics data, between the client140and each of the first physical machine110and the second physical machine120. The gateway130may be an independent physical component from the first physical machine110and the second physical machine120. In some embodiments, the gateway130may be allocated on the same physical component with one or more other components, for example with a first physical machine110, a second physical machine120, or the management system150.

The client140may be any device, e.g., a computer or communication device such as a desktop computer, a laptop computer, a computer tablet, or a smartphone. The client140communicates with the first physical machine110(the host) via the gateway130to process a 3D (or 2D) graphics video/image application, for example to display 3D video for a game or 3D images for a CAD tool on the device, via remote desktop or VDI through the cloud or Internet. Other applications that use the remote desktop or VDI and have similar bandwidth or data rate requirements may also use the architecture100.

The management system150may be any suitable physical or logical (software) component configured to control the operations of some of the other components/blocks, including the gateway130, the remote display driver114, and the dispatcher122. Upon detecting a request for rendering graphics from the client140, the management system150configures the gateway130and starts the dispatcher122and the remote display driver114to handle the request.

In an embodiment, when the VDI client140sends a request to the VDI host, the gateway130intercepts the request and forwards it to the VM112at a first physical machine110. The VDI management system150can also detect the request at the gateway130and initiate the remote display driver114. The VM112forwards the needed request information to the remote display driver114to render graphics for display. In turn, the remote display driver114communicates using RDMA with the render125at a second physical machine120. The RDMA permits the first physical machine110to use the GPU resources at the second physical machine120and bypass CPU processing at the first physical machine110. The RDMA communication also allows sending relatively large data at relatively high speed between the physical machines, for example within a data center. This high speed is suitable and sufficient to meet the data rate requirement, e.g., for 3D graphics rendering in 3D gaming, and achieve high quality user experience. The integration of the gateway130in the architecture100for distributed virtualization removes the need to send data back to the VM112or the first physical machine110before sending the data to the client140, which reduces delay. This also reduces the complexity and safety risks introduced by having otherwise multiple connections to the client140. Having a single connection between the gateway130and the client140(over the cloud) reduces safety risks (e.g., malicious attacks) since all other connections between the components can be sufficiently secured in a data center. The connection between the gateway130and the client140may be a transmission control protocol (TCP) connection. Alternatively, a more secure user datagram protocol (UDP) connection can be used between the gateway130and the client140.

FIGS. 2A and 2Bshow an embodiment method200for a protocol exchange between components of the architecture100. After a VDI connection is established successfully, e.g., via the cloud between the client140and the host or gateway130, the management system150detects the connection or request from the client140and, at step201, configures the gateway130using suitable integration parameters. The integration parameters may indicate addresses of the client140and the corresponding first host or physical machine110, and bind the two to a session associated with the request of the client140, e.g., using a session ID. The session ID is bound to the two entities and sent by the management system150to the gateway130. This enables the gateway130to identify the later received rendered data from the render server124as intended for the client140. The received data also indicates the same session, e.g., includes the session ID. Thus, the gateway130can match the data to the session of the client140(by matching the session ID in the rendered data), and hence forward the data properly to the client140. This configuration step eliminates having to resend the rendered data from the second physical machines120back to the host or first physical machine110and subsequently to the gateway130. Instead the gateway130sends back the rendered graphics data directly from the second physical machines120to the client140, which reduces burden on the VM, avoids further delay, and improves user experience. The step201can be performed using a handshake protocol or exchange between the management system150and the gateway130. At step202, the management system150sends the dispatcher122a command to ready a render or a render server124to connect to the remote display driver114. At step203, the dispatcher122sends a command to start the render server124. At step204, the management system150sends a notification to the remote display driver114to connect to the render server124that is ready. At step205, the remote display driver114gets ready to work and handle the request. This includes switching the display driver to the remote display driver mode (to connect to the render server124at a different physical machine120with GPU128).

At step206, an interface platform or operating system160(e.g., Windows™), e.g., at the client140or the first physical machine110, connects to the remote display driver114. At step207, the remote display driver114sends a command to the render server124to render graphics data. At step208, the render server124translates the command for rendering. At step209, the render server124initiates a command to start rendering (start the render125) using the native display driver126and the GPU128. At step210, the render server124captures the remote desktop screen or image (the result from the render125). At step211, the render server124compresses the screen image (e.g., in H.264 format) using the image compression block123. At step212, the render server124sends the screen image stream to the gateway130. At step213, other parts of the VDI protocol170may send VDI messages regarding the request or its rendered graphics to gateway130. At step214, the gateway130integrates all the received data to one connection or session of the client130. At step215, the gateway130sends the compressed rendered graphics for display to the client140. In other embodiments, some of the steps above may be omitted, combined, or implemented in a different order in any other suitable manner that serves the same outcome or purpose. Additional steps or commands may also be added to this protocol.

FIG. 3is a block diagram of a processing system300that can be used to implement various embodiments. The processing system300may be part of or correspond to the first host or physical machine110that lacks a GPU. Specific devices may utilize all of the components shown, or only a subset of the components and levels of integration may vary from device to device. Furthermore, a device may contain multiple instances of a component, such as multiple processing units, processors, memories, transmitters, receivers, etc. The processing system300may comprise a processing unit301equipped with one or more input/output devices, such as a network interfaces, storage interfaces, and the like. The processing unit301may include a central processing unit (CPU)310, a memory320, a mass storage device330, and an I/O interface360connected to a bus. The bus may be one or more of any type of several bus architectures including a memory bus or memory controller, a peripheral bus or the like.

The CPU310may comprise any type of electronic data processor. The memory320may comprise any type of system memory such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), a combination thereof, or the like. In an embodiment, the memory320may include ROM for use at boot-up, and DRAM for program and data storage for use while executing programs. In embodiments, the memory320is non-transitory. The mass storage device330may comprise any type of storage device configured to store data, programs, and other information and to make the data, programs, and other information accessible via the bus. The mass storage device330may comprise, for example, one or more of a solid state drive, hard disk drive, a magnetic disk drive, an optical disk drive, or the like.

The processing unit301also includes one or more network interfaces350, which may comprise wired links, such as an Ethernet cable or the like, and/or wireless links to access nodes or one or more networks380. The network interface350allows the processing unit301to communicate with remote units via the networks380. For example, the network interface350may provide wireless communication via one or more transmitters/transmit antennas and one or more receivers/receive antennas. In an embodiment, the processing unit301is coupled to a local-area network or a wide-area network for data processing and communications with remote devices, such as other processing units, the Internet, remote storage facilities, or the like.

FIG. 4is a block diagram of another processing system400that can be used to implement various embodiments. The processing system400may be part of or correspond to the second hosts or physical machines120including a GPU490. The GPU490is a processing unit configured or optimized for handling graphics data including rendering of 3D graphics into a suitable format for display. Other component of the processing system400include a CPU410, e.g., for more general data processing, a memory420, a mass storage device430, a network interface450, and I/O interface460to access nodes or one or more networks480. Such components may be similar to the corresponding components above of the processing system300.