Virtual machine system supporting a large number of displays

A method and system for supporting multiple displays in a virtual machine (VM) environment are disclosed. The system includes a client device coupled to a server hosting one or more VMs. The client device may include multiple displays anda client display manager coupled to the displays. The client display manager may be operable to establish display connections with virtual devices of a VM, receive data generated by the VM via the display connections, and forward the data to a display controller of a relevant display.

TECHNICAL FIELD

The embodiments of the invention relate generally to virtual machine systems and, more specifically, relate to a virtual machine system supporting a large number of displays.

BACKGROUND

In computer science, a virtual machine (VM) is a portion of software that, when executed on appropriate hardware, creates an environment allowing the virtualization of an actual physical computer system. Each VM may function as a self-contained platform, running its own operating system (guest OS) and software applications (processes). Typically, a virtual machine manager (VMM) manages allocation and virtualization of computer resources and performs context switching, as may be necessary, to cycle between various VMs.

A host machine (e.g., computer or server) is typically enabled to simultaneously run multiple VMs, where each VM may be used by a remote client. The host machine allocates a certain amount of the host's resources to each of the VMs. Each VM is then able to use the allocated resources to execute applications, including guest operating systems. The VMM virtualizes the underlying hardware of the host machine or emulates hardware devices, making the use of the VM transparent to the guest operating system or the remote client that uses the VM.

A VM may include one or more applications that generate data (e.g., images, video, etc.) to be displayed at a physical display device of the host machine. Usually, a display connection is established with the VM to receive this data and to send user input provided at the display device back to the VM. If the host machine has multiple display devices that render data generated by the VM, the display connection with the VM becomes heavily burdened, negatively impacting the end user experience.

Further, a VM may run in a remote server environment, in which a centralized server is partitioned into multiple VMs that may, for example, host virtual desktops, thereby providing a desktop for an end user of a remote client device. A remote client may have multiple displays to render data generated by the VM. However, currently there is no mechanism that can effectively display the output of a VM on multiple screens or display terminals of a remote client.

DETAILED DESCRIPTION

A method and system for supporting multiple displays in a virtual machine (VM) environment are disclosed. In one embodiment, a client is coupled to a server hosting one or more VMs. The client has a large number of displays that can render data generated by the VM. Each VM has multiple virtual devices associated with the displays of the client. In one embodiment, the client establishes display connections with virtual devices of the VM using at least one communication channel for each virtual device of the VM. Upon receiving data generated by the VM via a display connection, the client determines which display should present the data generated by the VM, and forwards the data to a display controller of the determined display. In one embodiment, separate channels are provided to communicate different types of data between the virtual device and the client (e.g., video data, cursor data, three-dimensional data, etc.).

By associating virtual devices of the VM with individual displays of the client and by using multiple channels to connect the virtual devices of the VM to the client, data generated by the VM is efficiently provided to multiple monitors of the client, and the overall end user experience is significantly improved. The above mechanism may also be applicable to a local host environment, in which the host includes a large number of displays that can render data generated by a VM running on the host.

In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.

FIG. 1is a block diagram of a virtual machine system100for supporting multiple displays in a remote host environment according to one embodiment of the invention. System100includes one or more clients108communicatively coupled to a host machine or a cluster of host machines101over a network105. Network105may be a local area network (LAN) or a wide area network (WAN) and may be a combination of one or more networks. Client108may be any computer system in communication with host machine101for remote execution of applications at host machine101.

The host machine101is configured to host one or more VMs102, each having a guest operating system104. Multiple guest operating systems and the associated VMs may be controlled by another operating system, known as a host OS or a virtual machine monitor (VMM) for managing the hosted VMs. The VMM may also be referred to as a hypervisor or a kernel-based virtual machine (KVM).

A guest OS104may be of the same or different type with respect to the host OS. For example, a guest OS may be a Windows™ operating system from Microsoft and a host OS may be a Linux operating system available from Red Hat. In addition, the guest OSes104can be of the same or different types.

VM102can be any type of virtual machines, such as, for example, hardware emulation, full virtualization, para-virtualization, and operating system-level virtualization virtual machines. Different VMs102hosted by server101may have the same or different privilege levels for accessing different resources.

The host101may be coupled to a host controller (not shown) via a network or directly. The host controller can manage the VMs102by, for example, adding a VM, deleting a VM, balancing the load on the server cluster, providing directory service to the VMs102, and perform other management functions.

Each VM102may include one or more applications107hosted by corresponding guest OSes104. Each VM102can communicate to one or more clients108. In one embodiment, each virtual machine102hosts or maintains a desktop environment providing virtual desktops for remote clients108. In some embodiments, client108may be a thin client with sufficient resources to enable a user to interact with a virtual desktop provided by VM102. A virtual desktop can represent an output (e.g., an image to be displayed) generated by a desktop application107running within VM102. Graphics data associated with the virtual desktop can be captured and transmitted to client108(e.g., in the form of commands), where the virtual desktop may be rendered and presented by a remote display application112. The remote display application108may be a thin client application such as a browser application. In one embodiment, the host101communicates with remote display application112using a remote access protocol (e.g., Remote Desktop Protocol (RDP), Simple Protocol for Independent Computing Environments (SPICE™ from Red Hat, etc.) that allows for display connection between the host101and the client.

Each client108is communicatively coupled to multiple displays110that present data rendered by the remote display application112. Each display110can be a display monitor, a window on a monitor screen, or any other entity that can receive graphic input and/or graphic commands. Each display110is associated with a display controller (not shown) that controls presentation of data on a relevant display110. In response to client side data, an application (e.g., application107) can generate output display commands (e.g., graphics commands), which may include one or more paint and/or draw operations, for example, in the form of executable instructions. The output display commands can then be transmitted (e.g., as graphics update commands) to the client108, and the remote display application112can collect the graphics commands and generate corresponding drawing commands for rendering data on the client display devices110.

Each VM102may have multiple virtual devices103that emulate physical devices such as display cards (e.g., PCI display cards). For example, a virtual device103may emulate graphic verbs such as drawing lines, ellipses, fill areas, display images, etc. In addition, the virtual devices104may emulate 3D verbs, such as drawing cubes, rotating a shape, etc. and may also provide video streaming verbs. In some embodiments, the virtual devices103use the local host hardware to accelerate some functions (e.g., local rendering).

As will be discussed in more detail below, each virtual device103can be connected with the client using a separate network connection or a shared network connection. In one embodiment, a virtual device103can be connected to multiple clients108and can transmit the output to the multiple clients108simultaneously, although not necessarily equally.

In one embodiment, the remote display application112of client108includes a client display manager109that forwards data to, and receives response from, display controllers of the client displays110. The client display manager109is responsible for establishing connections with virtual devices103that provide data to be rendered at the display devices110. In one embodiment, the client display manager109establishes a connection with a virtual device103using the IP address and port of the virtual device103. In one embodiment, each connection between the virtual device103and the client display manager109is provided via at least one dedicated channel (e.g., a socket). Alternatively, the virtual device103may share the network connection with other virtual devices103. Such a connection can be enabled using, for example, virtual channels (e.g., by adding a channel ID to transferred data).

In one embodiment, the host101includes a remote display system (RDS)106for each VM102. The RDS106may be part of the VMM or a hardware emulation layer, or run on top of the VMM. The RDS106oversees network connections of the VM102and assists the client display manager109in establishing display connections for the virtual devices103of the VM102. In particular, in one embodiment, the client display manager109establishes an initial connection with the host101and requests information about the available display channels that the client display manager109can connect to. The client display manager109then uses this information to establish display connections with the virtual devices103of the VM102. During the active display connection, the RDS106receives data (e.g., graphics commands) from the virtual devices103of the VM102, and communicates this data to the client display manager109. In one embodiment, the RDS106uses a shared connection to send graphics commands of multiple virtual devices103. Alternatively, the RDS106uses separate connections for different virtual devices103. In addition, in some embodiments, the RDS106uses different connections for different types of data (different types of graphics commands such as video, mouse, 3D data, etc.), where each type of data can be transmitted using a dedicated connection or a shared connection. In some embodiments, the RDS106also performs additional processing (e.g., compression, encryption, streaming, etc.) of the data received from the virtual devices103prior to communicating it to the client display manager109.

The client display manager109receives data (e.g., graphics commands) of the virtual devices103, and determines which display110should present this data. In one embodiment, in which the data is received via a dedicated channel (e.g., a socket), the client display manager109makes this determination based on the context of the dedicated channel (e.g., using the mapping between the socket ID and the display ID). In another embodiment, in which the data is received via a shared connection (e.g., using a virtual channel with a channel ID), the client display manager109makes this determination based on the identifier of the display device that is included in the received data (e.g., in the header of the graphics command issued by the virtual device103).

FIG. 2is a block diagram of a virtual machine system200for supporting multiple displays in a local host environment according to one embodiment of the invention. System200includes multiple displays201communicatively coupled to a host computer202. The displays201may include display monitors, windows on a monitor screen, or any other entity that can receive graphic input and/or graphic commands. Each display201is associated with a display controller (not shown) that controls presentation of data on a relevant display201.

Computer202hosts one or more VMs203, each having one or more applications207hosted by a guest OS205. The guest operating systems and the associated VMs are controlled by a host OS or a VMM. Each VM203has multiple virtual devices204that emulate physical devices such as display cards, similarly to virtual devices103ofFIG. 1.

In one embodiment, the host202includes display modules214associated with individual VMs203. The display modules214may be part of the VMM or a hardware emulation layer, plugins for the VMM or the hardware emulation layer, or independent modules running on top of the VMM. In another embodiment, one display module214is provided to operate with all VMs203of the host202.

The display module214establishes a display connection with each virtual device204. of the VM203In one embodiment, each display connection is dedicated to a specific virtual device204. Alternatively, a display connection can be shared by multiple the virtual devices204. Such a connection can be established using, for example, virtual channels (e.g., by adding a channel ID to the transferred data).

During the active display connection with the VM203, the display module214receives data (e.g., graphics commands) from the virtual devices204, and determines which display device201should present this data. In one embodiment, in which the data is received via a dedicated channel, the display module214makes this determination based on the context of the dedicated channel. In another embodiment, in which the data is received via a shared connection (e.g., using a virtual channel with a channel ID), the display module214makes this determination based on the identifier of the display device that is included in the received data.

FIG. 3is a block diagram illustrating exemplary display connections of a VM in a remote host system300according to an embodiment of the invention. System300includes a host302and a remote client320coupled to the host302via a network. The host302includes one or more VMs304. The client320has multiple displays324through330connected to the client display manager322. Displays324through330may include display monitors, windows on a monitor screen, and the like.

Each virtual device has at least one communication channel with the client display manager322. In the shown embodiment, each virtual device has multiple channels for communicating data of different types such as video data, mouse data, 3D data, etc. This separation of data into different data streams may enable different QoS levels and features for each stream. For example, the mouse data may be prioritized to make the machine more interactive, the keyboard data may be encrypted, the video data may be compressed, and the 3D data may be transmitted unchanged. Alternatively, more or less channels can be used for each virtual device.

A channel can be implemented using a dedicated connection (e.g., a socket) or a shared connection (e.g., a virtual channel with an additional channel ID field). In one embodiment, a virtual device can have channel connections with multiple clients320, and can transmit output to all the clients simultaneously, although not necessarily equally.

During the active display connection with the VM304, the client display manager322receives data (e.g., graphics commands) generated by the virtual devices, and determines which display should present this data. In one embodiment, in which the data is received via a dedicated channel (e.g., a socket), the client display manager322makes this determination based on the context of the dedicated channel (e.g., using the mapping between the socket ID and the display ID). In another embodiment, in which the data is received via a shared connection, the client display manager322makes this determination based on the identifier of the display device that is included in the received data (e.g., in the header of the graphics command sent by the virtual device).

The system300may operate in a wide area network (WAN) environment, as well as a local area network (LAN) environment. On LANs, where greater bandwidth is available, as compared to WANs, the host CPU resources are used relatively sparingly, while the bandwidth resources are used more freely. As such, data generated by each virtual device may not require processing by the guest OS, and can instead be transmitted directly to the matching display to handle the presentation of the content. In this way, relatively heavy or resource intensive content may be rapidly and effectively handled by multiple displays.

FIG. 4is a block diagram illustrating display connections of a virtual machine in a local host system400according to an embodiment of the invention. System400includes a host402running one or more VMs404. The host402has multiple displays424through430connected to a display module422. Displays424through430may include display monitors, windows on the screen of a display monitor, and the like.

Each virtual device has at least one communication channel with the display module422that provides functionality similar to that of client display manager322ofFIG. 3, except that it operates locally (e.g., using an API) when establishing display connections of the VM and handling data communicated during active display connections, instead of performing networking operations. In the shown embodiment, each virtual device has multiple channels for communicating data of different types such as video data, mouse data, 3D data, etc. Alternatively, a single channel can be used for each virtual device.

During the active display connection with the VM404, the display module422receives data (e.g., graphics commands) from the virtual devices, and determines which display should present this data. In one embodiment, in which the data is received via a dedicated channel, the display module422makes this determination based on the context of the dedicated channel. In another embodiment, in which the data is received via a shared connection, the display module422makes this determination based on the identifier of the display device that is included in the received data.

FIG. 5Ais a flow diagram illustrating a client-side method500for supporting multiple displays in a remote host environment, in accordance with one embodiment of the invention. Note that method500may be performed by processing logic which may include software, hardware, or a combination of both. For example, method500may be performed by system100ofFIG. 1.

Method500begins at block502where a client establishes display connections for multiple virtual devices of the VM. The client may establish at least one dedicated channel connection for a virtual device of the VM (e.g., via a socket). Alternatively, the client may establish a shared connection for multiple virtual devices (e.g., using virtual channels). In one embodiment, each display connection can be split into separate channels to communicate different types of data. In one embodiment, prior to establishing display connections, the client establishes an initial connection and uses it to obtain information about the channels that the client can connect to, including the available display channels associated with the virtual devices of the VM. The client then uses this information to establish display connections with the VM.

At block504, the client receives data (e.g., a graphics command) generated by the VM on the connection. At block506, the client determines which display(s) should present the data from this connection. As discussed above, this determination can be made based on the context of the connection or using a display ID that may be received as part of the command header. At block508, the client forwards the received command to a display controller of the appropriate display(s). At block510, the display controller executes the command on the display.

FIG. 5Bis a flow diagram illustrating a server-side method550for supporting multiple displays in a remote host environment, in accordance with one embodiment of the invention. Note that method550may be performed by processing logic which may include software, hardware, or a combination of both. For example, method550may be performed by system100ofFIG. 1.

Method550begins at block552where RDS receives a client request for information about available channels. In one embodiment, the RDS receives the client request via an initial (main) channel connection established by the client with the host. In addition to the main channel, various other types of channels can be available for the client to connect to. Some exemplary channel connections may include a display connection, a cursor connection, and inputs connection, a record connection, a playback connection, etc.

At block554, RDS sends information about available channels to the client, including information about display channels associated with multiple virtual devices of the VM. Each display channel may have a unique channel ID. In some embodiments, RDS subsequently sends notifications about added or removed channels to the client in real time.

At block556, RDS receives the client's request to establish display connections with the virtual devices of the VM for the client displays, and creates mapping between the virtual devices and the client displays based on the client request that identifies the displays.

Subsequently, during the active display connection, RDS sends data generated by the virtual devices to the client via the corresponding channel connection(s). In one embodiment, the RDS uses a shared connection to send graphics commands of multiple virtual devices. Alternatively, the RDS uses separate connections for different virtual devices. In addition, in some embodiments, the RDS uses different connections for different types of data (different types of graphics commands such as video, mouse, 3D data, etc.), where each type of data can be transmitted using a dedicated connection or a shared connection.

When receiving data back from the client, the RDS determines an appropriate virtual device based on the mapping between the virtual devices and the client displays, forward the data to the appropriate virtual device.

FIG. 6is a flow diagram illustrating a method600for supporting multiple displays in a local host environment, in accordance with one embodiment of the invention. Note that method600may be performed by processing logic which may include software, hardware, or a combination of both. For example, method600may be performed by system200ofFIG. 2.

Method600begins at block602where a display module of the host establishes display connections with multiple virtual devices of the VM. The display module may establish at least one dedicated channel connection for a virtual device of the VM. Alternatively, the display module may establish a shared connection for multiple virtual devices of the VM (e.g., using virtual channels). In one embodiment, each display connection can be split into separate channels to communicate different types of data.

At block604, the display module receives data (e.g., a graphics command) generated by the VM on the connection. At block606, the display module determines which display(s) should present data from this connection. As discussed above, this determination can be made based on the context of the connection or using a display ID that may be part of the received data. At block608, the display module forwards the received command to the display controller(s) of the appropriate display(s). The display controller(s) then execute the command on the display(s).

The computer system700may further include a network interface device708. The computer system700also may include a video display unit710(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device712(e.g., a keyboard), a cursor control device714(e.g., a mouse), and a signal generation device716(e.g., a speaker).

The data storage device718may include a machine-accessible storage medium728on which is stored one or more set of instructions (e.g., software722) embodying any one or more of the methodologies of functions described herein. The software722may also reside, completely or at least partially, within the main memory704and/or within the processing device702during execution thereof by the computer system700; the main memory704and the processing device702also constituting machine-accessible storage media. The software722may further be transmitted or received over a network720via the network interface device708.

The machine-readable storage medium728may also be used to store instructions to configure display devices in a VM system as described in embodiments of the invention and implemented by system100ofFIG. 1or system200ofFIG. 2, and/or a software library containing methods that call the above applications. While the machine-accessible storage medium728is shown in an exemplary embodiment to be a single medium, the term “machine-accessible storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-accessible storage medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instruction for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present invention. The term “machine-accessible storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media.