Virtual machine graphic resource usage

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for calculating graphics resources for a virtual machine. One of the methods includes determining resources available on a graphics card device included in a computer executing a plurality of virtual machines, each virtual machine configured to execute a virtual desktop; determining, based on data received from a hypervisor that manages execution of at least one of the plurality of virtual machines, a graphics profile for a virtual machine included in the plurality of virtual machines executing on the computer; determining a portion of the available resources on the graphics card device allocated to the virtual machine using the graphics profile; and computing an amount of resources on the graphics card device consumed by a virtual desktop of the virtual machine based on the portion of the available resources on the graphics card device allocated to the virtual machine.

RELATED APPLICATION

Benefit is claimed under 35 U.S.C. 9(a)-(d) to Foreign Application Serial No. 201641038044 filed in India entitled “VIRTUAL MACHINE GRAPHICS RESOURCE USAGE”, filed on Nov. 7, 2016, by VMware, Inc., which is herein incorporated in its entirety by reference for all purposes.

BACKGROUND

This specification relates to virtual machines.

A virtual machine is a software-based abstraction of a physical computer system. In general, any computer program that can be executed on a physical computer system can be executed in a virtual machine using virtualization software. Virtualization software is software that is logically interposed and interfaces with a virtual machine and a physical computer system. Each virtual machine is configured to execute an operating system (OS), referred to herein as a guest OS, and applications. A physical computer system, referred to herein as a host machine, can execute one or more virtual machines. A host system may include one or more host machines, e.g., servers.

A virtual machine can be accessed remotely through a network connection. For example, it is possible to use a remote client to access a virtual machine remotely. A remote client is a program that communicates user interface information with a remote system. Generally, the user interface information includes display data, which is received from the remote system and displayed locally to the user on a client device, and the remote client sends inputs generated by the user to the remote system. In this way, applications executing remotely from the client device can be accessed and interacted with by the user.

SUMMARY

In general, one innovative aspect of the subject matter described in this specification can be embodied in methods that include the actions of determining resources available on a graphics card device included in a computer executing a plurality of virtual machines, each virtual machine configured to execute a virtual desktop; determining, based on data received from a hypervisor that manages execution of at least one of the plurality of virtual machines, a graphics profile for a virtual machine included in the plurality of virtual machines executing on the computer; determining a portion of the available resources on the graphics card device allocated to the virtual machine using the graphics profile; and computing an amount of resources on the graphics card device consumed by a virtual desktop of the virtual machine based on the portion of the available resources on the graphics card device allocated to the virtual machine. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

The foregoing and other embodiments can each optionally include one or more of the following features, alone or in combination. The method may include determining, using the portion of the available resources on the graphics card device allocated to the virtual machine and the amount of resources on the graphics card device consumed by the virtual desktop of the virtual machine, whether to change the graphics profile for the virtual machine to a second graphics profile that indicates a different portion of resources on a second graphics card device that should be allocated to the virtual machine; and sending, to the hypervisor, second data for the second graphics profile to cause an execution of the virtual machine using the different portion of resources on the second graphics card device. The graphics card device may be the same model as the second graphics card device. Computing the amount of resources on the graphics card device consumed by the virtual desktop of the virtual machine may include computing the amount of resources on the graphics card device consumed by the virtual desktop of the virtual machine using the graphics profile during a first execution of the virtual machine. Sending, to the hypervisor, the second data for the second graphics profile to cause an execution of the virtual machine using the different portion of resources on the second graphics card device may include causing a second execution of the virtual machine on a second computer that includes the second graphics card device using resources allocated to the virtual machine based on the second graphics profile, the second execution of the virtual machine being a different execution than and occurring after the first execution of the virtual machine. The second graphics profile may be specific to a first type of graphics card device or a first graphics card device architecture. Causing the second execution of the virtual machine on the second computer that includes the second graphics card device using resources allocated to the virtual machine based on the second graphics profile may include determining, using the second graphics profile, that a particular computer that includes the first type of graphics card device or a particular graphics card device with the first graphics card device architecture is not available for the second execution of the virtual machine; determining that another graphics profile for a second type of graphics card device, different than the first type of graphics card device, or for a second graphics card device architecture, different than the first graphics card device architecture, identifies at least the resources identified by the second graphics profile in response to determining that a particular computer that includes the first type of graphics card device or a particular graphics card device with the first graphics card device architecture is not available for the second execution of the virtual machine; determining that the second graphics card device is specific to the other graphics profile; determining that the second computer includes the second graphics card device; and selecting the second computer for the second execution of the virtual machine in response to determining that the second computer includes the second graphics card device.

In some implementations, the method may include causing presentation of data identifying the portion of the available resources on the graphics card device allocated to the virtual machine and the amount of resources on the graphics card device consumed by the virtual desktop of the virtual machine. Determining the portion of the available resources on the graphics card device allocated to the virtual machine using the graphics profile may include determining a maximum quantity of virtual machines that can be assigned to the graphics card device using the graphics profile; and determining the portion of the available resources on the graphics card device allocated to the virtual machine using the maximum quantity of virtual machines that can be assigned to the graphics card device. The method may include determining a profile identifier for the graphics profile; and creating a mapping between the profile identifier and an identifier for the virtual machine upon execution of the virtual machine on the computer. Determining the graphics profile for the virtual machine included in the plurality of virtual machines executing on the computer may include determining, using the mapping between the profile identifier and the identifier for the virtual machine, the profile identifier. The method may include receiving the data from the hypervisor that identifies the graphics profile for the virtual machine.

The subject matter described in this specification can be implemented in particular embodiments and may result in one or more of the following advantages. In some implementations, the systems and methods described below may change a graphics profile used to allocate resources for execution of a virtual application based on the resources actually used during the execution to more efficiently allocate resources for execution of multiple different virtual applications. For example, a system may determine that a subset of the graphics resources on a graphics card device are not used for execution of a first virtual application to which the graphics resources were allocated and can be used instead for execution of a second virtual application, e.g., that could not otherwise be executed using the graphics card device, to reduce power consumed by the graphics card device, or both. A system may determine a more efficient allocation of graphics resources for execution of a virtual desktop or another graphics intensive application. In some implementations, a system may cause a particular host to execute virtual machines with the same graphics profile to increase efficient allocation of resources to virtual machines, reduce unused resources on physical graphics devices, reduce a number of hosts in the system that are being used and power required for those hosts, e.g., by disabling hosts that are not executing any virtual machines, or a combination of two or more of these. In some implementations, a system may notify a user device when a virtual machine is not being used and, in response to input from the user device, terminate a session for the virtual machine to allow the resources previously used to execute the virtual machine to be allocated to a different virtual machine or turned off.

DETAILED DESCRIPTION

When a client device requests a session for a virtual application that has a graphics rich user interface, e.g., a graphics rich virtual desktop, a host system may allocate graphics resources on a graphics device, e.g., a Peripheral Component Interconnect Express (PCIe) graphics device, to a virtual machine executing the virtual application. For example, the host system may determine a set of graphics resources that represent a virtual graphics device for the virtual machine. The virtual machine may use a portion of a physical graphics processing unit (pGPU) included on the graphics device for execution of a virtual graphics processing unit (vGPU) and allocate a portion of a frame buffer memory included on the graphics device for use by the vGPU.

The host system may use a graphics profile to determine properties of the virtual graphics device. For instance, the graphics profile may indicate a particular type of physical graphics device, physical resources from a physical graphics device that should be allocated to a virtual graphics device, or both. The graphics profile may indicate a percentage of physical graphics resources for the virtual graphics device. The graphics profile may indicate quantities of physical graphics resources, such as a number of pGPUs; a number of vGPUs; an amount of frame buffer memory; a number of processing units per second for processing, e.g., a number of polygons per second or instructions per second for the vGPU; a type of physical graphics device; a model for a physical graphics device; a vendor of a physical graphics device; or a combination of two or more of these.

The host system may monitor the virtual graphics device to determine whether to use different graphics properties for the virtual graphics device during a subsequent execution of the virtual machine. For example, the host system may determine whether more or fewer resources should be allocated to the virtual graphics device depending on the actual graphics resources used during execution of the virtual machine.

In some implementations, the virtual application may be any appropriate type of application. The virtual application may be a virtual desktop for a virtual machine. In some examples, the virtual application may be an application that requires dedicated graphics resources for efficient generation of a user interface. For instance, the virtual application may be a graphics design application executing on a virtual machine.

FIG. 1is an example of an environment100in which a host system104monitors graphics resources allocated to a virtual machine A120afor a client device102. For instance, the host system104allocates physical graphics resources to execution of the virtual machine A120aas one or more virtual graphics devices for the virtual machine A120a. The host system104may monitor the use of the virtual graphics devices for the virtual machine A120ato determine whether a different allocation of graphics resources may be more efficient for the virtual machine A120a. In some examples, the host system may be a virtual center or may be included in a virtual center. A virtual center may include multiple physical computers each of which may include one or more of the components described with reference to the host system.

For example, the host system104is an example of a system implemented as computer programs on one or more computers in one or more locations, in which the systems, components, and techniques described below are implemented. The host system104may use a single server computer or multiple server computers operating in conjunction with one another, including, for example, a set of remote computers deployed as a cloud computing service.

The host system104receives, at time TA, a request from the client device102for a virtual application. The virtual application may be a graphics rich virtual application. The virtual application may be a graphics rich virtual desktop. The client device102may include personal computers, mobile communication devices, and other devices that can send and receive data over a network124. For instance, the host system104receives the request from the client device102across the network124. The network124, such as a local area network (LAN), wide area network (WAN), the Internet, or a combination thereof, connects the client device102, and the host system104.

The host system104determines to allocate dedicated graphics hardware to the execution of the requested virtual application. For instance, the host system104includes multiple processors106, multiple storage devices108, and multiple physical graphics devices A-B110a-b. Each of the physical graphics devices A-B110a-bincludes one or more graphics processors A-B112a-b, e.g., pGPUs A-B112a-b, and a frame buffer memory A-B114a-b. The host system104uses the request to identify the requested virtual application and allocate resources for use executing the requested virtual application. The resources may include resources of the graphics devices A-B110a-b, resources of the processors106, resources of the storage devices108, or a combination of two or more of these.

In some implementations, each hypervisor A-B118a-bmanages resources on a single physical graphics device A-B110a-b. For example, a first hypervisor A118amanages resources for virtual machines A-B120a-ballocated resources from a first graphics device A110aand a second hypervisor B118bmanages resources for virtual machines C-D120c-dallocated resources from a second graphics device B110b. In some examples, a hypervisor, e.g., the first hypervisor A118a, may manage resources for multiple physical graphics devices, e.g., both the graphics devices A-B110a-b.

The host system104may use the request to determine a graphics profile for execution of the virtual application. For instance, the request may include a graphics profile identifier. The graphics profile may be specific to the virtual application, the client device102, or both. In some examples, the graphics profile identifier may be specified by user input and the request may include data that identifies the graphics profile identifier.

The host system104may use the graphics profile identifier to determine the graphics resources for allocation to the virtual application during execution. The host system104may access a graphics profile database116that includes a mapping of graphics profile identifiers to data indicating graphics resources for allocation of execution of a virtual application. An example of a mapping for four graphics profile identifiers is shown in Table 1, below.

The graphics profile database116may include a record for each graphics profile identifier, e.g., that uses the graphics profile identifier as a key. The record may identify the mapping for the specific graphics profile identifier, for instance by including the graphics profile identifier and values that represent properties of a vGPU, a frame buffer memory, or both. The record may indicate the specific resources for use during execution of corresponding virtual applications. In some examples, the record may indicate that the host system104should use a particular quantity of a particular resource for execution of a virtual application, e.g., 4 MB of physical frame buffer memory. The particular quantity may be a maximum quantity for the execution of the virtual application. As another example, the record may indicate that the host system104should use a particular percentage of a particular resource or of all resources, e.g., twenty-five percent of frame buffer memory and twenty-five percent of the pGPUs. For instance, the record may indicate that the graphics device should allocate four vGPUs to each pGPU and use twenty-five percent of the pGPU for execution of a corresponding virtual application. The host system104may generate the graphics profiles stored in the graphics profile database116. The host system may use data that generated based on an administrator input, a graphics device vendor specification, or both, to create the graphics profiles.

Each of the graphics profiles may be for a particular vendor or a particular graphics device A-B110a-b. The particular graphics device may be developed by a particular vendor, may be a model of graphics device manufactured by one of multiple different manufacturers, or both. For example, a first graphics device A110amay have three graphics profiles each of which indicate a different number of pGPUs and a different quantity of frame buffer memory for allocation to an application executed using the graphics profile. In some examples, a second graphics device B110bmay have five graphics profiles that each indicate percentages of the physical graphics resources, the graphics processors B112band the frame buffer memory B114b, that should be allocated to virtual applications. For instance, when a graphics device110includes four pGPUs and 16 GB of frame buffer memory, a first graphics profile may indicate allocation of an eighth of a single pGPU and 512 MB of frame buffer memory to execution of a virtual application. A second graphics profile may indicate allocation of one quarter of a single pGPU and 1 GB of frame buffer memory to execution of a virtual application. A third graphics profile may indicate allocation of one pGPU and 4 GB of frame buffer memory to execution of a virtual application.

The host system104provides information about the graphics resources to be used for execution of the virtual application to a hypervisor A118ato cause execution of the virtual application. For instance, the hypervisor A118areceives the graphics information indicated by the graphics profile for the virtual application. The hypervisor A118adetermines a server that includes available graphics resources specified by the graphics information. The hypervisor A118acauses the server to launch a virtual machine A120afor the virtual application using the graphics resources specified by the graphics information. For instance, the server allocates graphics resources specified by the graphics information to execution of the virtual machine A120afor the virtual application. The hypervisor A118amonitors execution of the virtual application on the server.

The host system104initiates a session, at time TB, for the virtual application with the client device102. For example, the host system104uses remote session communications protocols to provide the client device102with data for presentation of a user interface for the virtual application on a display. The virtual application may be a virtual desktop for the virtual machine A120aor an application executing on the virtual machine A120a. The virtual application may be a graphics intensive application, e.g., that requires dedicated graphics resources for execution.

The host system104, e.g., a resource analysis system122running on the host system104, may monitor the execution of the virtual application, the virtual machine A120a, or both, on the server included in the host system104. For instance, the resource analysis system122may communicate with the hypervisor A118ato determine the physical graphics resources allocated to the virtual machine A120a, the physical graphics resources actually used by the virtual machine A120aduring execution of the virtual application, or both.

The host system104includes multiple devices, e.g., computers, each of which execute different ones of the virtual machines A-D120a-d, execute one or more hypervisors A-B118a-bthat monitor the virtual machines A-D120a-d, or both. For example, the host system104may include a first server that executes the hypervisor A118a, a second server that executes the virtual machine A120a, and a third server that executes the resource analysis system122to analyze the execution of the virtual machine A120ato determine whether to use a different graphics profile during a subsequent execution of the virtual application.

The resource analysis system122may request and receive first data from the first server that identifies the graphics profile for the virtual application. For instance, after the session for the virtual application between the client device102and the host system104ends, the resource analysis system122may request the graphics profile identifier for the virtual application from the first server, e.g., from the hypervisor A118aexecuting on the first server.

The resource analysis system122may request and receive second data from the second server that indicates, graphics resources used for execution of the virtual application. In some examples, the resource analysis system122may request and receive the second data from the first server, e.g., from the hypervisor A118a. The resource analysis system122may send a request, after the session for the virtual application between the client device102and the host system104ends, to the second server for second data that indicates the graphics resources used during the execution of the virtual application. The second data that indicates the graphics resources used may include an average amount of frame buffer memory114used for execution of the virtual application, an average number of instructions per second for execution of the virtual application, detailed graphics resource usage data for a particular period of time that indicates specific amounts of resources used, or a combination of two or more of these. The particular period of time may be for the entire session or a subset of the session between the client device102and the host system104for the virtual application.

The resource analysis system122may compare the first data and the second data to determine whether to change a graphics profile for the virtual application. For instance, the resource analysis system122may determine whether execution of the virtual application requires more or less physical graphics resources, e.g., and the virtual application can be executed more efficiently, or whether the graphics profile should remain the same. In response, the resource analysis system122may change the profile the virtual application and, based on the new graphics profile, the physical graphics resources allocated during later execution of the virtual application.

FIG. 2shows an example of an environment200that includes a first graphics device A202and a second graphics device B216, both for execution of virtual machines. In some examples, the first graphics device A202and the second graphics device B216are manufactured by different vendors. In some other examples, the first graphics device A202and the second graphics device B216are manufactured by the same vendor and are different graphics device models that include different physical hardware.

A host system may assign execution of a virtual application to a server that includes the first graphics device A202. For instance, the host system may provide a hypervisor information about the virtual application including a graphics profile for the virtual application that indicates physical graphics resources for execution of the virtual application. The hypervisor may use the graphics profile to determine that a server, which includes the first graphics device A202, should execute the virtual application. For example, the hypervisor may determine that the first graphics device A202has available the necessary amount of physical graphics resources identified by the graphics profile for execution of the virtual application.

In response, the hypervisor provides the server with configuration data for the server, the first graphics device202, or both. The server can use the configuration data to allocate resources for execution of the virtual application. The configuration data may include data from the graphics profile for the virtual application.

For instance, the hypervisor may cause the first graphics device A202to allocate a first pGPU A1204aand a second pGPU A2204bfor use as a vGPU206for execution of the virtual application. The first graphics device A202may use a third pGPU A3204cand a fourth pGPU A4204dfor execution of one or more additional virtual applications, each executing on a corresponding virtual machine.

The hypervisor may cause the first graphics device A202to allocate a portion of frame buffer memory A210, included in the first graphics device A202, as first allocated frame buffer memory portion212for execution of the virtual application. For instance, the first graphics device A202may determine a start address and an end address for the first allocated frame buffer memory portion212from the frame buffer memory A210. In some examples, the start address, the end address, or both, for the first allocated frame buffer memory portion212are fixed for a current execution of the virtual application. For instance, during a session with a client device for which the virtual application is executed, the first graphics device A202may maintain the same start address and the same end address for the first allocated frame buffer memory portion212.

During the execution of the virtual application using the vGPU206and the first allocated frame buffer memory portion212, the hypervisor or another component of the host system monitors the resources on the first graphics device A202that are actually used for execution of the virtual application. For example, the hypervisor determines that the server uses a portion208of the vGPU206, e.g., the first pGPU A1204a, and a subset214of the first allocated frame buffer memory portion212. The hypervisor may determine the portion208of the vGPU206and the subset214of the first allocated frame buffer memory portion212as average resources, global maximum resources, local maximum resources, or a combination of two or more of these, used by the server for execution of the virtual application. For instance, the hypervisor may determine a global maximum as the subset214of the first allocated frame buffer memory portion212that represents the most graphics frame buffer memory used at any particular time during execution of the virtual application and an average portion208of the vGPU206used by the server for execution of the virtual application across the entire execution of the virtual application.

A host system, e.g., the resource analysis system122, may use the portion208of the vGPU206and the subset214of the first allocated frame buffer memory portion212, used during execution of the virtual application, to determine whether to change a graphics profile for the virtual application. For instance, the resource analysis system may receive, from the hypervisor, a graphics profile identifier for the virtual application and data that indicates the portion208of the vGPU206and the subset214of the first allocated frame buffer memory portion212used during execution of the virtual application. The resource analysis system may access a graphics profile database using the graphics profile identifier to determine a particular graphics profile that indicates the resources allocated for execution of the virtual application. The resource analysis system may compare the resources allocated for the execution of the virtual application with the data that indicates the portion208of the vGPU206and the subset214of the first allocated frame buffer memory portion212used during execution of the virtual application.

Using the comparison, the resource analysis system may determine whether the virtual application can be executed with a different set of resources than those identified in the particular graphics profile. For instance, the resource analysis system may determine that execution of the virtual application used only a first pGPU A1204aand not a second pGPU A2204bboth of which were allocated for the vGPU206. The resource analysis system may determine whether there is another graphics profile for the first graphics device A202, or another graphics device such as the second graphics device B216, that identifies resources closer to those actually used for execution of the virtual application, e.g., a graphics profile that identifies allocation of only one pGPU on the first graphics device A202for a vGPU and at least the subset214of the first allocated frame buffer memory portion212. The resource analysis system may determine that the other graphics will provide a more efficient use of the host system's resources for execution of the virtual application. When the resource analysis system determines that the particular graphics profile is most efficient, the resources analysis system may send the client device a message that indicates that the particular graphics profile is optimal or may take no action.

When the resource analysis system determines the other graphics profile for execution of the virtual application, the resource analysis system may provide, to the client device or another device, a message that identifies the other graphics profile. The message may indicate that the client device should use the other graphics profile for future execution of the virtual application. The message may indicate that use of the other graphics profile will likely cause the host system, or another host system, to allocate resources for use during execution of the virtual application more efficiently than the resources allocated during the previous execution of the virtual application.

When the resource analysis system provides the message to another device, e.g., an administrator device, the message may include, for a virtual application, an amount of current frame buffer memory allocated to the virtual machine; a frame buffer usage percentage; frequency of use data, e.g., hourly, daily, peak hourly frequency, etc.; a recommended frame buffer memory size, or a combination of two or more of these. The message may include, for a virtual application, an amount of current pGPUs or a property of the pGPUs allocated to the virtual machine; a pGPU usage percentage; frequency of use data, e.g., hourly, daily, peak hourly frequency, etc.; a recommended pGPU property, e.g., speed, number of cores, etc.; or a combination of two or more of these.

In some implementations, the message may indicate a cost savings or an additional cost for a change in a profile. For instance, the message may indicate that switching from a first profile to a second profile would reduce the physical resources allocated for execution of a virtual application and reduce costs by a particular amount.

When the resource analysis system provides the message to another device, e.g., an administrator device, the message may include information for multiple virtual applications, e.g., multiple graphics rich virtual desktops. Each of the multiple virtual applications may be monitored by a single hypervisor. Each of the multiple virtual applications may be executed on behalf of the same entity, e.g., client device, corporation, or user account. The message may cause the client device to present a dashboard of metrics associated with each of the multiple virtual applications. The metrics may include various properties about the physical graphics resources allocated to execution of the respective virtual application.

In some implementations, when the resource analysis system determines the other graphics profile for execution of the virtual application, the resource analysis system may provide the host system with data identifying the other graphics profile. The host system may use the other graphics profile automatically during, a subsequent execution of the virtual application for the client device, e.g., after receipt of a second request from the client device to execute the virtual application.

When the resource analysis system provides the message to another device, e.g., an administrator device, the resource analysis system may provide the client device with a message that indicates statistics of the execution of the virtual application. For instance, the resource analysis system may provide the client device with instructions to cause the presentation of a user interface with the statistics of the execution of the virtual application. The statistics may include data that represents properties of the vGPU206, properties of the first allocated frame buffer memory portion212, or both. The statistics may include data that represents properties of the portion208of the vGPU206, properties of the subset214of the first allocated frame buffer memory portion212, or both, that were used during execution of the virtual application. For instance, the instructions may cause presentation of a user interface that indicates an amount of memory included in the first allocated frame buffer memory portion212and the subset214of the first allocated frame buffer memory portion212that was actually accessed during execution of the virtual application.

FIG. 3depicts an example of a user interface300that includes statistics of the execution of a virtual machine. The user interface300includes three properties A, B, and C302-306, and respective statistics on the usage of those properties. In some examples, each of the three properties A, B, and C302-306may be properties for the execution of a single virtual machine. The first property A302may indicate that 62.5% of a portion of frame buffer memory allocated to execution of the virtual machine was used during the execution. The second property B304may indicate that, during peak usage, 93.75% of the vGPU cycles allocated for execution of the virtual machine were, used during execution. The third property C306may indicate that, during non-peak usage, about 40% of the vGPU cycles allocated for execution of the virtual machine were used at a minimum during execution. The percentages may be minimum percentages, maximum percentages, average percentages, or a combination of two or more of these.

In some examples, one or more of the three properties A, B, and C302-306may be properties for the execution of different virtual machines. For instance, the first property A302may indicate a portion of a frame buffer memory allocated to a first virtual machine, e.g., the virtual machine A120a, that was used during execution of the first virtual machine. The second property B304may indicate a portion of a frame buffer memory allocated to a second virtual machine, e.g., the virtual machine B120b, that was used during execution of the second virtual machine. The third property C306may indicate a portion of a frame buffer memory allocated to a third virtual machine that was used during the execution of the third virtual machine.

All of the virtual machines for which the user interface300includes information may be monitored by a single hypervisor. In some examples, some of the virtual machines for which the user interface300includes information may be monitored by different hypervisors. For instance, all of the virtual machines may be executed on behalf of a single client while two or more hypervisors may monitor the virtual machines.

The statistics included in the user interface300may be for a particular polling interface. For instance, the statistics may be for a polling interval, such as one hour, one day, one month, or another polling interval. The resources analysis system may receive input, e.g., user input, that defines the polling interval. In some examples, the resource analysis system may use a predetermined polling interval.

In some implementations, the resource analysis system may receive data that indicates user input accepting the change from the particular graphics profile to the other graphics profile for execution of the virtual application. For instance, the user interface300may include, for a second virtual machine with a resource that is using 100% or close to 100%, e.g., 90%, of a particular resource, e.g., frame buffer memory, a recommendation to change the profile for the respective virtual machine. The recommendation may identify a recommended profile. In some examples, the recommendation may request input identifying another profile, properties for another profile, e.g., as a search feature for another profile, or both.

The resource analysis system may receive the data from the client device, the other device, e.g., the administrator device, or both. The resource analysis system may store data in a database that indicates that the other graphics profile should be used for future execution of the virtual application, e.g., after disconnection of a current session for the virtual application when a new session for the virtual application is initiated.

In some examples, after providing the client device with the message that indicates the other graphics profile, the host system may receive a request for a second execution of the virtual application that identifies the other graphics profile. The host system may cause allocation of resources for the second execution of the virtual application using the other graphics profile.

In some implementations, when the host system determines that resources specific to a graphics profile for a virtual application are not available, the host system identifies other resources that are equivalent to the resources identified in the graphics profile for use during execution of the virtual machine. For example, a hypervisor may receive instructions from the host system to launch a session for the virtual application. The hypervisor determines the graphics profile for the virtual application, e.g., with a graphics profile identifier of K120Q. The hypervisor uses the graphics profile to determine whether graphics resources, identified by the graphics profile, are available for execution of the virtual application. Returning toFIG. 2, the hypervisor may determine that the graphics profile is specific to the first graphics device A202, e.g., all graphics devices with the same model, vendor, or both, as the first graphics device A202. The hypervisor determines that sufficient resources for the virtual application, as indicated in the graphics profile, are not available on any graphics devices of the same type as the first graphics device A202.

In response, the hypervisor uses graphics profile data, e.g., from the graphics profile database, to determine another graphics profile that is equivalent to the graphics profile, e.g., with a graphics profile identifier of K220Q. For instance, the hypervisor may use a mapping, e.g., stored in the graphics profile database, that indicates which graphics profiles are equivalent to other graphics profiles and can be used for execution of a virtual application. Table 2, below, indicates one example of two graphic profiles that are equivalent. Graphics profile K120Q and K220Q are equivalent because the same amount of frame buffer memory is allocated for execution of a corresponding virtual application. The graphics profiles may be equivalent because each pGPU is allocated the same quantity of vGPUs.

In some implementations the mapping may indicate that another graphics profile can be used instead of the graphics profile without a negative performance impact on a virtual application that will use resources identified by the other graphics profile compared to the virtual application using the original resources identified by the graphics profile. In some examples, the hypervisor may determine the graphics device properties identified in the graphics profile and another graphics profile that identifies the same graphics device properties. For instance, the graphics profile and the other graphics profile may indicate that 512 MB of frame buffer memory, two virtual display heads, one eighth of a pGPU should be allocated for execution of a corresponding virtual application. Both graphics profiles may indicate a maximum display resolution, per display head, of 2560 by 1600.

The hypervisor may determine that the graphics profile indicates allocation of a first pGPU A1204aand a second pGPU A2204bfor a vGPU206and a first allocated frame buffer memory portion212should be assigned to the vGPU206for execution of a virtual application. The hypervisor may determine that the other graphics profile, that is equivalent to the graphics profile, indicates allocation of a first different pGPU B1218aand a second different pGPU B2218b, on a second graphics device B216, as a second vGPU220along with a second allocated frame buffer memory portion224in a second frame buffer memory B222, included in the second graphics device B216.

The equivalence of the graphics profile and the other graphics profile may be approximate. For instance, the other graphics profile may indicate slightly more or slightly fewer resources. The other graphics profile may indicate that the second allocated frame buffer memory portion224has the same memory size or a different memory size, the same latency or a different latency, or a combination of these, as the first allocated frame buffer memory portion224. The other graphics profile may indicate that the second vGPU220has the same properties or different properties as the vGPU206. Some example properties may include a number of cores, a number of chip sockets, a number of instructions per second, a clock speed, or a combination of two or more of these.

The hypervisor may use the other graphics profile to determine a particular second graphics device B216that has resources available for execution of the virtual application. For instance, the hypervisor may determine that a particular server includes an array of second graphics devices B216and that one of the graphics devices in the array is not being used. The hypervisor may determine that the particular server has other resources available for execution of the virtual application, e.g., memory resources, processor resources, or both. The hypervisor assigns execution of the virtual application to the particular server to cause the particular server, and the one of the graphics devices in the array, to allocate resources for execution of the virtual application and launch a virtual machine, using the allocated resources, for the virtual application.

In some implementations, the hypervisor may use a database that indicates which physical graphics resources are in use and which graphics resources are available. The hypervisor may analyze the database, e.g., a table in the database, to que resources for use during execution of a virtual application. For instance, the hypervisor may query the database using data from a graphics profile, e.g., a physical device model or physical device vendor or another profile property, to determine available resources. The available resources may be on a per host level in a cluster, e.g., in the host system. The hypervisor may also determine other physical resources available for execution of a virtual application in addition to the physical graphics resources available for execution of the virtual application.

In some implementations, the hypervisor may use data representing the resources used by a virtual application during a previous execution when determining another graphics profile that is the equivalent of a graphics profile typically used for execution of the virtual application. For instance, the hypervisor may determine whether the portion208of the vGPU206, the subset214of the first allocated frame buffer memory portion212, or both, from a previous execution of the virtual application indicate that execution of virtual application uses most of or less than the resources identified by the graphics profile. The data representing the resources used by the virtual application during a previous execution may be included in a request to create a session for the virtual application, may be stored in a database, e.g., a record specific to the requesting device and the virtual application, or another appropriate location.

When the hypervisor determines that execution of the virtual application uses most of the resources identified by the graphics profile, the hypervisor may select another graphics profile that identifies the same or more resources than the graphics profile. For instance, when the graphics profile indicates that 1 MB of frame buffer memory should be allocated to execution of the virtual machine, the hypervisor may select another graphics profile that indicates that 1 MB or more, e.g., 2 MB, should be allocated to execution of the virtual machine.

When the hypervisor determines that execution of the virtual application uses less than the resources identified by the graphics profile, the hypervisor may select another graphics profile that identifies fewer resources than the graphics profile. For example, when the graphics profile indicates that 1 MB of frame buffer memory should be allocated to execution of the virtual machine, the hypervisor may select another graphics profile that indicates that 0.5 MB should be allocated to execution of the virtual machine.

In some implementations, a hypervisor may use a comparison of the resources allocated for the execution of a virtual application with the data that indicates the portion208of the vGPU206and the subset214of the first allocated frame buffer memory portion212used during execution of the virtual application to determine that more resources should be allocated to execution of the virtual application. For instance, the hypervisor may determine that, on average, the vGPU is processing instructions at maximum capacity and that another graphics profile, that identifies higher allocations of resources, should be assigned to the virtual application.

In some implementations, the host system may receive a request for data identifying resources allocated to a virtual application, resources used by the virtual application, or both. For example, the host system may receive a request from a computer, e.g., the client device or another device, that indicates a request for data identifying the resources used by the virtual application during execution. In response, the host system may generate instructions for presentation of a user interface that indicates the resources used by the virtual application, provide data to the computer that identifies the resources used by the virtual application, or both.

In some implementations, a host system may provision resources for a virtual application with a particular graphics profile on a physical graphics device that currently has physical graphics resources allocated to other virtual applications using the particular graphics profile. For instance, the host system may determine an efficient allocation of physical graphics resources by allocating resources on a particular physical graphics device to multiple virtual applications that each have the same graphics profile.

In some implementations, a host system may move or migrate a virtual application using a graphics profile for the virtual application. The host system may determine that the device, e.g., the physical graphics device or the computer that includes the physical graphics device, used to execute the virtual application failed. In response, the host system may determine to move execution of the virtual application to another device. The host system may determine whether another device is available for the graphics profile for the virtual application or whether the host system needs to determine an equivalent graphics profile that identifies graphics resources equivalent to those identified in the graphics profile for the virtual application. The host system may cause execution of the virtual application using the other device or a device for the equivalent graphics profile.

When the host system determines to migate a virtual application, e.g., in response to a request from a client device, the host system uses a graphics profile for the virtual application to determine another device to use for execution of the virtual application. A migration request may indicate that execution of the virtual application should move from a first data center to a second data center, from a data center in a first physical geographic location to another data center in a second physical geographic location, or both. The host system may tag the virtual application, the corresponding virtual machine on which the virtual application is executing, or both, with an identifier for the graphics profile to allow the host system to use the same graphics profile when recreating the virtual machine, e.g., at the second data center or the data center in the second physical geographic location. For instance, the host system, may use the tag to identify the graphics profile when the host system cannot move execution of the virtual application live and must recreate the virtual machine on a destination device.

FIG. 4is a flow diagram of a process400for changing a graphics profile for a virtual machine or a virtual application executing on the virtual machine. For example, the process400can be used by the host system104from the environment100. One example of a virtual application includes a virtual desktop for the virtual machine. The virtual desktop may be a graphics rich virtual desktop, e.g., that requires dedicated graphics resources for execution.

At402, a host system determines resources available on a graphics card device included in a computer executing multiple virtual machines, each virtual machine configured to execute a virtual desktop. The host system may determine a quantity of pGPUs included on the graphics card device; properties of the pGPUs, e.g., clock rate; a quantity of frame buffer memory, included on the graphics card device; properties of the frame buffer memory, e.g., speed; or a combination of two or more of these.

At404, the host system determines a profile identifier for a graphics profile for a virtual machine. For example, the host system may receive a request to execute a particular virtual machine and determine a profile identifier that is included in the request. In some implementations, the host system does not include a mapping of a profile identifier to the virtual machine before receiving the request. For instance, the host system does not store, in memory, data that indicates a profile identifier or a graphics profile to be used for execution of the virtual machine during a session with a client device from which the host system received the request.

At406, the host system creates a mapping between the profile identifier and an identifier for the virtual machine upon execution of the virtual machine on the computer. The host system may cause execution of the virtual machine, as one of the plurality of virtual machines, on the computer. A hypervisor, included in the host system, may cause the graphics card device to allocate the resources identified by the graphics profile for execution of the virtual machine. The hypervisor may create the mapping between the profile identifier and the identifier for the virtual machine.

The hypervisor may store the mapping in a memory. In some examples, the hypervisor may be the only component of the host system with direct access to the mapping, e.g., and other components of the host system need to request the mapping or data related to the mapping from the hypervisor. For instance, a resource analysis system, included in the host system, may request the profile identifier from the hypervisor to determine graphics resources allocated to the virtual machine during execution when the resource analysis system does not have direct access to the memory with the mapping. The resource analysis system may provide the hypervisor with the identifier for the virtual machine, or other data that identities the virtual machine such as a device that requested execution of the virtual machine. The hypervisor uses the data received from the resource analysis system to access the mapping and determine the profile identifier. In response, the resource analysis system receives the profile identifier from the hypervisor.

At408, the host system determines, based on data received from a hypervisor that manages execution of at least one of the virtual machines, a graphics profile for a virtual machine included in the plurality of virtual machines executing on the computer. The resource analysis system may use the profile identifier, received from the hypervisor, to access a graphics profile database. The resource analysis system may use the profile identifier as a key to select data for a graphics profile that is stored in the graphics profile database.

At410, the host system determines a maximum quantity of virtual machines that can be assigned to the graphics card device using the graphics profile. When the graphics profile indicates a percentage of the resources for the graphics card device that are allocated to each virtual machine, e.g., and the quantity of resources allocated to each virtual machine that uses the graphics card device is the same, the resource analysis system determines the maximum quantity of virtual machines using the percentage, e.g., by dividing one by the percentage value. When the graphics profile indicates one or more quantities of respective resources, the resource analysis system uses the quantities and amounts of resources included in the graphics card device to determine the maximum quantity of virtual machines.

In some implementations, the maximum quantity of virtual machines may be associated with the graphics profile, e.g., in the graphics profile database. The resource analysis system may request the maximum quantity of virtual machines from the graphics profile database using the profile identifier.

At412, the host system determines a portion of the available resources on the graphics card device allocated to the virtual machine. For instance, the resource analysis system may use the maximum quantity of virtual machines that can be assigned to the graphics card device and the resources available on the graphics card device to determine the portion of the available resources allocated to the virtual machine. When the graphics profile indicates a percentage of resources on the graphics card device that should be allocated to each of the virtual machines, the resource analysis system may use the total resources for the graphics card device and percentage value to determine the available resources on the graphics card device that are allocated to the virtual machine, e.g., without performing step410.

At414, the host system computes an amount of: resources on the graphics card device consumed by a virtual desktop of the virtual machine based on the portion of the available resources on the graphics card device allocated to the virtual machine. For example, the host system determines the resources actually used by the graphics card device during execution of the virtual desktop of the virtual machine. The resources actually used may be an average, a maximum, or a series of values that represent resources used during the execution of the virtual desktop during a session. In some examples, the resource analysis system may request the amount of graphics resources on the graphics card consumed by the virtual desktop of the virtual machine from the hypervisor, e.g., that is monitoring execution of the virtual machine. The hypervisor may use an application programming interface (API) to determine the graphics resources consumed by the virtual desktop. For instance, the API may include a method that allows the hypervisor to request the consumed graphics resources for the virtual desktop. The API may include a method that allows the hypervisor to receive a list of all running virtual machines, which the hypervisor is monitoring, that are using dedicated graphics resources and a wan percentage of the graphics resources allocated to each of the virtual machines.

In some examples, a hypervisor may monitor virtual machines that are allocated physical graphics resources for a particular vendor and no other vendors. In some examples, a hypervisor may monitor virtual machines that are allocated physical graphics resources for multiple vendors. The hypervisor may include a method to determine consumed resources for all of the vendors. The hypervisor may include one method for each of the vendors that allows the hypervisor to determine consumed resources for the particular vendor.

The virtual desktop may be an operating system user interface, e.g., which includes menu options to launch applications and present information about an operating system executing on the virtual machine. The virtual desktop may be a user interface for a virtual application other than an operating system, e.g., a user interface for a graphics design application.

At416, the host system determines, using the portion of the available resources on the graphics card device allocated to the virtual machine and the amount of resources on the graphics card device consumed by the virtual desktop of the virtual machine, whether to change the graphics profile for the virtual machine to a second graphics profile that indicates a different portion of resources on a second graphics card device that should be allocated to the virtual machine. For instance, the resource analysis system determines whether the second graphics profile indicates a more efficient allocation of resources for the virtual machine. The second graphics profile may identify more resources, fewer resources, or both, than the graphics profile. For instance, the second graphics profile may be for a different type of graphics device and indicate that more frame buffer memory should be allocated for execution of the virtual machine while a smaller vGPU is allocated for the execution of the virtual machine. The smaller vGPU may have fewer physical pGPU cores, a lower clock rate, or both.

In some implementations, the host system may determine that a particular virtual machine has not been or is not being used after execution, e.g., that minimal resources were consumed and there was not input received from a client device. The host system may provide a client device with a message that indicates that the particular virtual machine has not been or is not being used. The host system may receive a response that indicates that a session for the particular virtual machine should be terminated. In response, the host system may allocate the resources previously used for execution of the virtual machine to execution of a different virtual machine, turn off a host system that was previously executing the virtual machine, or place the host system in standby, e.g., reduce power provided to the host system.

The second graphics card device may be the same device as the first graphics card device. The second graphics card device may be the same model as the first graphics card device. The second graphics card device may be manufactured by a particular vendor that manufactured the first graphics card device. The second graphics card device may be manufactured by a different vendor than a particular vendor that manufactured the first graphics card device.

At418, the host system sends, to the hypervisor, second data for the second graphics profile to cause an execution of the virtual machine using the different portion of resources on the second graphics card device. The host system may receive, from the client device, a second request for a second execution of the virtual machine. The second request may identify the second graphics profile. In response to receipt of the second request, the host system may send the second data to the hypervisor to cause execution of the virtual machine.

In some examples, the host system may cause a second execution of the virtual machine on a second computer that includes the second graphics card device using resources allocated to the virtual machine based on the second graphics profile. The second execution of the virtual machine is a different execution than, and occurs after, the first execution of the virtual machine.

The order of steps in the process400described above is illustrative only, and changing the graphics profile for the virtual machine or the virtual application executing on the virtual machine can be performed in different orders. For example, the host system may determine a graphics profile for a virtual machine and then determines resources available on a graphics card device.

In some implementations, the process400can include additional steps, fewer steps, or some of the steps can be divided into multiple steps. For example, the host system may perform steps402,408,412, and414without performing any of the other steps in the process400.