Subsystem-level power management in a multi-node virtual machine environment

A computer-implemented method includes capping the amount of power available to each of a plurality of compute nodes, and managing power allocation among subsystems within each of the compute nodes according to the requirements of workloads assigned to each of the compute nodes. The method further comprises reporting an actual performance level and performance capability for each subsystem within each of the plurality of compute nodes, and monitoring parametric data for a particular workload. A target compute node is identified from among the compute nodes, wherein the target compute node would be capable of performing the particular workload if power was reallocated from a first subsystem to a second subsystem within the target compute node. The particular workload is then assigned to the target compute node. Optionally, assigning the particular workload may include migrating the workload to the target compute node from another of the compute nodes.

BACKGROUND

Field of the Invention

The present invention relates to power management in a multi-node virtual machine environment.

Background of the Related Art

In a cloud computing environment, a user is assigned a virtual machine somewhere in the computing cloud. The virtual machine provides the software operating system and has access to physical resources, such as input/output bandwidth, processing power and memory capacity, to support the user's application. Provisioning software manages and allocates virtual machines among the available computer nodes in the cloud. Because each virtual machine runs independent of other virtual machines, multiple operating system environments can co-exist on the same computer in complete isolation from each other.

BRIEF SUMMARY

One embodiment of the present invention provides a computer-implemented method, comprising capping the amount of power available to each of a plurality of compute nodes, and managing power allocation among subsystems within each of the plurality of compute nodes according to the requirements of workloads assigned to each of the plurality of compute nodes. The method further comprises reporting an actual performance level and performance capability for each of the subsystems within each of the plurality of compute nodes, and monitoring parametric data for a particular workload. A target compute node is identified from among the plurality of compute nodes, wherein the target compute node would be capable of performing the particular workload if power was reallocated from a first subsystem to a second subsystem within the target compute node. The method may then assign the particular workload to the target compute node.

DETAILED DESCRIPTION

One embodiment of the present invention provides a computer-implemented method, comprising capping the amount of power available to each of a plurality of compute nodes, and managing power allocation among subsystems within each of the plurality of compute nodes according to the requirements of workloads assigned to each of the plurality of compute nodes. The method further comprises reporting an actual performance level and performance capability for each of the subsystems within each of the plurality of compute nodes, and monitoring parametric data for a particular workload. A target compute node is identified from among the plurality of compute nodes, wherein the target compute node would be capable of performing the particular workload if power was reallocated from a first subsystem to a second subsystem within the target compute node. The method may then assign the particular workload to the target compute node. Optionally, assigning the particular workload to the target compute node may include migrating the workload to the target compute node from another of the plurality of compute nodes.

The amount of power available to each of a plurality of compute nodes is capped, for example, at a fixed amount of power subject to periodic increases or decreases. In a multi-node environment, such as a multi-server blade chassis, a chassis management module may be responsible for determining an appropriate amount of power for each compute node or other information technology equipment (ITE) and instructing the individual compute nodes to cap their power consumption at the amount of the “power cap.” The chassis management module may determine a “power cap” for each compute node in consideration of various factors, including a cumulative power cap amount for all of the compute nodes in the chassis. A chassis management module may be instructed to limit power consumption to a chassis power cap by a remote management node.

Power allocation among subsystems within each of the plurality of compute nodes is managed according to the requirements of workloads assigned to each of the plurality of compute nodes. For example, each compute node will have a management controller, such as a baseboard management controller (BMC), which is capable of monitoring and controlling various operating characteristics of subsystems within the compute node. In accordance with specific embodiments of the invention, a management controller may control the power consumption or operating mode of multiple subsystems. The subsystems preferably include at least a processor and memory, but may further include input/output adapters, power delivery components, and cooling devices. The management controller may allocate power among the subsystems in a manner that allows the subsystems to perform the workloads while limiting the total power consumption to the power cap assigned to the compute node. For example, if the compute node is assigned to run a workload that is processor intensive, then the management controller may, if needed, allocate unallocated power under the compute node's power cap to the processor. Furthermore, managing power allocation among subsystems within each of the plurality of compute nodes may include reallocating power from the first subsystem to the second subsystem within the target compute node. For example, the management controller may reduce an amount of power allocated to the memory so that an equal amount of power may instead be allocated to the processor. In another example, the subsystems of each of the plurality of compute nodes include a processor and a memory module, wherein managing power allocation among the subsystems within each of the plurality of compute nodes includes controlling the voltage and frequency of the processor and controlling the throughput of the memory module.

As stated above, the method further comprises reporting an actual performance level and performance capability for each of the subsystems within each of the plurality of compute nodes. Such information may be reported to, and collected by, a chassis management module. Furthermore, parametric data for a particular workload may be monitored by the chassis management module, whether the workload is currently running on a compute node or is a new workload yet to be run. A target compute node may be identified from among the plurality of compute nodes, wherein the target compute node would be capable of performing the particular workload if power was reallocated from a first subsystem to a second subsystem within the target compute node. For example, a chassis management module may, in view of the capability of a management module for a compute node to reallocate power among its subsystems, consider assigning a workload to a target compute node that can perform the particular workload even if the present allocation of power among subsystems in the target compute node would not be sufficient to perform the particular workload. Once the particular workload has been assigned to the target compute node, the target compute node may reallocate power to its subsystems in a manner that facilitates the performance of the particular workload. Accordingly, power may be reallocated from a first subsystem to a second subsystem within the target compute node in response to the workload placing a demand on the second subsystem that requires more power than a previous power allocation to the second subsystem. Optionally, the power may be reallocated from the first subsystem to the second subsystem without allocating any additional power to the target compute node.

In a further embodiment, the target compute node may request an increase in its capped amount of power in response to the target compute node reallocating power among subsystems to reduce stranded power within the target compute node and running workloads that have a collective power requirement exceeding the power cap after having reallocated power among the subsystems. In other words, after a management controller of a compute node has already reallocated power among its subsystems so that the compute node is making the best use of its power cap, the management controller may send a request for an increase in its power cap to the chassis management module if additional power is needed to meet the power requirements of the workloads assigned to the compute node. However, the management controller preferably will not request, and the chassis management module preferably will not grant, an increase in the compute node power cap if the relevant subsystem did not have additional unused capabilities that could make use of the power.

In another embodiment, the method caps the collective amount of power available to the plurality of compute nodes. As mentioned above, such a power cap may be implemented by a remote management node. If the plurality compute nodes have collective workload demands that exceed the power cap of the plurality of compute nodes (i.e., a chassis power cap), then one or more high priority compute nodes among the plurality of compute nodes may be given a high priority. Accordingly, the method may include increasing the capped amount of power available to the one or more high priority compute nodes in response to the one or more high priority compute nodes requesting more power, and reducing the capped amount of power available to lower priority compute nodes among the plurality of compute nodes in order to keep the total power consumption for the plurality of compute nodes within the capped collective amount of power.

In yet another embodiment, the method may reduce the capped amount of power available to a subset of the plurality of compute nodes in response to a partial loss of power. A chassis power supply, or multiple chassis power supplies, distributes power to each of the compute nodes in the chassis. When a power supply fails, there is less power available. However, the present embodiment would not reduce power to all compute nodes, but rather reduces the capped amount of power available to a subset of the compute nodes. For example, a high priority compute node that is running one or more high priority workloads may be allowed to continue operating without any reduction in its power cap.

In a still further embodiment, each of the plurality of compute nodes report power allocation data to a chassis management module, wherein the power allocation data for each compute node includes the amount of power allocated to each subsystem and the amount of power being used by each subsystem, such as a processor and a memory module. The chassis management module will preferably maintain access to data identifying the performance capability of each subsystem in each of the plurality of compute nodes. The data reported allows the chassis management to determine which of the compute nodes is best suited to run a workload. For example, a workload having a high priority may be assigned to a compute node that is operating in performance mode.

In another embodiment, a profile may be maintained for each workload that is being run by the plurality of compute nodes, wherein each profile characterizes the resource requirements of a workload. Optionally, the method may include monitoring the resource requirements of a new workload that is being run by a compute node, and maintaining a profile characterizing the resource requirements of the new workload.

In yet another embodiment, each of the compute nodes may be ranked as a function of performance capabilities of the subsystems in each compute node. Then, the ranking of compute nodes may be used to identify one or more of the plurality of compute nodes that is capable of running a particular workload based on the profile maintained for the particular workload.

In an additional embodiment, the capped amount of power available to the target compute node may be maintained or increased in response to a partial loss of power to the plurality of compute nodes, wherein assigning the particular workload to the target compute node includes migrating the workload to the target compute node from a compute node among the subset of compute nodes in response to a partial loss of power.

A further embodiment responds to a partial loss of power or a loss of communication between a chassis management control and the target compute node, by allowing the target compute node to continue operating with the capped amount of power and reducing the capped amount of power to the other compute nodes in the plurality of compute nodes.

It should be understood that although this disclosure is applicable to cloud computing, implementations of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

FIG. 4depicts an exemplary computing node (or simply “computer”)102that may be utilized in accordance with one or more embodiments of the present invention. Note that some or all of the exemplary architecture, including both depicted hardware and software, shown for and within computer102may be utilized by the software deploying server150, as well as the provisioning manager/management node222and the server blades204a-nshown inFIG. 5. Note that while the server blades described in the present disclosure are described and depicted in exemplary manner as server blades in a blade chassis, some or all of the computers described herein may be stand-alone computers, servers, or other integrated or stand-alone computing devices. Thus, the terms “blade,” “server blade,” “computer,” and “server” are used interchangeably in the present descriptions.

Computer102includes a processor unit104that is coupled to a system bus106. Processor unit104may utilize one or more processors, each of which has one or more processor cores. A video adapter108, which drives/supports a display110, is also coupled to system bus106. In one embodiment, a switch107couples the video adapter108to the system bus106. Alternatively, the switch107may couple the video adapter108to the display110. In either embodiment, the switch107is a switch, preferably mechanical, that allows the display110to be coupled to the system bus106, and thus to be functional only upon execution of instructions (e.g., virtual machine provisioning program—VMPP148described below) that support the processes described herein.

System bus106is coupled via a bus bridge112to an input/output (I/O) bus114. An I/O interface116is coupled to I/O bus114. I/O interface116affords communication with various I/O devices, including a keyboard118, a mouse120, a media tray122(which may include storage devices such as CD-ROM drives, multi-media interfaces, etc.), a printer124, and (if a VHDL chip137is not utilized in a manner described below), external USB port(s)126. While the format of the ports connected to I/O interface116may be any known to those skilled in the art of computer architecture, in a preferred embodiment some or all of these ports are universal serial bus (USB) ports.

As depicted, computer102is able to communicate with a software deploying server150via network128using a network interface130. Network128may be an external network such as the Internet, or an internal network such as an Ethernet or a virtual private network (VPN).

Application programs144in the system memory of computer102(as well as the system memory of the software deploying server150) also include a virtual machine provisioning program (VMPP)148. VMPP148may include code for implementing the processes described below, such as those described in reference toFIGS. 6-9. VMPP148is able to communicate with a vital product data (VPD) table151, which provides required VPD data described below. In one embodiment, the computer102is able to download VMPP148from software deploying server150, including in an on-demand basis. Note further that, in one embodiment of the present invention, software deploying server150performs all of the functions associated with the present invention (including execution of VMPP148), thus freeing computer102from having to use its own internal computing resources to execute VMPP148.

Also stored in the system memory136is a VHDL (VHSIC hardware description language) program139. VHDL is an exemplary design-entry language for field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and other similar electronic devices. In one embodiment, execution of instructions from VMPP148causes the VHDL program139to configure the VHDL chip137, which may be an FPGA, ASIC, or the like.

In another embodiment of the present invention, execution of instructions from VMPP148results in a utilization of VHDL program139to program a VHDL emulation chip151. VHDL emulation chip151may incorporate a similar architecture as described above for VHDL chip137. Once VMPP148and VHDL program139program VHDL emulation chip151, VHDL emulation chip151performs, as hardware, some or all functions described by one or more executions of some or all of the instructions found in VMPP148. That is, the VHDL emulation chip151is a hardware emulation of some or all of the software instructions found in VMPP148. In one embodiment, VHDL emulation chip151is a programmable read only memory (PROM) that, once burned in accordance with instructions from VMPP148and VHDL program139, is permanently transformed into a new circuitry that performs the functions needed to perform the processes of the present invention.

A cloud computing environment allows a user workload to be assigned a virtual machine (VM) somewhere in the computing cloud. This virtual machine provides the software operating system and physical resources such as processing power and memory to support the user's application workload.

FIG. 5depicts an exemplary blade chassis that may be utilized in accordance with one or more embodiments of the present invention. The exemplary blade chassis202may operate in a “cloud” environment to provide a pool of resources. Blade chassis202comprises a plurality of blades204a-n(where “a-n” indicates an integer number of blades) coupled to a chassis backbone206. Each blade supports one or more virtual machines (VMs). As known to those skilled in the art of computers, a VM is a software implementation (emulation) of a physical computer. A single hardware computer (blade) can support multiple VMs, each running the same, different, or shared operating systems. In one embodiment, each VM can be specifically tailored and reserved for executing software tasks 1) of a particular type (e.g., database management, graphics, word processing etc.); 2) for a particular user, subscriber, client, group or other entity; 3) at a particular time of day or day of week (e.g., at a permitted time of day or schedule); etc.

As depicted inFIG. 5, blade204asupports VMs208a-n(where “a-n” indicates an integer number of VMs), and blade204nsupports VMs210a-n(wherein “a-n” indicates an integer number of VMs). Blades204a-nare coupled to a storage device212that provides a hypervisor214, guest operating systems, and applications for users (not shown). Provisioning software from the storage device212allocates boot storage within the storage device212to contain the maximum number of guest operating systems, and associates applications based on the total amount of storage (such as that found within storage device212) within the cloud. For example, support of one guest operating system and its associated applications may require 1 GByte of physical memory storage within storage device212to store the application, and another 1 GByte of memory space within storage device212to execute that application. If the total amount of memory storage within a physical server, such as boot storage device212, is 64 GB, the provisioning software assumes that the physical server can support 32 virtual machines. This application can be located remotely in the network216and transmitted from the network attached storage217to the storage device212over the network. The global provisioning manager232running on the remote management node (Director Server)230performs this task. In this embodiment, the computer hardware characteristics are communicated from the VPD151to the VMPP148. The VMPP148communicates the computer physical characteristics to the blade chassis provisioning manager222, to the management interface220, and to the global provisioning manager232running on the remote management node (Director Server)230.

Note that chassis backbone206is also coupled to a network216, which may be a public network (e.g., the Internet), a private network (e.g., a virtual private network or an actual internal hardware network), etc. Network216permits a virtual machine workload218to be communicated to a management interface220of the blade chassis202. This virtual machine workload218is a software task whose execution, on any of the VMs within the blade chassis202, is to request and coordinate deployment of workload resources with the management interface220. The management interface220then transmits this workload request to a provisioning manager/management node222, which is hardware and/or software logic capable of configuring VMs within the blade chassis202to execute the requested software task. In essence the virtual machine workload218manages the overall provisioning of VMs by communicating with the blade chassis management interface220and provisioning management node222. Then this request is further communicated to the VMPP148in the computer system. Note that the blade chassis202is an exemplary computer environment in which the presently disclosed methods can operate. The scope of the presently disclosed system should not be limited to a blade chassis, however. That is, the presently disclosed methods can also be used in any computer environment that utilizes some type of workload management or resource provisioning, as described herein. Thus, the terms “blade chassis,” “computer chassis,” and “computer environment” are used interchangeably to describe a computer system that manages multiple computers/blades/servers.

FIG. 6is a diagram of a multi-node chassis240in accordance with one embodiment. The multi-node chassis240includes a chassis management module242and a plurality of compute nodes or ITEs250(ITE-1 through ITE-n). In accordance with various embodiments of the invention, the chassis management module (CMM)242includes control logic244for executing CMM level power management and workload management and control logic246for maintaining a matrix of power usage against performance for each subsystem of each compute node in the chassis240.

FIG. 7is a diagram of a compute node or ITE250in accordance with one embodiment. The compute node250includes a management controller252with control logic254for handling ITE communication to and from the ITE, such as communication with the chassis management module244(SeeFIG. 6). Outgoing communications may include, without limitation, power capping256to a processor260and memory262. Incoming communications may include, without limitation, power requirement triggers258from a processor power/utilization monitor264and a memory power/utilization monitor266.

FIG. 8is a diagram of the chassis management module240in communication with a group of compute nodes250that are each capable of controlling subsystem power allocation. Each compute node250has a management controller252that maintains a power matrix270. The power matrix270for a given compute node250identifies each subsystem, as well as the power capability (max power), allocation (subsystem power cap), and current use (power consumption) for each subsystem. As shown, the power matrix270includes only a processor subsystem and a memory subsystem (consistent withFIG. 7), but other subsystems may be included, so long as the compute node has the capability of power capping that subsystem and monitoring power consumption of that subsystem.

The chassis management module240includes a power management and workload management module244for handling virtual machines on the compute nodes250, and power matrix data246that represents the collection of data from the power matrices of all of the compute nodes250. The power matrix data246is used on the power management and workload management module244in its issuance of power caps to individual compute nodes, handling of power requests receiving from individual compute nodes (increases/decreases of power caps), and workload assignments to individual compute nodes. These aspects and functions may be performed consistent with any of the embodiments of the invention described herein.

FIG. 9is a flowchart of a method280in accordance with an embodiment of the present invention. Step282caps the amount of power available to each of a plurality of compute nodes. In step284, power allocation among subsystems within each of the plurality of compute nodes is managed according to the requirements of workloads assigned to each of the plurality of compute nodes. An actual performance level and performance capability for each of the subsystems within each of the plurality of compute nodes is reported in step286. Step288monitors parametric data for a particular workload.

A target compute node from among the plurality of compute nodes is identified in step290, wherein the target compute node would be capable of performing the particular workload if power was reallocated from a first subsystem to a second subsystem within the target compute node. Then, in step292the particular workload is assigned to the target compute node.