Distributed power budgeting

A distributed power management computer program product is configured to collect power consumption data that indicates power consumption by at least a plurality of the components of a node. The program code can be configured to provide, to each of a plurality of controllers associated with a respective one of the plurality of components, the power consumption data. The program code can be configured to determine a node power consumption. The program code can be configured to determine a power differential as a difference between the node power consumption and an upper power consumption threshold of the node. The program code can be configured to determine a proportion of the node power consumption consumed by a first component. The program code can be configured to compute a local power budget for the first component.

BACKGROUND

Embodiments of the inventive subject matter generally relate to the field of computer system power management and more particularly, to distributed power management informed with global power consumption information.

Various techniques have been used in an attempt to reduce power consumption while maintaining system performance. For example, some systems use dynamic voltage and frequency scaling (DVFS). DVFS modifies the voltage supplied to the system and the frequency at which the processor operates. DVFS may allow a system to operate in low-power states, and only have increased power consumption when necessary. For example, during a high workload period, the voltage supplied to the system and the frequency at which the processor operates may be high. During a low workload period, DVFS may decrease the voltage supplied to the system and decrease the frequency at which the processor operates. In addition to DVFS, some systems utilize power capping mechanisms. Such mechanisms impose an upper limit, or cap, on the power consumption of the system.

SUMMARY

Embodiments of the inventive subject matter include collecting, from each of a plurality of controllers of a node having a plurality of components, component power consumption. Each of the plurality of controllers is associated with a respective one of the plurality of components. The component power consumptions are provided to each of the plurality of controllers. A node power consumption for the node is determined based, at least in part, on the component power consumption. The power cap is determined for the plurality of components. A power differential power is determined as a difference between the node power consumption and the power cap for the plurality of components. A proportion of the node power consumption consumed by the component is determined based on the component power consumption of the component. A local power budget is computed for the component based, at least in part, on the power differential and the proportion of the node power consumption consumed by the component.

Embodiments of the inventive subject matter include an apparatus comprising a plurality of processing components. A distributor to provide to each of the plurality of controllers, an indication of power consumption for the apparatus based, at least in part, on component power consumptions of the plurality of processing components. A plurality of controllers, each of which is associated with a respective one of the plurality of processing components. A power differential is determined by the plurality of controllers as a difference between a power cap for the apparatus and a power consumption sum based, at least in part, on the component power consumptions. A power consumption sum attributable to the respective one of the plurality of processing components associated with the controller is determined by the respective one of the plurality of controllers. A local power budget for the respective one of the plurality of processing components associated with the controller is computed based, at least in part, on the power differential and the proportion of the power consumption sum attributable to the respective one of the plurality of processing components associated with the controller.

DESCRIPTION OF EMBODIMENT(S)

The description that follows includes exemplary systems, methods, techniques, instruction sequences and computer program products that embody techniques of the present inventive subject matter. However, it is understood that the described embodiments may be practiced without these specific details. For instance, although examples suggest a 1:1 relationship between component controllers and components, a component controller can be associated with multiple components. In other instances, well-known instruction instances, protocols, structures and techniques have not been shown in detail in order not to obfuscate the description.

Often times, power management mechanisms are centralized—that is, a central controller or microprocessor manages the power supplied to each component of the system. Centralized power management can be effective in controlling power consumption of the system, but can also be costly from a performance standpoint. Because the central controller or microprocessor manages power allocation for each component, the central controller or microprocessor is heavily occupied with managing power consumption for the multiple components of the system. In systems where power allocation is modified frequently, power management spends resources computing the power allocations for the components and incurs communication overhead for communicating the power allocations throughout the system

Distributing control and management of power budgets for components allows for scalability in a complex system with a multitude of components, as well as reducing processing overhead. Information about power consumption by the multitude of components in a system is collected, and then distributed to controllers for each of the components. This distribution of system wide power consumption information provides a controller of a component a node level view of power consumption. For this description, a node includes a group of components. Examples of a node include a server, a rack server, a board with multiple microprocessors, a blade server, a multi-core processor, etc. Examples of components include a processor, a memory module, an input/output hub controller, a processor core, core memory, etc. With the power consumption information, the controller for the component (“component controller”) can compute a power budget for the component based on a proportion of the node power consumption attributable to the component. This component level proportional power budgeting allows for consistent performance. Components using a greater amount of power will be given greater component power budgets and components consuming smaller amounts of power (e.g., idle components) will be given smaller component power budgets. The greater power budgets allow working components to consume greater amounts of power when available and to reduce the impact of reductions when node power consumption exceeds the node power cap. Distributing the task of determining the component power budgets avoids a single point of failure in power management. Tasking a processor with distributing the power consumption information instead of computing several power budgets frees resources (e.g., cycles) for other tasks. Or a less resource intensive component can be tasked with distributing the power consumption information.

FIG. 1depicts a conceptual diagram of an example node with distributed local power budgeting based on node level visibility of power consumption. The node ofFIG. 1comprises a power subsystem104, multiple components, and a distributor.FIG. 1only depicts three components102,110, and112to avoid adding unnecessary complexity to the figure. Each of the components102,110, and112has respective component controllers108,114, and116. The component controllers can be implemented with any one of a proportional controller, a proportional-integral controller, and a proportional-integral-derivative controller.FIG. 1depicts example operations at stages A-D. The stages are examples and are not necessarily discrete occurrences over time (e.g., operations of different stages may overlap).

At a stage A, the power subsystem104collects component power consumption data from each of the component controllers108,114, and116at periodic intervals. For instance, the power subsystem104collects power consumption data from all of the components controllers108,114,116within a time period x. The power consumption data indicates power consumption that can be attributed to a single component. But the power may be consumed by the component and sub-components and/or other coupled components. For instance, power consumption data may indicate an amount of power consumption attributable to a processor component. But the power is actually consumed by the processor component, off-chip cache of the processor component, and random access memory used by the processor component. Collection of the power consumption data can be in accordance with a push method, pull method, or a combination of both. The component102,110, and112can be configured to automatically transmit component power consumption data to the power subsystem104at periodic intervals. The power subsystem104can actively request component power consumption data from the component(s)102,110, and112at periodic intervals. The power subsystem104also collects background power consumption data (e.g., fan power consumption). In addition to collecting power consumption data, the power subsystem104filters the power consumption data to remove electrical noise that may occur in the power consumption data.

At stage B, the power subsystem104supplies the distributor106with the collected power consumption data. Again, a pull method, push method, or combination of push and pull may be used to convey the collected power consumption data from the power subsystem104to the distributor106. For instance, the distributor106may poll the power subsystem104for the collected power consumption data at the expiration of each periodic interval. As another example, the power subsystem104may send a message (e.g., interrupt) to the distributor106in response to collecting the power consumption data from the components and the background power consumption data. At stage C, the distributor106distributes the collection of component power consumption data to the component controllers108,114, and116. If the upper power consumption threshold for the node (“node power cap”) is not already known or readily accessible by each controller (e.g., the node power cap is set at boot-up in memory available to the controllers), the distributor106also communicates the node power cap to each of the component controllers108,114, and116.

At stage D, each of the component controllers108,114, and116compute a proportional component power budget for their associated components102,110, and112. The below tables illustrate proportional component power budgets based on node power consumption.

Table 1 indicates component power consumptions for a node. The node has four cores and eight components (two components per core—a processor component and a memory component). The power consumption of each component (in Watts) during the time period 0 is indicated in the “T0” column. During time period 0, Core0consumed a total of 260 W (processor component: 170 W, memory component 90 W). As indicated by the “Workload” column, Core0was working and not in an idle state during time period 0. Cores1-3, however, were in an idle state during time period 0, and consumed less power than Core0(160 W per core—each processor component of Cores1-3: 110 W, each memory component of Cores1-350 W). Table 1 also includes the background power consumed during time period 0. The node power consumption was 840 W during time period 0 (including background power consumption of 100 W).

Assuming the node has a node power cap of 1000 W, the node underutilized the total power capacity by 160 W during time period 0. After receiving the power consumption data (depicted in Table 1), each of the component controllers of the node calculates a component power budget for time period 1. Table 2 shows a computation of proportional component power budgets on a pro-rata basis.

As discussed previously, the excess power to be allocated for time period 1 is 160 W. During time period 0, Core0's processor component consumed 170 W. This represents roughly 20% of the node power consumed during time period 0. Therefore, Core0's processor component will be budgeted roughly 20% of the 160 W of excess power during time period 1, or roughly 32 W of the excess power. Core0's processor component will be budgeted roughly 202 W for time period 1.

The above example illustrates the efficiency of locally computing power budget computations relative to a centralized approach. Instead of one of the cores or another processing being burdened with computing the power budgets for 4 cores (i.e., 8 components), each component controller computes a power budget for only two components. Consequently, each component controller makes only five power budgeting calculations per time period:1. Determine the reciprocal of the node power consumption (“1/Node”)2. Determine the processor component's fractional power consumption for time period 0 (“Frac.p”) by multiplying the processor component's power consumption for time period 0 by 1/Node3. Determine the processor component's proportion of the excess power for time period 1 by multiplying the excess power available for time period 0 by Frac.p4. Determine the memory component's fractional power consumption for time period 0 (“Frac.m”) by multiplying the memory component's power consumption for time period 0 by 1/Node5. Determine the memory component's proportion of the excess power for time period 1 by multiplying the excess power available for time period 0 by Frac.m
These five calculations are estimated to consume less than 100 cycles. Assuming 2 ns per cycle, the time consumed by a process requiring 100 cycles is 200 ns. If, for example, each time period is 250 μs, at less than 100 cycles, the processing time for each component controller to calculate power budgets is less than one percent of the 250 μs time period (<200 ns/250 μs→ or <0.8%). This results in a processing consumption of less than 1% in terms of the component controllers instructions per second.

In some cases, a node may consume more power than the given total power capacity defined or set for the node. Embodiments can use the distributed computation of proportional component power budgets to throttle power consumption in a proportional manner across the components. An example of this is depicted in Tables 3 and 4.

Table 3 indicates component power consumption for a node during a time period 2. Again, the node has four cores and eight components (two components per core—a processor component and a memory component). Unlike Table 1, Table 3 indicates that both Core0and Core1are working and not in an idle state during time period 2. During time period 2, Core0and Core1each consume a total of 340 W (each processor component: 220 W, each memory component: 120 W). As in Table 1, both Core2and Core3were in an idle state during time period 2, and consumed less power than Core0and Core1(160 W per core—each processor component of Cores2-3: 110 W, each memory component of Cores2-3: 50 W). The node power consumption was 1100 W during time period 2 (including background power consumption of 100 W).

Again, assuming the node has a defined node power cap of 1000 W, the node exceeded the node power cap by 100 W during time period 2. After receiving the power consumption data (depicted in Table 3), individual component controllers of the node calculate component power budgets for time period 3. Table 4 shows a computation of proportional component power budgets on a pro-rata basis. It should be noted that background power is not reduced. Consequently, to bring the node power consumption within the 1000 W node power cap, the node power consumption, excluding background power consumption, must be brought below 900 W. In other words, the component power budget of each component for time period 3 will be reduced by a portion of the 100 W of excess power used based on their power consumption during time period 2.

As previously discussed, the excess power consumed by the node during time period 2 was 100 W. Consequently, node power consumption for time period 3 will be reduced by 100 W. During time period 2, Core0's processor component consumed 220 W. This represents roughly 20% of the node power consumed during time period 2. Core0's processor component's power budget for time period 3 will be reduced by roughly 20% of the 100 W of excess power consumed during time period 2, or roughly 20 W. Core0's processor component will be budgeted roughly 200 W for time period 3.

The power budgeting discussed can conserve resources in several scenarios. In a first scenario, a component is consuming relatively little power and the node power consumption is below the node power cap. Because the node power is below the node power cap, it is not necessary for a control system to be engaged. Thus, the component is able to run at its maximum performance.

In a second scenario, a component is consuming relatively greater power and the node power consumption is below the node power cap. Because the node power consumption is below the node power cap, it is not necessary for a control system to be engaged. Thus, the component is able to run at its maximum performance.

In a third scenario, a component is consuming relatively little power and the node power consumption is above the node power cap. Because the node power consumption is above the node power cap, a control system will be engaged and the node power consumption will be reduced. In this scenario, the component controller associated with the component consuming relatively little power can decrease the power budget for the component over subsequent time periods, aiding in bringing the node power consumption below the node power cap. Additionally, if the component consuming relatively little power experiences a sudden increase in workload, the node power cap may not be breached based on the budget given to that component during lower workload periods. This can prevent the node power consumption from breaching the node power cap when components experience a sudden increase in workload.

In a fourth scenario, a component is consuming relatively greater power and the node power consumption is above the node power cap. Because the node power consumption is above the node power cap, a control system will be engaged and the node power consumption will be reduced. In this scenario, the component controller associated with the component consuming relatively greater power can decrease the power budget to the component over subsequent time periods, aiding in bringing the node power consumption below the node power cap. Because the power budgeting is proportional, the component consuming relatively great power will still be budgeted a large portion of the power, relative to the other components. Once below the node power cap, the power budget of the component consuming relatively great power can be increased to increase performance. If the node power cap is once again breached, the process can repeat itself.

FIG. 2is a flow diagram of example operations for local updating of component power budgeting based on visibility of node power consumption in a given time period. The flow begins at block202.

At block202, the distributor obtains node power consumption data, including component power consumptions and background power consumption, from a power subsystem. A variety of implementations are possible for the distributor to obtain the node power consumption data. For instance, the power subsystem may signal the distributor when all of the node power consumption data for a given time period is available to be read by the distributor. Alternatively, the power subsystem can write the node power consumption data in a batch or incrementally (e.g., write the data as it is received from each component controller) to a memory location accessible by the distributor or a memory location that is part of the distributor. The flow continues at block204.

At block204, the distributor calculates the node power consumption. The distributor computes a sum of the component power consumptions and the background power consumption. The flow continues at block206.

At block206, the distributor determines the node power cap defined for the node. The node power cap may be set in many ways. For example, the node power cap may be set based on the node's physical constraints. The node power cap may be set in accordance with a policy or firmware. The flow continues at block208.

At block208, the distributor provides the node power consumption and the node power cap to each of the component controllers. The distributor can broadcast the node power consumption and the node power capacity to each of the component controllers. For instance, the distributor sends messages or packets carrying the node power consumption and the node power cap to the component controllers. As another example, the distributor writes the node power consumption and node power cap into registers of each of the component controllers. The flow continues at block210.

The operations from block210and subsequent to block210represent operations of one of the component controllers. At block210, the component controller obtains the node power consumption and the node power cap from the distributor. As stated with respect to block208, the component controller may receive the node power consumption and the node power cap over a bus, read the node power consumption and the node power cap from a local register, etc. The flow continues at block212.

At block212, the component controller calculates the component power budget for a component(s) associated with the component controller for a next time period. This calculation is performed based on the node power consumption for the given time period and the proportion of the node power consumption used by the component(s) during the given time period. The flow continues at block214.

At block214, the calculated component power budget is used for the associated component for the next time period. For instance, the controller activates an actuator to modify operation of a component in accordance with the calculated component power budget for the succeeding time period. The controller itself may actuate the change in power consumption, or the controller may pass an indication of the calculated component power budget to another device that modifies operation of the associated component accordingly.

FIG. 3depicts a conceptual diagram of an example node with distributed local power budgeting based on node level visibility of power consumption communicated with a notification technique. The node ofFIG. 3comprises a power subsystem304, multiple components302,312, and314, a distributor306, and memory310. The memory310may be shared across the components302,312,314. The memory310may be distinct memories accessible by individual ones of the components and the distributor306.FIG. 3only depicts three components302,312, and314to avoid adding unnecessary complexity to the figure. Each of the components has respective component controllers308,316, and318.FIG. 3depicts example operations depicted as occurring over stages A-F. The stages are examples and are not necessarily discrete occurrences over time (e.g., operation of different stages may overlap).

At stage A, the power subsystem304collects component power consumption data from each of the component controllers308,316, and318and background power consumption at periodic intervals. The background power consumption is not necessarily collected from the component controllers308,316, and318. The power consumption data indicates power consumption that can be attributed to a single component. But the power may be consumed by the component and sub-components and/or other coupled components. For instance, power consumption data may indicate an amount of power consumption attributable to a processor. But the power is actually consumed by the processor, off-chip cache of the processor, and random access memory used by the processor. Collection of power consumption data can be a push method, pull method, or a combination of both. The components302,312, and314can be configured to automatically transmit component power consumption data to the power subsystem304at periodic intervals. The power subsystem304can actively request component power consumption data from the components302,312, and314at periodic intervals. The power subsystem,304also collects background power consumption data (e.g., fan power consumption). In addition to collecting power consumption data, the power subsystem304filters power consumption data to remove electrical noise.

At stage B, the power subsystem304supplies the distributor306with the collected power consumption data. Again, a pull method, a push method, or a combination of push and pull may be used to convey the collected power consumption data from the power subsystem304to the distributor306. For instance, the distributor306may poll the power subsystem304for the collected power consumption data at the expiration of each periodic interval. As another example, the power subsystem304may send a message (e.g. interrupt) to the distributor306in response to collecting the power consumption data, including the background power consumption data.

At stage C, the distributor306writes the collection of power consumption data to memory310. For example, the distributor306writes the collection of power consumption data to a reserved section of the memory310. Or the distributor306writes the collection of power consumption to available free space in the memory310.

At stage D, the distributor306notifies the component controllers302,312and314that the collection of power consumption data has been written to the memory310. The notification indicates the address or addresses of the memory310in which the data has been written. If the power cap for the node is not already known by each controller (e.g., the power cap is set at boot-up in memory accessible by the controllers, is defined in a register by firmware, etc.), the distributor306also writes the node power cap to the memory310and notifies the component controllers302,312, and314of the location in memory in which the node power cap has been written.

At stage E, the component controllers308,316, and318access memory310and retrieve the collection of power consumption data and the node power cap. For example, the component controllers308,316, and318access the memory310with the address provided by the distributor306. As another example, the component controllers308,316, and318access the memory310with an address for a reserved section of the memory310that is stored in a local register.

At stage F, each of the component controllers308,316, and318compute a proportional component power budget for their associated components302,312, and314. As discussed previously (see discussion ofFIG. 1and Tables 1-4), the component power budgets are based on node power consumption awareness at a component level.

FIG. 4is a flow diagram of example operations for local updating of component power budgets based on visibility of node power consumption in a given time interval with a notification technique. The flow begins at block402.

At block402, the distributor obtains node power consumption data, including component power consumption and background power consumption from a power subsystem for the given time period. A variety of implementations are possible for the distributor to obtain the node power consumption data. For instance, the power subsystem may signal the distributor when all of the node power consumption data for a given time period is available to be read by the distributor. Alternatively, the power subsystem can write the node power consumption data in a batch or incrementally (e.g., write the data as it is received from each component controller) to a memory location accessible by the distributor or a memory location that is part of the distributor. The flow continues at block404.

At block404, the distributor determines the node power cap for the node. The node power cap may be set in accordance with different techniques as mentioned with respect toFIG. 2. The flow continues at block406.

At block406, the distributor writes the node power consumption data and the node power cap to memory. The distributor can write the collection of component power consumptions to a segment of the memory reserved for the power consumptions, the background power consumption to another reserved segment of the memory, and the node power cap to a different reserved segment of the memory. The flow continues at block408.

At block408, the distributor notifies the component controllers that the component power consumptions for the given time period and the total power capacity have been written to memory. The notification can be in the form of an interrupt, alerting the component controllers that component power consumption for the given time period and the total power capacity have been written to memory. The notification includes a location in memory from which the component controllers can retrieve the component power consumptions for the first given time period and the node power cap. Some embodiments reserve x segments of memory to preserve the power consumption data across x time intervals. Power consumption data for a time interval would not be overwritten until the x segments are filled. Some embodiments write timestamps with the power consumption data. The flow continues at block410.

The operations from block410and subsequent to block410represent operations of one of the component controllers. The component controllers asynchronously access the memory. At block410, the component controller obtains the node power consumption data and the node power cap from the memory. The flow continues at block412.

At block412, the component controller calculates the node power consumption for the given time period. The component controller computes a sum of the component power consumptions and the background power consumption for the given time period. The flow continues at block414.

At block414, the component controller calculates a component power budget for component(s) associated with the component controller for a next time period. This calculation is performed based on the node power consumption for the given time period and the proportion of the node power consumption attributable to the component(s) associated with the component controller during the given time period. The flow continues at block416.

At block416, the calculated component power budget is used for the next time period for the associated components. For instance, the component controller activates an actuator to modify operation of the associated component(s) in accordance with the calculated power budget for the next time period. The controller itself may actuate the change in power consumption, or the controller may pass an indication of the calculated component power budget to another device that modifies operation of the associated component(s) accordingly.

The flowcharts are provided as examples and are not intended to limit scope of the claims. For example, embodiments can pass the power consumption data from a distributor to a memory location through an adder that generates a sum of the component power consumptions and background power consumption into a memory space defined for a node power consumption. Embodiment can write the individual power consumptions, or only communicate the node power consumption without granularity of the component power consumptions to the component controllers. In addition, embodiments can configure the component controllers to compute component power budgets based on proportional node power consumption without the background power consumption. For instance, the power cap may be defined for non-background power consumption.

Although the examples in this description depict only one distributor in a node, embodiments can employ one or more additional distributors. For example, another processor may be designated as a backup distributor for a failover scenario. As another example, components of a node may be logically divided into different groups of components and a different distributor be assigned responsibility for each logical grouping of components.

Although the examples in this description depict power budgeting based on past power consumption, embodiments can employ power budgeting in an oversubscription scenario. For example, a system having a plurality of power supplies may lose one or more of the power supplies. In such a scenario, the component controllers may receive an interrupt request (“IRQ”). Upon receiving the IRQ, the component controllers can immediately throttle their associated components, decreasing the power consumed by their associated components, and thus the node power consumption. After receiving, from the distributor, a node power cap reflective of the power supply failure, the component controllers can calculate new power budgets for their associated components based on the node power cap reflective of the power supply failure.

The examples above also describe a distributor supplying the collected power consumption data to the component controllers. In some embodiments, the distributor may transmit the node power consumption (i.e., total power consumption by the node) to the component controllers without the breakdown of power consumption across controllers. In other embodiments, a distributor transmits both the node power consumption and the collection of component power consumption to the component controllers.

Although the examples above describe proportional power budgeting of excess power in a pro-rata manner, embodiments can employ proportional budgeting that is not strictly pro-rata. For example, certain components may be allocated a predetermined percentage of the excess, or may be allocated a predetermined percentage in addition to their pro-rata share. In other embodiments, certain components may be allocated a predetermined power level, or may be allocated a predetermined power level in addition to their pro-rata share.

FIG. 5depicts an example computer system500. The computer system500includes a processor having multiple cores502and512(possibly including other processors, additional cores, etc.). The computer system includes memory526. The memory526may be system memory (e.g., one or more of cache, SRAM, DRAM, zero capacitor RAM, Twin Transistor RAM, eDRAM, EDO RAM, DDR RAM, EEPROM, NRAM, RRAM, SONOS, PRAM, etc.) or any one or more of the above already described possible realizations of machine-readable storage media. The computer system also includes a bus504, a network interface522(e.g., an ATM interface, an Ethernet interface, a Frame Relay interface, SONET interface, wireless interface, etc.), and a storage device(s)528(e.g., optical storage, magnetic storage, etc.). The computer system500includes multiple cores502and512. Each core502and512has a processor component506and514, a memory component508and516, and a component controller510and518, respectively. The computer system500also includes a distributor520and a power subsystem524. The power subsystem524collects component power consumption data from the component controllers510and518, as well as background power consumption data for the computer system500. The power subsystem524communicates the component power consumptions and the background power to the distributor520. In some embodiments, the distributor calculates node power consumption and distributes the node power consumption to the component controllers510and518. The component controllers510and518then calculate and update the power budget for the components506,508,514, and516with which they are associated. Further, realizations may include fewer or additional components not illustrated inFIG. 5(e.g., video cards, audio cards, additional network interfaces, peripheral devices, etc.). The core502and512, the storage device(s)528, and the network interface522are coupled to the bus504. Although illustrated as being coupled to the bus504, the memory526may be coupled to the cores502and512.

Computer program code for carrying out operations for aspects of the present inventive subject matter may be written in any combination of one or more programming languages, including an object oriented programming language such as JAVA (an object-oriented, class-based computer programming language), SMALLTALK (an object-oriented, dynamically typed computer programming language), C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).