Multi-level power monitoring, filtering and throttling at local blocks and globally

Power management for a multi-processor chip includes a centralized global power manager that monitors global power for the whole chip, and local power managers. Local power managers manage power for local blocks such as processor cores, caches, and memory controllers. When a local block executes an instruction or accesses memory, an event is generated and looked up in a local power estimate table. A local power estimate for that event is sent to the global power manager, which sums all local power estimates received from all local blocks. An exponential moving average (EMA) is generated and compared to a global power threshold. When global power is over the threshold, local targets are sent to power managers that generate and monitor local power averages that must remain under the local target. The local block is throttled by the local power manager to reduce power when the local target is exceeded.

FIELD OF THE INVENTION

This invention relates to power management, and more particularly to global and local power management.

BACKGROUND OF THE INVENTION

Improvements in semiconductor processing technologies and design tools have allowed for more and more functions to be integrated together on a single integrated circuit (IC) chip. Such improvements allowed for cache memories to be integrated onto the same chip as a central processing unit (CPU) core. More recently, multiple CPU cores are being integrated onto the same chip along with one or more memories.

As this trend continues, multiple CPU cores and multiple memory blocks will be integrated together.FIG. 1shows a multi-processor system chip. Processor cores10,10′,10″ are integrated together onto IC chip20. Each processor core may execute a separate stream of instructions and each accesses its own local cache memory, caches12,12′,12″. When data is not found in the local cache memory (a cache miss), memory controllers14,14′,14″ fetch the desired data from an external memory, such as using an external bus to a large external main memory.

Snoop tags16contain directory information about the entries currently being stored in caches12,12′,12″. Cache coherency is achieved through the use of snoop tags16, perhaps in conjunction with external directories and other controllers.

The higher integration of functions onto a single silicon die is primarily achieved by using a higher density of transistors. Transistors generate heat as a by-product as do resistances. Transistors operating at a very high frequency may get too hot, even causing some localized melting or other damage, reducing long term reliability. Such heat is drawn away through a package containing the IC chip, but the amount of heat dissipated is limited.

Power management of highly integrated systems is critical to prevent such damage. Power management unit18may be included on IC chip20. Power management unit18may be activated by a sensor such as a temperature detector or by a software monitor or program to reduce power consumption on IC chip20. For example, power management unit18may cause processor cores10,10′,10″ to enter a lower-power state when the sensed temperature is above a limit. The low-power state may be a reduced frequency of operation or periodically halting operation to allow IC chip20to cool.

A wide variety of lower-power modes and power management units and techniques are known. However, most are relatively simple and are designed for use with simple systems, such as single-processor chips. While such single-processor power-management techniques could simply be replicated for each of the multiple CPU's on a multi-processor chip, the power management achieved would be disjointed and lack coordination among the many processor-memory clusters sharing the same IC chip die.

What is desired is power management for a more complex multi-processor chip. A power management system that can control multiple CPU cores and multiple memory blocks that can operate independently of one another is desirable. Centrally-coordinated power management of multiple processor-memory clusters is desirable.

DETAILED DESCRIPTION

The inventors have discovered that power consumption can be monitored on a global, or chip-wide level. Power consumption can then be controlled on a local level. Power is thus monitored globally but controlled locally.

The local level can have separate power monitors and controllers for each processor core and each memory block. Power is monitored for the whole chip by a single central power monitor at the global level, but controlled for many local blocks at the local level.

The global power consumption can be compared to one or more global power thresholds. When these global thresholds are exceeded, power must then be reduced. Power is reduced by sending local power targets to each local block. Each local block then monitors its own power consumption. When its local power consumption exceeds its local target, then the local block reduces its power consumption. Local power consumption can be reduced by local throttling, such as by reducing the operating frequency of the local CPU core or local memory block, or completely stopping one or more clocks.

The term “threshold” used herein refers to a limit compared at the global level. The term “target” used herein refers to a limit compared at the local level.

FIG. 2is a block diagram of a multi-processor chip with a global power monitor and multiple local power managers. Local blocks that perform functions include multiple CPU's, including processor core10, multiple caches such as cache12, and one or more memory controller14. As these local blocks perform functions, they generate events. Events may be generated in a variety of ways, including, such as when a processor fetches or executes an instruction, when a cache access occurs, when the memory controller accesses an external memory, or when a local temperature sensor measures a temperature exceeding a defined threshold. Events may be signaled when some number of such functions occur, such as one event being generated for every 8 instructions fetched, or for every 64 bytes read.

When processor core10generates an event, the event is looked up in local power table38. Local power table38stores power estimates for each kind of event. The local power estimate found in local power table38is sent to global power averager34in the global power management unit.

Likewise, when cache12generates an event, such as when cache12is read, the event is looked up in its local power table38′. Local power table38′ stores power estimates for each kind of cache event. The local power estimate found in local power table38′ is also sent to global power averager34in the global power management unit.

When memory controller14generates an event, the event is looked up in local power table38″ for memory controller14. Local power table38″ stores power estimates for each kind of event. The local power estimate found in local power table38″ is also sent to global power averager34in the global power management unit.

All the local power estimates from all the local power tables are summed by global power averager34. An average such as a running average or exponential moving average (EMA) is also generated by global power averager34. The average power filters out fluctuations in the instantaneous sum of local power estimates.

This average power from global power averager34, the filtered sum of all local power estimates from local blocks, is the global power estimate. The global power estimate is sent to central power manager30and compared to one or more global power thresholds from global power table32. When the global power estimate is below the global power thresholds, the chip is operating safely below the power limits and is not in danger of overheating. Operation can continue, with the local blocks continuing to estimate their local power and sending these local power estimates to global power averager34for summing and averaging. Central power manager30compares the global power estimate from global power averager34to global power thresholds from global power table32to ensure that the global power thresholds are not being exceeded.

When the global power estimate exceeds the global power thresholds from global power table32, then power must be reduced. Local power targets are read from global power table32and are sent to local power tables38,38′,38″. Local power managers36,36′,36″ are also activated by central power manager30.

Once activated, local power managers36,36′,36″ compare local power estimates to the local power targets stored in local power tables38,38′,38″. Local power managers36,36′,36″ filter and average the local power estimates from local power tables38,38′,38″ that are generated by events from the local block. When the local power average exceeds the local power target, local power managers36,36′,36″ throttle their local block to reduce its power consumption.

For example, processor core10may be executing instructions at a high rate, sending many events to local power table38and causing the local power estimate generated by local power manager36to rise above the local power target for the local block, CPU_A. Then local power manager36throttles processor core10to reduce its power consumption.

Each of local power managers36,36′,36″ operates independently of the others. Thus local power manager36′ may not have to throttle cache12while local power manager36is throttling processor core10. The local power targets can be separately set by the system's programmer. This allows high-power local blocks such as processor core10to have a more restrictive local power target than other local blocks such as cache12. It may be more effective to throttle a high-power local block than a local block that draws little power. The local power target for low-power local blocks can be set high enough so that the low-power local blocks do not have to be throttled.

FIG. 3is a flowchart of operation of global power estimates and local power targets using the global power monitor and multiple local power managers ofFIG. 2. Each of the many local blocks generates a local power estimate as events occur, step130. These local power estimates are sent to the central power manager and summed and averaged or filtered, step132, to generate a global power average.

The current global power average is compared to one or more global power thresholds or limits, step134. When the current global power average exceeds a global power threshold, step135, then power throttling by local blocks is activated. Local power targets are sent from the central power manager to the local power managers, step136.

Once activated, each local power manager generates a local power average and compares it to the local power target for that local block. When the local power average is over the local target, the local power manager throttles the local block to reduce its power consumption, step138. Power may be throttled by inserting or increasing a number of idle cycles for a period of time, by reducing execution frequency, stopping one or more clocks, pausing or delaying operation or memory accesses, etc.

FIG. 4shows the centralized global power manager in more detail. Local power estimates from the many local blocks are received by global power adder42, which forms a sum of all local estimates. This sum is the global power estimate for the current period of time, or sample period, and may be thought of as an instantaneous power estimate.

Averager44generates an exponential moving average (EMA) from the new global power estimate from global power adder42. An old EMA, such as for the last sampling period, is read from a record in global power table32, and a decay parameter is also read from the record. The old EMA and decay parameter are used by averager44to calculate the new average power. Bit-shifts may be used by averager44in response to the decay parameter to vary the amount (decay period) or speed of the averaging.

The new average power EMA from averager44is stored in the record in global power table32as the current EMA, over-writing the old EMA that was read. Comparator48compares the current EMA generated by averager44to the power threshold from the record read from global power table32. When the new average power EMA is above the power threshold, central power manager30is alerted.

Central power manager30can then react to the global power average going over the global power threshold by activating local throttling. Local power targets are read from the record in global power table32and sent to the local power managers for the local blocks. The local power managers then monitor their local power estimates and throttle or reduce the power consumption of their local block when their local power target is over the local target.

The local power targets may differ for the various local blocks. For example, local targets for cache blocks may be higher or less restrictive than power targets for processor cores. Local power targets for some processor cores may differ from targets for other processor cores. Some local targets may be set to a maximum value to effectively disable local throttling for those local blocks.

More than one global power average may be generated. For example, one global power average may have a relatively slow averaging period of 1 second, while another global power average has a faster averaging period of 1 milli-second (1 ms). Each global power average has a separate record in global power table32.

For example, first record52has a 1 ms averaging (decay) period while second record54has a 1 second averaging period. The global threshold for first record52is set to correspond to 100 Watts, while the global threshold for second record54is set to a lower 80 Watts. The second record is the system cooling limit and has a longer averaging time period, while the first record is a power-supply limit and has a faster averaging period.

Another (third) record could be added with an even faster averaging period to represent a capacitive limit of power that can be instantaneously supplied by capacitors, which is a higher instantaneous power than the power supply can supply over a longer period of time. A guard-band record could be added with a lower threshold to activate less-severe local power targets before a higher threshold is reached when more severe local power targets must be applied. The decay periods for the guard-band record could be the same as the first or second record, but have different values for the local power targets.

Local estimates may remain valid until the next time a global sum is generated. Alternately, for longer time periods in which local estimates change, global power adder42may keep a running sum of all local estimates as local events generate local power estimates that are sent to global power adder42and accumulated over the sampling period. Then several EMA's may be generated from one global power estimate from global power adder42, but with different decay periods or shifts.

FIG. 5shows a local power management unit in more detail. As events occur in the local block, such as cache or memory accesses or instruction fetching or execution, the event is looked up in local power estimate table50. Estimated power or energy that is consumed by processing an event is stored in local power estimate table50for each kind of event that the local block can perform. When the local block performs the event, the power estimate for that event is output from local power estimate table50as the local power estimate. This local power estimate is sent to the global power manager and is summed by global power adder42(FIG. 4).

Ideally, normal operation does not cause the global power thresholds to be reached. Then local monitoring and throttling do not have to be activated. The local block simply looks up events in local power estimate table50and sends these estimates to the global power manager for summing. Power monitoring is performed globally.

However, when the global power threshold is exceeded, power must be reduced. Then the local power managers are activated and local power targets are sent. Each local power manager must monitor its local power and throttle the local block's operation to stay under the local power target.

When one of the global power thresholds is exceeded, a local power target is read from global power table32(FIG. 4) and sent to the local block's power manager to be stored in local power target table62. Several local targets may be stored in local power target table62in different records. For example, fast record56contains power target A1and uses a 1 ms averaging period, while slow record58stores a different local power target A2with a slower 1 second averaging period. EMA_A1is the faster 1 ms local power average stored in fast record56, while EMA_A2is the slower 1 second local power average stored in slow record58.

Averager64generates an exponential moving average (EMA) from the new local power estimate from local power estimate table50. An old local EMA, such as for the last sampling period, is read from a record in local power target table62, and a decay parameter is also read from the record. The old local EMA and local decay parameter are used by averager64to calculate the new average power. Bit-shifts may be used by averager64in response to the decay parameter to vary the amount (decay period) or speed of the averaging.

The new local power EMA from averager64is stored in the record in local power target table62as the current EMA, over-writing the old EMA that was read. Comparator68compares the current EMA generated by averager64to the local power target from the record read from local power target table62. When the new local power EMA is above the local power target, local power manager60is alerted.

Local power manager60reacts to the local power average going over the local power target by activating local throttling of the local block. Local power manager60can throttle or reduce the power consumption of their local block by a wide variety of methods. For example, local power manager60may activate inserting of a predetermined number of idle cycles over a period of time, or halting the clock to the local block for a period of time, such as X cycles for every 100 cycles. One or more clocks to the local block may also be slowed down or stopped, or instruction dispatches or memory accesses delayed.

FIGS. 6A-Bare a flowchart of global power monitoring and local throttling.FIG. 6Ashows global monitoring. When an event occurs in a local block such as a processor or a cache, step102, then the event is looked up in that local block's local power estimate table, step104. The local power estimate is read from the local power estimate table and sent to the centralized global power manager, step106.

The global power manager receives local power estimates from many local blocks and sums these local power estimates, step108. A global power estimate is obtained for the current sampling period. There may be several different global power thresholds programmed into records in a global power table, and different decay periods for EMA's may be used. A stored EMA for each record is read, and the current global power estimate is averaged into the old EMA to get a new EMA that is stored in the record, step110.

The averaging period for that record's EMA determines the shift amount or relative weighting of the current estimate and the old EMA. Slow averages use a smaller weighting of the current global power estimate and hence take longer to adjust, while faster averages use a larger weighting of the current power estimate and can change more rapidly.

The new EMA is compared to the global power threshold for that EMA's record in the global power table, step112. When the threshold is not exceeded, global monitoring continues by repeating the steps ofFIG. 6A.

When the global threshold for that EMA's record in the global power table is exceeded, step112, then the steps ofFIG. 6Bare taken. The local power targets are read from the over-threshold EMA's record in the global power table and send to the local blocks for storage in the local power target tables. The local power managers are activated to monitor and throttle the local blocks, step116. Local EMA's are generated from the local power estimate by the local power manager for different averaging periods, step118.

The local EMA's are compared to the local targets, step120. Local monitoring continues when the local targets are not exceeded. When a local target is exceeded, step120, then local throttling occurs, step122. Local throttling can be achieved by reducing a clock rate, access, or execution rate of the local block, stopping one or more clocks, or by powering down the local block for a period of time, or by using a lower-power mode for the local block. Lower-power modes for the local block may include techniques such as reducing the operating voltage(s) or adjusting the substrate bias voltage(s) for the local block. Local monitoring continues to determine if the local throttling was effective. The local throttle can be turned on and off as the local average oscillates above and below the local target.

FIGS. 7A-Bhighlight generation of an exponential moving average (EMA) of a power value. InFIG. 7A, a new power estimate may be a local power estimate or may be a global power estimate that is the sum of many local power estimates. Shifter74shifts the new power estimate to the right by a number of bit-positions that is equal to a decay value. For a sample window of N samples or clock periods, the decay DECAY may be set as:
2^DECAY=(N−1)/2

Register70holds the old EMA value of this power estimate. The old EMA value may be read from a power table. Adder73sums the un-shifted old EMA from register70with the shifted new power estimate. Shifter75right-shifts the old EMA from register70by a number of bit-positions DECAY that is the same shift as performed by shifter74. This shifted old EMA is subtracted by subtractor72from the sum from adder73to generate the new EMA for this power value. The new EMA can be stored in register70and written as the new EMA to a record in a power table.

FIG. 7Bshows an example of this operation. The new sample, or new power estimate is shifted to the right by one bit position. The old EMA is likewise right-shifted by one bit-position before subtracting, but not before adding. This has the same effect as exponentially averaging the new power estimate with the old EMA.

The EMA operation is simple to implement with a programmable processor or with hardwired logic, yet provides an accurate average for use in power management. The amount of averaging can be adjusted by adjusting the decay parameter DECAY.

ALTERNATE EMBODIMENTS

Several other embodiments are contemplated by the inventors. For example other kinds of local or global filters may be used besides EMA, such as a normal average of the last N samples. Local or global filters that are priority encoding filters or filters that drop or ignore the smallest local power estimates and only process the inputs can be used. The sum preceding the filter may ignore these smallest inputs, especially when they are much smaller than the largest inputs. Thus the sum may be a rough sum or estimated sum rather than a fully-inclusive and exact sum.

Local blocks that perform functions could be other kinds of blocks such as function-specific blocks and other processors besides CPU's, such as digital-signal processor (DSP) processors, caches, snoops, bus-interface units, I/O controllers, or memory controllers. Examples of local blocks that perform functions may include cache snoop tags, cache directory controllers, compression/decompression engines (codecs), security acceleration engines including those for secure-sockets layer (SSL), IO controllers, off-chip bus controllers, etc. These local blocks, which perform functions, may thus perform a wide variety of functions, including programmable, instruction-processing, memory lookup and referencing, security processing, compression, decompression, input-output, communications, and other functions. There may be other blocks included on the chip that do not include local power managers and thus do not participate in local or global power management.

The central power manager could perform global throttling. For example, the power-supply voltage to the entire chip could be reduced, or the global clock slowed down. The central power manager could receive external signals such as a temperature indicator or an external alarm signal from a power supply or power controller. A record in global power table32could have local targets that are activated when the external alarm or event occurs. Alternately, the external event could alter one or more global EMA's or could be fed into one of the local block's local power estimate table50, perhaps being given a large weight. Multiple targets and multiple thresholds could be activated. Then the target or threshold with the lowest value may be triggered before those with higher values. A programmable local power target that is not revised by the central power manager may be added. This allows a local block to be power managed regardless of global management.

The central power manager may perform priority encoding of local power estimates rather than summing and filtering the local power estimates. The central power manager could receive local estimates and look then up in a table to determine what action to take. Various other kinds of central management may be performed.

Some events may not have an entry in local power estimate table50or may use a generic entry and a generic estimate. The power estimates may vary in accuracy and may be quite inaccurate for some events while being more accurate for other events. Local power managers could receive inputs from local sensors, such as a local temperature sensor for the temperature of the local block. Temperature inputs could generate events or replace local events or act as additional local events.

The local power estimates do not have to arrive at the global power averager at exactly the same time. The global power averager could keep a running sum of all local estimates that arrive during a period of time, such as before the next average is generated. Various pipelining may also be used. The format of records in the power tables may vary significantly, and records and data may be re-arranged or stored in nested structures such as linked lists or relational tables in a database. The power table itself could be stored on-chip or externally off-chip, or a combination thereof. The decay parameter may be allowed to have just a few values and may not necessarily be stored in the power table.

Comparison of thresholds and generation of filtered power averages could be performed at different times. For example, in the case of EMA, first all EMA's are generated for all decay periods, then each EMA is compared to the thresholds. Rather than store all local power targets in global power table32, the local power targets could be stored in local power target tables62and then activated when needed.

The power estimates, thresholds, and targets may represent real energy or power values such as in Joules or Watts, or may be arbitrary values or weights that are relative to one another. The exact values may be set by simulation and later adjusted by experimentation or evaluation of actual chips. The programmer or system software may adjust these values by writing new values to global power table32. Some records may be disabled, such as by using a disable bit or programming thresholds to a predetermined values such as 0000 or FFFF. The term power is sometimes used broadly to include energy, and sometimes may refer to energy consumed over an indefinite period of time rather than energy over a predetermined period of time.

Local power tables could have several records with several local targets and local EMA's, or could have just one target. Combination of events could also have an entry in the table. Different kinds of accesses or instructions could have different power estimates in local power table38. Each local block can have a different local power table38, or local power table38could be shared among several local blocks such as a cluster of one or more local blocks. The same or differing values may be programmed into local power tables38for different local blocks. A local or global power table may be programmable by software, or may be pre-defined, such as when constructed as a PLA, ROM, electrical fuses, laser fuses, or other non-software programmable structures.

Local power target table62could also contain fields that indicate the type of local throttling to be performed when the local target is exceeded. For example, a field could indicate the percentage of idle cycles to insert. More idle cycles could be inserted for violation of the slow-average local power target than for the faster-average target. Thus the amount of throttling is then increased when the local target for the slow moving average is violated. A field could indicate an amount to alter or an absolute target for the operating or substrate bias voltages.

Some processor cores may be allocated higher local power targets than other processor cores. For example, a first processor core may perform more critical operations such as executing operating system commands, while other processors execute instruction threads for less-important user programs. The power target for the first processor can be set higher than the local power targets for the thread-executing processors, causing the thread-processing processors to be throttled more than the first processor.

Rather than have the global summer sum all local power estimates from all local blocks, a subset of the local power estimates may be summed. For example, only power estimates from processor blocks may be summed while cache power estimates are ignored. More than one kind of global power average may be generated. One power average may include the sum of estimates for all processor blocks, while another power average may include the sum of estimates for cache blocks but not for processor blocks. This allows power to be separately monitored for processor and for cache local blocks.

The power estimate from the memory controller may include power consumed off-chip by an array of DRAM memory chips or memory modules. Other kinds of local blocks may also have their power estimated. For example, an I/O controller block may have its power estimated. The estimate may or may not include external power consumption, such as by external line or bus drivers.

Local power management may only be needed in rare circumstances, but not during normal operating loads. The local targets may be applied during exceptionally high load times and later removed after a period of time of lower global average power. The low-power modes of each local block may be programmable, such as throttling by 30%, 50%, etc. of the available cycles. The rate of servicing read or write requests to cache or memory may be reduced for throttling.

Other decay and EMA calculations and may be substituted. The EMA generator may be implemented in a variety of ways, including programmable logic, hardware logic gates and blocks, firmware, and software. Rather than have just two levels, local and global, a third level or a hierarchy of levels may be added. For example, several chips in a system could send chip-global power estimates to a system power manager that then performs functions of the global power manager.