Source: http://www.google.com/patents/US7788519?dq=5958006
Timestamp: 2013-12-11 20:30:30
Document Index: 313999498

Matched Legal Cases: ['Application No. 200410070913', 'Application No. 93120990', 'Application No. 93120990', 'Application No. 200410070913', 'Application No. 2002', 'Application No. 2002', 'Application No. 10', 'Application No. 10', 'Application No. 200710106805', 'Application No. 11', 'Application No. 200410070913', 'Application No. 200410070913']

Patent US7788519 - Method, system, and apparatus for improving multi-core processor performance - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Advanced Patent Search | Sign inAdvanced Patent SearchPatentsA system, apparatus, and method for a core rationing logic to enable cores of a multi-core processor to adhere to various power and thermal constraints....http://www.google.com/patents/US7788519?utm_source=gb-gplus-sharePatent US7788519 - Method, system, and apparatus for improving multi-core processor performancePublication numberUS7788519 B2Publication typeGrantApplication numberUS 11/686,861Publication dateAug 31, 2010Filing dateMar 15, 2007Priority dateJul 15, 2003Also published asCN1577280A, CN100555227C, CN101320289A, DE112004001320B3, US7389440, US7392414, US20050050310, US20060117199, US20060117200, US20060123263, US20060123264, US20070198872, WO2005010737A2Publication number11686861, 686861, US 7788519 B2, US 7788519B2, US-B2-7788519, US7788519 B2, US7788519B2InventorsDaniel W. Bailey, Todd Dutton, Tryggve FossumOriginal AssigneeIntel CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (33), Non-Patent Citations (25), Referenced by (1), Classifications (29) External Links: USPTO, USPTO Assignment, EspacenetMethod, system, and apparatus for improving multi-core processor performanceUS 7788519 B2Abstract A system, apparatus, and method for a core rationing logic to enable cores of a multi-core processor to adhere to various power and thermal constraints.
1. A method for disabling clocks to at least one processor core of a plurality of processor cores comprising:
calculating an executing core limit based on a workload;
asserting a signal to one processor core based at least in part on executing an operation that is dependent on a previous memory operation, the signal to disable the clocks to the processor core;
blocking an instruction issue operation in response to the signal; and
transitioning the core to a cache coherent low power state of operation;
defining an operating point based at least in part on the executing core limit and a voltage and frequency pair, and adjusting the operating point based on analysis of a number of cores status with respect to a waiting or rationing queue.
2. A method for disabling clocks to at least one processor core of a plurality of processor cores comprising:
calculating a predetermined executing core limit based on a workload;
asserting a signal to one processor core based at least in part on executing an operation that is dependent on a memory operation, the signal to disable the clocks to the processor core;
blocking an instruction issue operation in response to the signal;
assigning an identifier for the disabled processor core; and
de-asserting the signal for the disabled processor core upon completion of the memory operation if a number of executing cores is less than or equal to the predetermined executing core limit;
defining an operating point based at least in part on the predetermined executing core limit and a voltage and frequency pair, and adjusting the operating point based on analysis of a number of cores status with respect to a waiting or rationing queue.
3. A core rationing logic to disable clocks to at least one processor core of a plurality of processor cores comprising:
a workload circuit to calculate an executing core limit;
a transmitter circuit to assert a signal to one processor core based at least in part on executing an operation that is dependent on a memory operation, the signal to disable the clocks to the processor core; and
a comparison circuit to de-assert the signal for the disabled processor core upon completion of the memory operation if a number of executing cores is less than or equal to the executing core limit.
4. The logic of claim 3 further comprising the workload circuit to define an operating point based at least in part on the executing core limit and a voltage and frequency pair, and adjusting the operating point based on analysis of a number of cores status with respect to a waiting or rationing queue.
5. A core rationing logic to disable clocks to at least one processor core of a plurality of processor cores comprising:
a transmitter circuit to assert a signal to one processor core based at least in part on executing an operation that is dependent on a memory operation, the signal to disable the clocks to the processor core;
a comparison circuit to de-assert the signal for the disabled processor core upon completion of the memory operation if a number of executing cores is less than or equal to the executing core limit;
an assignment circuit to assign an identifier for the disabled processor core; and
the transmitter circuit to de-assert the signal for the disabled processor in response to the signal de-asserted by the comparison circuit.
6. The logic of claim 5 further comprising the workload circuit to define an operating point based at least in part on the executing core limit and a voltage and frequency pair, and adjusting the operating point based on analysis of a number of cores status with respect to a waiting or rationing queue. Description
This application is a divisional of U.S. patent application Ser. No. 10/621,228 filed Jul. 15, 2003, now abandoned.
The present disclosure pertains to the field of power management. More particularly, the present disclosure pertains to a new method and apparatus for improving multi-core processor performance despite power constraints.
Power management schemes allow for reducing power consumption to achieve low power applications for various types of and systems and integrated devices, such as, servers, laptops, processors and desktops. Typically, software methods are employed for systems and integrated devices to support multiple power states for optimizing performance based at least in part on the Central Processing Unit (CPU) activity.
Present power management schemes either decrease voltage or frequency or both for reducing power consumption. However, this results in decreased overall performance. Also, some methods incorporate analog designs that have various challenges relating to loop stability for transient workloads, calibration, and tuning.
With the introduction of processors with multiple cores, power management becomes a major concern because of the increase in cores operating at high frequencies and voltages and need to adhere to various power constraints, such as, thermal limits, maximum current, and Vcc range.
FIG. 1 illustrates a flowchart for a method utilized in accordance with an embodiment
FIG. 2 illustrates a bar chart utilized in accordance with an embodiment.
FIG. 3 illustrates a bar chart utilized in accordance with an embodiment.
FIG. 4 illustrates an apparatus in accordance with one embodiment.
DETAILED DESCRIPTION The following description provides method and apparatus for improved multi-core processor performance despite power constraints. In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate logic circuits without undue experimentation.
As previously described, a problem exists for improving processor performance while adhering to power constraints. The present methods incorporate lowering the voltage or frequency at the expense of overall performance. In contrast, the claimed subject matter improves overall performance while adhering to power constraints. For example, a concept of �rationing the number of executing cores for a processor system� allows for increasing frequency as a result of disabling clocks to cores that are idle as they wait for a memory transaction to complete. For example, the claimed subject matter exploits the idle time period of processor cores by disabling the clocks to the core, that results in less power dissipation. Thus, a higher frequency can be utilized as a result of the decrease in power dissipation. In one embodiment, an appropriate executing core limit is calculated for the workload. Also, in the same embodiment, the number of executing cores are less than or equal to the number of available and ready threads. A thread is an independent set of instructions for a particular application.
In one embodiment, the claimed subject matter facilitates selecting a voltage/frequency operating point based on a prediction of the activity level of the threads running on all of the cores collectively. For example, TPC-C threads tend to be active 50-60% of the time, and spend 40-50% of their time idle, waiting for memory references to be completed. In such an environment, one would specify an executing core limit that would be, in one embodiment, 60% of the total number of cores on the die; if there were 8 cores, one would set the executing core limit to, in this case, five. One would then specify a voltage-frequency operating point that corresponds to having only five cores active and three cores inactive (low power state) at a time; this is a significantly higher operating frequency than one would specify if one was allowing all eight cores to be simultaneously active. The core rationing logic constrains the operations of the die, guaranteeing that no more than five cores (in this case) are active at any given moment. Statistics are gathered regarding the occupancy of the Waiting and Rationing queues (which will be discussed further in connection with FIG. 1); at intervals these statistics are analyzed to determine whether the operating point (executing core limit and its associated voltage/frequency pair) should be changed. If the Waiting queue tends to be empty and the Rationing queue tends to be full, that is an indication that cores are not making progress when they could be, and that to improve performance the executing core limit should be raised and the voltage/frequency reduced; conversely, if the Rationing queue tends to be empty, and the Waiting queue tends to be full, this may be an indication that one can increase performance by reducing the executing core limit and increasing the voltage/frequency point.
FIG. 1 illustrates a flowchart for a method utilized in accordance with an embodiment. In one embodiment, the flowchart depicts a method for a state diagram.
In the same embodiment, the state diagram illustrates a predetermined state machine for a processor core in a system. In this same embodiment, the state machine facilitates the �rationing of the cores� to improve processor performance as a result of disabling clocks to cores that are waiting for a memory transaction to complete.
In one embodiment, the state diagram has four defined states, such as, a Core Unassigned state 202, an Executing state 204, a Rationing FIFO Queue state 206, and a Waiting state 208. Initially, the Core Unassigned state is defined as follows: each core does not have an assigned thread. Subsequently, in the event that a core has a thread assigned to it, the claimed subject matter transitions to the Rationing FIFO Queue state 206. In one embodiment, FIFO is defined as a First In First Out.
Upon transitioning to the Rationing FIFO Queue state, a comparison between the number of executing cores and an executing core limit (ECL) is determined. In one embodiment, a processor or system specification determines the proper executing core limit in order to adhere to thermal power considerations. In one embodiment, the ECL is determined by a formula depicted later in the application. If the number of executing cores is less than ECL, the particular core transitions to the Executing state 204 if the core was the next one to be processed in the FIFO queue. Otherwise, the core remains in the Rationing FIFO queue 206.
Upon entering the Executing state, the core remains in this state unless an event occurs, such as, a memory reference and overheating event, and/or a fairness timeout. For example, a fairness timeout may be utilized to prevent a possible live lock state. In this context, a memory reference refers to a read or write operation to a particular memory address that does not reside in any cache memory coupled to the processor (�a miss in all levels of cache memory�). Therefore, an access to main memory is initiated.
If an event occurs as previously described, the core transitions to the Waiting state 208. Upon completion of the event, the core transitions to the Rationing FIFO queue state. This sequence of cycling between states 204, 206, and 208 occurs until the particular thread is completed. Upon completion of the thread, the core transitions to the Core Unassigned State.
However, the claimed subject matter is not limited to the four defined states in the state diagram. The claimed subject matter supports different amounts of states. FIG. 1 merely illustrates an example of limiting the number of executing cores to be less than the available number of threads. For example, one embodiment would allow for multiple waiting states. Alternatively, the waiting states could be replaced by another queue state.
Also, other embodiments of state diagrams would allow multiple priority levels for cores, as well as having different waiting queues depending on the nature of the event that provoked exit from the executing state (memory wait, thermal wait, ACPI wait, etc).
Typically, a core executes a memory read or write operation and subsequently executes an operation that is dependent on that operation (for example, it makes use of the data returned by a memory read operation). Subsequently, the core �stalls� waiting for that memory operation to be completed. In such a case, it asserts a signal to the central core rationing logic indicating that it is stalled; this is the indication that it is eligible to be disabled by the core rationing logic. The core rationing logic responds to this signal by �napping� the core in question�it asserts a �nap� signal to the core, which causes the core to block instruction issue and then transition into a (cache-coherent) low power state. Furthermore, the core rationing logic puts an identifier for that core in the Waiting queue. When the memory operation completes, the core deasserts the �stall� signal; the core rationing logic responds to this by moving the identifier for that core from the Waiting queue to the Rationing queue. If the number of currently executing (not �napped�) cores is less than or equal to the Executing Core Limit, the core rationing logic removes the oldest identifier from the Rationing queue, and deasserts the �nap� signal to that core.
FIG. 2 illustrates a bar chart utilized in accordance with an embodiment. In one embodiment, the bar chart depicts a percentage time spent executing for a 16-core multiprocessor as calculated by a Monte Carlo simulation for a variety of workloads. The independent axis illustrates the ECL for 2, 4, 6, 8, 10, 12, 14, and 16. Also, there is a bar for each ECL at a different workload as simulated with a memory reference duty cycle (with respect to executing time) of 1%, 30%, 40%, and 50%.
Analyzing the 50% memory reference duty cycle highlights the fact that the percentage time executing saturates at 50%. Thus, processing the memory references consumes half of the executing time when the ECL is equal to the number of available threads.
FIG. 3 illustrates a bar chart utilized in accordance with an embodiment. In addition to FIG. 2, FIG. 3 illustrates the total performance as calculated by the product of the percentage time executing and the frequency. The total performance also incorporates the fact that frequency is inversely proportional to the ECL. As previously described, this relationship exists because as one reduces the number of executing cores, this results in reducing power dissipation. Therefore, the frequency can be increased to remain at the steady-state thermal limit.
Also, FIG. 3 depicts the maximum percentage time executing is 70% for the 30% memory reference duty cycle. Also, the product of the saturation limit and the number of threads demarcates the onset of saturation. Of particular note is the onset of saturation because this may be the region for improved or optimum performance.
In one embodiment, a self optimization formula is utilized to determine the appropriate ECL. In the formula, N depicts the number of threads that have context: % E depicts the percentage executing time; and % M depicts the percentage memory reference time. The formula is:
int(N�(%E/(%E+%M)))
FIG. 4 depicts an apparatus in accordance with one embodiment. In one embodiment, the apparatus depicts a multi-core processor system with a plurality of processors 410 coupled individually to an independent bank of Level 3 (L3) Cache memory. In the same embodiment, a plurality of four busses form two counter rotating �rings��a Request/Response (REQ0/RSP0) ring (402 and 404) in the clockwise direction, and a Request/Response ring (REQ1/RSP1) (406 and 408) in the counterclockwise direction. The circle in between the �P�s and the �C�s represents a pair of state devices for each ring. Thus, a set of circular pipelines are utilized for passing information from each processor core/cache bank to any other processor core/cache bank. The system interface logic contains the memory controllers for memory DIMMs, the router logic to handle the interconnection links to other processor dies and/or I/O subsystems, and assorted other system control logic (including the central core rationing controller).
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5168554Oct 13, 1989Dec 1, 1992International Business Machines CorporationConverting trace data from processors executing in parallel into graphical formUS5442775 *Feb 8, 1994Aug 15, 1995Meridian Semiconductor, Inc.Two clock microprocessor design with stallUS5737615Apr 12, 1995Apr 7, 1998Intel CorporationMicroprocessor power control in a multiprocessor computer systemUS5752030 *Aug 4, 1993May 12, 1998Hitachi, Ltd.Program execution control in parallel processor system for parallel execution of plural jobs by selected number of processorsUS5913068Aug 28, 1996Jun 15, 1999Kabushiki Kaisha ToshibaMulti-processor power saving system which dynamically detects the necessity of a power saving operation to control the parallel degree of a plurality of processorsUS5968167Apr 3, 1997Oct 19, 1999Videologic LimitedMulti-threaded data processing management systemUS6357016Dec 9, 1999Mar 12, 2002Intel CorporationMethod and apparatus for disabling a clock signal within a multithreaded processorUS6549954Aug 23, 1999Apr 15, 2003Advanced Micro Devices, Inc.Object oriented on-chip messagingUS6550020Jan 10, 2000Apr 15, 2003International Business Machines CorporationMethod and system for dynamically configuring a central processing unit with multiple processing coresUS6574739Apr 14, 2000Jun 3, 2003Compal Electronics, Inc.Dynamic power saving by monitoring CPU utilizationUS6640282Jan 25, 2001Oct 28, 2003Hewlett-Packard Development Company, L.P.Hot replace power control sequence logicUS6804632Dec 6, 2001Oct 12, 2004Intel CorporationDistribution of processing activity across processing hardware based on power consumption considerationsUS6883107Mar 8, 2002Apr 19, 2005Intel CorporationMethod and apparatus for disabling a clock signal within a multithreaded processorUS6889319Dec 9, 1999May 3, 2005Intel CorporationMethod and apparatus for entering and exiting multiple threads within a multithreaded processorUS6920572Nov 8, 2001Jul 19, 2005Texas Instruments IncorporatedUnanimous voting for disabling of shared component clocking in a multicore DSP deviceUS6931506Feb 20, 2003Aug 16, 2005Stmicroelectronics SaElectronic device for data processing, such as an audio processor for an audio/video decoderUS6934727Jan 2, 2003Aug 23, 2005Microsoft CorporationDynamic synchronization of tablesUS6971034Jan 9, 2003Nov 29, 2005Intel CorporationPower/performance optimized memory controller considering processor power statesUS6990598 *Mar 21, 2001Jan 24, 2006Gallitzin Allegheny LlcLow power reconfigurable systems and methodsUS7093147Apr 25, 2003Aug 15, 2006Hewlett-Packard Development Company, L.P.Dynamically selecting processor cores for overall power efficiencyUS7318164 *Dec 13, 2001Jan 8, 2008International Business Machines CorporationConserving energy in a data processing system by selectively powering down processorsUS7480911 *May 9, 2002Jan 20, 2009International Business Machines CorporationMethod and apparatus for dynamically allocating and deallocating processors in a logical partitioned data processing systemUS20020018877Jul 27, 2001Feb 14, 2002Woodall Calvin L.Reduced motion and anti slip padUS20030014467Jun 6, 2002Jan 16, 2003Kiichiro HanzawaMultitasking operating system capable of reducing power consumption and vehicle electronic control unit using sameUS20030079151Oct 18, 2001Apr 24, 2003International Business Machines CorporationEnergy-aware workload distributionUS20030084154Oct 31, 2001May 1, 2003International Business Machines CorporationEnergy-induced process migrationUS20030115495Dec 13, 2001Jun 19, 2003International Business Machines CorporationConserving energy in a data processing system by selectively powering down processorsUS20040128663Dec 31, 2002Jul 1, 2004Efraim RotemMethod and apparatus for thermally managed resource allocationJP2003029886A Title not availableJPH09185589A Title not availableJPH11280891A Title not availableWO2002039242A1Oct 31, 2001May 16, 2002Millennial Net IncNetworked processing system with optimized power efficiencyWO2005010737A2Jul 14, 2004Feb 3, 2005Intel CorpA method, system, and apparatus for improving multi-core processor performance* Cited by examinerNon-Patent CitationsReference1CN Pat. Appl. No. 200410070913.7 First Office Action mailed Jun. 23, 2006, 9 pgs (includes English translation).2Fifth Office Action from Counterpart China Patent Application No. 200410070913.7, dated Mar. 2, 2009 (4 pgs. Translation included).3Foreign Office Action from Counterpart Great Britain Patent Application No. GB0602753.6, dated May 23, 2007 (4 pgs).4Foreign Office Action from Counterpart Great Britain Patent Application No. GB0602753.6, dated Nov. 1, 2007 (3 pgs).5Foreign Office Action from Counterpart Taiwan Patent Application No. 93120990, dated Apr. 26, 2007 (5 pgs. Translation included).6Foreign Office Action from Counterpart Taiwan Patent Application No. 93120990, dated Jul. 26, 2006 (6 pgs. Translation included).7IEEE, "IEEE 100 The Authoritative Dictionary of IEEE Standards Terms", IEEE, IEEE 100, The Authoritative Dictionary of IEEE Standards Terms, 2000, Standards Information Network, 7th edition, pp. 904, (2000), 3 pgs.8Notice of Allowance from U.S. Appl. No. 11/336,015, mailed Nov. 5, 2007, 11 pgs.9Office Action from Counterpart China Patent Application No. 200410070913.7, dated Sep. 28, 2008 (6 pgs. Translation included).10Office Action from Counterpart Japanese Patent Application No. 2002-520257, dated Feb. 25, 2009 (3 pgs. Translation included).11Office Action from Counterpart Japanese Patent Application No. 2002-520257, dated Jul. 29, 2008 (3 pgs. Translation included).12Office Action from Counterpart Korea Patent Application No. 10-2006-7000942, dated Feb. 26, 2007 (5 pgs. Translation included).13Office Action from Counterpart Korea Patent Application No. 10-2006-7000942, dated Mar. 19, 2008 (5 pgs. Translation included).14Office Action from foreign counterpart China Patent Application No. 200710106805.4, mailed Feb. 12, 2010, 4 pgs.15Office Action from foreign counterpart German Patent Application No. 11 2004 001 320.8-53, mailed May 10, 2010, 4 pgs.16Office Action from U.S. Appl. No. 11/336,015, mailed Apr. 16, 2007, 13 pages.17Office Action from U.S. Appl. No. 11/336,302, mailed Apr. 16, 2007, 11 pgs.18Office Action from U.S. Appl. No. 11/336,303, mailed Jun. 13, 2007, 5 pgs.19Office Action from U.S. Appl. No. 11/336,681, mailed Jan. 12, 2007, 12 pgs.20Office Action mailed Nov. 22, 2006 from U.S. Appl. No. 11/336,303, 8 pgs.21Second Foreign Office Action from Counterpart China Patent Application No. 200410070913.7, dated May 16, 2007 (8 pgs. Translation included).22Sholander, P., et al., "The Effect of Algorithm-Agile Encryption on ATM Quality of Service", IEEE, 1997, pp. 470-474.23Third Foreign Office Action from Counterpart China Patent Application No. 200410070913.7, dated Jun. 8, 2007 (9 pgs. Translation included).24U.S. Appl. No. 11/336,302 Office Action mailed Nov. 6, 2006, 7 pgs.25U.S. Appl. No. 11/366,015 Office Action mailed Oct. 25, 2006, 7 pgs.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8185758 *Jun 30, 2011May 22, 2012Intel CorporationMethod and system for determining an energy-efficient operating point of a platform* Cited by examinerClassifications U.S. Classification713/500, 713/320, 712/200, 712/32, 712/28, 711/163, 711/118, 711/167, 713/300International ClassificationG06F1/32, G06F1/00, G06F1/26, G06F12/00, G06F15/00Cooperative ClassificationY02B60/1278, Y02B60/1217, G06F1/3228, G06F1/3287, Y02B60/1282, G06F1/329, Y02B60/1285, G06F1/324, G06F1/3296, Y02B60/144European ClassificationG06F1/32P5F, G06F1/32P1D, G06F1/32P5T, G06F1/32P5S, G06F1/32P5VRotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google