Patent Document

BACKGROUND OF THE INVENTION 
       [0001]    1. Field of the Invention 
         [0002]    The present invention relates generally to interconnected processing systems, and more particularly, to processing systems that dynamically control I/O interface performance based on a prediction of I/O requirements. 
         [0003]    2. Description of Related Art 
         [0004]    Interfaces within and between present-day integrated circuits have increased in operating frequency and width. In particular, in multiprocessing systems, both wide and fast connections are provided between many processing units. Data width directly affects the speed of data transmission between systems components, as does the data rate, which is limited by the maximum frequency that can be supported by an interface. However, such fast and wide interconnects are significant power consumers in a computer system formed from interconnected processing units. 
         [0005]    The processing units in a multi-processing system, particularly a symmetric multi-processing system (SMP) may need to communicate at any time, since, for example, when close affinity is provided between processors, a processor might access memory that is located on a remote node, but that is otherwise available in the processor&#39;s memory space. Therefore, for the above and other reasons, present-day multi-processing systems typically keep the physical layer of the interfaces operational and cycle idle data patters on the interconnects in order to maintain calibration of the links when transmissions are not being made over the interface physical layer. However, such operation wastes power, generates heat, and raises background noise levels (electromagnetic emissions) in the system. The alternative of placing the interface physical layers in a power-managed state would lead to unacceptable latency for transactions. 
         [0006]    It is therefore desirable to provide a method, interface and computer system that more effectively manage the state of interface physical link layers in a multi-processing system to reduce power consumption and background noise levels. 
       BRIEF SUMMARY OF THE INVENTION 
       [0007]    The above-mentioned objective of providing improved performance and/or power efficiency of a system interconnect physical layer between processing units is provided in a method, and a computer system and an interface that implement the method. 
         [0008]    The method is a method of managing the state of a physical link layer of external interfaces that interconnect processing units of a computer system. The physical link layers have dynamically adjustable bandwidth. The method detects events other than I/O requests that occur in a processing unit that are indicators of potential future transactions on one of the external interfaces connected to the processing unit. The method predicts, from the detected events, that future transactions will likely occur on the interface, and in response, controls the dynamically adjustable bandwidth of physical link layer of the interface to accommodate the future transactions by increasing the dynamically adjustable bandwidth of the first physical link layer interface. After the future transactions have occurred, the dynamically adjustable bandwidth of first physical link layer is restored to a lower value. 
         [0009]    The foregoing and other objectives, features, and advantages of the invention will be apparent from the following, more particular, description of the preferred embodiment of the invention, as illustrated in the accompanying drawings. 
     
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
         [0010]    The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of the invention when read in conjunction with the accompanying Figures, wherein like reference numerals indicate like components, and: 
           [0011]      FIG. 1  is a block diagram of a computer system in which techniques in accordance with embodiments of the invention are implemented. 
           [0012]      FIG. 2  is a block diagram showing details of a processing unit  10  that can be used to implement processing units  10 A- 10 D of  FIG. 1 . 
           [0013]      FIG. 3  is a block diagram of a controller  30  that can be used to implement controller  30 A and/or  30 B within processing unit  10 A of  FIGS. 1-2 . 
           [0014]      FIG. 4  is a flowchart showing an exemplary method of operating a processing system. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0015]    The present invention encompasses techniques for controlling the bandwidth, including the width and/or frequency of links, such as parallel busses or serial connections, that interconnect processing units in a processing system. Non I/O (input/output) transaction events occurring within the processing units are used to predict when I/O transactions are likely to occur over the links and the prediction is used to control the bandwidth of the links to accommodate the predicted transactions. The techniques thus can reduce power consumption and radiated emissions by maintaining the links in a lower power or inactive state between use. 
         [0016]    With reference now to the figures, and in particular with reference to  FIG. 1  a distributed computer system in accordance with an embodiment of the present invention is shown. A first processing unit  10 A includes a processor core  12  coupled to a memory  14  that stores program instructions for execution by processor  12 . The program instructions may include program instructions forming computer program products that perform portions of the techniques disclosed herein within processing units  10 A- 10 D. Processing unit  10 A also includes a network interface (NWI)  16  that couples processing unit  10 A to interface links  11 , which are wired or wireless links to other processing units  10 B,  10 C, and provide for access between processing unit  10 A and resources such as remote memory  14 A within processing unit  10 B. Links  11  have dynamically adjustable bandwidth/power consumption, which is controlled as disclosed below. Other processing units  10 B- 10 D are of identical construction in the exemplary embodiment, but embodiments of the invention may be practiced in asymmetric distributed systems having processing units with differing features. The distributed computer system of  FIG. 1  also includes other resources such as I/O devices  19 , including graphical display devices, printers, scanners, keyboards, mice, which may be coupled to the links  11  or one of nodes  10 A- 10 D. Processing units  10 A- 10 D are also coupled to storage devices  18 , for storing and retrieving data and program instructions, such as storing computer program products in accordance with an embodiment of the invention. 
         [0017]    Referring now to  FIG. 2 , details within a processing unit  10  that can be used to implement processing units  10 A- 10 D are shown. Within processing unit, controllers  30 A,  30 B are shown to illustrate two possible locations of a controller that manages the bandwidth of a physical link layer  24  of interface  11  according to one or more control signals bw. Within one or more of core  12 , memory  14  and network interface  16 , logic, control logic detects events that are indicative of future external bus transactions that are likely to be issued over interface  11 . For example, a controller  30 A within core  12  might detect that certain instructions are being executed, or memory ranges are being read or written, that correspond to operations that will generate I/O transactions over interface  11 . For example, controller  30 A may detect that a direct-memory access (DMA) buffer is being allocated, or a DMA channel being initialized in bus I/O unit  20  or elsewhere within processing unit  10  for transfer to buffers  21  that supply data to, or receive data from, a logical link layer  22  of network interface  16 . Controller  30 A may be coupled to one or more trace array units  13  within core  12  to capture state information that is indicative of the events, and combine the state contained in the trace array to provide detected events as input for predicting a required bandwidth of interface  11  in the near future. System level events such as a hypervisor executing within processing unit  10  starting a thread with an association to remote memory, or the association of remote memory to a running thread can be used to predict and trigger an increase in link bandwidth between the core on which the thread is running and the location of the remote memory, so that when the inevitable memory accesses by the thread occur, the link is operating at sufficient bandwidth. Similarly, a controller  30 B within arbiter  26  of logical link layer  22  may detected that the logical link layer  22 , and thus interface  11  is being arbitrated for and therefore physical link layer  24  will soon need to be active for a number of transactions. In another example, controller  30 B may count idle cycles of logical link layer  22  to determine a required bandwidth for physical link layer  24 . Alternatively, or in combination, controller  30 B within network interface  16  (whether or not within arbiter  26 ) might also be connected to detect activity in buffers  21  with write operations anticipating upcoming output operations, or initialization of the buffer indicating a future read transaction that will be commanded by core  12  or another actor within processing unit  10 . 
         [0018]    Processing unit  10  of  FIG. 2  is used to illustrate control of one of links  11  between two of processing units  10 A- 10 D, but the techniques of the present invention extend to connection of memories, peripherals and other functional units within a computer system or other electronic device and are not to be construed as limiting as to the particular system in which they are implemented. Links  11  between processing units  10 A- 10 D are, in the example, made by a uni-directional physical layer interconnect of wired signals connected between processing units  10 A- 10 D, however, the techniques of the present invention extend to non-physically connected (wireless) interfaces having multiple datapaths and to bi-directional interfaces, as well. In order to support the adjustable bandwidth of links  11 , processing units  10 A- 10 D may include elastic interface (EI) units with adjustable operating frequency and/or selectable width as described in detail in U.S. Pat. No. 8,050,174 entitled “SELF HEALING CHIP-TO-CHIP INTERFACE”, U.S. Pat. No. 7,117,126 entitled “DATA PROCESSING SYSTEM AND METHOD WITH DYNAMIC IDLE FOR TUNABLE INTERFACE CALIBRATION” and in U.S. Pat. No. 7,080,288 entitled “METHOD AND APPARATUS FOR INTERFACE FAILURE SURVIVABILITY USING ERROR CORRECTION.” The disclosures of the above-referenced U.S. Patents are incorporated herein by reference. 
         [0019]    Referring now to  FIG. 3 , details of a controller  30  that may be used to detect events and predict future transactions on a physical layer of interface  11  is shown. Controller  30  may, for example, implement controller  30 A within core  12  as shown in  FIG. 2 . Controller  30  is also provided only as one example of an architecture that may be implemented in discrete logic, for example as a state machine, or may be implemented in firmware or software as program instructions executed by core  12  or another processor within processing unit  10 , such as a core within logical link layer  22  or a service processor coupled to core  12 . As an example of a mechanism for detecting events, a bus snooper  31  observes transactions on an internal or external bus of core  12 , such as a bus that couples core  12  to memory  14 . In another example a hypervisor  34  reports thread state change or remote memory association events, such as the above-described connection between a thread executing within processing unit  10  and a remote memory. When an event detector  32 A detects that a combination of events indicates a likelihood that a number of transactions will soon occur over interface  11 , a counter  35 A in prediction unit  34  is incremented. Similarly, another event detector  32 B receives indications of activity at logical link layer  22  and determines whether to increment another counter  35 B based on whether the activity indicates that a number of transactions will occur over interface  11 . A bandwidth profile calculator  33  determines from the values of counters  35 A and  35 B, which may be periodically reset, or reset according to another mechanism, the bandwidth that is likely needed over interface  11 . Bandwidth profile calculator  33  provides a control signal to a physical link layer bandwidth control circuit that sets the operating frequency and/or width of the physical link layer of interface  11  appropriately to balance power consumption (or generated noise, etc., depending on the particular system criteria) with the bandwidth supplied over interface  11  for the transactions. A timer  37  is provided to restore the bandwidth to an initial value after a predetermined or programmable interval. In one exemplary implementation, timer  37  controls a time between intervals of full-bandwidth or partial-bandwidth operation as commanded by bandwidth profile calculator  33  and a low-power shutdown state. The width of the intervals can also be set by bandwidth profile calculator, so that interface  11  is cycled between the low-power state and the full-bandwidth or partial-bandwidth state in order to complete transactions that are allowed to accumulate in buffers  21  between the intervals of full-bandwidth or partial-bandwidth operation. In all of the cases above, the actual demand generated by I/O requests is generally combined with the predicted demand to determine an appropriate link bandwidth. 
         [0020]    Referring now to  FIG. 4 , a method of operating a processing system is illustrated in a flowchart. First, interface links between processing units are initialized and calibrated at a nominal interface width and frequency (step  50 ). During operation, events are detected that indicate I/O is likely to occur over one or more of the links (step  51 ). The events are logically combined and counter to generate predictors that indicate a bandwidth that will be needed for the one or more links (step  52 ). Once the predictor is over a threshold value (decision  53 ) or the link utilization is over a threshold value (decision  54 ), the bandwidth of the physical layer (PHY) is raised for a predetermined time period (step  55 ). After the predetermined time period has elapsed (decision  56 ) the bandwidth of the physical layer is lowed to the previous bandwidth (step  57 ). Until the scheme is ended or the system is shut down (decision  58 ), steps  51 - 57  are repeated. 
         [0021]    As noted above, portions of the present invention may be embodied in a computer program product, e.g., a program executed processors having program instructions that direct the operations outlined in  FIG. 4 , by controlling the interfaces of  FIG. 2  and  FIG. 3 . The computer program product may include firmware, an image in system memory or another memory/cache, or stored on a fixed or re-writable media such as an optical disc having computer-readable code stored thereon. Any combination of one or more computer readable medium(s) may store a program in accordance with an embodiment of the invention. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. 
         [0022]    In the context of the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. 
         [0023]    While the invention has been particularly shown and described with reference to the preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form, and details may be made therein without departing from the spirit and scope of the invention.

Technology Category: y