Abstract:
In an information handling system, a plurality of system resources are usable by at least one processor. An affinity structure includes elements describing the relative accessibility of the plurality of system resources to the processor. An affinity manager adjusts the affinity structure in response to a least a first one of the plurality of system resources operating outside of at least one operating parameter.

Description:
BACKGROUND 
     The description herein relates to information handling systems and the handling of resources in such systems. 
     As the value and use of information continue to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system (IHS) generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, IHSs may also vary regarding what information is handled, how the information is handled, how much information is processed, stored, or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in IHSs allow for IHSs to be general or configured for a specific user or specific use such as financial transaction processing, airline reservations, enterprise data storage, or global communications. In addition, IHSs may include a variety of hardware and software components that may be configured to process, store, and communicate information and may include one or more computer systems, data storage systems, and networking systems. 
     IHSs contain resources, such as memory resources, input/output (I/O) devices and ports, etc., that can experience errors. Some resources are designed such that minor errors can be corrected. For instance, in the case of memory resources, many systems store data in memory along with an ECC (Error Correcting Code) that allows single-bit errors in a data word to be corrected. In the case of network interfaces, various packet protocol layers have the capability to detect errors, with at least some protocols responding to an error by dropping an offending packets and/or requesting that the packet be resent. 
     SUMMARY 
     A method of operating an IHS comprises monitoring a system resource for operation according to at least one operating parameter. When the system resource is operating outside of the operating parameter, an affinity parameter for the system resource is adjusted. At least one component of the system reduces usage of the first system resource according to the affinity parameter. For instance, in one embodiment the system resource can be a memory device or portion thereof that is experiencing errors. After a given error rate or number of errors is reached, the system places the memory device in a different proximity domain that makes the memory device appear to the operating system as if it has a reachability from a processor than is less than the device&#39;s actual physical reachability. In response, the operating system will tend to allocate memory from a closer proximity domain first, but can still use the memory device under heavy load. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram illustrating an embodiment of an IHS. 
         FIG. 2  illustrates a portion of an IHS having a plurality of memory modules, according to an embodiment. 
         FIG. 3  shows a system resource allocation table (SRAT), according to an embodiment. 
         FIG. 4  shows a system locality distance information table (SLIT), according to an embodiment. 
         FIG. 5  contains a flowchart for a method of dynamic affinity adjustment, according to an embodiment. 
         FIG. 6  illustrates an SRAT after modification to alter the affinity of two system resources, according to an embodiment. 
         FIGS. 7 and 8  illustrate, respectively, an SRAT and SLIT according to a different embodiment that alters the affinity of the two system resources by modifying the SLIT. 
         FIG. 9  contains a flowchart for a method of dynamic affinity adjustment for restoring a resource to a closer proximity position, according to an embodiment. 
         FIG. 10  illustrates a portion of an IHS with a plurality of Non-Uniform Memory Access (NUMA) nodes, according to an embodiment. 
         FIGS. 11 and 12  illustrate, respectively, an SRAT and SLIT example for the IHS of  FIG. 10 , according to an embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     For purposes of this disclosure, an IHS includes any instrumentality or aggregate of instrumentalities operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, or utilize any form of information, intelligence, or data for business, scientific, control, or other purposes. For example, an IHS may be a personal computer, a network storage device, or any other suitable device and may vary in size, shape, performance, functionality, and price. The IHS may include random access memory (RAM), one or more processing resources such as a central processing unit (CPU) or hardware or software control logic, read only memory (ROM), and/or other types of nonvolatile memory. Additional components of the IHS may include one or more disk drives, one or more network ports for communicating with external devices as well as various I/O devices, such as a keyboard, a mouse, and a video display. The IHS may also include one or more buses operable to transmit communications between the various hardware components. 
       FIG. 1  is a block diagram of one typical IHS. The IHS  100  includes a processor  110  such as an Intel Pentium series processor or one of many other processors currently available. A memory I/O hub chipset  120  connects to processor  110  over a front-side bus  115 . Memory I/O hub  120  provides the processor  110  with access to a variety of resources. Main memory  130  connects to memory I/O hub  120  over a memory bus. A graphics processor  140  also connects to memory I/O hub  120 , allowing the graphics processor to communicate, e.g., with processor  110  and main memory  130 . Graphics processor  140 , in turn, provides display signals to a display device  150 . 
     Other resources can also be coupled to the system through memory I/O hub  120 , including an optical drive  160  or other removable-media drive, one or more hard disk drives  165 , one or more network interfaces  170 , one or more USB (Universal Serial Bus) ports  180 , and a super I/O controller  190  to provide access to user input devices  195 , etc. 
     Not all IHSs include each of the components shown in  FIG. 1 , and other components not shown may exist. Furthermore, some components shown as separate may exist in an integrated package or be integrated in a common integrated circuit with other components. As can be appreciated, many systems are expandable, and include or can include a variety of components, including redundant or parallel resources. 
     The Advanced Configuration and Power Interface (ACPI) standard has been developed to allow operating system-directed configuration and power management on computers. The ACPI standard is described in the  Advanced Configuration and Power Interface Specification , rev. 3.0a, Dec. 30, 2005, incorporated herein by reference. ACPI, among other things, describes an interface between an ACPI-compatible operating system and the system BIOS (Basic Input/Output System). This interface uses tables to describe system information, features, and methods for controlling those features. 
     Some IHSs, e.g. some servers, use designs that have multiple nodes, each with one or more processors and local memory, arranged in what is known as a Non-Uniform Memory Access (NUMA) architecture. The nodes are interconnected by NUMA interconnects, such that the memory local to one node is also accessible to the other nodes, although generally with a longer access time than a local memory access. A large memory space is thus distributed among the nodes, with memory devices existing at two or more proximities to each processor. In a NUMA system, it is most efficient for processes running on a certain node to use memory local to that node, or if not local, as close to the node as possible. Accordingly, it is desirable for an operating system to have proximity information available for the memory units and/or other redundant resources on the system. ACPI provides tables that allows processors, memory, and other resources to be associated with a node, and relative distances between the various nodes to be understood by the operating system. Two optional ACPI tables, the System Resource Affinity Table (SRAT) and the System Locality Distance Information Table (SLIT), contain information used by the operating system during initialization to determine resource affinity and proximity. 
     The SRAT follows a table format including a defined header that identifies a variable number of entries known as Static Resource Allocation Structures. Two types of structures are allowed to populate the entries: Processor Local APIC/SAPIC Affinity Structures and Memory Affinity Structures (APIC and SAPIC refer respectively to two different interrupt models, the Intel Advanced Programmable Interrupt Controller and Intel Streamlined Advanced Programmable Interrupt Controller). A Processor Affinity Structure identifies a specific processor by APIC or SAPIC ID, and associates that processor with a proximity domain. A Memory Affinity Structure identifies a memory range by Base Address and Length, and associates that memory range with a proximity domain. Each structure contains other fields and exists in a specific format defined in the ACPI Specification, the details of which are not necessary for an understanding of the embodiments described below. 
     The SLIT follows a table format that includes a defined header indicating the table length, the number N of System Localities (Proximity Domains) in the system, and an N×N matrix of relative proximity values, Entry□□. The value stored at Entry[x][y] represents the relative proximity of System Localities x and y. The proximity of a System Locality to itself (Entry[x][x]) is defined as a relative value of 10, with proximities lower than 10 disallowed. Other proximities are scaled accordingly, e.g., a proximity between two localities of 30 indicates that the locality is about three times slower to reach than a local access. A proximity of 255 is entered in Entry[x][y] when System Locality y is unreachable from System Locality x. 
     Referring now to  FIG. 2 , an exemplary portion of an IHS  200  is illustrated, including a processor  110  connected by a front side bus  115  to a memory I/O hub  120 , and a main memory subsystem comprising two memory buses MB 0  and MB 1  and six Dual In-line Memory Modules (DIMMs) DIMM 0 -DIMM 5 . Memory bus MB 0  is illustrated as a multi-drop memory bus coupling memory I/O hub  120  to DIMMs DIMM 0 -DIMM 3 . Memory bus MB 1  is illustrated as a multi-drop memory bus coupling memory I/O hub  120  to DIMMs DIMM 4  and DIMM 5 , with two empty slots for additional DIMMs. Other exemplary systems may have a different number of memory buses, a different number of drops or slots for modules, may use other form factors such as SIMMs (Single In-line Memory Modules) or non-modular memory devices fixed to a motherboard, and/or may use point-to-point memory buses. 
     IHS  200  uses a design that allows each address on each DIMM to be accessed just as easily and quickly as each other address, assuming that the DIMM and/or bus serving that DIMM is available. In this sense, the physical “proximity” of each DIMM to processor  110 , or other devices that access main memory, is equivalent. Were this configuration to be described in an ACPI SRAT, IHS  200  would be described in the table as having one “node” and one proximity domain, to which the single processor and six DIMMs are assigned. Since all DIMMs belong to the same proximity domain as processor  110 , a single Memory Affinity Structure could be used in an ACPI SRAT to describe the range of memory addresses covered by all six DIMMs. 
     In one embodiment, multiple Memory Affinity Structures are used to describe the memory in the local proximity domain. For instance, a simplified SRAT  300  is shown in  FIG. 3 , illustrating only MAS entries and no PAS entries or header. Each MAS entry is further simplified to show only the fields representing a proximity domain and the fields representing the Base Address and Length (labeled “Address Range”). In SRAT  300 , six MAS entries are used, one each for the address ranges corresponding to each DIMM. Accordingly, each MAS entry associates one of the DIMMs with Proximity Domain  0 , the domain of the single processor  110 . 
       FIG. 4  depicts an exemplary simplified SLIT  400  that is to be used with SRAT  300  (only the Entry matrix is shown). SLIT  400  expresses proximities for three Proximity domains  0 ,  1 , and  2 . The proximity of domains  1  and  2  is set to 20, and the proximity of domains  2  and  3  is set to 30. Note that domains  1  and  2  are indicated as unreachable from each other. 
     In IHS  200 , one or more of the processor, hub, and/or DIMMs test or monitor the addressable memory locations on each DIMM for errors. For instance, when the system is turned on a Power-On Self Test (POST) routine will write and then read bit patterns to/from each memory location, and will detect any memory locations that do not return the same bit pattern that was written. In many systems, data words are also stored in the DIMMs with extra bits comprising an ECC, e.g., that allows a system component to detect single and some multi-bit errors, and correct single-bit errors in data words. Other parity/error detection schemes exist in some memory designs, and in other types of resources such as those that handle data packets comprising multiple data words. 
     In one embodiment, hardware and/or software detects memory errors. A System Management Interrupt passes control to a BIOS routine that acts on the memory errors, determining whether one of the DIMMs is producing an undesirable number of errors. The BIOS routine can respond by moving the address range of the DIMM from its actual physical Proximity Domain  0  to a virtual Proximity Domain ( 1  or  2  in this example), and notifying the OS that the SRAT has been updated. This causes the OS to reduce usage of the offending DIMM relative to other DIMMs that are operating with better performance, and yet allows the DIMM to remain available for use by the OS under high load situations. 
       FIG. 5  contains a flowchart  500  for one method implemented in a SMI handler routine. This example responds to the detection and correction of a correctable (e.g., single-bit) error detected in a resource, such as a DIMM. The SMI handler routine is configured with the ability to save resource state, e.g., for each MAS entry in SRAT  300 . The state can include logging of each error detected by resource, a filtered error rate, a moving time window and associated error count in the window, or an absolute count of the number of errors detected. 
     The SMI handler routine is programmed with error parameters that will result in the relegation of a resource to a lower proximity domain. For instance, a hard threshold such as 10 errors total could trigger relegation of the resource, as could a given error rate. Should the error parameters be exceeded, the affinity of the resource is adjusted to place the resource further from the CPUs, and the OS is notified to update its affinity tables (e.g., SRAT and/or SLIT). The event is logged, and the SMI handler then performs other tasks as needed before returning control to the OS. 
     As the OS normally only checks the SRAT and SLIT during initialization, the OS is preferably modified to receive an indication that the SRAT and SLIT should be reloaded after dynamic modification. Alternately, permission to reboot could be requested, or the SRAT and SLIT could be retained but not consulted by the OS until reboot, for some other reason, causes the tables to be reloaded. 
     In some embodiments, resources can be tested against multiple error parameters, divided, e.g., by error severity or error frequency. A resource that has already been relegated, and continues to accumulate errors, can be relegated yet again to place it even further from the CPUs. Likewise, detected uncorrectable errors could trigger the relegation of a memory range to a far distant virtual proximity domain, indicating that the OS should only use the memory range for fault-tolerant memory applications. 
     An example of two stages of relegation is shown in the modified SRAT  600  of  FIG. 6 . In this example, correctable errors have accumulated to beyond a first threshold on DIMM 4  and beyond a second threshold on DIMM 1 . The SMI handler of  FIG. 5  has dynamically modified SRAT  300  to produce SRAT  600 , with DIMM 4  now assigned to virtual Proximity Domain  1  and DIMM 1  now assigned to virtual Proximity Domain  2 . Referring back to SLIT  400 , the OS now will see accesses to DIMM 4  as twice as costly and accesses to DIMM 1  as three times as costly as accesses to DIMM 0 , DIMM 2 , DIMM 3 , and DIMM 5 . 
     Another approach to achieve the same end is shown in  FIGS. 7 and 8 . An SRAT  700  assigns each DIMM to its own virtual Proximity Domain, none of which are the same as the Proximity Domain  0  of the processor. SLIT  800 , however, is configured such that each virtual Proximity Domain has the minimum distance of 10 from Proximity Domain  0 , unless relegated. Thus in SLIT  800 , DIMM 1  has been reassigned a proximity of 30 by modifying entries [0][2] and [2][0] of SLIT  800 , and DIMM 4  has been reassigned a proximity of 20 by modifying entries [0][5] and [5][0] of SLIT  800 . 
     Alternate embodiments can have coarser or finer granularity or dynamically change the number of MAS entries to respond to resource problems. In the  FIG. 2  embodiment, for instance, two MAS entries could be instantiated, one for MB 0  and one for MB 1 , with errors tracked for each bus. Alternately, sections of each DIMM, such as each 64 MB address range, can each receive their own MAS entry and be tracked individually for errors. MAS entries also can be carved from a larger entry by tracking smaller divisions of memory than are represented in the original MAS entries. When a smaller division experiences problems, the larger entry containing that address range is divided into two or three MAS entries, allowing the problematic address range to be handled separately. 
     Embodiments can optionally include a mechanism for improving the proximity of a resource after a demonstrated period without errors.  FIG. 9  illustrates a section of a resource state handler  900  as described above, for tracking the error performance of resources. Resource state handler also considers, for resources that have been relegated to a more distant proximity, whether the stability of the resource has improved (e.g., no new errors have been detected). The handler then considers, if an improvement has occurred, whether the improvement is such that an affinity upgrade is warranted. This step can be implemented by requiring a certain period of time without errors, or an error rate decreasing below a low threshold. 
     When the resource is to be upgraded, the resource handler adjusts affinity for the resource, e.g., by modifying the SRAT or SLIT as described above, and notifying the OS as described above. 
     Although the above examples use a system with one processor and one physical resource System Locality, other examples can be implemented on a system with multiple physical resource System Localities.  FIG. 10  illustrates components of a NUMA IHS  1000 , including four processor groups PG 0 -PG 3 , four local memories LM 0 -LM 3  connected to respective processor groups, and six NUMA interconnects Nlx,y, where x and y are varied such that each processor group shares an interconnect with each other processor group. Each processor group and local memory is assigned to a corresponding SRAT system locality, i.e., PG 0  and LM 0  are assigned to system locality SL 0 , etc. Additionally, each processor group is “connected” to a “virtual” local memory in a corresponding virtual system locality, i.e., PG 0  corresponds to a virtual local memory VLM 0 , located in a virtual system locality SL 5 , and so forth. The virtual local memory VLM 0  is virtual in that it actually represents addressable regions of LM 0  that have been “moved” to a different SRAT system locality due to the detection of correctable errors. Depending on the performance of the physical local memory components, none, some, or all of virtual system localities SL 5 -SL 8  may be populated with address ranges from corresponding local memories LM 0 -LM 3 . 
       FIGS. 11 and 12  show exemplary SRAT and SLIT contents for the system  1000  configuration shown in  FIG. 10 . SRAT  1100  contains twelve entries. Four entries are for the four processor groups PG 0 -PG 3  (if any processor group contains more than one processor, additional entries would be required), assigning them respectively to System Localities SL 0 -SL 3 . Four entries are for the sections of the four local memories LM 0 -LM 3  that are deemed reliable, assigning them respectively to System Localities SL 0 -SL 3 . Using the techniques described in the preceding examples, each of these entries could be divided into multiple entries for multiple address ranges. The remaining four entries are for the sections of the four local memories LM 0 -LM 3  that are deemed unreliable, assigning them respectively to System Localities SL 4 -SL 7 . Again, the number of entries will depend on the existence of and number of unreliable address ranges in each local memory. 
     SLIT  1200  contains an 8×8 matrix of proximity values. The diagonal is populated with proximity values of 10, ensuring that the OS will prefer to allocate each processor its own reliable local memory first. A proximity value of 20 is assigned to the remaining matrix elements with both column and row less than four, ensuring that the OS will as a second preference allocate each processor reliable memory local to one of the other processors. A proximity value of 30 is assigned to matrix elements associating a processor system locality with its corresponding virtual system locality (e.g., SL 0  and SL 4 , SL 1  and SL 5 , etc.) This ensures that the OS will as a third preference allocate each processor its own unreliable local memory. Fourth, a proximity value of 40 is assigned to matrix elements associating a processor system locality with another processors corresponding virtual system locality, ensuring that such memory is allocated last. Lastly, a proximity value of 255 is assigned to the cross-terms for the virtual system localities. 
     Other configurations are possible for a NUMA system, including assigning all unreliable memory to a common “non-preferred” locality that contains no processors. Furthermore, this approach can be used with resources other than memory, such as network interfaces and other memory-mapped resources. Other types of errors that can trigger an affinity parameter adjustment can include packet errors detected through a cyclic redundancy check, or parameters that indicate a resource is running abnormally. For instance, a component that is running hot could be moved to a more distant proximity to decrease the chances that a critical routine would be affected if the component were to fail. 
     In many embodiments, once the proximity of a resource is distance, usage of the resource will be reduced by the operating system preferring to allocate other similar but non-problematic resources first in the future. Under heavy load when all resources are needed the reduced usage may not be detectable, but on average the resource will experience reduced usage. Optionally, the OS may choose to copy data out of a resource to a more reliable location when the resource is downgraded. 
     Many other features of the described systems and methods may be varied as design parameters. Although an ACPI-based implementation is described, other implementations are possible. Those skilled in the art recognize that various features and elements are alternately implementable using hardware, BIOS, or operating system approaches. 
     Although illustrative embodiments have been shown and described, a wide range of other modification, change and substitution is contemplated in the foregoing disclosure. Also, in some instances, some features of the embodiments may be employed without a corresponding use of other features. Accordingly, it is appropriate that the appended claims be constructed broadly and in manner consistent with the scope of the embodiments disclosed herein.