Abstract:
Disclosed are a concurrent selftest engine and its applications to verify, initialize and scramble the system memory concurrently along with mainline operations. In prior art, memory reconfiguration and initialization can only be done by firmware with a full system shutdown and reboot. The disclosed hardware, working along with firmware, allows us to do comprehensive memory test operations on the extended customer memory area while the customer mainline memory accesses arc running in parallel. The hardware consists of concurrent selftest engines and priority logic. Great flexibility is achieved by the new design because customer-usable memory area can be dynamically allocated, verified and initialized. The system performance is improved by the fact that the selftest is hardware-driven whereas in prior art, the firmware drove the selftest. More comprehensive test patterns can be used to improve system memory RAS as well.

Description:
TRADEMARKS 
       [0001]    IBM® is a registered trademark of International Business Machines Corporation, Armonk, N.Y., U.S.A. Other names used herein may be registered trademarks, trademarks or product names of International Business Machines Corporation or other companies. 
       BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    This invention relates to computer system design, and particularly to the system that have large central storage. 
         [0004]    2. Description of Background 
         [0005]    A method for testing a memory device which has a plurality of memory locations each having a corresponding memory address is known as a memory selftest from U.S. Pat. No. 5,033,048 granted Jul. 16, 1991. 
         [0006]    IBM has supplied a memory selftest hardware engine to customers for many years. IBM&#39;s hardware which is provided to a customer usually is more than the customer has required at purchase, and the customer generally pays for a configuration of the hardware system in accordance with what he needs based on the real time workloads. The hardware system will release reserved resources on-demand in accordance with such reconfiguration and initialization which has been done by firmware at IML. Memory sub-system resources of a Central Storage belong to this category where the customer is allowed to access only the memory he has purchased. Once the customer&#39;s needs expand and he is willing to buy more memory, the memory sub-system can be reconfigured to release more reserved memory for his use. On the other hand, should the customer&#39;s needs diminish, the memory sub-system can be reconfigured to have smaller amount of memory available as well. 
         [0007]    Once more reserved memory is released to customer, the newly allocated memory needs to be tested via Test Block instruction, fixed by DRAM sparing if necessary, and initialized. Also, once any unused memory is reclaimed back. The data stored in that memory region needs to be erased or perhaps scrambled. 
         [0008]    As we have said, in prior IBM machines, such reconfiguration and initialization were done by firmware. It involved a full system shutdown and reboot. Also, the time to test a memory region was really slow due to the fact that it was firmware-driven. Also, the test patterns used for the test were very limited. Those hardware memory selftest engines which existed were only run during system initial machine load (IML) time or to scrub memory during customer operations. 
         [0009]    To solve this problem, we have developed and introduced a concurrent memory selftest for use in the IBM z9-109 mainframe system. The current selftest engine, working along with firmware, allows us to do comprehensive memory test operations on the extended customer memory area before it is released to him while the customer mainline memory accesses are running in parallel. The memory about to be allocated could be tested, repaired by sparing if necessary, and cleared. Or the data in the memory just de-allocated can be erased or scrambled. The concurrent selftest activities are totally transparent to any customer operations. Only a fraction of system total memory bandwidth is used to achieve this work. Because the selftest sequences are done by hardware, the time needed to inspect the entire memory region about to be allocated is substantially reduced. 
       SUMMARY OF THE INVENTION 
       [0010]    The shortcomings of the prior art are overcome and additional advantages are provided through the provision of our new method that concurrently tests and repairs memory. The memory can now be dynamically allocated or de-allocated because of customers&#39; demands, as well as run during system initial machine load (IML) time or to scrub memory during customer operations. 
         [0011]    Selftest needs to be performed on newly allocated area to check and initialize the memory. The concurrent selftest activities are totally transparent to any customer operations. Only a fraction of system total memory bandwidth is used to achieve this work. 
         [0012]    System and computer program products corresponding to the above-summarized methods are also described and claimed herein. 
         [0013]    Additional features and advantages are realized through the techniques of the present invention. Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention. For a better understanding of the invention with advantages and features, refer to the description and to the drawings. 
       Technical Effects 
       [0014]    As a result of the summarized invention, technically we have achieved a solution which dynamically checks and repairs the newly allocated memory based on customers&#39; demand. This method improves system performance, as well as the system Reliability, Availability and Serviceability (RAS). The design is flexible and efficient. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]    The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which: 
           [0016]      FIG. 1  illustrates one example of Memory traffic flows in a z9-109 memory controller with memory selftest engines 
           [0017]      FIG. 2  illustrates one example of typical z9-109 memory architecture. 
           [0018]      FIG. 3  illustrates one example of a block diagram of the concurrent selftest engine 
           [0019]      FIG. 4  illustrates one example of a z9-109 system memory configuration 
       
    
    
       [0020]    The detailed description explains the preferred embodiments of the invention, together with advantages and features, by way of example with reference to the drawings. 
       DETAILED DESCRIPTION OF THE INVENTION 
       [0021]    We implement our invention with concurrent selftest hardware provided with the system which contains of two major pieces of hardware: selftest engine and priority logic. When concurrent selftest is needed, the hardware selftest engine is first setup by firmware. Generally, the starting and ending addresses, address mode, and data mode are initialized. After the setup under the firmware the selftest engine will start sending fetch and store commands to the priority logic in the background. The priority logic will take the commands from the selftest engine and regular mainline traffic, prioritize them, and send them sequentially over to the Processor Memory Arrays (PMA) section of the memory sub-system. 
         [0022]    Turning now to the drawings in greater detail, it will be seen that in  FIG. 1  there is a system block diagram that shows how the memory traffic is handled. 
         [0023]    In the z9-109 implementation, the MSC (Main Storage Controller) chip has an X port and a Y side each independently controlling a PMA. Within the hardware we have provided a plurality of ports for a memory region of the global system storage, each of these ports has a concurrent selftest engine which is assigned to test a memory region within a set of DRAMs on the PMA to which it is assigned. There are X and Y ports of the Controller which operate independently, and the engines in both the X and Y ports can be operating in parallel as well. There are two MSC chips to a node, and both MSC chips in a node can be operating in parallel, as can the 4 nodes in a system. That adds up to 16 selftest engines running allocated to memory regions concurrently to quickly verify the quality of the pre-allocated extended memory. See the illustrations described below with respect to the Figures for the selftest hardware engines in a system. 
         [0024]      FIG. 2  shows an example of the memory architecture of z9-109 implementation. 
         [0025]    Concurrent Selftest Engine 
         [0026]    Below are the detailed explanations of each of the components The concurrent selftest engine is the core of the hardware employed to test and repair memory and to dynamically allocate or de-allocate memory regions because of customers&#39; demands. Once setup, it will generate the memory fetch and store commands to the priority logic. For memory stores, the selftest engine can use fixed or random data patterns. For memory fetches, the hardware memory selftest engine (in a manner different from the selftest engine of U.S. Pat. No. 5,003,048) will check the data validity either by bit comparing or by ECC checking, and update the selftest status based on the results. 
         [0027]      FIG. 3  shows the block diagram of the concurrent selftest engine. 
         [0028]    The firmware implements the following setup parameters which are used in a hardware concurrent selftest engine. The settings are divided into 4 categories. 
         [0029]    Address Control Parameters 
         [0030]    1. Starting Address 
         [0031]    It defines starting address of the extended memory region that the concurrent selftest will be working on. 
         [0032]    2. Ending Address 
         [0033]    It defines ending address of the extended memory region that the hardware concurrent selftest engine will be working on. 
         [0034]    4. LICCC (Licensed Internal Code Configuration Control) Address 
         [0035]    It defines the upper limit of the customer address space. It is used as a control to prevent any selftest accesses from entering the customer&#39;s address range. Any setup error or internal control error results in a specification error status being posted to the firmware. 
         [0036]    Data Control Parameters 
         [0037]    4. Data Generation Mode 
         [0038]    For selftest writes, the firmware setup controls the data and requires it to be either fixed data pattern or random data pattern. In fixed data pattern mode, the data generated will be from Data Pattern Parameter. In random data pattern mode, the data will be calculated by a random data generator. 
         [0039]    5. Data ECC Mode: ECC/Compare Mode 
         [0040]    The firmware defines the way data is sent to or returned from memory. In ECC mode, the data will be transferred along with an ECC code. On a fetch operation the fetch ECC station will check the ECC results. In compare mode, the data will be transferred as 144 bit data without ECC. On a fetch operation, the data is compared against a known data pattern to verify its validity, 
         [0041]    6. Data Pattern 
         [0042]    The data control parameter holds the implemented data pattern. It is used in fixed data pattern mode and also used as the starting point by the random data generator in random data mode. 
         [0043]    7. Random Data Generation Mask 
         [0044]    The random data generation mask is used by the random data generator to generate random data patterns. 
         [0045]    Operation Sequence Control Parameters 
         [0046]    8. Gap Control 
         [0047]    The firmware gap control is used to introduce artificial gaps between the commands that the hardware memory selftest engine sends to memory. This would affect the data bandwidth that the engine uses comparing to the overall memory data bandwidth. This would affect the system performance since the concurrent selftest engine shares the same memory and memory ports with mainline function. In concurrent mode, speed in completing the testing is generally not a factor. Thus, the gap is generally set fairly large to limit the data bandwidth usage. 
         [0048]    9. Start/Stop Bits 
         [0049]    Start/stop bits are the main switch to turn on/off the selftest engine. 
         [0050]    Status and Error Reporting Registers 
         [0051]    10. Status Register 
         [0052]    A status register stores the current status of the concurrent selftest engine and the overall testing results. Firmware can poll this register periodically to watch the selftest progress and check the overall selftest results. 
         [0053]    11. Bit Error Counters 
         [0054]    Each data bit has a corresponding bit error counter that keeps track of how many errors have occurred during the memory selftest. During the concurrent selftest in compare mode, should miscompares occur, the selftest engine will increment the count for the corresponding bit. In ECC mode, the counters also increment when data CE is detected. 
         [0055]    Priority Logic 
         [0056]    The main function of the hardware priority logic is to merge the memory command stream from selftest engine with the mainline memory command stream together. The priority logic can be programmed to treat the selftest commands with normal priority or lower priority. 
         [0057]    In normal priority mode, the priority logic will treat both selftest command and mainline command in the same manner. The commands arc basically executed based on the availability of the DRAM banks only. The memory bandwidth used by selftest commands is mainly controlled by the ‘gap control’ parameter of the selftest engine, 
         [0058]    In low priority mode, the priority logic will give the selftest command lower weight than the regular mainline commands. The selftest command will only be executed if there are no outstanding mainline commands pending. This will minimize the performance impact that concurrent selftest posts would cause on the mainline memory operations. 
         [0059]    The other function that the priority logic provides is by the hardware that handles the memory bank/rank conflicts. Traditionally, all the incoming mainline commands are targeting different memory banks by design. However, we have added memory selftest commands in background could target a memory bank that is currently being used by regular mainline commands. When such a conflict occurs, the priority logic will delay sending out the later coming command until its target memory bank is freed. 
         [0060]    Firmware 
         [0061]    Firmware is the driving force for the concurrent selftest. Basically, when such selftest is needed, firmware first sets up the selftest engine with parameters detailed in the above section. Once the concurrent selftest is initiated, all the hardware memory selftest engines on each memory port run in parallel. The firmware periodically polls the selftest status. Once all the engines finished the tests on its own memory port, the firmware can retrieve all the error status information out and takes indicated and proper actions based on the results, e.g. sparing the DRAM chips and other operations. 
         [0062]    Applications 
         [0063]    The central storage regions can be categorized as follow. The selftest engine will mainly work on the inactive regions and the unassigned regions, once the system storage configuration is changed on-demand by the customer. 
         [0064]      FIG. 4  shows a typical storage configuration of a z9-109 system 
         [0065]    The performance gets boosted substantially since the activities are done by hardware and no firmware code is involved during the selftest execution. The concurrent selftest engine can be used for the following scenarios: 
         [0066]    1. Concurrently Verify/Test the Newly Allocated Memory Region 
         [0067]    (Once the new memory is allocated, the concurrent selftest is performed to verify the memory content has any defects or not. This has performance advantage over the existing implementation.) 
         [0068]    2. Concurrently Initialize the Newly Allocated Memory Region per Architecture. 
         [0069]    (Once the newly allocated the memory has tested defect-free, the memory needs to be initialized with a certain data pattern before being turned over to customer usage. The data pattern is determined per system architecture.) 
         [0070]    3. Concurrently Clear an Unused Memory Region that Application is no Longer Active. 
         [0071]    (For data security reason, the concurrent selftest can be used to clear a chunk of memory with a fixed data pattern thus erasing all the leftover customer information.) 
         [0072]    4. Concurrently Scramble an Unused Active Memory Region 
         [0073]    (This new capability is useful for data security. The concurrent selftest can be used to clear a chunk of memory with a random data pattern thus erasing all the leftover customer information.) 
         [0074]    The capabilities of the present invention are and can be implemented in software, firmware and hardware as a combination thereof using the hardware memory selftest engine. 
         [0075]    As one example, one or more aspects of the present invention can be included in an article of manufacture (e.g., one or more computer program products) having, for instance, computer usable media for implementing the invention. The media has embodied therein, for instance, computer readable program code means for providing and facilitating the capabilities of the present invention. The article of manufacture can be included as a part of a computer system or sold separately. 
         [0076]    Additionally, at least one program storage device readable by a machine, tangibly embodying at least one program of instructions executable by the machine to perform the capabilities of the present invention can be provided. 
         [0077]    The flow diagrams depicted herein are just examples. There may be many variations to these diagrams or the steps (or operations) described therein without departing from the spirit of the invention. For instance, the steps may be performed in a differing order, or steps may be added, deleted or modified. These steps can be provided as a service to the customer. All of these variations are considered a part of the claimed invention. 
         [0078]    While the preferred embodiment to the invention has been described, it will be understood that those skilled in the art, both now and in the future, may make various improvements and enhancements which fall within the scope of the claims which follow. These claims should be construed to maintain the proper protection for the invention first described.