Abstract:
The invention relates to a method for storage management in a data processing system having a plurality of storage devices with different performance attributes and a workload. The workload is being associated with respective sets of data blocks to be stored in said plurality of storage devices. The method comprises the steps of dynamically determining performance requirements of the workload and dynamically determining performance attributes of the storage devices. The method further comprises the step of allocating data blocks to the storage devices depending on the performance requirements of the associated workload and the performance attributes of the storage devices.

Description:
BACKGROUND 
       [0001]    1. Field of the Invention 
         [0002]    The present invention relates to a data storage subsystem also known as an auxiliary storage manager. The auxiliary storage manager is responsible for moving frames from central storage to auxiliary storage (AUX). The auxiliary storage is usually a file on a direct access storage devices (DASD) or hard disk. 
         [0003]    2. Description of the Related Art 
         [0004]    One example of implementation of such a subsystem is in the IBM® operating system z/OS®. The IBM® operating system z/OS® is a virtual storage system which comprises mapping virtual storage pages to frames in central storage or slots in auxiliary storage. A virtual storage system, such as the IBM® operating system z/OS®, only loads those portions of a program that are needed for the momentary operation of the program into central storage. The inactive pieces of the program, however, are kept in auxiliary storage. An auxiliary storage manager is responsible for moving storage pages or data blocks from the central storage to the auxiliary storage when not currently needed. Another component of the z/OS operating system is the workload manager. It controls access to system resources for the work executing on z/OS based on administrator-defined goals. 
         [0005]    New storage technologies provide non-volatile storage devices with different access speeds. For example new devices such as Flash memories are faster than conventional slower DASD (direct access storage devices) or hard disk drives. Therefore the need for a suitable workload allocation arises. 
         [0006]    Different state of the art documents are concerned with storage or memory allocation. Document US 2009/0019097 describes a system and method for memory allocation management. Memory allocation requests comprise parameters for indicating requirements including a priority, a mandatory status etc. Each of a plurality of memories, such as scratch memory, persistent memory, etc., is characterized by operating speed, capacity and suitability of application types. Memory allocations are optimized by examining the parameters and a memory map is generated. 
         [0007]    Another state of the art document, U.S. Pat. No. 7,062,628, describes a method and apparatus for storage pooling and provisioning for journal based storage and recovery. A set of interconnected storage systems supporting different types of storage devices and different performance attributes are intelligently applied to process types, such as journal entries. The processes are ranked according to a predetermined priority ranking. The highest priority process is matched with available devices from storage pools that rank highest in the processes performance priority. The storage pools are ranked according to capacity, reliability, and access rate. 
         [0008]    Document U.S. Pat. No. 6,760,910 describes a workload distribution management method to enhance shared resource access in a multi-system environment and to meet a common performance standard. The method comprises dynamically tracking use by a plurality of work classes of a plurality of resources in the multi-system. Each work class comprises at least one work unit. A system resource manager dynamically forms a plurality of sets of the shareable resources and dynamically associates each work class with a set of the shareable resources based on resources currently employed by the at least one work unit. 
         [0009]    The problem is that new storage technologies provide non-volatile storage devices with different access speeds, for example, a Flash memory is faster than conventional slower DASD (direct access storage devices) or hard disk drives. To enable fast data processing, the time needed for moving pages from the auxiliary storage back to the central storage should be as short as possible. A complete replacement of the slower auxiliary storage devices with the faster ones may become too expensive. Therefore, an economic way to distribute pages over the storage devices with different access speeds is desired. 
       SUMMARY 
       [0010]    The present invention provides a method for storage management that utilizes different access speeds of the storage mediums to its advantage. 
         [0011]    More specifically, the present invention provides a method for storage management in a data processing system having a plurality of storage devices with different performance attributes and a workload. The workload is being associated with respective sets of data blocks to be stored in the plurality of storage devices. The method comprises the steps of dynamically determining performance requirements of the workload and dynamically determining performance attributes of the storage devices. The method further comprises the step of allocating data blocks to the storage devices depending on the performance requirements of the associated workload and the performance attributes of the storage devices. 
         [0012]    Through the claimed method new storage technologies can be advantageously applied. The different access speeds provided by non-volatile storage devices, for example, a Flash memory being faster than conventional slower DASD (direct access storage devices) or hard disk drives can now be adequately allocated to the performance requirements of the workloads. For fast data processing requirements, the time needed for moving pages from the auxiliary storage back to the central storage can now be as short as possible. 
         [0013]    In a further embodiment of the claimed invention the step of dynamically determining the performance requirements of the workload comprise the steps of monitoring performance of the workload and repeatedly determining whether the performance of the workload meets the performance requirements of the workload. The method further comprises the steps of changing performance requirements of the storage devices to be allocated to the monitored workload depending on the previous determining step. 
         [0014]    This further embodiment of the invention constantly checks if a workload is performing within its performance requirements. If this is not the case, then the storage allocation is adjusted accordingly. 
         [0015]    In a further embodiment of the claimed invention the method further comprises the steps of determining performance requirements of data blocks of the workload, determining a pattern of performance requirements of the data blocks of said workload and allocating data blocks of the workload to storage devices based on the pattern. 
         [0016]    Determining the pattern of allocation of data block of a certain workload allows for the possibility of shortcutting the method. If the same workload for, which such a pattern has been determined, is repeated, then the pattern of storage allocation may be used on the data blocks of that workload. 
         [0017]    In yet a further embodiment of the claimed invention the performance attributes of the plurality of storage devices comprise at least one of the following:
       an access rate,   a free storage capacity, and   a reliability.       
 
         [0021]    In a further embodiment of the claimed invention the data processing system comprises a central storage to store data blocks currently needed for executing the workload and wherein said storage devices are auxiliary storages to store data blocks not currently needed for executing the workload. 
         [0022]    In a further embodiment of the claimed invention the plurality of storage devices comprise at least a fast storage device and a slow storage device, wherein the fast storage device has a higher access rate than the slow storage device. 
         [0023]    In a further embodiment of the claimed invention the method further comprises the steps determining priorities for plural workloads, wherein critical workloads have a higher priority than non-critical workloads and allocating the workloads to the storage devices depending on the priority of the workloads. 
         [0024]    As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon. 
         [0025]    Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. 
         [0026]    A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. 
         [0027]    Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. 
         [0028]    Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user&#39;s computer, partly on the user&#39;s computer, as a stand-alone software package, partly on the user&#39;s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user&#39;s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). 
         [0029]    Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. 
         [0030]    These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
         [0031]    These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks. 
         [0032]    The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0033]    Preferred embodiments of the invention are illustrated in the accompanying figures. These embodiments are merely exemplary, i.e. they are not intended to limit the content and scope of the appended claims. 
           [0034]      FIG. 1  shows the main components of an embodiment of the advanced auxiliary subsystem. 
           [0035]      FIG. 2  shows in more detail the interaction between the auxiliary storage manager and the workload manager of a preferred embodiment of the invention. 
           [0036]      FIG. 3  shows a preferred embodiment of the workload manager decision logic. 
           [0037]      FIG. 4  shows a preferred embodiment of building an allowed usage table. 
       
    
    
     DETAILED DESCRIPTION 
       [0038]    Shown in  FIG. 1  are the components which comprise an advanced subsystem. The main components of an advanced subsystem are the workload manager  101  and the auxiliary storage manager  102 . Also shown in  FIG. 1  are the different storage devices,  110  is the central or real storage,  115  is the fast auxiliary storage, while  116  is the slow auxiliary storage. 
         [0039]    The workload manager  101  classifies all incoming workload based on policies defined by the customer. The advanced auxiliary storage subsystem works with two performance requirements: importance (IMP) and storage critical (SC). 
         [0040]    The workload manager uses 7 levels of importance. Importance 0 has the highest importance in the system and importance 6 (discretionary) has the lowest importance in the system. 
         [0041]    The storage critical (SC) is an indicator of the importance of the data set to the functioning of the application. It has two settings, critical and non-critical. When the workload has the storage critical indicator set in the workload manager policy, then the workload will only lose storage to other work with equal or greater importance. If the importance of the other work is lower than that of the workload, then no changes will happen. The storage critical indicator is also used as part of the workload placement decision. 
         [0042]    The workload manager  101  also works with a performance index (PI). The performance index is an indicator as to how well the workload meets the requirements in the workload manager  101  policy. PI&gt;1 indicated that the workload doesn&#39;t meet the goals, while PI&lt;1 indicates that the workload overachieves the goal. The workload manager  101  analyses the performance index and depending on the value the workload manager tries to determine the reasons for the particular PI value. 
         [0043]    The workload manager  101  of the advanced auxiliary subsystem monitors the usage of the real  110  and auxiliary storages  115  and  116 . This monitoring is represented in  FIG. 1  by the lines  120 ,  125  and  126  respectively. The monitoring of the storages is necessary, because the current usage of each storage influences the decision made by the workload manager concerning the placement of new frames. 
         [0044]    The auxiliary storage manager  102  is responsible for taking  130  a frame from the central/real storage  110  and store  135 ,  136  the frame in the auxiliary storage  115 ,  116 . In the case of an advanced subsystem the placement  115 ,  116  is done based on decisions made by the workload manager  101 . Prior to storing  135 ,  136  the frame, the auxiliary storage manager  102  communicates  140  with the workload manager  101 . The auxiliary storage manager  102  passes the address space ID (ASID) of the frame owner to the workload manager  101 . 
         [0045]    The workload manager  101  has to answer such a request with a placement decision (slow  115  or fast  116  auxiliary storage). Then the auxiliary storage manager  102  stores  135 ,  136  the frame previously retrieved  130  from the real storage  110  in the auxiliary storage  115 ,  116  based on the decision of the workload manager  101 . 
         [0046]    Shown in  FIG. 2  is a detailed flow of an auxiliary subsystem. When the auxiliary storage manager  102  retrieves a frame, the workload of the frame is determined  200  by analyzing the address space (ASID). In step  201  the auxiliary storage manager  101  creates a slot placement request and passes the slot placement request to the workload manager  101 . Based on the current usage of the fast auxiliary storage  220 , a quick decision  221  or a full decision  300 ,  221  or  222  is made by the workload manager. If more than 50% of the fast auxiliary storage  221  is free, the decision is always to place the slot on the fast auxiliary storage  221 . 
         [0047]    When the fast auxiliary storage is filled above 50%, then the “importance dependent” slot placement takes place in  300 . 
         [0048]    Now referring to  FIG. 3 , the workload manager decision logic  300  converts  310  the passed ASID to the related importance (IMP)  310 . This is done because all further processing is importance related. Then the workload manager logic  300  analyses in step  320  if the address space has the storage critical (SC) indicator set or if it has the Performance Index (PI)&gt;1 and shows delays. If either case is true, then the advanced importance (AP_IMP) is set to the importance minus one in step  321 . Otherwise the advanced importance (AP_IMP) is set to the importance in step  322 . Finally in step  330  the current fast auxiliary storage usage (in percent) is compared with the allowed usage for the advanced importance (AP_IMP) of the slot. The allowed usage is in a table  490 , which is rebuilt every 2 seconds by the workload manager  101  (see  FIG. 4 ). 
         [0049]    If the current auxiliary storage usage is higher than the allowed usage for the slot, then the workload manager returns to the auxiliary storage manager  101  with the command “Use slow auxiliary storage” in step  222 . Otherwise it returns with the command “Use fast auxiliary storage” in step  221 . Either command is then passed to the auxiliary storage manager through  140 . 
         [0050]    Shown in  FIG. 4  is the building an allowed usage table  490 . Every 2 seconds the workload manager  101  builds a new allowed usage table  490 , which is a 7×2 matrix. The first column contains the allowed usage percentage by importance (AP_IMP) and the second column contains the number of pageable frames of all address spaces by importance. 
         [0051]    The build of an allowed usage table process starts with clearing the pageable frame column. This is done in step  401  of  FIG. 4 . Then in step  410  the process starts at address space  1 . The other address spaces are looped through in steps  411 ,  412  and  413 . In particular, the following is done for all address spaces:
   1 step  411  extracts the importance for the current address space.   2 step  412  increase the pageable frames count in the allowed usage table, by the pageable frames of the current address space. Also, the row in the allowed usage table is indexed by the importance. (IMP index  frames=IMP index  frames+pageable frames of the current address space)   3 step  413  continues with the next address space until the last address space has been processed.   
 
         [0055]    In step  410  the advanced importance (AP_IMP) for all 7 rows is calculated. This done by the following formula: 
         [0000]        AP   —   IMP   row =( IMP   row  frames×100)/SUM( IMP 0 frames to  IMP 6 frames)
 
         [0056]    Finally in step  430  the process waits 2 seconds and restarts the rebuilding the allowed usage table after the 2 seconds. 
         [0057]    The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. 
       REFERENCE SIGNS 
       [0000]    
       
           101  workload manager 
           102  auxiliary storage manager 
           110  real storage 
           115  auxiliary storage fast 
           116  auxiliary storage slow 
           120  monitoring real storage 
           125  monitoring auxiliary storage fast 
           126  monitoring auxiliary storage slow 
           130  auxiliary storage manager receiving (stealing) frame 
           135  auxiliary storage manager storing frame to auxiliary storage slow 
           136  auxiliary storage manager storing frame to auxiliary storage fast 
           140  communication 
           200  steal frame and identify ASID 
           201  request frame placement decision and pass ASID 
           202  process stolen frame as decided by the workload manager 
           220  auxiliary storage (fast) usage &lt;50% 
           221  use fast auxiliary storage 
           222  use slow auxiliary storage 
           300  workload manager decision logic 
           310  convert ASID to related IMP 
           320  (SC=on) or (PI&gt;1 and delay) 
           321  AP_IMP=IMP−1 
           322  AP_IMP=IMP 
           330  auxiliary storage (fast) usage &lt;allowed usage for AP_IMP 
           400  start 
           401  clear pageable frame colon in allowed usage table 
           410  select 1. address space 
           411  convert ASID to IMP 
           412  increase the IMP related frame count in the allowed usage table by number of pageable frames from the current address space 
           413  more address spaces 
           420  calculate the IMPn % in the allowed usage table for all importances 
           421  advanced importance calculation 
           430  schedule next allowed usage update table after 2 seconds 
           490  allowed usage table