Abstract:
The invention relates to a system and method for prioritizing one or more data processing operations in a computer storage system, the computer storage system including a plurality of modules, the method comprising receiving a command indicating one or more data processing operations to which priority is to be assigned and interfacing with each of the modules so as to prioritize the one or more data processing operations over other data processing operations.

Description:
RELATED APPLICATIONS 
     This patent application claims priority to Indian patent application serial number 1917/CHE/2007, having title “Prioritising Data Processing Operations”, filed on 27 Aug. 2007 in India (IN), commonly assigned herewith, and hereby incorporated by reference. 
     BACKGROUND OF THE INVENTION 
     The increasing use of applications with significant data throughput requirements, such as online transaction processing applications, combined with the increasing use of file servers and rich media data servers, has led to an input/output (I/O) intensive application space which demands fast processing of large volumes of data. Such I/O intensive environments can have large, sustained workloads involving a wide range of I/O data transfer sizes as well as periodic burst accesses, depending on the applications involved. 
     When applications which are executing in parallel demand fast processing, an acceptable level of performance on a per application basis becomes a major customer requirement. Even in systems with resources such as large memory, multiple CPUs and associated resource management utilities, I/O subsystem bottlenecks may exist because the I/O system as a whole may not be configured to run optimally. Furthermore, applications for which I/O operations are critical, such as online transaction processing, currently compete equally for resources with applications for which I/O is non-critical, negatively impacting the critical application&#39;s requirements. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Embodiments of the invention will now be described, by way of example, with reference to the accompanying drawings, in which: 
         FIG. 1  illustrates a storage area network according to an embodiment of the present invention; 
         FIG. 2  is a high-level overview of a processing system; 
         FIG. 3  illustrates the configuration of a host, intelligent switch and storage device of the storage area network of  FIG. 1  in more detail; 
         FIG. 4  is a flow diagram illustrating the process of resource tuning for prioritisation according to an embodiment of the present invention; 
         FIG. 5  is a flow diagram illustrating a host storage stack layer registration process according to an embodiment of the invention; 
         FIG. 6  is a flow diagram illustrating a network storage stack layer registration process according to an embodiment of the invention; 
         FIG. 7  is a flow diagram illustrating the steps performed in processing user prioritisation instructions according to an embodiment of the invention; and 
         FIG. 8  is a flow diagram illustrating the steps performed in synchronised tuning of the host and network stack layers according to an embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     Referring to  FIG. 1 , a storage area network (SAN)  1  includes a plurality of hosts  2   a ,  2   b ,  2   c , each having two host bus adaptors (HBAs)  3   a ,  4   a ,  3   b ,  4   b ,  3   c ,  4   c  connected via a plurality of first data links  5  to a plurality of intelligent switches  6   a ,  6   b . The intelligent switches  6   a ,  6   b  route data between the hosts  2   a ,  2   b ,  2   c  and a plurality of storage elements  7   a - d  connected to the intelligent switches  6   a ,  6   b  via a plurality of second data links  8 . The plurality of hosts  2   a - c  are, in the present example, servers, although they may alternatively be client computers. The storage elements  7   a - 7   c  comprise disk drive arrays  7   a ,  7   b ,  7   c  and a tape library  7   d . The first and second data links  5 ,  8  between the hosts  2   a - c , intelligent switches  6   a ,  6   b  and storage arrays  7   a - d  are, in the present example, fibre channel connections, but can alternatively be other connections such as ethernet connections, SCSI (Small computer System Interface) interfaces or TCP/IP connections, as known to those skilled in the art. The network of HBAs  3   a - 3   c ,  4   a - 4   c , data-links  5 ,  8 , and switches  6   a ,  6   b  that connect the hosts  2   a - 2   c  and the storage elements  7   a - 7   d  make up what is known as the fabric of the storage area network  1 . 
     A third data link  9  is provided between the hosts  2   a - c , switches  6   a ,  6   b  and storage arrays  7   a - d , and is used to carry storage network stack resource control data according to the present invention, as will be described in more detail below. The third data link  9  in the present example comprises a TCP/IP data link, although the fibre channel connections  5 ,  8  can alternatively be used, or alternative connections known to those skilled in the art. 
     Each host  2   a ,  2   b ,  2   c , is allocated a storage area which can be accessed through the fabric of the SAN  1 . The storage area may be distributed over different storage elements  7   a - 7   d . A LUN or Logical Unit Number (also referred to as Logical Unit i.e. LU), corresponds to an actual or virtual portion of a storage element. For example, a LUN may correspond to one or more disks in a disk drive array. 
     The hosts  2   a - 2   c  may be connected in a local area network (LAN) to a number of client computers (not shown). The client computers may be directly connected to the SAN  1  with their own data links, such as fibre channel links, or indirectly connected via the LAN and the hosts  2   a - 2   c.    
       FIG. 2  is a high-level overview of a processing system, such as those operating on the hosts  2   a - 2   c , illustrating the inter-relationship between software and hardware. The system includes hardware  20 , a kernel  21  and application programs  22 . The hardware is referred to as being at the hardware level  23  of the system. The kernel  21  is referred to as being at the kernel level or kernel space  24  and is the part of the operating system that controls the hardware  20 . The application programs  22  running on the processing system are referred to as being at a user level or user space  25 . 
       FIG. 3  illustrates the host  2   a , intelligent switch  6   a  and storage array  7   a  of  FIG. 1  in more detail. The host  2   a  runs the HP UX™ operating system and includes a number of applications  26  operating in user space  27 , the applications generating I/O operation requests  28  to be processed in kernel space  29 . The kernel space  29  includes a file system  30 , volume manager  31 , multipathing driver  32 , device driver  33 , in the present example a SCSI device driver, and host bus adaptor driver  34 , each forming a layer of what is referred to as the host storage software stack. Other layers are possible in addition to or in place of the layers illustrated in  FIG. 3 , for instance a filter driver or a device driver above or below any of these layers performing specialised operations, or specific to a mass storage protocol such as iSCSI. Also, a fibre channel driver and/or an Infiniband driver could form part of the host storage software stack. 
     The kernel space  29  also includes the HBAs  3   a ,  4   a  of the host  2   a , which are controlled by the host bus adaptor driver  34  and connect to data link  5  linking the host  2   a  to the intelligent switch  6   a  and, in turn, via data link  8  to storage array  7   a.    
     Each of the layers of the host storage software stack has a respective tuner module  35 - 39  arranged to support tuning of the layer for prioritisation control. The tuners interface with a stack interface service (SIS)  40  implemented as a pseudo driver module in kernel space  29 . The stack interface service  40  receives commands from a priority and resource manager  41 , also implemented as a pseudo driver module in kernel space  29 . The stack interface service  40  is arranged to receive data from a user administrative control application  42  running in user space  27  and from a SAN resource agent  43 . The user administrative control application  42  receives user inputs  44  via a user input interface which could be a command line interface or a graphical user interface. 
     The intelligent switch  6   a  and the storage array  7   a  are referred to as layers of the network storage software stack and are also each provided with a respective tuner  45 ,  46  arranged to support tuning of their resources for prioritisation control. The network stack layers  6   a ,  7   a  are also provided with a respective network stack agent  47 ,  48 , which interfaces with the tuner  45 ,  46 . The SAN resource agent  43  is configured as a daemon process running in user space  27  and is used to interface with the network stack agents  47 ,  48  of the SAN components via the third data link  9 . 
       FIG. 4  is a flow diagram illustrating the process of resource tuning for prioritisation at the host  2   a , intelligent switch  6   a  and storage array  7   a  illustrated in  FIGS. 1 and 3 . 
     In a first stage in the process, the layers  30 - 34  of the host storage software stack register with the stack interface service  40  (step S 1 ). 
       FIG. 5  illustrates the steps involved in the registration of each of the host storage stack layers  30 - 34 . The SIS driver  40  is loaded into the kernel  29  along with the priority and resource manager  41  (step S 1 . 1 ). The individual layer  30 - 34  is then loaded into the kernel  29  (step S 1 . 2 ). When a layer is loaded in the kernel, this layer will register with the SIS  40  using an ioctl interface (step  1 . 3 ). In particular, the storage layer issues an ioctl call to the driver  40  having the following parameters: 
     
       
         
               
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 ioctl 
                 (Device Descriptor, 
               
               
                   
                   
                 Stack Identification No., 
               
               
                   
                   
                 Control Option, 
               
               
                   
                   
                 Supported Attributes, 
               
               
                   
                   
                 Tuning Function Pointer (Function Parameters) 
               
               
                   
                   
                 ) 
               
               
                   
                   
               
             
          
         
       
     
     The definitions of the storage layer ioctl parameters are set out below in table 1.0. 
     
       
         
               
               
               
             
           
               
                 TABLE 1.0 
               
               
                   
               
               
                 No. 
                 Parameter 
                 Definition 
               
               
                   
               
             
             
               
                 1 
                 Device Descriptor 
                 Stack interface service driver descriptor 
               
               
                 2 
                 Stack 
                 Indicates the layer that is registering with 
               
               
                   
                 Identification No. 
                 the SIS driver 
               
               
                 3 
                 Control Option 
                 Indicates to the SIS driver whether the 
               
               
                   
                   
                 layer opts in or out of resource control 
               
               
                 4 
                 Supported Attributes 
                 The attributes which this tuner can support, 
               
               
                   
                   
                 for instance application id/process id, I/O 
               
               
                   
                   
                 type, I/O request size, storage device or 
               
               
                   
                   
                 LUN 
               
               
                 5 
                 Tuning Function 
                 A function pointer supplied by the stack 
               
               
                   
                 Pointer 
                 layer for tuning control. 
               
               
                   
               
             
          
         
       
     
     Each host storage stack layer therefore indicates, via the ioctl call, whether or not it can control its resources so as to prioritise I/O operations based on attributes such as a particular application or process identification number, an input/output type or request size or an end LUN. The ‘control option’ parameter is used to specify whether the layer opts in or out of resource control in general and the ‘supported attributes’ parameter is used to specify the particular attribute(s) of I/O operations that the layer can support for prioritisation. The tuning function pointer is used to control the tuning of the layer for resource prioritisation when required. 
     The stack interface service  40  communicates the ioctl call parameters received from each stack layer to the priority and resource manager  41  using a function call exposed by the priority and resource manager  41  to the SIS  40  (step S 1 . 4 ). 
     The priority and resource manager  41  records the host storage stack layers that register, together with an indication as to whether they have opted for resource control via the ‘control option’ parameter (step  1 . 5 ) and, if so, records the attributes that can be controlled, as specified in the ‘supported attributes’ parameter (step S 1 . 6 ). 
     Referring again to  FIG. 4 , following the host registration process, a network stack layer registration process is performed (step S 2 ). 
       FIG. 6  illustrates the network stack layer registration process. The SAN resource agent  43  issues a request for registration to the network stack agent  47 ,  48  of each network stack layer  6   a ,  7   a  (step S 2 . 1 ). The network stack agent  47 ,  48  of each layer responds with a PriorityControlResponse indicating that registration is complete (step S 2 . 2 ). 
     The PriorityControlResponse returned to the SAN resource agent  43  includes a header, stack identification number, control option and supported attributes, as defined in table 2.0 below. 
     
       
         
               
               
               
             
           
               
                 TABLE 2.0 
               
               
                   
               
               
                 No. 
                 Parameter 
                 Definition 
               
               
                   
               
             
             
               
                 1 
                 Header 
                 Indicates to the stack interface service that 
               
               
                   
                   
                 this is a network stack response 
               
               
                 2 
                 Stack 
                 Indicates the layer that is registering with 
               
               
                   
                 Identification No. 
                 the SIS driver (intelligent switch, storage 
               
               
                   
                   
                 array etc) 
               
               
                 3 
                 Control Option 
                 Indicates to the SIS driver whether the 
               
               
                   
                   
                 layer opts in or out of resource control 
               
               
                 4 
                 Supported Attributes 
                 The attributes which this tuner can support, 
               
               
                   
                   
                 for instance a storage device, end LUN, or 
               
               
                   
                   
                 an array controller port 
               
               
                   
               
             
          
         
       
     
     The SAN resource agent  43  receives the PriorityControlResponse and communicates the data received in the response to the stack interface service  40  via an ioctl call (step S 2 . 3 ). The parameters in the ioctl call between the SAN resource agent  43  and the stack interface service  40  are parameter numbers 1 to 4 listed in table 1.0, namely the device descriptor, stack identification number, control option and supported attributes, where the stack identification number, control option and supported attributes are taken from the PriorityControlResponse. 
     The stack interface service  40  communicates the ioctl parameters to the priority and resource manager  41  (step S 2 . 4 ) using a function call exposed by the priority and resource manager  41  to the SIS  40 . The priority and resource manager  41  records the network stack layers that register, together with an indication as to whether they have opted for resource control via the ‘control option’ parameter (step S 2 . 5 ) and, if so, records the attributes that can be controlled, as specified in the ‘supported attributes’ parameter (step S 2 . 6 ). 
     Referring to  FIG. 4 , once the SAN stack layer registration process (S 2 ) is complete, user instructions can be received via the user administrative control module  42  (step S 3 ). 
     Via the user administrative control module  42 , users, for instance system administrators, can select tuning of the resources of the host and network stack layers according to a number of factors. For instance, the user can specify tuning to prioritise the resources based on the attributes of I/O operations to be processed, such as the I/O type, for instance read or write, the I/O operation size, for instance greater or less than 4 Kb, 8 Kb, 16 Kb etc, the end LUN, an application name or a process identification number. The user can also specify a validity period for which the priority request is required and a quantity indicating the proportion of the resource which is to be allocated to the prioritised I/O operation, for instance allocated as a percentage. 
       FIG. 7  illustrates the steps performed in processing user instructions. 
     User requirements for prioritisation control are received as user inputs  44 , via an input device such as a keyboard or mouse (not shown), at the user administrative control module  42  (step S 3 . 1 ). Once the user administrative control module  42  determines that user requirements have been received (step S 3 . 2 ) it communicates the requirements to the stack interface service driver via an ioctl call (step S 3 . 3 ) of the form: 
     
       
         
               
               
               
             
           
               
                   
                   
               
             
             
               
                   
                 ioctl 
                 (Device Descriptor, 
               
               
                   
                   
                 Command, 
               
               
                   
                   
                 Attribute type, 
               
               
                   
                   
                 Attribute value, 
               
               
                   
                   
                 Validity period, 
               
               
                   
                   
                 Resource quantity 
               
               
                   
                 ) 
               
               
                   
                   
               
             
          
         
       
     
     The definition of the parameters of the ioctl call between the user administrative control module and the stack interface service module are set out in table 3.0 below. 
     
       
         
               
               
               
             
           
               
                 TABLE 3.0 
               
               
                   
               
               
                 No. 
                 Parameter 
                 Definition 
               
               
                   
               
             
             
               
                 1 
                 Device Descriptor 
                 SIS driver descriptor 
               
               
                 2 
                 Command 
                 Indicates to the SIS whether the user 
               
               
                   
                   
                 wants priority to be turned on or off 
               
               
                 3 
                 Attribute type 
                 The attribute type to be prioritised 
               
               
                 4 
                 Attribute value 
                 The value for the attribute type 
               
               
                 5 
                 Validity period 
                 The duration for which the priority request 
               
               
                   
                   
                 is valid 
               
               
                 6 
                 Resource quantity 
                 The percentage of the resource which is to 
               
               
                   
                   
                 be allocated for prioritisation 
               
               
                   
               
             
          
         
       
     
     The stack interface service  40  communicates the user control parameters it receives in the ioctl call to the priority and resource manager  41  (step S 3 . 4 ) using a function call exposed by the priority and resource manager  41  to the SIS  40 . The priority and resource manager  41  records the user control parameters (step S 3 . 5 ) and identifies any derived values from the parameters (step S 3 . 6 ). For instance, an end primary attribute such as an end LUN would result in a derived attribute of the port of an array controller which is associated with the end LUN. The priority resource manager  41  then determines (step S 3 . 7 ) whether priority control according to the user request is possible and returns an indication to the user via the stack interface service  40  and user administrative control module  42  that the request has either failed (step S 3 . 8 ) or has been successful (step S 3 . 9 ). 
     Referring to  FIG. 4 , once the user instructions have been received and processed, and in the case that the user request is successful (step S 3 . 9 ), synchronised tuning of the resources of the host and network stack layers can be performed (step S 4 ). 
       FIG. 8  illustrates the steps performed in synchronised tuning of the host and network stack layers. 
     Referring to  FIG. 8 , the priority and resource manager  41  activates or deactivates the tuning function of the host storage stack layers that have registered for resource control by setting the value of the tuning function pointer provided in the ioctl call of each host storage stack layer accordingly, so as to implement the desired prioritisation (step S 4 . 1 ). The priority and resource manager  41  also issues a PriorityControlCommand to each registered layer of the network stack, via the SAN resource agent and the respective network stack agents (step S 4 . 2 ). 
     The PriorityControlCommand has the parameters set out in table 4.0 below. 
     
       
         
               
               
               
             
           
               
                 TABLE 4.0 
               
               
                   
               
               
                 No. 
                 Parameter 
                 Definition 
               
               
                   
               
             
             
               
                 1 
                 Header 
                 Indicates the destination stack layer 
               
               
                 2 
                 Control 
                 Tuner on or off 
               
               
                 3 
                 Attribute type 
                 For instance, whether based on array 
               
               
                   
                   
                 controller port or end LUN 
               
               
                 4 
                 Attribute value 
                 Value of the specified attribute type 
               
               
                   
               
             
          
         
       
     
     The priority and resource manager  41  then updates a priority resource control map that it stores which associates user needs with stack tuner support allocations, and links the associated host and/or network stack layers that are supporting each priority request (step S 4 . 3 ). A typical resource control map would have the values set out in table 5.0 below. 
     
       
         
               
               
               
               
               
               
             
           
               
                 TABLE 5.0 
               
               
                   
               
               
                   
                 Tuner 
                 Attribute 
                 Attributes 
                   
                 Validity 
               
               
                 Layer Id 
                 function/request 
                 Specified 
                 Supported 
                 % 
                 Period 
               
               
                   
               
             
             
               
                 File 
                 FS_ResourceControlFunction 
                 Application 
                 Application 
                 50% 
                 60 
               
               
                 System 
                   
                 Id, end 
                 Id 
                   
                 minutes 
               
               
                   
                   
                 LUN 
               
               
                 HBA I/F 
                 HBA_IF_ResourceControlFunction 
                 End LUN 
                 End LUN 
                 50% 
                 60 
               
               
                 driver 
                   
                   
                   
                   
                 minutes 
               
               
                 Storage 
                 ARRAY_ResourceControlRequest 
                 End LUN 
                 End LUN 
                 50% 
                 60 
               
               
                 Array 
                   
                   
                   
                   
                 minutes 
               
               
                   
               
             
          
         
       
     
     Meanwhile, the host and network stack layers implement the priority control based on their received user requirements (steps S 4 . 4  and S 4 . 5 ). Prioritisation of I/O throughput is accordingly implemented across the storage software layers of a computer storage system in a synchronised manner, based on user attribute requirements. 
     In an example of the present invention, three applications A1, A2 and A3 operate in the user space  27 . A1 uses a logic unit number LUN1 for read operations and A2 uses the logic unit number LUN1 for write operations. A3 uses LUN2 for both read and write operations. The user could, for instance, specify: 
     Attribute types: Application ID, I/O type, end LUN 
     Attribute value: A1, Reads, LUN1 
     The resulting priority resource allocation would accordingly be 100% of the file system resources allocated to A1 and 100% of the driver resources for I/O operations to LUN1 for reads only. 
     Alternatively, the user could specify: 
     Attribute types: Application ID, end LUN 
     Attribute value: A1, LUN1 
     The resulting priority resource allocation would accordingly be 100% of the file system resources allocated to A1 and 50% of the driver resources for I/O operations to A1 and A2 (since both access LUN1). Net priority for A1 will therefore be 100% of the file system resources and 50% of the driver resources. 
     In the case that a user choice is incorrectly specified, for instance in the case that an impossible combination is specified such as A1 with attributes I/O type and end LUN and values ‘writes’ and ‘LUN2’, the priority and resource manager  41  observes read and write patterns to end LUNs from A1 and highlights the possibility of an anomaly to the user. 
     Current work load managers, such as the HPUX™ process resource manager support specific CPU/Memory and Disk I/O bandwidth prioritisation localised to one layer of the storage software stack. The user administrative control module  42  according to embodiments of the present invention is arranged to expose an application programmable interface (API) for priority attribute specification to enable tuning control by work load managers. This can enable the prioritisation infrastructure of the present invention to interact with existing work load managers in conjunction with CPU/Memory allocations so that prioritisation clashes do not occur. 
     The user administrative control module  42  also indicates to users that manually tuning layers of the host and/or network stack from user space can override the tuning performed by the prioritisation infrastructure of the present invention or may be overridden by the prioritisation infrastructure of the present invention. 
     The priority and resource manager  41 , according to embodiments of the invention, is arranged to control each of the layers in the host stack and network. Each layer, in turn, controls and assigns its internal resources based on their availability and controllability to that layer for priority support. For example, in the host stack the multipathing layer  32  can be controlled by detecting the turn around time of data being routed and choosing paths with the shortest turn-around time for allocation to prioritised applications. The paths allocated to prioritised I/O operations will therefore be dynamic in nature, as determined by the multipathing layer. Considering another example, in the network stack, the tuner  45  in the intelligent switch  6   a  is, in certain embodiments, arranged to monitor slow response paths and to take the decision to reroute prioritised I/O operations appropriately to less loaded paths. Similarly, the array tuner  46  can also allocate more I/O command processing resources to a LUN used by high priority applications, or assign ownership of a LUN to a controller which is loaded less that other controllers. In all of the examples above, resource control is dependent on the capabilities of that layer, as would be appreciated by those skilled in the art. 
     Whilst specific examples of the host and network layer registration processes have been provided, the invention is not limited to these examples. For example, the SIS driver module  40  and priority and resource manager  41  need not be loaded prior to the layers  30  to  34 ,  6   a ,  7   a , of the host and network stacks, but can be loaded afterwards. The SIS driver module  40  can, in this case, be configured to trigger the layers to register through a further ioctl interface. Other arrangements are also possible, as would be apparent to those skilled in the art. 
     Also, whilst the invention has been described with regards to implementing resource prioritisation on host  2   a , intelligent switch  6   a  and storage array  7   a  of  FIG. 1 , the invention can also be applied to the resources of other combinations of hosts  2   a ,  2   b ,  2   c , intelligent switches  6   a ,  6   b  and storage elements  7   a ,  7   b ,  7   c ,  7   d.    
     The present invention may be implemented as computer program code stored on a computer readable medium. Furthermore, whilst the invention has been described based on a Unix-based operating system, namely HPUX™, it can also be applied for other operating systems, the implementations of which would be apparent to the skilled person from the present disclosure.