Patent Document

BACKGROUND  
       [0001]     1. Field of the Invention  
         [0002]     The present invention relates to computing systems, and more particularly to optimizing direct memory access (“DMA”) channel arbitration.  
         [0003]     2. Background of the Invention  
         [0004]     Storage area networks (“SANs”) are commonly used where plural memory storage devices are made available to various host computing systems. Data in a SAN is typically moved from plural host systems (that include computer systems, servers etc.) to the storage system through various controllers/adapters.  
         [0005]     Host systems typically include several functional components. These components may include a central processing unit (CPU), main memory, input/output (“I/O”) devices, and streaming storage devices (for example, tape drives). In conventional systems, the main memory is coupled to the CPU via a system bus or a local memory bus. The main memory is used to provide the CPU access to data and/or program information that is stored in main memory at execution time. Typically, the main memory is composed of random access memory (RAM) circuits. A computer system with the CPU and main memory is often referred to as a host system.  
         [0006]     Host systems often communicate with storage systems via a host bus adapter (“HBA”, may also be referred to as a “controller” and/or “adapter”) using an interface, for example, the “PCI” bus interface. PCI stands for Peripheral Component Interconnect, a local bus standard that was developed by Intel Corporation®. The PCI standard is incorporated herein by reference in its entirety. Most modern computing systems include a PCI bus in addition to a more general expansion bus (e.g. the ISA bus). PCI is a 64-bit bus and can run at clock speeds of 33 or 66 MHz.  
         [0007]     PCI-X is another standard bus that is compatible with existing PCI cards using the PCI bus. PCI-X improves the data transfer rate of PCI from 132 MBps to as much as 1 GBps. The PCI-X standard was developed by IBM®, Hewlett Packard Corporation® and Compaq Corporation® to increase performance of high bandwidth devices, such as Gigabit Ethernet standard and Fibre Channel Standard, and processors that are part of a cluster.  
         [0008]     Various other standard interfaces are also used to move data from host systems to storage devices. Fibre channel is one such standard. Fibre channel (incorporated herein by reference in its entirety) is an American National Standard Institute (ANSI) set of standards, which provides a serial transmission protocol for storage and network protocols such as HIPPI, SCSI, IP, ATM and others. Fibre channel provides an input/output interface to meet the requirements of both channel and network users.  
         [0009]     iSCSI is another standard (incorporated herein by reference in its entirety) that is based on Small Computer Systems Interface (“SCSI”), which enables host computer systems to perform block data input/output (“I/O”) operations with a variety of peripheral devices including disk and tape devices, optical storage devices, as well as printers and scanners.  
         [0010]     A traditional SCSI connection between a host system and peripheral device is through parallel cabling and is limited by distance and device support constraints. For storage applications, iSCSI was developed to take advantage of network architectures based on Fibre Channel and Gigabit Ethernet standards. iSCSI leverages the SCSI protocol over established networked infrastructures and defines the means for enabling block storage applications over TCP/IP networks. iSCSI defines mapping of the SCSI protocol with TCP/IP.  
         [0011]     DMA modules are used by HBAs to perform data transfers between memory locations, or between memory locations and an input/output port. A DMA module functions without involving a microprocessor by initializing control registers in the DMA unit with transfer control information. The transfer control information generally includes source address (the address of the beginning of a block of data to be transferred), the destination address, and the size of the data block. DMA units provide address and bus control signals to and from a device for a read and/or write cycle.  
         [0012]     Specific channels are implemented in a DMA unit to allow storage devices to transfer data directly to and from memory storage devices. A channel can be activated by a DMA request signal (DREQ) from a storage device or a host system. The DMA unit receives the DREQ, provides a DMA acknowledged signal (DACK), and transfers the data over the channel to or from the storage device.  
         [0013]     HBAs typically use multiple DMA channels and have an arbitration module that arbitrates for access to the PCI (or PCI-Express) link. This allows an HBA to arbitrate and switch contexts (between channels) by actively processing command, status and data. Multiple channels are serviced in periodic bursts. After each arbitration cycle there is additional time for re-loading a data pipeline when connecting to selected channel, even if the selected channel is the same as the previously selected channel in the previous arbitration cycle. Hence if multiple channels are not being used, the minimum arbitration cycle adds latency (of plural clock cycles) and affects overall performance. The reason for the latency is that certain resources have to be loaded and updated for every DMA cycle. When consecutive requests occur from the same DMA channel and no other channel is requesting access, there is an additional penalty each time the same channel is serviced because no data is transferred by another channel during the time when the active channel is re-initialized (or “re-armed”).  
         [0014]     A HBA often has to perform frequent context switching between DMA channels, especially when status, command and data are processed at the same time. In other instances, the HBA may operate using predominantly a single DMA channel. This occurs for example, where a large data transfer occurs and a particular DMA unit gets access. These instances (i.e. single channel use or frequent context switching) are not predictable. Conventional HBAs fail to auto-sense single channel usage and adjust arbitration cycles accordingly.  
         [0015]     Also, industry standards (for example, PCI-Express standard) provide transaction rules as to when memory read/write requests must terminate based on maximum payload, maximum read request size and address/data alignment. Because of these rules, a large DMA request may have to be segmented into smaller blocks. In conventional systems, completion for a first data segment is received before another request is granted for the next segment to the same DMA channel. This is because only one outstanding request per DMA channel is allowed. The request-response sequence is repeated until the entire DMA request is completed. If a single channel is repeatedly used for a large data transfer, the request-response-request cycle negatively affects overall performance.  
         [0016]     Therefore, what is required is a system and method that can automatically sense if the same DMA channel is being used for a data transfer, disable certain standard request and segmentation rules for a certain period and efficiently transfer data by decreasing arbitration frequency.  
       SUMMARY OF THE INVENTION  
       [0017]     In one aspect of the present invention, a host bus adapter coupled to a network and a host computing system is provided. The host bus adapter includes a direct memory access (“DMA”)mode detection module that receives a DMA channel identifier information from an arbitration module that receives requests from plural DMA channels, wherein the DMA mode detection module includes a DMA counter that counts a number of times a single DMA channel is exclusively serviced by the arbitration module and if the DMA counter value is equal to a threshold value, then the DMA mode detection module enables a single channel mode during which standard transaction rules are ignored for determining DMA request lengths for transferring data. The single channel mode is enabled for a certain duration.  
         [0018]     The host bus adapter includes a rule based segmentation logic that may be enabled and/or disabled by host bus adapter firmware and/or detection of a single channel mode condition.  
         [0019]     The DMA mode detection module includes a register that stores the threshold value and the threshold value is programmable. The plural DMA channels include a DMA channel in a receive and transmit path of the host bus adapter.  
         [0020]     In another aspect of the present invention, a system that allows a host computing system to communicate with plural devices over a network is provided. The system includes a host bus adapter including a DMA mode detection module that receives a DMA channel identifier information from an arbitration module that receives requests from plural DMA channels, wherein the DMA mode detection module includes a DMA counter that counts a number of times a single DMA channel is exclusively serviced by the arbitration module and if the DMA counter value is equal to a threshold value, then the DMA mode detection module enables a single channel mode during which standard transaction rules are ignored for determining DMA request lengths for transferring data.  
         [0021]     The host bus adapter includes a rule based segmentation logic that may be enabled and/or disabled by host bus adapter firmware and/or detection of a single channel mode condition.  
         [0022]     The DMA mode detection module includes a register that stores the threshold value and the threshold value is programmable.  
         [0023]     A host computing system that can communicate with plural devices over a network is provided. The computing system is coupled to a host bus adapter that includes a DMA mode detection module that receives a DMA channel identifier information from an arbitration module that receives requests from plural DMA channels, wherein the DMA mode detection module includes a DMA counter that counts a number of times a single DMA channel is exclusively serviced by the arbitration module and if the DMA counter value is equal to a threshold value, then the DMA mode detection module enables a single channel mode during which standard transaction rules are ignored for determining DMA request lengths for transferring data.  
         [0024]     The host bus adapter includes a rule based segmentation logic that may be enabled and/or disabled by host bus adapter firmware and/or detection of a single channel mode condition.  
         [0025]     In yet another aspect of the present invention, a method for transferring data between a host computing system and plural devices using a HBA is provided. The method includes, determining if a same DMA channel in the HBA has been exclusively serviced for a certain duration; and enabling a single channel mode during which standard transaction rules are ignored for determining DMA request lengths for transferring data between the host computing system and the plural devices.  
         [0026]     This brief summary has been provided so that the nature of the invention may be understood quickly. A more complete understanding of the invention can be obtained by reference to the following detailed description of the preferred embodiments thereof concerning the attached drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0027]     The foregoing features and other features of the present invention will now be described with reference to the drawings of a preferred embodiment. In the drawings, the same components have the same reference numerals. The illustrated embodiment is intended to illustrate, but not to limit the invention. The drawings include the following Figures:  
         [0028]      FIG. 1A  is a block diagram showing various components of a SAN;  
         [0029]      FIG. 1B  is a block diagram of a host bus adapter that uses DMA mode selection, according to one aspect of the present invention;  
         [0030]      FIG. 1C  shows a block diagram of plural DMA units using a DMA mode selection module, according to one aspect of the present invention;  
         [0031]      FIG. 1D  shows a block diagram of the DMA mode selection module, according to one aspect of the present invention; and  
         [0032]      FIG. 2  is a process flow diagram of executable steps for DMA processing, according to one aspect of the present invention. 
     
    
       [0033]     The use of similar reference numerals in different figures indicates similar or identical items.  
       DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0034]     To facilitate an understanding of the preferred embodiment, the general architecture and operation of a SAN, and a HBA will be described. The specific architecture and operation of the preferred embodiment will then be described with reference to the general architecture of the host system and HBA.  
         [0035]     SAN Overview:  
         [0036]      FIG. 1A  shows a SAN system  100  that uses a HBA  106  (referred to as “adapter  106 ) for communication between a host system (for example,  200 ) with host memory  101  to various storage systems (for example, storage subsystem  116  and  121 , tape library  118  and  120 ) using fibre channel storage area networks  114  and  115 . Host memory  101  includes a driver  102  that co-ordinates all data transfer via adapter  106  using input/output control blocks (“IOCBs”). Servers  117  and  119  can also access the storage sub-systems using SAN  115  and  114 , respectively.  
         [0037]     A request queue  103  and response queue  104  is maintained in host memory  101  for transferring information using adapter  106 . Host system communicates with adapter  106  via a PCI bus  105  through a PCI core module (interface)  137 , as shown in  FIG. 1B .  
         [0038]     HBA  106 :  
         [0039]      FIG. 1B  shows a block diagram of adapter  106 . Adapter  106  includes processors (may also be referred to as “sequencers”) “XSEQ”  112  and “RSEQ”  109  for receive and transmit side, respectively for processing data received from storage sub-systems and transmitting data to storage sub-systems. Transmit path in this context means data path from host memory  101  to the storage systems via adapter  106 . Receive path means data path from storage subsystem via adapter  106 . It is noteworthy, that only one processor is used for receive and transmit paths, and the present invention is not limited to any particular number/type of processors. Buffers  111 A and  111 B are used to store information in receive and transmit paths, respectively.  
         [0040]     Beside dedicated processors on the receive and transmit path, adapter  106  also includes processor  106 A, which may be a reduced instruction set computer (“RISC”) for performing various functions in adapter  106 .  
         [0041]     Adapter  106  also includes fibre channel interface (also referred to as fibre channel protocol manager “FPM”)  113 A that includes an FPM  113 B and  113  in receive and transmit paths, respectively. FPM  113 B and FPM  113  allow data to move to/from storage systems.  
         [0042]     Adapter  106  is also coupled to external memory  108  and  110  via connection  116 A ( FIG. 1A ) (referred interchangeably, hereinafter) and local memory interface  122 . Memory interface  122  is provided for managing local memory  108  and  110 . Local DMA module  137 A is used for gaining access to move data from local memory ( 108 / 110 ).  
         [0043]     Adapter  106  also includes a serial/de-serializer  136  for converting data from 10-bit to 8-bit format and vice-versa.  
         [0044]     Adapter  106  also includes request queue DMA channel  0   130 , response queue DMA channel  131 , request queue ( 1 ) DMA channel  132  that interface with request queue  103  and response queue  104 ; and a command DMA channel  133  for managing command information. These DMA channels are coupled to arbiter  107  that receives plural requests from DMA channels and grants access to a certain channel.  
         [0045]     Both receive and transmit paths have DMA modules  129  and  135  that are used to gain access to a channel for data transfer in the receive/transmit paths. Transmit path also has a scheduler  134  that is coupled to processor  112  and schedules transmit operations.  
         [0046]     A host processor (not shown) sets up shared data structures in buffer memory  108 . A host command is stored in buffer  108  and the appropriate sequencer (i.e.,  109  or  112 ) is initialized to execute the command.  
         [0047]     Various DMA units (or channels, used interchangeably throughout this specification) (for example,  129 ,  130 ,  131 ,  132 ,  133  and  135 ) send a request to arbiter  107 . The requests are analyzed based on established standard arbitration rules. When a request is granted, the DMA unit is informed of the grant and memory access is granted to a particular channel.  
         [0048]     DMA Mode Selection Module  107 A (“Module  107 A”)  
         [0049]     In one aspect of the present invention, a DMA mode selection module  107 A is provided that automatically detects (“auto-senses”) when a particular DMA channel is being granted consecutive (i.e. back-to-back) access for a certain period of time. Module  107 A can enable a “single channel” mode that circumvents various standard rules, for example, turn-off data block segmentation, which reduces the number of arbitration cycles and the turnaround time spent for the same DMA channel to be re-initialized.  
         [0050]     Module  107 A also senses when multiple channels are being used again after a single channel mode is enabled. When this occurs, the single channel mode is disabled and standard segmentation techniques are used for a large data transfer.  
         [0051]      FIG. 1C  shows arbiter  107  being functionally coupled with module  107 A. Plural DMA channels (for example,  129  and  135 ) are coupled with arbiter  107 . Each DMA channel has a request pipeline (for example,  129 B and  135 B) and a segmentation module (for example  129 A and  135 A). The segmentation module segments a DMA transfer into segments (or blocks) to meet data transfer rules and/or if a DMA transfer is too large. Sequencer  109  and  112  send channel task commands ( 129 C and  135 C) to DMA channels  129  and  135 , respectively. The commands are used to generate a request to arbiter  107 .  
         [0052]     Segmentation modules  129 A and  135 B also operate based on certain rules, for example, when and how data blocks should be segmented. These rules can be turned on or off by the firmware. Some of these rules are based data block length and address limitations as imposed by the PCI-Express standard protocol. Rules may also be enabled/disabled by detection of a single channel mode condition that is described below.  
         [0053]     When a channel is granted access, the information is passed on to module  107 A. If the same channel is being granted access repeatedly and there are no competing requests, module  107 A enables a single channel mode select signal  107 F. This allows segmentation module to stop segmenting data blocks (and/or divide data blocks into larger segments) and transfer data in larger blocks to avoid arbitration cycle. The single channel mode is enabled for a finite time only so that once other channel request access, then there is no backlog.  
         [0054]      FIG. 1D  shows a block diagram for module  107 A. Module  107 A&#39;s auto-sensing mode can be enabled or disabled by adapter  106  firmware. Arbiter  107  provides a channel&#39;s identifier (“Channel ID”)  107 J every time access is granted to a channel. A threshold register (shown as PST_THR)  107 D can be programmed by firmware with a persistence threshold value. Register  107 D is controlled by firmware using signal/command  107 H. Command/signal  107 I is used to load the threshold value.  
         [0055]     A DMA sequence counter  107 B (DMA_SEQ_CNT) maintains a running count each time a DMA request is granted by arbiter  107  (shown as  107 K). Counter  107 B is enabled by firmware using command/signal  107 G. Compare module  107 N compares the Channel ID of a current DMA request with the Channel ID  107 J. If the DMA channel ID  107 J is the same for a current channel ( 107 P) as for the last channel and counter  107 B value (i.e.  107 L) is less than register  107 D value (i.e.,  107 M) as determined by logic  107 E, then counter  107 B is increased. If the Channel ID  107 J is different then counter  107 B is reset, for example, 1. When  107 L is equal to or greater than  107 M, then the single mode select signal  107 F is generated and sent to segmentation logic (for example,  129 A or  135 A).  
         [0056]      FIG. 2  shows a flow diagram of executable process steps for auto-sensing DMA channel usage and enable a single channel mode.  
         [0057]     The process starts when counter  107 B is enabled by firmware in step S 200  by command/signal  107 G and in step S 201 , a threshold value is loaded in register  107 D by command/signal  107 I.  
         [0058]     In step S 202 , the process determines if a current DMA channel ID ( 107 P) is the same as the Channel ID for the previous request ( 107 J). This is performed by compare module  107 N. If the Channel ID is different, then in step S 203  counter  107 B is reset.  
         [0059]     If the Channel ID is the same, signifying that the same DMA channel is being serviced, then in step S 202 A, the DMA counter  107 B is incremented.  
         [0060]     In step S 204 , counter value  107 L is compared with threshold value  107 M by logic  107 E. If  107 L is not equal to  107 M, then DMA requests are arbitrated and data blocks segmented using standard rules, and counter  107 B is increased.  
         [0061]     If  107 L is equal to  107 M, then is step S 206  the single DMA channel mode is enabled. When single channel mode is not enabled then certain transaction rules are followed. For example, 
    (a) Read requests must not exceed the maximum read request size;     (b) Write requests must not exceed the maximum payload size;     (c) Read and write requests must not overlap 4 kilo byte (“KB”) address segments; and/or     (d) Write requests must terminate at 128 Byte (“B”) boundaries as often as possible (i.e., unless a last request in a sequence that does not end at a  128 B address).    
 
         [0066]     If  107 L is equal to  107 M, then the single channel mode is enabled by command/signal  107 F.  
         [0067]     When single channel mode is enabled, the foregoing transaction rules are not used to determine the beginning and the end of each DMA request. This will cause the arbitration frequency to decrease since the size of transfers will increase.  
         [0068]     It is noteworthy that single channel mode may only be allowed for a certain duration by firmware of adapter  106 . Normal request length segmentation/arbitration occurs after single channel mode is automatically disabled.  
         [0069]     Although the present invention has been described with reference to specific embodiments, these embodiments are illustrative only and not limiting. Many other applications and embodiments of the present invention will be apparent in light of this disclosure and the following claims.

Technology Category: g