Abstract:
A host bus adapter (“HBA”) is provided with a programmable trace logic that can be enabled or disabled by firmware running on the HBA and if enabled can receive trace information from at least one processor, which is stored in a local memory buffer controlled by a local memory interface. A receive and transmit path processor data is traced and stored in the local memory buffer. The trace logic includes an arbitration module that receives trace data from plural sources and the trace data is stored in a first in first out based buffer before being sent to a direct memory access arbiter module and then to an external memory. Trace data as stored in the external memory includes a trace data source identity value, and a time stamp value indicating when data was collected.

Description:
BACKGROUND  
       [0001]     1. Field of the Invention  
         [0002]     The present invention relates to storage systems, and more particularly, to maintaining trace information in host bus adapters (“HBAs”).  
         [0003]     2. Background of the Invention  
         [0004]     Storage area networks (“SANs”) are commonly used where plural memory storage devices are made available to various host computing systems. Data in a SAN is typically moved from plural host systems (that include computer systems) to the storage system through various controllers/adapters (including HBAs).  
         [0005]     Various standard interfaces are used to move data from host systems to storage devices. Fibre channel is one such standard. Fibre channel (incorporated herein by reference in its entirety) is an American National Standard Institute (ANSI) set of standards, which provides a serial provides a serial transmission protocol for storage and network protocols such as HIPPI, SCSI, IP, ATM and others. Fibre channel provides an input/output interface to meet the requirements of both channel and network users.  
         [0006]     Host systems often communicate with storage systems via a HBA using the “PCI” bus interface. PCI stands for Peripheral Component Interconnect, a local bus standard that was developed by Intel Corporation®. The PCI standard is incorporated herein by reference in its entirety. Most modern computing systems include a PCI bus in addition to a more general expansion bus (e.g. the ISA bus). PCI is a 64-bit bus and can run at clock speeds of 33 or 66 MHz.  
         [0007]     PCI-X is a standard bus that is compatible with existing PCI cards using the PCI bus. PCI-X improves the data transfer rate of PCI from 132 MBps to as much as 1 GBps. The PCI-X standard was developed by IBM®, Hewlett Packard Corporation® and Compaq Corporation® to increase performance of high bandwidth devices, such as Gigabit Ethernet standard and Fibre Channel Standard, and processors that are part of a cluster.  
         [0008]     The iSCSI standard (incorporated herein by reference in its entirety) is based on Small Computer Systems Interface (“SCSI”), which enables host computer systems to perform block data input/output (“I/O”) operations with a variety of peripheral devices including disk and tape devices, optical storage devices, as well as printers and scanners. A traditional SCSI connection between a host system and peripheral device is through parallel cabling and is limited by distance and device support constraints. For storage applications, iSCSI was developed to take advantage of network architectures based on Fibre Channel and Gigabit Ethernet standards. iSCSI leverages the SCSI protocol over established networked infrastructures and defines the means for enabling block storage applications over TCP/IP networks. iSCSI defines mapping of the SCSI protocol with TCP/IP.  
         [0009]     The iSCSI architecture is based on a client/server model. Typically, the client is a host system such as a file server that issues a read or write command. The server may be a disk array that responds to the client request.  
         [0010]     HBAs today perform complex operations and are key to the overall efficiency of a SAN. HBAs may use more than one processor whose operation should be tracked to perform diagnostics in case of a failure or otherwise. HBA processors use program counters that track various processor-executed operations. However, conventional HBAs do not provide an efficient system for tracing multiple processors or providing the trace information in a user-friendly interface.  
         [0011]     Therefore, there is a need for a system and method that can trace multiple processors in an HBA.  
       SUMMARY OF THE INVENTION  
       [0012]     A system for storing trace information is provided. The system includes, a programmable trace logic that can be enabled or disabled by firmware running on a HBA and if enabled can receive trace information from at least one processor, which is stored in a local memory buffer controlled by a local memory interface. A receive and transmit path processor data is traced and stored in the local memory buffer.  
         [0013]     In yet another aspect, a host bus adapter (“HBA”) is provided with a programmable trace logic that can be enabled or disabled by firmware running on the HBA and if enabled can receive trace information from at least one processor, which is stored in a local memory buffer controlled by a local memory interface.  
         [0014]     In yet another aspect of the present invention, a local memory interface for storing processor trace information is provided. The interface includes,  
         [0015]     a programmable trace logic that can be enabled enabled or disabled by firmware running on a HBA and if enabled can receive trace information from at least one processor, which is stored in a local memory buffer controlled by the local memory interface.  
         [0016]     The trace logic includes an arbitration module that receives trace data from plural sources and the trace data is stored in a first in first out based buffer before being sent to a direct memory access arbiter module and then to an external memory. Trace data as stored in a circular memory buffer includes a trace data source identity value, and a time stamp value indicating when data was collected  
         [0017]     This brief summary has been provided so that the nature of the invention may be understood quickly. A more complete understanding of the invention can be obtained by reference to the following detailed description of the preferred embodiments thereof concerning the attached drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0018]     The foregoing features and other features of the present invention will now be described with reference to the drawings of a preferred embodiment. In the drawings, the same components have the same reference numerals. The illustrated embodiment is intended to illustrate, but not to limit the invention. The drawings include the following include the following Figures:  
         [0019]      FIG. 1A  is a block diagram showing various components of a SAN;  
         [0020]      FIG. 1B  is a block diagram of a host bus adapter that includes trace logic, according to one aspect of the present invention;  
         [0021]      FIG. 1C  shows a block diagram of a local memory interface, according to one aspect of the present invention;  
         [0022]      FIG. 1D  shows a block diagram of trace logic, according to one aspect of the present invention;  
         [0023]      FIG. 1E  shows a block diagram of trace data format that is stored in external memory, according to one aspect of the present invention;  
         [0024]      FIG. 1F  shows a table with an example of code associated with the source of trace data, collected according tone aspect of the present invention; and  
         [0025]      FIGS. 2-15  show various registers that are used in various adaptive aspects of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0026]     To facilitate an understanding of the preferred embodiment, the general architecture and operation of a system using storage devices will be described. The specific architecture and operation of the preferred embodiment will then be described with reference to the general architecture.  
         [0027]     It is noteworthy that a host system, as referred to herein, may include a computer, server or other similar devices, which may be coupled to storage systems. Host system includes a host processor, memory, random access memory (“RAM”), and read only memory (“ROM”), and other components.  
         [0028]      FIG. 1A  shows a system  100  that uses a controller/adapter  106  (referred to as “adapter  106 ) for communication between a host system (not shown) with host memory  101  to various storage systems (for example, storage subsystem  116  and  121 , tape library  118  and  120 ) using fibre channel storage area networks  114  and  115 . Host memory  101  includes a driver  102  that co-ordinates all data transfer via adapter  106  using input/output control blocks (“IOCBs”).  
         [0029]     A request queue  103  and response queue  104  is maintained in host memory  101  for transferring information using adapter  106 . Host system communicates with adapter  106  via a PCI bus  105  through a PCI interface  107  (or PCI-X bus and PCI-X bus interface) and PCI core module  137 , as shown in  FIG. 1B .  
         [0030]      FIG. 1B  shows a block diagram of adapter  106 . Adapter  106  includes processors (may also be referred to as “sequencers”)  112  and  109  for receive and transmit side, respectively for processing data received from storage sub-systems and transmitting data to storage sub-systems. Transmit path in this context means data path from host memory  101  to the storage systems via adapter  106 . Receive path means data path from storage subsystem via adapter  106 . It is noteworthy, that only one processor is used for receive and transmit paths, and the present invention is not limited to any particular number/type of processors. Buffers  111 A and  111 B are used to store information in receive and transmit paths, respectively.  
         [0031]     Beside dedicated processors on the receive and transmit path, adapter  106  also includes processor  106 A, which may be a reduced instruction set computer (“RISC”) for performing various functions in adapter  106 , as described below. It is noteworthy that all the processors ( 109 ,  112  and  106 A) have program counters for tracking various operations (“trace information”).  
         [0032]     Adapter  106  also includes fibre channel interface (also referred to as fibre channel protocol manager “FPM”)  113 A that includes an FPM  113 B and  113  in receive and transmit paths, respectively. FPM  113 B and  113  allow data  113  allow data to move to/from storage systems  116 ,  118 ,  120  and  121 .  
         [0033]     Adapter  106  is also coupled to external memory  108  and  110  via connection  116 A (referred interchangeably, hereinafter) and local memory interface  122 . Adapter  106  to store firmware trace results, according to one aspect of the present invention, uses external memory  108 .  
         [0034]     Memory interface  122  is provided for managing local memory  108  and  110  and includes the trace logic for recording processor events, according to one aspect of the present invention. Local DMA module  137 A is used for gaining access to move data from local memory ( 108 / 110 ).  
         [0035]     Adapter  106  also includes a serial/de-serializer  136  for converting data from 10-bit to 8-bit format. Both receive and transmit paths have direct memory access (“DMA”) via modules  129  and  135 . Transmit path also has a scheduler  134  that is coupled to processor  112  and schedules transmit operations.  
         [0036]     Adapter  106  includes request queue DMA channel  0   130 , response queue DMA channel  131 , request queue ( 1 ) DMA channel  132  that interface with request queue  103  and response queue  104 ; and a command DMA channel  133  for managing command information.  
         [0037]      FIG. 1C  shows a block diagram of memory interface  122  that arbitrates between requests to access local memory from various DMA channels via DMA interface  142  that interfaces with local DMA module  137 A. Registers  138  store configuration information that is received from processor  106 A.  
         [0038]     Arbiter  139  is provided to manage access to local memory that is shared by plural DMA channels. Priorities may be pre-programmed using processor  106 A.  
         [0039]     Control logic  140  interfaces with all the modules of interface  122  and loads firmware trace, according to one aspect of the present invention.  
         [0040]     Firmware trace module  141  provides a mechanism to transfer trace information regarding processor  106 A,  109 ,  112 , and modules  130 ,  131  and  132  to an external memory (for example,  108  and  110 ). Trace information can be used for later analysis. Logic  141  receives processor  106 A program counter data  141 A and bi-directional data  141 B and  141 A. Trace module  141  can use the request/response DMA channel ( 130 - 131 ) to move trace information to memory  108 / 110 .  
         [0041]      FIG. 1D  provides a detailed block diagram of trace logic  141 . Module  141  includes a trace arbiter module  141 A that receives trace information from various sources. In one aspect, each source is provided two trace trace registers that may be written by adapter  106  firmware or local DMA module  137 A.  
         [0042]     As shown in  FIG. 1D , trace data,  106 B and  106 C from processor  106 A,  112 A and  112 B from processor  112 ,  109 A and  109 B from processor  109 ,  130 A and  130 B from request queue module  130 ,  132 A and  132 B from request module ( 1 )  130 B, and  131 A and  131 D from response queue module  131 , respectively, enter arbiter  141 A. Trace information is then moved into temporary memory  141 B, which may be a first in first out (FIFO) module that is used to hold data before it is moved into local memory  108  or  109  through arbiter  139  that is controlled by logic  140 .  
         [0043]     In one aspect, a circular buffer  108 A is used to store trace data, which is maintained by the firmware of adapter  106 . Firmware defines the location and size of buffer  108 A by setting up a Starting and Ending Address registers. A segment size may be set and every time a segment size data block is stored, the segment count is incremented and an interrupt generated to processor  106 A.  
         [0044]     It is noteworthy that module  141  can be programmed for 1-word or 2-word transfers. If a 1-word transfer is selected, the trace information results in an IOCB address from processor  106 A memory pointer. If a 2-word transfer is selected, the data results in an out-pointer (for example, 21 bits and an IOCB address) from processor memory  106 A.  
         [0045]      FIG. 1E  shows a block diagram of trace data format that is stored in external memory. Trace data includes a code  150  that denotes the source of the data, as shown in the table of  FIG. 1F . For example, code “000” denotes that the trace data is from RISC  106 A, “001” denotes that trace data is from processor  112  and so forth.  
         [0046]     A timer counter value  151  provides a time stamp for the data, i.e., when the data was actually recorded. Program counter or IOCB address  152  denotes the actual address of the IOCB or the program counter. Trace data  153  includes the actual data or an IOCB address.  
         [0047]      FIGS. 2-15  show registers  138  that are used in various adaptive aspects of the present invention.  FIG. 2  shows a listing of various registers that are used and described herein.  FIG. 3  shows a table with control register values that enable and/or disables trace data collection, according to one aspect of the present invention. Various bit values, for example, “bit 8” if set enables trace information collection from processor  106 A.  
         [0048]      FIG. 4  shows a circular buffer  108 A start address register, which holds the start address in buffer  108 A.  FIG. 5  holds the end address in buffer  108 A.  
         [0049]      FIG. 6  holds the memory address of buffer  108 A where data is written, while  FIG. 7  provides the size of RAM buffer segments.  FIG. 8  shows the register that is used to hold the number of segments that are being stored in buffer  108 A at any given time.  
         [0050]      FIG. 10  shows a register that is written with “dummy” data when a 1-word trace is performed on processor  106 A.  FIG. 11  shows a register that holds trace data from processor  106 A, while performing a 2-word trace.  
         [0051]      FIG. 12  shows a register that contains dummy data when performing a 1-word trace involving processor  112 .  FIG. 13  shows trace data involving processor  112  and is written when a 2-word trace is performed.  
         [0052]      FIG. 14  shows a register that contains dummy data when performing a 1-word trace involving processor  109 .  FIG. 15  shows trace data involving processor  109  and is written when a 2-word trace is performed.  
         [0053]     Firmware running on processor  106 A converts data in local memory. Firmware can parse data stored in buffer  108 A by using a graphical user interface (“GUI”). The GUI allows a user to filter the data and easily interpret the interpret the data since it is correlated with program counters and is time stamped.  
         [0054]     Although the present invention has been described with reference to specific embodiments, these embodiments are illustrative only and not limiting. Many other applications and embodiments of the present invention will be apparent in light of this disclosure and the following claims.