Abstract:
The present invention provides a mechanism for fast routing of data in a Storage Area Network. A protocol interface module (PIM) interfaces with outside networks and the storage devices, such as over fiber channel (FC). The PIM encapsulates received data into a streaming protocol, enabling storage processors to direct data to/from the appropriate physical disk in a similar manner to the directing of network messages over the Internet or other network.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     The present application is a continuation of co-pending, commonly assigned, patent application Ser. No. 10/238,927 entitled “METHOD AND APPARATUS FOR TRANSFERRING INFORMATION BETWEEN DIFFERENT STREAMING PROTOCOLS AT WIRE SPEED,” filed Sep. 9, 2002, which in itself claims priority to U.S. Provisional Patent Application 60/317,817, entitled “METHOD AND APPARATUS FOR PROCESSING FIBRE CHANNEL FRAMES AT WIRE SPEED,” filed Sep. 7, 2001 the disclosures of which are hereby incorporated herein by reference. 
    
    
     BACKGROUND OF THE INVENTION 
     The present invention relates to storage area networks (SAN), and in particular to virtual SANs. 
     Storage virtualization describes the process of representing, to a user, a number of discrete physical storage devices as a single storage pool having a single set of characteristics. For example, in a storage area network connecting host computers with storage devices, the user perceives a single block of disk space with a defined reliability (e.g., 100 GB at RAID 1), regardless of whether the data is stored on a single RAID 1 disk array or is split across multiple, separate disk arrays. 
     In the above situation, each host computer must be aware of the storage devices connected to the storage area network because each host computer manages the storage virtualization that is presented to its users. When the storage devices connected to the storage area network are modified (such as a new device being added or an existing device being removed), each host computer must be reconfigured to accommodate the modification. Such reconfiguration involves work by network administrators and ensures that changes in the network are not seamless. 
       FIG. 1  shows a schematic view of information transfer by a conventional storage area network. Specifically, conventional Storage Area Network (SAN)  20  comprises storage server  22  in communication with external data source  24  such as a minicomputer, and external data storage device  26  such as a disk array, through port  23  in communication with fibre channel (FC) controller  28 . 
     Communication between FC controller  28  and CPU  30  of storage server  22  of SAN  20  shown in  FIG. 1  takes place typically along a PCI bus  32 . Conventionally, the transfer of data in and out of storage server  4  is performed utilizing a store-and-forward algorithm. Specifically, FC controller  28  sends and receives data directly to and from memory  34  of host server  22 . Similarly, FC controller  28  forwards received commands to memory  34  of host server  22 , interrupting operation of CPU  30  of host server  22  until all information has arrived. 
     Data handling techniques employed by the conventional storage area network shown in  FIG. 1  offer certain disadvantages. One disadvantage of the configuration shown in  FIG. 1  is the consumption of attention of the CPU of the storage server to manage operation of the FC processor. For example, TABLE 1 below lists a typical sequence of twelve steps 1-12 which occur when the FC controller responds to an FCP_CMND from a host requesting data. 
     
       
         
               
               
             
               
               
             
           
               
                 TABLE 1 
               
               
                   
               
               
                 Step# 
                 Step 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 1 
                 FCP_CMND frame arrives 
               
               
                 2 
                 Command arrives. Interrupt is generated. 
               
               
                 3 
                 CPU interrogates the interrupt cause (command). 
               
               
                 4 
                 CPU reads memory to examine the FCP_CMND 
               
               
                 5 
                 CPU allocates memory for a Write command OR wireless data to 
               
               
                   
                 memory for a Read command. 
               
               
                 6 
                 CPU tells controller that buffers are available for processing. 
               
               
                 7 
                 Controller reads memory (data) and ships data out. 
               
               
                 8 
                 Transfer complete. Interrupt is generated. 
               
               
                 9 
                 CPU interrogates interrupt cause (transfer complete). 
               
               
                 10 
                 CPU writes status to memory. 
               
               
                 11 
                 CPU tells controller that the STATUS frame is in memory. 
               
               
                 12 
                 Controller reads memory (data) and ships data out. 
               
               
                   
               
             
          
         
       
     
       FIG. 1  shows this flow of interaction between a FC controller chip  28  and the host CPU  30 . In this case the CPU is involved with at least three memory access steps as well as accessing the FC controller through the PCI bus at least four times. The FC controller itself has to access the memory at least three times. 
     Further compounding the problem, the FC controller generates an interrupt. This causes the CPU to take the penalty of processing interrupts, which is heavy since a stack has to be saved into memory. When the JO traffic from the FC controller is heavy, interrupts are generated frequently; and the CPU spends a lot of time just saving the contexts. 
     As the CPU handles all commands, commands requiring a lot of processing block the simpler RD and WR commands that are communicated much more frequently. 
     The PCI bus can only have a certain amount of load. This limits the number of devices that can sit on the bus. A PCI bridge  36  can help solve this problem, but this causes latency. 
     The CPU in this case usually runs a real time or general purpose operating system (OS). CPU cycles are shared between the OS and IO handling. Interrupts are frequently taken by the CPU for things such as timers or serial port traffic. 
     Frames are processed in a “store and forward” approach. The entire frame has to be in memory before the CPU is informed of it&#39;s arrival. This further detracts from the speed of operation of the system shown in  FIG. 1 , as each incoming frame must first be buffered and then retrieved in order for proper routing to occur. 
     Still another disadvantage of the conventional approach shown in  FIG. 1  is complexity, as the processor must include sufficient memory capacity in the form of buffering hardware to store the necessary quantity of information. 
     Another disadvantage associated with the system shown in  FIG. 1  is the bottleneck created by the single FC controller, which is allocated to both receive incoming information from external hosts, and to issue commands to retrieve data from the storage devices. 
       FIG. 2  shows an example of a conventional virtual storage system  60  which utilizes off-the-shelf PC hardware using general Operating Systems (OS). In the implementation shown in  FIG. 2 , there are two separate FC controllers  28   a  and  28   b . FC controller  28   a  is in communication with storage device (INITIATOR)  26  through port  63   a . FC controller  28   b  is in communication with Host (TARGET)  24  through port  63   b.    
     A disadvantage of the conventional approach shown in  FIG. 2 , however, is that the two separate FC controllers  28   a  and  28   b  are now contending for access to memory  34 , internal bus  32 , and CPU  30 . Both controllers  28   a  and  28   b  are also interrupting CPU  30  with work requests. 
     To preform virtualization, the CPU has to bind the Host that is sending the command with the LUN number in the command. The FC controller would send an identifier indicating which host sent the command. The CPU would then have to perform a hash lookup using the handle and the LUN number. The identifier used by the controller is usually a pointer to its internal device context. 
     With FC, the link may bounce. The FC controllers will then perform discovery again. This time however, a different identifier may be chosen by the FC controller for that same Host. The CPU would then have to modify its tables so that the correct binding will be established. 
     Given the disadvantages associated with conventional storage systems, embodiments of the present invention are directed toward improvements in these and other areas. 
     BRIEF SUMMARY OF THE INVENTION 
     The present invention provides a mechanism for fast routing of data in a Storage Area Network. A protocol interface module (PIM) interfaces the storage processors with outside networks and the storage devices, such as over fiber channel (FC). The PIM encapsulates received frames or just the payload of the frames into a streaming protocol, enabling storage processors to direct data to/from the appropriate physical disk in a similar manner to the directing of network messages over the Internet or other network. 
     In one embodiment, the storage processor is a network processor, with multiple, parallel protocol processors. In addition, a general purpose CPU is provided. The PIM inspects the incoming message, and inserts a frame type in the header. The storage processor uses a hardware classifier to read the frame type and, if it is a read or write operation, assign it to one of the protocol processors. Otherwise, all of the commands are sent to the general purpose CPU. 
     In one embodiment, the network processors handle the virtualization, first determining which VLUN from the host ID, LUN field and port. Information is then fed into a hardware tree search engine to determine the physical location of the data. The commands making up the read or write operation are then routed to the appropriate storage processor connected to the data storage itself (e.g., RAID controller). 
     To distribute tasks, the storage processor connected to the PIM need not do the virtualization and processing of storage commands itself. Rather, it could route some frames to another storage processor connected to a switch fabric for processing. The PIM receiving the original message places an identification of the appropriate storage processor, where it has previously been assigned, into the header of the frame. The PIM receives a frame of information including a header and a payload. The PIM rapidly inspects only the header of the received data frame in order to identify relevant contextual information. Based upon recognized contextual information, the PIM references a look up table to identify corresponding frame destination information. The PIM then generates a descriptor from the destination information, encapsulates the descriptor within a frame according to a streaming protocol, and forwards the encapsulated frame to the storage processor for routing to the appropriate destination. Use of the look up table in combination with the rapidly-discerned header context information avoids buffering of the entire incoming data frame, thereby enhancing speed and efficiency of information transfer. 
     In one embodiment, the internal protocol is packet over SONET (POS), and the SAN is connected to the host and to the storage controllers over fiber channel (FC). By using the PIMs to perform conversion from one protocol to the other, a common protocol, such as SCSI, can be used for all the storage processors, allowing for modularity regardless of the particular system design. Other outside connections than FC could be used, such as Ethernet, etc. The present invention thus provides many speed advantages, by doing certain operations in hardware, and doing others using network processors and performing the virtualization and storage (e.g., RAID) operations using a network, streaming protocol. A streaming protocol, such as POS, assumes the other side is ready to receive the data and streams it across, with any flow control being done in hardware as specified by the POS protocol. Additionally, separate paths are used for receive and transmit to achieve full duplex. 
     A fuller understanding of the present invention may be obtained by reference to the following drawings and related detailed description. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       For a more complete understanding of the present invention, reference is now made to the following descriptions taken in conjunction with the accompanying drawing, in which: 
         FIG. 1  is a simplified schematic view of one example of a conventional storage area network. 
         FIG. 2  is a simplified schematic view of another example of a conventional storage area network. 
         FIG. 3  is a simplified schematic view of one embodiment of a storage area network in accordance with the present invention. 
         FIG. 4  is a block diagram of the storage server of  FIG. 3 . 
         FIG. 5  is a block diagram of the software modules of a storage server. 
         FIG. 6  is a block diagram of upstream and downstream storage processors in an embodiment in accordance with the present invention. 
         FIG. 7  is a block diagram of the PIM and SP architecture according to an embodiment of the invention. 
         FIG. 8  is a block diagram of an embodiment of in accordance with the present invention with a switch fabric and multiple external protocols. 
         FIG. 9  is a block diagram of an embodiment in accordance with the present invention without a switch fabric. 
         FIG. 10  is a block diagram of an embodiment in accordance with the present invention with multiple ports per SP. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     System Overview 
     As described generally above, an embodiment of a storage area network in accordance with the present invention may include an interface processor configured to transfer information at wire speed between an external streaming protocol and an internal streaming protocol that is different from the external streaming protocol. 
       FIG. 3  shows a storage server  100  according to an embodiment of the present invention. The figure shows a storage area network (SAN)  102 , a number of physical storage devices  104 , and a number of host computers  106 . 
     The storage server  100  is also referred to as a Virtual Storage Exchange (VSX) or Confluence Virtual Storage Server (CVSS). The storage server  100  provides storage virtualization to servers in a homogeneous as well as a heterogeneous environment, providing a solution to large data centers, ISPs, SSPs, and ASPs in the area of network storage. 
     The SAN  102  can be any type of computer network. It is referred to as a storage area network in the present application because that is its relevant function with respect to the embodiments of the present invention. In an embodiment of the present invention, the SAN  102  is a Fibre Channel network, the host computers  106  and the storage devices  102  are configured to communicate with a Fibre Channel network, and the storage server  100  is also configured to communicate with a Fibre Channel network. Thus, the storage server  100  can be easily added to an existing SAN. 
     The physical storage devices  104  include tape drives, disk arrays, JBODs (“just a bunch of disks”), or other types of data storage devices. The physical storage devices  104  can be connected directly to the host computers  106  via the SAN  102  or can be indirectly connected to the host computers  106  via the SAN  102  and the storage server  100 . As discussed above in the Background, management of storage virtualization is burdensome when the storage devices  104  are directly connected to the host computers  106  via the SAN  102 . The present invention improves management of storage virtualization by using the storage server  100  to indirectly connect the storage devices  104  to the host computers  106 . 
     The host computers  106  can be servers or stand-alone computers. The host computers  106  can be directly connected to the SAN  102  or indirectly connected via a switch, router, or other communication link. 
       FIG. 4  is a block diagram of the storage server  100  showing the hardware components related to embodiments of the present invention, including a storage processor (SP)  110 , a line card (LC)  112 , a virtual server card (VSC)  114 , and a switch fabric  116 . 
     The storage server  100  may include one or more storage processors  110 . The storage processors  110  process the storage commands and data to be stored as information flows between the host computers  106  and the storage devices  104 . One or more of the storage processors  110  may be included on each line card  112 . The storage server  100  includes space for numerous line cards  112 , so the capabilities of the storage server  100  can be modularly increased by adding more line cards  112  or more storage processors  110 . Each storage processor  110  is associated with one or more ports of the storage server  100 . 
     The storage server  100  may include one or more virtual server cards  114 . The virtual server cards control the operation of the storage server  100  and control the line cards  112 , which perform the actual work of transferring commands and data. 
     The switch fabric  116  connects the storage processors  110 . The switch fabric switches information received at one port to another port of the storage server  100 . For example, when a host computer  106  wants to read data stored on the storage area network  102 , its request is processed by the storage processor  110  associated with the port associated with that host computer  106 . That storage processor  110  is referred to as the upstream storage processor  110 . The upstream storage processor  110  communicates with a downstream storage processor  110  associated with the port associated with the storage device  104  storing the data to be read, via the switch fabric  116 . Then the switch fabric  116  transfers the data read from the storage device to the host computer  106 , via the downstream and upstream storage processors  110 . 
       FIG. 5  is a block diagram of the storage server  100  showing the functionality relevant to embodiments of the present invention. The functions of the storage server  100  may be implemented by one or more processors that execute processing according to one or more computer programs, microcode segments, hardware structures, or combinations thereof. The functions relevant to the present invention are the media unit (MU) manager  120 , the virtual logical unit number (virtual LUN or VLUN) manager  122 , and the physical logical unit number (physical LUN or PLUN) manager  124 . 
     Additional details of the storage server  100  are provided in other applications assigned to the present assignee and filed on Feb. 13, 2002 that claim the benefit from the above-noted Provisional Application No. 60/268,694 and are hereby incorporated herein by reference as follows: U.S. nonprovisional patent application Ser. No. 10/076,855 entitled “Storage Virtualization and Storage Management to Provide Higher Level Storage Services”; U.S. nonprovisional patent application Ser. No. 10/076,909 entitled “Method and Apparatus for Identifying Storage Devices”; U.S. nonprovisional patent application Ser. No. 10/077,482 entitled “System and Method for Policy Based Storage Provisioning and Management”; U.S. nonprovisional patent application Ser. No. 10/077,181 entitled “Virtual Data Center”; U.S. nonprovisional patent application Ser. No. 10/076,906 entitled “Failover Processing in a Storage System”; U.S. nonprovisional patent application Ser. No. 10/077,199 entitled “RAID at Wire Speed”, and U.S. nonprovisional patent application Ser. No. 10/076,878 entitled “Method for Device Security in a Heterogeneous Storage Network Environment”. 
     The PLUN manager  124  manages data and command transfer to and from the storage devices  104 . Each storage device  104  may have associated therewith a PLUN that is used for identifying each particular storage device  104 . 
     The VLUN manager  122  manages data and command transfer to and from the host computers  106 . Each host computer  106  may be associated with one or more VLUNs. Each VLUN represents a virtual address space (e.g., gigabytes of storage) with defined attributes (e.g., performance parameters, reliability level, etc.). As such, each host computer  106  exchanges data and commands with the storage server  100  with reference to a particular VLUN. 
     Abstract block-storage addressing is achieved via a data structure known as a media unit (MU). The MU manager  120  basically translates between VLUNs and PLUNs. The MU manager  120  is responsible for managing the address space of all the storage devices  104  (physical LUNs) connected to the storage server  100 . The MU manager  120  also manages the address space of the storage constructs built within the storage server  100 , including slices, concatenations, RAID0 (stripes) and RAID1 (mirrors). 
     The MU manager  120  uses an abstract block-storage addressing technique that enables address spaces to be treated in a logical manner, regardless of the underlying storage constructs or physical LUNs. These logical address spaces can be combined together into more complex and feature rich storage constructs, which are also treated simply as abstract block-storage address spaces. 
     Used in conjunction with a virtual LUN, these logical address spaces can be configured to appear as LUNs on a multi-ported storage device. This process of presenting physical LUNs as logical address spaces on virtual devices is referred to as storage virtualization. 
       FIG. 5  shows the relationship between physical media units and the other services. The PLUN manager  124  manages PLUNs, the MU manager  120  manages media units, and the VLUN manager  122  manages VLUNs. 
     In addition,  FIG. 5  shows the relationships between PLUNs, media units, and VLUNs. Generally, a PLUN directly corresponds to a storage device, such as a disk or a disk array. Such a direct one-to-one is relationship generally shown in the following figures. However, a PLUN can also be associated with a portion of the storage device. Multiple PLUNs can be associated with different portions of a single storage device. 
     Each physical media unit (first-level media unit) generally directly corresponds to a single, respective PLUN. Each VLUN is generally associated with a single, respective media unit. 
     FC Frame Processing Description 
       FIG. 6  is a block diagram illustrating a simplified view of the SAN in  FIG. 4 , showing only a single upstream SP  120  and a single downstream SP  122 , interconnected by switch fabric  116 . In an example process, host  106  transfers data over a connection  124  (such as fiber channel) to a PIM  126 , which is a fibre channel ASIC (FC ASIC) in this embodiment. A second FC ASIC  128  is available for connection to other hosts. Host  106  may be a computer itself, or a network card connected to a network of computers. 
     The PIM, or FC ASIC, communicates with storage processor  120  over interface lines  130 . This interface uses a standard internal protocol which is a streaming, packetized protocol, preferably packet over SONET (POS). The upstream storage processor  120  then performs a virtualization and sends the data to be read or written to the appropriate downstream SP  122  connected to the actual disks  132 ,  134 , on which the data resides. The connection to these disks is through additional FC ASICs  136 ,  138 . These then connect through a separate FC port  140  to a storage controller  142 , such as a Redundant Array of Independent Disks (RAID) controller. Connections  124  and  140  could be a protocol other than FC. Examples of other protocols than FC include, but are not limited to, iSCSI, InfiniBand, iFCP, and FCIP. These different streaming protocols may utilize different frame formats. 
       FIG. 7  illustrates in more detail aspects of the PIM and SP of  FIG. 6 . PIM or FC ASIC includes a fiber channel port  144  on one side and a POS port  146  on the other. Internal to the ASIC is a field extraction engine  148 . This inspects the incoming FC frame, and determines what type of action is required, such as read, write, status, or other command. A packet classifier  150  then inserts a frame type code into a Point-to-Point-Protocol (PPP) field encapsulating the FC frame in a POS frame. 
     When SP  120  receives the encapsulated POS frame through a POS port  152 , a hardware classifier  154  examines the frame type and determines where to send it. A read or write command is sent to one of multiple protocol processors  156 . Other commands are provided to a general purpose CPU  158 . Protocol processor  156  determines which VLUN is associated with the frame using the host ID, LUN field and the port associated with the frame. Once the VLUN is determined, a tree search engine  160  is invoked to determine the physical LUN locations for routing purposes. The read or write operation can then be provided through a switch port  162 , using the POS protocol, to switch fabric  116 . In one embodiment, the read or write operation is first directed to a virtual cache  164 . For further description of the virtual cache, see copending application entitled, “Storage Area Network Data Cache”, U.S. patent application Ser. No. 10/234,560, filed Sep. 6, 2002, incorporated herein by reference. If the data is not found in the cache, it is then routed to the indicated downstream SP  122 . 
     In one embodiment, the SP  120  connected to PIM  126  need not be the one which performs the storage functions. Rather, the storage functions could be distributed to any SP connected to switch fabric  116 . The SP that gets assigned will have a routing tag, which is provided back through SP  120  to PIM  126 , so that the next associated frame can have the appropriate routing tag attached by PIM  126 . For a fuller description of the routing tags and their implementation, reference should be made to copending application “Silicon-Based Storage Virtualization Server,” Ser. No. 10/077,696, filed Feb. 13, 2002. Reference should also be made to copending application “Data Mover Mechanism to Achieve SAN RAID to Wire Speed”, U.S. patent application Ser. No. 10/224,618, filed Aug. 19, 2002. The disclosures of the above-referenced applications are hereby incorporated herein by reference. 
     In one embodiment, storage processor  128  is a network processor, such as that made by IBM, the IBM MP4GS3. (E.g., the IBM 32NPR161EPXCAC133). This network processor includes 16 protocol processors and an embedded Power PC processor. 
     The POS connections shown in  FIG. 7  are full duplex, with hardware flow control. This way, the PIM can assume that the SP is ready to receive the frame, relying on hardware flow control to kick in if the SP cannot receive it. 
     The FC ASIC sends the server WWN instead of a device context identifier. The WWN is a worldwide unique identifier. It is issued by IEEE. This way, when a link bounce occurs, the tables do not have to be reprogrammed since the WWN of the server is the same. 
       FIG. 8  shows an alternate embodiment, wherein external minicomputer  702  communicates information utilizing the FC streaming protocol, while external disk array storage device  704  communicates information utilizing the iSCSI streaming protocol. Internally, information is communicated between storage processors  706  and  708  of VSX  700  utilizing a streaming protocol (POS) that is different from either FC or iSCSI. 
     While  FIG. 6  shows an embodiment of an apparatus in accordance with the present invention wherein multiple storage processors as connected to each other through a switch fabric, this is also not required. Storage processors of the VSX could be connected back-to-back and the apparatus and method would fall within the scope of the present invention. This is shown in  FIG. 9 , wherein storage processors  802  and  804  of VSX  800  are connected back-to-back. 
     In addition, while  FIG. 6  shows an embodiment of an apparatus in accordance with the present invention wherein each interface processor is in communication with a single storage processor, this is also not required. In actual implementation, each storage processor has sufficient processing power and bandwidth to handle multiple external and internal ports as shown in  FIG. 10 . 
     The approaches just described offer a number of advantages over the “store and forward” algorithm that is conventionally employed to transfer information. One advantage of embodiments of the present invention is enhanced speed of transfer. Specifically, the use of cut through mode enables packets or frames of data to be routed from source ports to destination ports even before the whole frame of information has arrived. 
     A second advantage of embodiments in accordance with the present invention is reduced complexity. Specifically, the elimination of the need to store and forward each incoming frame significantly reduces the data buffering or caching hardware required. 
     A third advantage of embodiments in accordance with the present invention is efficiency. Specifically, the internal switch fabric enables distributed process, since control and data information packets can be routed from any SP to any SPat wire speed. This is especially efficient for clustering of SPs in order to allow sharing of processing power. It also facilitates alternate path access during error recovery. 
     A fourth advantage of embodiments in accordance with the present invention is scalability. Specifically, the routing systems and methods just described are scalable because the addition of ports also adds processing power in the form of storage processors. 
     A fifth advantage of embodiments in accordance with the present invention is flexibility. For example, each SP of the VSX can be programmed to handle transfer between different varieties of streaming protocols. Moreover, the capacity to utilize such an additional streaming protocol can be readily incorporated into an apparatus in accordance with the present invention by changing a new protocol PIM with the new protocol on the external interface, and changing the internal protocol on the SP interface. 
     Framing 
     The header of a frame may include a variety of forms of control information that provide a context for the frame. One type of control information is the source ID of the frame, which identifies the initiator of the frame (i.e. the network address of the Initiator) to the receiver. In the frame format associated with the FC streaming protocol, the source ID is called “SID”. 
     A second type of control information is the frame originator&#39;s job ID or handle, which identifies the specific job or operation in progress among the plurality of concurrent operations. The job ID or handle allows an incoming frame to be associated with a particular job or operation in progress. In the frame format associated with the FC streaming protocol, the job ID or handle is called “OXID”. 
     A third type of control information present in the frame header is the destination ID, which identifies the final receiver of the frame (i.e. the network address of the Destination.) In the frame format associated with the FC streaming protocol, the destination ID is called “DID”. 
     A fourth type of control information is the receiver&#39;s job ID or handle. The frame receiver&#39;s job ID or handle identifies the specific job or operation in progress among the plurality of concurrent operations, so an incoming frame can be identified with which job or operation. In the frame format associated with the FC streaming protocol, the receivers job ID or handle is called “RXID”. 
     To summarize, for the FC streaming protocol, the SID and OXID control information together identify the specific job on the originator side. The DID and RXID control information identify the corresponding job ID on the target side. 
     A fifth type of control information describes the frame type. Examples of frame types include but are not limited to Command Frames, Data Frames, Status Frames, and Acknowledgment Frames. 
     The content of the payload depends on the frame type identified in the header. For example, the payload of a Command Frame contains the command code and its parameters. The payload of a Data Frame contains data. 
     The integrity of the contents of the entire frame may be ensured by such mechanisms as CRC or Check-Sum. 
     The FC ASIC provides the following features to assist in the SP processing of the frames. 
     It provides the FCP frame type in a code inserted into the PPP protocol field. The types that should be identified by the FC ASIC are: 
     FCP_CMND—0x2B00, 
     FCP_STATUS—0x2B01, 
     FCP_DATA—0x2B03, 
     RDY/ACK—0x2B02, and 
     NON SCSI, CONTROL, EVENT_REPORT—0X2B04. There will be a subtype to distinguish the three types. 
     When sending data frames to the SP, each frame is numbered sequentially. This is a simple way for the SP to see if there are missing data frames. This is the frame_id in the TAGS field. Some frames take longer to process than others. With a hardware “tagging” facility, the frames can be kept in order of arrival at the ingress side even though the firmware has completed processing and has queued the frame to the fabric. The hardware “tagging” will ensure that the frame is sent to the fabric in the order received. 
     The SP will send the FC ASIC an SP handle and routing information. The SP handle will be sent along with the FC ASIC handle. The FC ASIC will use the latest handle and routing information sent by the SP. The routing information may change during the command duration. The routing information contains a remote IoCB lookup, TB number, target DMU/DSU. This mechanism is used to minimize the time to look for an IoCB before shipping the data frame. 
     The FC ASIC provides a CRC over the data portion of the data frame. This is in addition to the CRC added by the POS Interface. This CRC is used to validate the integrity of data from the upstream SP to the downstream SP. This CRC is referred to as DTCRC. 
     The FC ASIC validates the DTCRC provided in a data frame. This is in addition to the CRC applied by the POS Interface. Since the headers will change, the value of the DTCRC remains intact. 
     The FC ASIC informs the SP if it receives a data frame with an invalid DTCRC. This way, the problem can be reported back to the host. 
     When the FC ASIC sends the SP a WRITE command, it needs to wait for a RDY acknowledgment from the SP informing it of the number of bytes the SP is willing to take in. The SP will send more RDY frames when the byte count has finished. This support is needed in order to support requests that span disks. The SP will only send an RDY up to the disk boundary. This way, the SP does not have to split a frame. 
     The FC ASIC tells the SP through control frames the FC topology: 
     loop/fabric/point2point. 
     In a loop topology, the FC ASIC goes through the Loop Initialization and provides the SP through a directed frame of the loop position map. 
     FC ASIC Sending a Write Command 
     A command descriptor will be sent to the SP. The SP will send a response back to the FC ASIC with the number of bytes it is willing to accept. This frame is referred to as a RDY frame. With the RDY frame, the SP will insert the SP handle. This SP handle should be used by the FC ASIC when sending data related to this command. 
     The FC ASIC will send data to the SP up to the amount specified in the response. The FC ASIC will attach the SP command handle with each data frame. This will allow the SP to associate the data frames with a particular command. 
     When the FC ASIC has finished sending data up to the amount specified in the RDY frame, the SP will send another RDY frame telling the FC ASIC the amount of data it is willing to accept. This will continue until the write command is complete. This mechanism is similar to the XFER_RDY used in Fibre Channel. 
     When the SP completes the write operation, it will send a STATUS response to the FC ASIC. After sending out the STATUS frame, the FC ASIC will generate an ACK frame to the SP to indicate the completed transaction. The SP will at this point free up its resources. 
     Frame ID Numbering 
     Frames may be dropped due to the lack of buffer resources or collisions. In order to detect this condition, the ACK/RDY, Data and Status frames have a frame id number. 
     The SP observes the frame ID from the FC ASIC and checks it to make sure that it is the expected value. The expected value is 1 more than the previous frame ID. 
     The FC ASIC similarly checks the SP frame ID to make sure that it is the expected value. The expected value is 1 more than the previous frame ID. 
     When the FC ASIC sends a READ Command to the SP, the SP will respond with Data Frames. The 1st Data Frame will have a frame ID number of 0, the 2nd Data Frame a frame ID of 1 and so on. Once data transfer is complete, the SP will send the Status Frame. The frame ID of this frame will be 1 more than the last Data Frame. 
     CRC Protection 
     In order to provide end-to-end data protection the FC ASIC will provide CRC over the FC Data in the data frames. The SP may or may not examine the CRC. It will just route the data frame including the CRC to the egress side. This is the DTCRC. 
     On the egress side, the SP will modify the headers and send the data frame to the FC ASIC. The FC ASIC will perform the CRC verification. 
     If there is an error, the FC ASIC will generate a Status frame indicating a CRC Failure. 
     On the positive side, the SP is not burdened with performing CRC calculations. 
     As will be understood by those with skill in the art, the present invention may be embodied in other specific forms without departing from the essential characteristics thereof. For example, a general purpose CPU could be used instead of a network processor in the SP of the described embodiment. Additionally, an internal streaming protocol other than POS could be used, such as Ethernet over SONET or SDH. Accordingly, the foregoing description is intended to be illustrative, but not limiting, of the scope of the invention which is set forth in the following claims.