Source: http://www.google.com/patents/US7957409?dq=US+6,313,853
Timestamp: 2016-06-26 09:29:43
Document Index: 169525675

Matched Legal Cases: ['Application No. 2003219522', 'Application No. 2003291522', 'Application No. 2', 'Application No. 200380109101', 'Application No. 07007583', 'Application No. 07007583', 'Application No. 07007583', 'Application No. 200380109101', 'Application No. 03', 'Application No. 03', 'Application No. 200380109101', 'Application No. 200380109101']

Patent US7957409 - Methods and devices for transmitting data between storage area networks - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsMethods and devices are provided for efficient transmission of data between storage area networks. According to some aspects of the invention, novel methods are provided for processing data packets sent by, or received from, a storage area network. Some such aspects of the invention involve storing a...http://www.google.com/patents/US7957409?utm_source=gb-gplus-sharePatent US7957409 - Methods and devices for transmitting data between storage area networksAdvanced Patent SearchPublication numberUS7957409 B2Publication typeGrantApplication numberUS 10/351,167Publication dateJun 7, 2011Priority dateJan 23, 2003Fee statusPaidAlso published asCN1742469A, US8724656, US20040146063, US20110206059Publication number10351167, 351167, US 7957409 B2, US 7957409B2, US-B2-7957409, US7957409 B2, US7957409B2InventorsAli Golshan, Neelima Mehta, Pags Krishnamoorthy, Madhuri Kolli, Devi Prasad Ivaturi, Venkatesh JanakiramanOriginal AssigneeCisco Technology, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (28), Non-Patent Citations (33), Classifications (34), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetMethods and devices for transmitting data between storage area networks
US 7957409 B2Abstract
Methods and devices are provided for efficient transmission of data between storage area networks. According to some aspects of the invention, novel methods are provided for processing data packets sent by, or received from, a storage area network. Some such aspects of the invention involve storing a packet (or a portion of a packet) in a single memory location during an encapsulation or de-encapsulation process. Instead of repeatedly copying the packet during processing, pointer information is passed along that indicates the single memory location. In some aspects of the invention, the segment boundaries of a packet are retained after data transmission. If data in the packet need to be re-transmitted, the packet is re-transmitted with the same segment boundaries.
Storage area networks (“SANs”) are becoming increasingly popular networks for businesses, universities and governmental entities. Such networks are typically connected via optical fiber, which allows for high-speed data transmission. Many SANs use the Fiber Channel (“FC”) protocol for data transmitted via optical fiber.
The present invention provides methods and devices for more efficient transmission of data between storage area networks. According to some aspects of the invention, novel methods are provided for processing data packets sent by, or received from, a storage area network. Some such aspects of the invention involve storing a packet (or a portion of a packet) in a single memory location during an encapsulation or de-encapsulation process. Preferably, a “scratch pad” space is reserved in the memory location, within which header information may be written. Instead of repeatedly copying the packet during processing, pointer information is passed along that indicates the single memory location.
According to some aspects of the invention, a “slim” TCP stack is provided which eliminates the overheads associated in context switches (from interrupt-level to process-level), found in conventional TCP stacks, by doing the TCP processing at interrupt level. The invention also provides a method to eliminate buffer copies, found in a conventional TCP stacks. Both these factors—no buffer copies and interrupt-level processing—reduce a large amount of processing overhead. Elimination of the socket layer, which, too, is otherwise found in conventional TCP stacks, adds to the foregoing benefits. One advantage of eliminating the socket layer is the elimination of socket buffers and the associated buffer-to-buffer copying.
Other aspects of the invention provide a method for controlling data transmitted between storage area networks. Since the TCP processing is done in an interrupt context, it is necessary to ensure that a restricted amount of time is spent in a single interrupt context. The “slim” TCP helps towards this goal. Additionally, the method includes the following steps: on receiving the first FCIP packet, a TCP header may be added to the FCIP packet, and the FCIP packet is transmitted to the Internet, all in the same interrupt context. For subsequent FCIP packets, transmission will be done on receiving acknowledgements for previously sent packets according to some aspects of the invention. The interrupt context may assign a higher priority to command packets than to data packets.
The steps of receiving, adding and transmitting may be considered a “loop.” Accordingly, before transmitting an FCIP packet to the Internet, to restrict the processing done in a single interrupt context, it may be determined whether a maximum number of loops has occurred during the interrupt context.
FIG. 1 depicts system 100 according to some aspects of the present invention. System 100 includes storage area network (“SAN”) 105, which is located in San Francisco in this example. SAN 105 includes a plurality of hosts, data storage devices, switches and/or routers, servers and other components well known to those of skill in the art. Here, SAN 105 is interconnected using optical fiber. A Fiber Channel (“FC”) protocol is used for relaying information within SAN 105. SAN 110 is a similar storage area network located in New York. Obviously, SAN 105 and SAN 110 could be located anywhere in the world. Moreover, while only two SANs are illustrated in FIG. 1, any number of SANs could be interconnected in system 100.
It is necessary for FC frames 112 and 135 to be encapsulated or otherwise transformed into a format recognizable by IP cloud 120. This is accomplished by encapsulating the FC frames into “FCIP” packets 117 and 122. Here, this encapsulation is performed by cards 115 and 130 of network devices 118 and 125, respectively. Network devices 118 and 125 may be routers, switches or other types of network devices known in the art. Cards 115 and 120 may be port adapter cards such as the Fiber Channel Port Adapter (“FCPA”) in the Cat6500 switch, which is manufactured by the assignee. The encapsulation process will be explained in more detail with reference to FIGS. 2 and 3.
In a typical TCP connection, a “client” network device initiates transmissions and a “server” network device enters a “listen” mode during such transmissions. Preferred embodiments of the present invention do not use this client/server model. Instead, cards 115 and 130 simultaneously transmit in “interrupt” context, which will be described in more detail below.
In some embodiments, each TCP connection is defined by four properties, which are collectively referred to herein as a “four-tuple.” These properties are source port, destination port, source IP address and destination IP address. According to some embodiments of the invention, the command connection and the data connection have different four-tuples, including different ports, but have the same IP address. A “five-tuple” is a four-tuple plus a specified protocol type, which will be TCP protocol according to preferred aspects of the invention.
With a typical client/server configuration, the client initiates the connection while the server is passively listening to the network. This is known as a “passive open” configuration. However, according to some embodiments of the present invention, the TCP end points are treated as peers rather than as a client or server. Both peers simultaneously attempt to establish connections. This condition is referred to as a “simultaneous open.” Some aspects of the invention remove the complexity of the socket layer functions such as “listen”, “accept” and “bind” for the establishment of the connection. The TCP code runs at interrupt level, segments are processed as and when they are received Therefore, processing time is minimized and data transfer between SANs is accelerated.
In some such embodiments, this “peer level” configuration is established by removing the socket layer interface from the TCP stack. FIG. 5A illustrates the structure of conventional TCP stack 500, which includes link layer 505 (which is an Ethernet layer in this example), IP layer 510, TCP layer 515, socket layer 520 and application layer 525. Those of skill in the art will understand that variations of conventional TCP stack 500 exist. For example, while Ethernet layer 505 is a common link layer, there are other link layers known in the art that serve a similar purpose, such as serial line IP (“SLIP”) and point-to-point protocol (“PPP”).
In some embodiments of the present invention, FCIP Module 317 includes logic for distinguishing FC data frames from FC control frames, e.g., by parsing the Fiber Channel header of the FC frames. FCIP Module 317 sends data frames to a first buffer maintained by FC Driver 315 and FCIP Module 317 sends control frames to a second buffer maintained by FC Driver 315. In some preferred embodiments, these buffers are referred to as “FIFO buffers,” wherein “FIFO” means “first in, first out.” FC Driver 315 knows that frames in the first buffer are data frames and that frames in the second buffer are control frames. The interaction of the FIFO buffers with other elements of the system will be described below with reference to FIGS. 7 through 11.
However, the boundaries between segments are not stored for subsequent re-transmission. If no acknowledgment (“ACK”) packet were received to indicate that a particular segment had reached its destination, a new packet would be created for re-transmission of the segment. The new packet would probably include a different amount of data than the original segment. For example, if no ACK were received for the segment that included bytes 1-20, bytes 1-30 might be re-transmitted as one segment.
FIGS. 8-11 provide additional details regarding pointer manipulation according to some aspects of the invention. FIG. 8A depicts buffer 705 and memory 735 at a first time and FIG. 8B depicts buffer 705, memory 735 and packet structure (“PAK”) 810 at a second time.
In ordinary TCP processing, a high amount of overhead per packet is required to process a packet in the interrupt context. Part of this overhead is due to the need to change between normal “process context” and interrupt context. For example, suppose process A is occurring when an interrupt is received. Process A is halted and the state of process A is saved. Then, the system switches to interrupt context and the service routine associated with the interrupt is scheduled and processed. After the interrupt process is completed, the system returns to process context. A scheduler must then reschedule process A and retrieve the state of process A at the time the process was halted. Accordingly, changing contexts adds a lot of overhead.
One way of controlling interrupt processing is by imposing a maximum lifetime on unidirectional interrupt processing in order to prevent packets from being dropped. This lifetime may be measured in CPU cycles or “loops.” According to some aspects of the invention, after TCP Module 320 has caused one such packet to be sent, TCP Module 320 asks FC Driver 315 if it has another packet to send. Suppose there is another packet to send. This packet, which is associated with another part of the buffer, is then processed and sent. If this process were continued until the buffers were drained of packets, it would be very likely that incoming packets in the other direction, i.e., from the IP cloud into the Line Card, would be dropped, due to interrupt processing in the other direction getting all the CPU cycles.
Therefore, in preferred aspects of the invention, this “loop” is only permitted to happen a certain number of times before interrupt processing of outgoing packets is terminated, each loop being associated with a single packet. According to some such aspects, command packets have a higher priority than data packets and accordingly are allowed a larger number of loops before the processing stops. According to one such aspect of the invention, the maximum number of loops is 4 for command packets and 3 for data packets.
Another way of controlling interrupt processing of data being sent to the IP Cloud is by placing limitations on transmitted data. For example, some aspects of the invention limit the amount of data sent before receiving an acknowledgment packet (an “ACK”) according to a “TCP sliding window.” For example, the limitation may be the equivalent of 64 kb of data sent before receiving an ACK. After the data transmission limit has been reached, the interrupt processing of packets in the FC Driver' memory is suspended. After receiving an ACK, the TCP sliding window is “opened,” the TCP Module queries the FC Driver as to whether it has any packets to send. If the answer is “No,” the interrupt process is re-enabled. Then, packets from the FC side may once again be processed in interrupt context and transmitted until the transmission limit is attained.
According to some aspects of the present invention, receiving any incoming data will prevent the data transmission limit from being attained. For example, when a data packet is received by IP Driver 330, TCP Module 320 needs to cause an ACK packet to be sent. Before sending the ACK packet, TCP Module 320 queries FC Driver 315 to determine whether there is a packet in its memory which could be “piggy-backed” and sent along with the ACK packet to the IP Cloud.
The interrupt may be conceived as a notification mechanism between an FC port adaptor and the FC Driver. The interrupt processing is enabled only at certain times. For example, when the system comes on line, the interrupt “notification mechanism” is enabled. At this time, when the FC Driver receives a packet, it is passed immediately through the steps of method 600 and sent to the IP cloud.
The interfaces 1268 are typically provided as interface cards (sometimes referred to as “line cards”). Generally, they control the sending and receiving of data packets over the network and sometimes support other peripherals used with the network device 1260. Among the interfaces that may be provided are FC interfaces, Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various very high-speed interfaces may be provided such as fast Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces, HSSI interfaces, POS interfaces, FDDI interfaces, ASI interfaces, DHEI interfaces and the like.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5742607Dec 20, 1995Apr 21, 1998Intel CorporationMethod and apparatus for controlling two way communication via disparate physical mediaUS5987519Sep 19, 1997Nov 16, 1999Georgia Tech Research CorporationTelemedicine system using voice video and data encapsulation and de-encapsulation for communicating medical information between central monitoring stations and remote patient monitoring stationsUS6006275 *Oct 6, 1998Dec 21, 1999Compaq Computer CorporationNetwork connector operable in bridge mode and bypass modeUS6400730 *Feb 8, 2000Jun 4, 2002Nishan Systems, Inc.Method and apparatus for transferring data between IP network devices and SCSI and fibre channel devices over an IP networkUS6401171 *Feb 26, 1999Jun 4, 2002Cisco Technology, Inc.Method and device for storing an IP header in a cache memory of a network nodeUS6611495 *Jun 7, 1999Aug 26, 2003Telefonaktiebolaget Lm Ericsson (Publ)System and method for improved data transfer in packet-switched communication networksUS6760769Aug 30, 2002Jul 6, 2004Adaptec, Inc.Apparatus and methods for transmitting data at high speed using TCP/IPUS6985490Jul 11, 2001Jan 10, 2006Sancastle Technologies, Ltd.Extension of fibre channel addressingUS7096247 *Aug 30, 2002Aug 22, 2006Adaptec, Inc.Apparatus and methods for receiving data at high speed using TCP/IPUS7106751Feb 19, 2002Sep 12, 2006Hitachi, Ltd.Apparatus for linking a SAN with a LANUS7114009 *Mar 16, 2001Sep 26, 2006San Valley SystemsEncapsulating Fibre Channel signals for transmission over non-Fibre Channel networksUS7171440 *Jul 15, 2002Jan 30, 2007The Boeing CompanySystem and method for virtual packet reassemblyUS7197047Apr 30, 2002Mar 27, 2007Nishan Systems, Inc.Method and apparatus for transferring data between IP network devices and SCSI and fibre channel devices over an IP networkUS7308001Nov 15, 2002Dec 11, 2007Computer Network Technology CorporationFibre channel frame batching for IP transmissionUS20020078028Dec 14, 2001Jun 20, 2002Trevalon Inc.Network serverUS20020141353 *Mar 19, 2002Oct 3, 2002Reiner LudwigMethod and device for improving a data throughputUS20020156924 *Apr 23, 2001Oct 24, 2002Moshe CzeigerMethod for communicating between fibre channel systemsUS20020181506 *Jun 4, 2001Dec 5, 2002Koninklijke Philips Electronics N.V.Scheme for supporting real-time packetization and retransmission in rate-based streaming applicationsUS20030028663May 22, 2002Feb 6, 2003Mullendore Rodney N.High jitter scheduling of frames in an arbitrated loopUS20030084219Oct 26, 2001May 1, 2003Maxxan Systems, Inc.System, apparatus and method for address forwarding for a computer networkUS20030093541 *Jan 18, 2002May 15, 2003Lolayekar Santosh C.Protocol translation in a storage systemUS20030204620 *Apr 29, 2002Oct 30, 2003Intel CorporationNetwork device with improved routing characteristicsUS20040013117 *Jul 18, 2002Jan 22, 2004Ariel HendelMethod and apparatus for zero-copy receive buffer managementUS20040146063Jan 23, 2003Jul 29, 2004Cisco Technology, Inc.Methods and devices for transmitting data between storage area networksUS20050232269May 26, 2005Oct 20, 2005Maxxan Systems, Inc.System, apparatus and method for address forwarding for a computer networkUS20060209840 *Sep 12, 2005Sep 21, 2006Slt Logic LlcSystem and method for providing transformation of multi-protocol packets in a data streamUS20070118665 *Jan 22, 2007May 24, 2007Philbrick Clive MTCP/IP offload device with fast-path TCP ACK generating and transmitting mechanismWO2001028179A2Oct 12, 2000Apr 19, 2001Bluearc Uk LimitedApparatus and method for hardware implementation or acceleration of operating system functions* Cited by examinerNon-Patent CitationsReference1AU 2nd Office Action dated Aug. 12, 2009, AU Patent Application No. 2003219522.2AU Examiner Report mailed Jul. 15, 2008, and received Aug. 11, 2008, from Australian Application No. 2003291522.3Canadian Office Action dated Dec. 7, 2009, CA Patent Application No. 2,514,086.4Chinese Office Action dated Aug. 5, 2010, CN Serial No. 200380109101.2.5Cisco Systems, "Cisco Catalyst Switching Solutions Guide, Intelligent Switching for the Evolving Network: Easing network migration, adaptation, and expansion." Printed Mar. 29, 2004, www.cisco.com/go/catalyst2950, pp. 1-12.6CN Rejection Decision dated Jun. 19, 2009. CN Patent Application No. 200380109101.2.7Examiner's Communication and extended European Search Report, pursuant to Rule 44a EPC, mailed Jul. 25, 2007, received Aug. 1, 2007, from European Patent Application No. 07007583.3, Methods and Devices for Transmitting Data Between Storage Area Networks, 12 pp.8Examiner's Communication dated Apr. 4, 2008 from EP Application No. 07007583.3.9Examiner's Communication pursuant to Article 94(3) EPC, dated Apr. 4, 2008, from European Patent Application No. 07007583.3, 3 pp.10First OA issued Apr. 11, 2008, for the CN Patent Application No. 200380109101.2.11First Official Action issued Nov. 30, 2005, for the European Patent Application No. 03 768 926.2, filed Nov. 14, 2003, Methods and Devices for Transmitting Data Between Storage Area Networks 11 pp.12Golshan, et al., "Methods and Devices for Transmitting Data Between Storage Area Networks", U.S. Appl. No. 10/350,574, filed Jan. 23, 2003, 52 pages.13Monia, Charles et al., IETF, Standard Track, "iFCP-A Protocol for Internet Fibre Channel Networking", IP Storage Working Group, Dec. 2002, pp. 1-104.14Monia, Charles et al., IETF, Standard Track, "iFCP—A Protocol for Internet Fibre Channel Networking", IP Storage Working Group, Dec. 2002, pp. 1-104.15Monia, et al., "iFCP-A Protocol for Internet Fibre Channel Storage Networking", IETF-IP Storage Working Group, Dec. 31, 2002.16Notice of Allowance, mailed Dec. 28, 2009 from U.S. Appl. No. 10/350,574.17Notice of Grant from EP Patent Application No. 03 768 926.2-2416 mailed Feb. 19, 2007.18Notification of Reexamination issued May 4, 2010, for CN Patent Application No. 200380109101.2.19PCT International Preliminary Examination Report dated Oct. 25, 2004 from PCT Application No. PCT/US03/036182.20PCT International Search Report from PCT Application No. PCT/US03/036182 mailed May 8, 2004.21PCT Written Opinion from PCT Application No. PCT/US03/036182 mailed May 8, 2004.22 *Rajagopal et al, Fiber Channel over TCP/IP (FCIP), Aug. 2001.23Second Chinese Office Action issued Oct. 10, 2008, from Chinese Application No. 200380109101.2 (Chinese and English Translation).24US Notice of Allowance, mailed Apr. 6, 2010 from U.S. Appl. No. 10/350,574.25US Office Action mailed Aug. 4, 2008, for U.S. Appl. No. 10/350,574.26US Office Action, mailed Aug. 2, 2007 from U.S. Appl. No. 10/350,574.27US Office Action, mailed Aug. 4, 2008 from U.S. Appl. No. 10/350,574.28US Office Action, mailed Feb. 6, 2007, from U.S. Appl. No. 10/350,574.29US Office Action, mailed Jan. 14, 2008 from U.S. Appl. No. 10/350,574.30US Office Action, mailed Jan. 22, 2009 from U.S. Appl. No. 10/350,574.31US Restriction Requirement, mailed Aug. 17, 2009 from U.S. Appl. No. 10/350,574.32Voruganti, et al., "An Analysis of Three Gigabit Networking Protocols for Storage Area Networks", IBM Almaden Research Lab, San Jose, CA. IEEE 2001.33Voruganti, K. and Sarkar, P., "An Analysis of Three Gigabit Networking Protocols for Storage Area Networks", IEEE 0-7803-7001, May 2001, pp. 259-265.* Cited by examinerClassifications U.S. Classification370/429, 370/392, 370/474, 370/386, 370/466, 370/401, 370/471, 370/469International ClassificationH04L12/54, H04L12/28, H04L29/08, H04J3/24, H04L29/06, H04J3/16, H04L12/50, H04L12/46, H04L1/18Cooperative ClassificationH04L69/28, H04L69/329, H04L69/16, H04L69/161, H04L69/162, H04L67/1097, H04L69/163, H04L29/06, H04L12/4633, H04L1/188European ClassificationH04L29/06, H04L29/08N9S, H04L12/46E, H04L1/18T5, H04L29/06J7, H04L29/06J3, H04L29/06J3SLegal EventsDateCodeEventDescriptionJan 23, 2003ASAssignmentOwner name: CISCO TECHNOLOGY, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOLSHAN, ALI;MEHTA, NEELIMA;KRISHNAMOORTHY, PAGS;AND OTHERS;REEL/FRAME:013708/0370;SIGNING DATES FROM 20030121 TO 20030122Owner name: CISCO TECHNOLOGY, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOLSHAN, ALI;MEHTA, NEELIMA;KRISHNAMOORTHY, PAGS;AND OTHERS;SIGNING DATES FROM 20030121 TO 20030122;REEL/FRAME:013708/0370Dec 8, 2014FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services