Source: https://patents.google.com/patent/US7433326B2/en
Timestamp: 2019-04-21 20:48:36+00:00

Document:
2004-07-06 Assigned to CISCO SYSTEMS, INC. reassignment CISCO SYSTEMS, INC. MERGER (SEE DOCUMENT FOR DETAILS). Assignors: ANDIAMO SYSTEMS, INC.
2005-06-27 Assigned to CISCO TECHNOLOGY, INC. reassignment CISCO TECHNOLOGY, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CISCO SYSTEMS, INC.
Methods and devices are provided for detecting whether peer ports interconnecting two network devices can perform a novel protocol called Exchange Peer Parameters (“EPP”). If the peer ports are so configured to perform EPP, EPP services are exchanged between the peer ports. In a first phase, information is exchanged about peer port configurations of interest. In a second phase, the results of the exchange of information are applied to hardware and/or software of the respective ports, as needed.
This application claims priority to U.S. Provisional Application No. 60/429,897, filed Nov. 27, 2002, which is hereby incorporated by reference for all purposes.
The present invention generally relates to data networks. More specifically, the invention relates to the configuration of routers, switches and other network devices within such data networks.
Several limitations may be encountered when configuring networks such as local area networks, storage area networks and the like. There are a variety of network devices, such as routers, switches, bridges, etc., which may be used to configure such networks. Some of these network devices have greater capabilities than others. For example, some devices may readily be configured to support logical networks superimposed upon a physical network (e.g., virtual local area networks (“VLANs”) or virtual storage area networks (“VSANs”)) and some may not.
In one example, it would be useful to transport packets of different frame types using the same interswitch link instead of dedicating inter-switch links for different frame types. For example, it would be desirable if links between network devices could carry both Ethernet and Fiber Channel (“FC”) frames.
It is also important to determine as quickly as possible whether a network device has certain capabilities. For example, it would be very useful to determine quickly whether a peer port of another network device is configured (or could be configured) to carry frames of particular VLANs or VSANs, and to configure the network device as needed. Otherwise, various problems (including dropped frames) will ensue if the network device is connected to other devices that are transmitting frames for the wrong VLAN or VSAN. However, testing and configuring network devices for such capabilities can be time-consuming.
According to some aspects of the invention, a new protocol, known herein as Exchange Peer Parameters (“EPP”), is provided for communication between peer ports of network devices that form part of the fabric of a network. In some embodiments, EPP protocol is used to exchange information and/or to configure E or F ports of an FC network.
Methods and devices are provided for detecting whether an attached peer port of a network device can exchange peer parameters with the corresponding port according to a novel Exchange Peer Parameters (“EPP”) protocol. If the peer port is so configured, EPP service exchanges are performed with the peer port. In a first. phase, information is exchanged about peer port configurations of interest. In a second phase, the results of the exchange of information are applied to hardware and/or software of the peer ports, as needed.
According to some aspects of the invention, when an inter-switch link is formed, a port of a peer network device is interrogated to determine whether it can support EPP protocol. If so, EPP service exchanges are performed with the peer port.
According to other aspects of the invention, configuration information is exchanged between peer ports in a network after an inter-switch link has been formed between the peer ports and after data frames have been transmitted to and from the peer network device. Such an information exchange may occur, for example, when the trunk mode of one of the ports has been changed during operation of the port. The results of the exchange of information are applied to hardware and/or software of the peer ports, as needed.
According to some implementations of the invention, methods and devices are provided for configuring a port of a network device in trunking mode so that all frames are transmitted in a novel format known as extended inter-switch link (“EISL”) format, which will be discussed in more detail below. According to some such aspects of the invention, when an inter-switch link is formed, a port of a peer network device is interrogated to determine whether it can be a trunking port. If so, the port is configured to be in trunking mode using the EPP protocol.
According to some preferred aspects of the invention, the EPP protocol is used after the Exchange Switch Capabilities (“ESC”) protocol. ESC may be used to exchange a set of protocols supported by the switch. EPP is one such protocol in the set of protocols. The EPP protocol is used, for example, to determine whether a port of a network device is configurable for supporting VLANs, VSANs and/or EISL. The EPP protocol can be used, for example, to configure an E or F port for EISL. If an E port is so configured, the port is referred to as a “trunking E port” or a TE port.
According to some implementations of the invention, a method is provided for modifying configurations of peer ports interconnecting network devices. The method includes: determining that the interconnected peer ports, comprising a first port of a first network device and a second port of a second network device, can support Exchange Peer Parameters protocol; exchanging configuration information using the Exchange Peer Parameters protocol between the interconnected peer ports; and configuring the interconnected peer ports according to the exchanged information.
The determining step can involve exchanging information between the first port and the second port via, for example, Exchange Link Parameter protocol or Exchange Switch Capability protocol. The exchanging step can involve exchanging frames in, for example, type-length-value format or a fixed frame length format.
The configuration information can include, for example, virtual storage area network information or trunk mode information. The configuration information can be exchanged when the interconnected peer ports are being initialized or when the interconnected peer ports have already been initialized. The configuration step can include configuring the hardware and/or the software of the interconnected peer ports according to the exchanged information.
Alternative implementations of the invention provide a method for modifying a configuration of a network device. The method includes: determining that a first expansion port of a first network device, the first expansion port attached to a second expansion port of a second network device, can be configured to transmit frames in Extended Interswitch Link format; and configuring the first expansion port to transmit frames in Extended Interswitch Link format.
The determining step can include exchanging trunk mode information between the first expansion port and the second expansion port via Exchange Peer Parameters protocol. The configuring step can include configuring the hardware and/or software of the first expansion port to enable transmission of frames in Extended Interswitch Link format. The configuring step can involve informing the second expansion port via Exchange Peer Parameters protocol that the configurations have been applied to the first expansion port.
Some embodiments of the invention provide a computer program for causing a first expansion port of a first network device to modify a configuration of a second expansion port of a second network device. The computer program causes the first expansion port to perform the following steps: determining that the second expansion port can be configured as a trunking port for transmitting frames in Extended Interswitch Link format; and configuring the second expansion port as a trunking port.
The determining step may involve exchanging information between the first expansion port and the second expansion port via Exchange Link Parameter protocol or via Exchange Switch Capability protocol. The configuring step can include exchanging information between the first expansion port and the second expansion port via Exchange Peer Protocol.
Alternative aspects of the invention provide a carrier wave embodying an encoded data signal for modifying a configuration of a network device. The encoded data signal includes: a command code field for identifying whether a command is from a synchronization phase or a commit phase of a process for configuring an expansion port of the network device; and a command identifier field for indicating whether a request to perform part of the process has been accepted or rejected.
The encoded data signal may also include trunk configuration information. The trunk configuration information can include, e.g., administratively configured trunk mode information for trunk mode negotiation, virtual storage area network list information, or port virtual storage area network information. The administratively configured trunk mode information can include a setting selected from the group consisting of ON, OFF and AUTO.
Yet other embodiments of the invention provide an apparatus for modifying a configuration of a network device. The apparatus includes: a mechanism for determining that the interconnected peer ports, comprising a first port of a first network device and a second port of a second network device, can support Exchange Peer Parameters protocol; a mechanism for exchanging configuration information using the Exchange Peer Parameters protocol between the interconnected peer ports; and a mechanism for configuring the interconnected peer ports according to the exchanged information. These mechanisms may or may not be separate devices, according to the implementation.
Still other embodiments of the invention provide a first network device for modifying a configuration of a second network device. The first network device is configured to perform the following steps: determining that a port of the second network device can support Exchange Peer Parameter protocol; and causing the port to be configured based on configuration information exchanged between the first network device and the port via Exchange Peer Parameters protocol.
The determining step can include exchanging information between the first network device and the port via Exchange Link Parameter protocol or Exchange Switch Capability protocol. The configuring step can include exchanging information between the first network device and the port via Exchange Peer Parameter protocol.
A further understanding of the nature and advantages of the present invention may be realized by reference to the remaining portions of the specification and the drawings.
FIG. 1 illustrates a storage area network.
FIG. 2 depicts an EISL frame.
FIG. 3 illustrates a simplified frame having an EISL header.
FIG. 4 illustrates an exemplary stack for implementing an exchange peer protocol (“EPP”).
FIG. 5 is a flow diagram that outlines the processes of determining that a device can be configured for EPP and implementing EPP.
FIG. 5A is a diagram of a time-length-value frame.
FIG. 6 is a table that indicates how differences are resolved between a local trunk mode and a peer trunk mode.
FIG. 7 is a diagram that indicates VSAN bit map information from port A and port B and the resulting VSAN intersection bit map.
FIG. 7A is a flow chart that outlines a process for implementing the EPP SYNC and commit phases after a link has previously been established.
FIG. 8 is a flow chart that outlines the EPP process for an initiating port.
FIG. 9 is a flow chart that outlines the EPP process for a receiving port.
FIG. 10 is a table that describes one example of an EPP header.
FIG. 11 depicts a network device that may be configured to perform the methods of the present invention.
FIG. 1 indicates network 100, which is a storage area network (“SAN”) according to some preferred aspects of the present invention. Although the following description will focus on SANs and their corresponding protocols, etc., the present invention is applicable to other networks, such as LANs.
SAN 100 includes nodes 105 and 110, which may be host devices such as personal computers. SAN 100 also includes nodes 115, 120 and 125, which are storage devices in this instance. Although Internet 130 is not part of SAN 100, it is connected to SAN 100 via node 131. Similarly, nodes 105 through 125 are connected to SAN 100 via ports 106, 111, 116, 121 and 126, respectively.
SAN 100 also includes network devices 135, 140 and 145. Such network devices may be of any kind known in the art, such as routers, switches, bridges, etc. These network devices are connected to their respective nodes by fabric ports. For example, network device 135 is connected to nodes 105 and 110 by fabric ports 150 and 155, respectively. Such ports are designated with an “F” in FIG. 1.
Connections between network devices are made by expansion ports or “E” ports. Connections between E ports are referred to as Inter-Switch Links (“ISLs”). For example, network device 135 is connected to network device 140 via an ISL between E port 160 of network device 135 and E port 170 of network device 140. Similarly, the connection between network device 140 and 145 is made by an ISL between E ports 175 and 180.
As is well known in the art, connections between network devices and nodes of storage area networks are commonly made via optical fiber. Data are transmitted on such networks according to various formats, but most commonly using the Fiber Channel protocol.
Some network devices may be configured to support a novel frame format, known as extended inter-switch link (“EISL”) format, which is the subject of other pending patent applications assigned to Andiamo Systems. The description of some embodiments and applications of EISL in U.S. patent application Ser. No. 10/034,160 is hereby incorporated by reference for all purposes. In one example, the EISL format allows a single network device to process frames or packets having different formats. For example, a network device configured to support EISL may process both FC frames and Ethernet frames. The EISL format also supports VLANs, VSANs and similar features.
An EISL format allows the implementation of a fibre channel network with features and functionality beyond that provided by ISL format. In one example, the EISL format allows a port (known herein as a “trunking port”) to transport frames of more than one format. For example, a trunking port can switch Ethernet and Fiber Channel (“FC”) frames and is adaptable to transmitting frames of other formats as they are developed. An EISL header is used on EISL links to enable this transportation of different frame types. In another example, the EISL format allows the implementation of multiple virtual storage area networks (VSANs) on a single physical network. In still other examples, the EISL format provides mechanisms for implementing forwarding mechanisms such as Multi-Protocol Label Switching (MPLS) or Time To Live (TTL) fields specifying how packets should be forwarded and when packets or frames should be dropped. Any format allowing for the implementation of multiple virtual storage area networks on a physical fibre channel network while also allowing the transmission of different frame types, forwarding fields, and/or time to live, etc. is referred to herein as an EISL format.
FIG. 2 indicates one example of an EISL frame. One of skill in the art will appreciate that the size, sequence and functionality of the fields within this EISL frame can vary from implementation to implementation. For example, the numbers of bits indicated for each field are different in alternative EISL frames.
The EISL frame 200 is bounded by start of frame delimiter (“SOF”) 205 an end of frame delimiter (“EOF”) 280. These delimiters enable an EISL-capable port to receive frames in a standard format at all times. If an EISL-capable port is not in EISL mode and receives frames in the EISL format, it accepts the frame according to some aspects of the invention. However, the port may not be able to send frames in EISL format.
In this embodiment, EISL header 260 includes VSAN field 240, which specifies the virtual storage area network number of payload 270. A VSAN allows for multiple logical or “virtual” storage area networks to be based upon a single physical storage area network. Accordingly, VSAN field 240 of EISL header 260 indicates the virtual storage area network to which this frame belongs.
MPLS label stack field 265 provides a common forwarding mechanism for both FC and Ethernet frames. Cyclic redundancy check (“CRC”) field 275 is used for error detection.
Exchange Link Parameter (“ELP”) protocol is an existing FC protocol that is used for communication with E ports. Similarly, Exchange Switch Capability (“ESC”) protocol is an existing FC protocol that is used for communication between E ports. These protocols can be used to exchange information regarding the capabilities of network devices.
According to some aspects of the invention, a new protocol, known herein as exchange peer protocol (“EPP”), is provided for communication between E ports. According to some preferred aspects of the invention, the EPP protocol is used after the ESC protocol. In such implementations, ESC protocol is used to determine if a network device is capable of performing EPP protocol exchange. The EPP protocol may be used, for example, to determine the port VSAN of a peer port of a network device or to determine whether the peer port is configurable for supporting EISL. When the peer port is enabled for EISL, the peer port is referred to as a “trunking port”.
FIG. 3 illustrates a simplified version of an EISL frame. Here, frame 300 includes EISL header 305, header 310 and payload 315. Header 310 may be, for example, an FC header or an Ethernet header. According to some aspects of the present invention, field 320 is a field of payload 315. In one example, field 320 is a service access point (“SAP”) field, which is a part of a fiber channel frame that is reserved for services that may be defined by a client. Field 320, according to some aspects of the invention, is an SAP field used for encoding EPP. According to some such aspects of the invention, field 320 is an EPP header and payload 315 includes an EPP payload, which will be described in more detail below.
FIG. 4 illustrates stack 400 according to some embodiments of the present invention. Stack 400 includes physical layer 405. For simplicity, all of the fiber channel layers are illustrated as a single layer, FC 2 layer 410. Switch Interlink Services (“SW_ILS”) layer 415 provides functionality for ELP 420 and ESC 425, according to the standard FC format. Layer 415 also provides a mechanism for vendors to add their own protocols, such as EPP_ILS 430 in this example. The EPP protocol frames exchanged according to SW_ILS service specification are called EPP_ILS frames.
However, not all ports will recognize SW_ILS. Accordingly, in other implementations of the present invention, other formats or services may be used to provide EPP services. For example, other implementations of the invention use Extended Link Services (ELS) format to provide EPP services.
FIG. 5 is a flow diagram that depicts an exchange of information between two E ports according to some aspects of the present invention. E port A may be, for example, port 160 of FIG. 1 and E port B may be, for example, E port 170 of FIG. 1. In other embodiments, one or both ports are F ports and may exchange frames using, for example, ELS format.
The information exchanged in section 505 of FIG. 5 represents the detection phase of EPP, wherein the EPP capability of an attached peer port is detected. Detection phase 505 is performed using ELP and ESC according to one implementation of this method.
Area 510 represents the SYNC phase of EPP, wherein configuration information of interest to the peer port is exchanged. According to some such embodiments, the configuration information is exchanged in time-length-value (“TLV”) format, which will be described below with reference to FIG. 5A.
At time 520, a link has been established between port A and port B. In step 525, port A sends an ELP request to port B. In this instance, port A has initiated the process. However, as will be explained in more detail below, the present invention includes a mechanism for dealing with situations in which both ports A and B have simultaneously initiated the process. ELP request 525 includes link-level parameters such as buffer-to-buffer credit (indicating how much data can be transmitted from one buffer to another before new credits are required).
In step 530, port B sends information to port A indicating an acceptance of the ELP request. In essence, step 525 involves the sending of port A's link-level parameters to port B and step 530 involves the sending of port B's link-level parameters to port A. In step 535, port A sends an acknowledgement to port B. At this time, port A knows port B's link configuration and port B knows port A's link configuration.
Then, in step 540, port A sends other information regarding the configuration of the network device that includes port A. In this step, port A indicates the services/protocols that port A can support. In some embodiments, the information will include a vendor string that indicates the particular vendor and model number of the network device and its capabilities. In one such embodiment, step 540 includes the transmission of services/protocols that port A can support in code/service pairs. Some codes may be standard FC codes which correspond with standard FC services (e.g., FSPF). However, one such code is a unique code that corresponds with EPP.
In step 545, port B sends an acceptance to port A and also sends information regarding the vendor and switch capabilities of the switch associated with port B. In this example, both port A and port B support EPP. Accordingly, detection phase 505 was successful and in steps 530 and 545, port B accepted port A's request and ESC information, respectively. However, port B could have rejected either of those requests. Alternatively, port B could have selected a different service if port B did not support EPP.
The combination of a request and an acceptance (or of a request and a rejection) will sometimes be referred to herein as an “exchange.” In the embodiment described with respect to FIG. 5, the exchanges are performed according to an SW_ILS format, as described above.
After determining that port B supports EPP and that port B could be configured to be a trunking port, port A sends an EPP_SYNC_ILS to port B in step 550 and EPP SYNC_ILS phase 510 begins. In this embodiment, the EPP_SYNC_ILS includes configuration information for use by Port B in configuring itself to be a trunking E port. However, in other embodiments, EPP may be used for port VSAN consistency checks without configuring port B as a trunking port.
FIG. 5A illustrates frame 585 in type-length-value (“TLV”) format, which is a preferred format for data exchanged between ports A and B during SYNC phase 510. Type field 590 encodes how value field 592 is to be interpreted. In other words, type field 590 indicates what kind of value will be encoded in value field 592. Length field 591 indicates the length of value field 592, e.g., in bytes. Value field 592 is a payload that encodes information to be interpreted as specified by type field 590.
TLV format is inherently quite flexible, because both the type and length of value field 592 can be varied. However, in other embodiments of the invention, fixed-length frames may be used for the same purpose.
FIG. 6 is table that indicates trunk mode negotiation according to some aspects of the present invention. If the sending trunk mode (here, port A) has an admin trunk mode setting of “OFF,” then the sending port will be treated as a non-trunking port. If the admin trunk mode of the sending port is “ON,” the sending port will be treated as a trunking port if the receiving port (here, port B) has an admin trunk mode of “ON” or “AUTO.” If the sending port has an admin trunk mode of “AUTO,” the receiving port must have an admin trunk mode of ON for the sending trunk mode to operate as a trunking port. Otherwise, the receiving port will operate as a normal port.
Referring again to FIG. 5, in step 555 port B sends an acknowledgement to port A. In step 560, port B sends its own configuration information, which may include trunking configuration information as described above, to port A.
In addition to exchanging admin trunk mode information, ports A and B may exchange VSAN list information during SYNC phase 510. The exchange of VSAN list information according to one such implementation will now be explained with reference to FIG. 7. In this example, ports A and B exchange bit maps that indicate which VSANs to allow. Here, port A sends bit map 705 to port B in which bits 1 through 5 have a value of “1,” indicating that VSANs 1 through 5 should be allowed. Port B, in turn, sends bit map 710 indicating that VSANs 4 through 8 should be allowed. In preferred implementations, the bit maps indicate the status of more (or less) than 8 VSANs and include a correspondingly greater (or smaller) number of bits.
Both ports A and B, or the network devices associated with the respective ports, then compute an intersection bit map that indicates the VSANs common to both ports. In this case, intersection bit map 715 indicates that VSANs 4 and 5 are both allowed. In some embodiments of the present invention, the intersection bit map is computed at the end of the EPP_SYNC phase. In other embodiments of the present invention, the intersection bit map is computed at other times. However, this process should occur prior to the beginning of the commit phase.
After the intersection bit map has been computed, the network devices associated with ports A and B each will store the intersection bit map in memory and only VSANs 4 and 5 will be permitted to send data frames along this data path. VSANs 4 and 5 are known as “operational VSANs” on the link between port A and port B.
According to some embodiments of the present invention, the configuration information exchanged during SYNC phase 510 includes port VSAN information. In some such aspects of the invention, port VSAN information is particularly important when the ports are functioning as non-trunking ports. If ports are functioning as trunking ports, the EISL header will contain a VSAN number indicating the VSAN to which the frame belongs.
However, according to some aspects of the invention, if the ports are not functioning as trunking ports, there will be no EISL header and consequently no VSAN number. If a port is not trunking, frames will be transmitted in the native FC format, not in EISL format. However, a VSAN will be implicitly associated with each frame. This VSAN is the port VSAN of the receiving port.
By default, every E port has a port VSAN number equal to 1. However, various port VSAN numbers may be assigned. If there is a mismatch between port VSAN numbers, various actions may take place according to various aspects of the present invention. According to some such aspects, a system administrator would be notified if, for example, a port having a port VSAN number of 1 sent a packet to a port having a port VSAN number of 2. According to other aspects of the invention, one or more of the ports would be brought down in the event of such a port VSAN mismatch.
At the end of step 560, port A knows the configuration of port B and port B knows the configuration of port A. In step 565, port A sends an acknowledgement to port B indicating that it has received port B's EPP_SYNC configuration information. Then, the EPP_SYNC phase of the process has concluded. On completion of SYNC phase 510, ports A and B will evaluate the configuration information that needs to be applied.
In the current example, ports A and B are configured to become trunking E ports. Accordingly, prior to EPP_Commit phase 515, port B is configured to be a trunking E port in programming step 568. According to some aspects of the invention, programming step 568 involves hardware programming necessary for supporting trunking mode operation and the preparation of EISL frames. In one instance, when the port is enabled for trunking mode, all frames are transmitted in EISL format.
When step 568 is complete, the EPP_Commit phase commences in step 570 by the sending of an EPP_Commit request from port B to port A. After port A receives the EPP_Commit request, port A performs its own programming operation in step 572, which is parallel to the programming step 568 of port B: according to some aspects of the invention, programming step 572 involves hardware programming necessary for supporting trunking mode operation and the preparation of EISL frames. In one instance, when the port is enabled for trunking mode, all frames are transmitted in EISL format. Then, in step 575, port A sends an SW_ACC to port B, indicating that port A has completed its programming step.
At some time after ports A and B have been transmitting data, an operator may decide to reconfigure some aspect of the ports. For example, the VSAN number may change on one or both of the ports and a new intersection bit map would need to be computed. If this is the case, the foregoing process need not go back through the ELP and ESC phases, but may proceed directly to the EPP_SYNC and EPP_Commit phases.
This process will be outlined with reference to FIG. 7A. In step 750, a network administrator changes the local admin trunk mode of port A from “AUTO” to “ON.” In step 755, the EPP SYNC process begins with a parallel to step 550 of FIG. 5, in which the new local admin trunk mode of port A is transmitted to port B. In step 760, port B sends an “ACK” to port A. In this example, the peer admin trunk mode (of port B) remains set to “AUTO.” Consequently, port B sends its peer admin trunk mode to port A in step 765, port A responds with an “ACK” in step 770 and both ports change their operational trunk mode to T (trunking) in step 775. The necessary EPP commit programming for trunking operation is performed in step 780.
FIG. 8 is a flow chart that depicts the process flow of an EPP method from the initiating port's perspective, according to one aspect of the present invention. The first step is step 805, the ready state. In step 810, an EPP_SYNC request is sent to the receiving port. In step 815, the initiating port requests an acceptance from the receiving port for the EPP_SYNC request. If the response is received within a predetermined time, the response is evaluated in step 820. If the response is not received within the predetermined time, the method proceeds to step 830 and the initiating port enters a retry waiting state.
Sometimes port B will send its own EPP_SYNC request during the time port A is awaiting a response to port A's EPP_SYNC request. This circumstance is known as a “collision.” In the event of a collision, in step 816 port A determines whether to accept the EPP_SYNC request from port B. If port A does accept the EPP_SYNC request from port B, the process continues to step 910 of FIG. 9, which is described below. If port A does not accept the EPP_SYNC request from port B, port A sends a rejection (e.g., an “SW_RJT”) to port B in step 817. Then, the process returns to step 815.
In step 835, it is determined whether the retry count or time is exceeded. If this retry count is exceeded, a failure will be reported and the system will return to a ready state. If the retry count is not exceeded, the EPP_SYNC request will be sent once again in step 810 and the process will proceed from step 810.
In step 820, if the response is determined to be acceptable, the method proceeds to step 825, where the system waits for an EPP_Commit state. If the response is determined not to be acceptable in step 820, an SW_RJT response is sent to the receiving port and the initiating port returns to the ready state of step 805.
If an EPP_Commit is received by the initiating port in step 825, then the process continues to step 840, wherein hardware programming is performed on the initiating port. In step 845, it is determined whether the hardware programming is completed. If not, the method proceeds to step 855, wherein the hardware programming step is reported and the system enters the retry condition of step 830. If the hardware programming is a success, the method proceeds to step 850 and an SW_ACC response for the EPP_Commit is transmitted to the receiving port.
The process then continues to step 860, wherein the initiating port waits for an acknowledgement from the receiving port. If the acknowledgement is not received within a predetermined time, then the process proceeds to step 855. If the acknowledgement is received within the predetermined time, the initiating port returns to the steady state of step 805.
FIG. 9 indicates the EPP process from the perspective of the receiving port. In step 905, the receiving port is in a ready state. In step 910, an SW_ACC is sent to the initiating port for the EPP_SYNC. In step 915, the receiving port waits for an acknowledgement for the SW_ACC response. If this response is not received within a predetermined time, there is a timeout and the receiving port returns to the ready state of step 905. If the acknowledgement is received within the predetermined, the method proceeds to step 920 and hardware programming is performed on the receiving port. In 925 it is determined whether the hardware programming is completed. If not, a failure report is made in step 930 and the receiving port returns to a ready state in step 905. If the hardware programming is done, the method proceeds to step 935 and an EPP_Commit is sent to the initiating port.
In step 940, the receiving port waits for an SW_ACC for the EPP_Commit that it has sent to the initiating port. If no such response is received within a predetermined time, the process proceeds to step 930 and a failure is reported. The receiving port then returns to the ready state of step 905. If a response is received during the predetermined time, then the method proceeds to step 945 and the response is evaluated. If the response is determined to be acceptable, a success is notified. In step 950, if the response is not determined to be acceptable, an error is reported and the system returns to the ready state of 905.
FIG. 10 indicates the components, values and sizes of EPP header fields according to some embodiments of the present invention. Other embodiments may have more or fewer fields. Moreover, the fields may have lengths other than those depicted in FIG. 10.
In one implementation of the present invention that uses SW_ILS, the command identifier field indicates values chosen from a range of vendor specific command identifiers. The command identifier may indicate, for example, an EPP request, an SW_RJT (reject) or an SW_ACC (accept). In one embodiment, the value of the command ID is 0X71000000. The revision field identifies the revision of the EPP service. For the first revision, the value is 1. The revision number should be changed every time there is a change in the EPP header.
As noted above, in some implementations EPP uses a two-phase mechanism to establish the operating environment. The first phase is the synchronizing phase (EPP_SYNC), where the configuration information on both sides is synchronized. The second phase is the commit phase (EPP_COMMIT), where the actual hardware programming is performed. The EPP command code field is used to identify whether the EPP request sequence is from the EPP_SYNC phase or the EPP_COMMIT phase.
The session field is used to identify a particular session on the side that initiated the EPP request sequence. In some cases of error or failure, EPP will retry its protocol exchange. The session number will be changed for each retry of the EPP operation. This feature helps identify stale sessions.
The worldwide name (WWN) indicates the WWN of the network device to which the port belongs. According to some aspects of the present invention, the WWN information is used for resolving “collisions” of simultaneous EPP_SYNC requests.
The payload length field is used to identify the total length of the payload, including the EPP header. The reserved field is reserved for future use.
There will be times when 2 ports will simultaneously send EPP requests to one another. Such “collisions” will be resolved based on the WWN of the network device with which the port is associated. The port within the network device having the lower WWN will send an SW_ACC to the other port. The port whose network device has the WWN will send SW_RJT to the other port, with a reason code indicating collision.
Generally, the techniques of the present invention may be implemented on software and/or hardware. For example, they can be implemented in an operating system kernel, in a separate user process, in a library package bound into network applications, on a specially constructed machine, or on a network interface card. In a specific embodiment of this invention, the technique of the present invention is implemented in software such as an operating system or in an application running on an operating system.
A software or software/hardware hybrid implementation of the techniques of this invention may be implemented on a general-purpose programmable machine selectively activated or reconfigured by a computer program stored in memory. Such a programmable machine may be a network device designed to handle network traffic, such as, for example, a router or a switch. Such network devices may have multiple network interfaces including frame relay and ISDN interfaces, for example. Specific examples of such network devices include routers and switches.
For example, the methods of this invention may be implemented in specially configured network devices such as the MDS 9000 family of switches manufactured by Cisco Systems, Inc. of San Jose, Calif. A generalized architecture for some such machines will appear from the description given below. In an alternative embodiment, the techniques of this invention may be implemented on a general-purpose network host machine such as a personal computer or workstation. Further, the invention may be at least partially implemented on a card (e.g., an interface card) for a network device or a general-purpose computing device.
Referring now to FIG. 11, a network device 1160 suitable for implementing the techniques of the present invention includes a master central processing unit (CPU) 1162, interfaces 1168, and a bus 1167 (e.g., a PCI bus). When acting under the control of appropriate software or firmware, the CPU 1162 may be responsible for implementing specific functions associated with the functions of a desired network device. For example, when configured as an intermediate router, the CPU 1162 may be responsible for analyzing packets, encapsulating packets, and forwarding packets for transmission to a set-top box. The CPU 1162 preferably accomplishes all these functions under the control of software including an operating system (e.g. Windows NT), and any appropriate applications software.
CPU 1162 may include one or more processors 1163 such as a processor from the Motorola family of microprocessors or the MIPS family of microprocessors. In an alternative embodiment, processor 1163 is specially designed hardware for controlling the operations of network device 1160. In a specific embodiment, a memory 1161 (such as non-volatile RAM and/or ROM) also forms part of CPU 1162. However, there are many different ways in which memory could be coupled to the system. Memory block 1161 may be used for a variety of purposes such as, for example, caching and/or storing data, programming instructions, etc.
The interfaces 1168 are typically provided as interface cards (sometimes referred to as “line cards”). Generally, they control the sending and receiving of data packets over the network and sometimes support other peripherals used with the network device 1160. Among the interfaces that may be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various very high-speed interfaces may be provided, such as fast Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces, HSSI interfaces, POS interfaces, FDDI interfaces, ASI interfaces, DHEI interfaces and the like. Generally, these interfaces may include ports appropriate for communication with the appropriate media. In some cases, they may also include an independent processor and, in some instances, volatile RAM. The independent processors may control such communications intensive tasks as packet switching, media control and management. By providing separate processors for the communications intensive tasks, these interfaces allow the master microprocessor 1162 to efficiently perform routing computations, network diagnostics, security functions, etc.
Although the system shown in FIG. 11 illustrates one specific network device of the present invention, it is by no means the only network device architecture on which the present invention can be implemented. For example, an architecture having a single processor that handles communications as well as routing computations, etc. is often used. Further, other types of interfaces and media could also be used with the network device.
Regardless of the network device's configuration, it may employ one or more memories or memory modules (such as, for example, memory block 1165) configured to store data, program instructions for the general-purpose network operations and/or other information relating to the functionality of the techniques described herein. The program instructions may control the operation of an operating system and/or one or more applications, for example.
Because such information and program instructions may be employed to implement the systems/methods described herein, the present invention relates to machine-readable media that include program instructions, state information, etc. for performing various operations described herein. Examples of machine-readable media include, but are not limited to, magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROM disks; magneto-optical media; and hardware devices that are specially configured to store and perform program instructions, such as read-only memory devices (ROM) and random access memory (RAM). The invention may also be embodied in a carrier wave traveling over an appropriate medium such as airwaves, optical lines, electric lines, etc. Examples of program instructions include both machine code, such as produced by a compiler, and files containing higher level code that may be executed by the computer using an interpreter.
While the invention has been particularly shown and described with reference to specific embodiments thereof, it will be understood by those skilled in the art that changes in the form and details of the disclosed embodiments may be made without departing from the spirit or scope of the invention. For instance, it will be appreciated that at least a portion of the functions described herein that are performed by a network device such as a router, a switch and/or selected components thereof, may be implemented in another device. For example, these functions can be performed by a host device (e.g., a personal computer or workstation). Such a host can be operated, for example, by a network administrator. Considering these and other variations, the scope of the invention should be determined with reference to the appended claims.
configuring the first port to transmit frames in Extended lnterswitch Link format.
2. The method of claim 1, wherein the determining step comprises exchanging information between the first port and the second port via Exchange Link Parameter protocol.
3. The method of claim 1, wherein the determining step comprises exchanging information between the first port and the second port via Exchange Switch Capability protocol.
4. The method of claim 1, wherein the exchanging step comprises exchanging frames in type-length-value format.
5. The method of claim 1, wherein the exchanging step comprises exchanging frames in a fixed frame length format.
6. The method of claim 1, wherein the configuration information comprises virtual storage area network information.
7. The method of claim 1, wherein the configuration information comprises trunk mode information.
8. The method of claim 1, wherein the configuration information is exchanged when the interconnected peer ports are being initialized.
9. The method of claim 1, wherein the configuration information is exchanged when the interconnected peer ports have already been initialized.
10. The method of claim 1, wherein the configuration step further comprises configuring the hardware of the interconnected peer ports according to the exchanged information.
11. The method of claim 1, wherein the configuration step further comprises configuring the software of the interconnected peer ports according to the exchanged information.
12. The method of claim 1, wherein the configuring step comprises configuring the hardware and/or software of the first port to enable transmission of frames in Extended lnterswitch Link format.
13. The method of claim 1, wherein the configuring step comprises informing the second port via Exchange Peer Parameters protocol that the configurations have been applied to the first port.
configuring the second expansion port as a trunking port.
15. The machine-readable medium of claim 14, wherein the determining step comprises exchanging information between the first expansion port and the second expansion port via Exchange Link Parameter protocol.
16. The machine-readable medium of claim 14, wherein the determining step comprises exchanging information between the first expansion port and the second expansion port via Exchange Switch Capability protocol.
17. The machine-readable medium of claim 14, wherein the configuring step comprises exchanging information between the first expansion port and the second expansion port via Exchange Peer Protocol.
means for configuring the first port to transmit frames in Extended lnterswitch Link format.
causing the port to be configured to transmit frames in Extended lnterswitch Link format.
20. The first network device of claim 19, wherein the determining step comprises exchanging information between the first network device and the port via Exchange Link Parameter protocol.
21. The first network device of claim 19, wherein the determining step comprises exchanging information between the first network device and the port via Exchange Switch Capability protocol.
22. The first network device of claim 19, wherein the configuring step comprises exchanging information between the first network device and the port via Exchange Peer Parameter protocol.
24. The apparatus of claim 18, wherein the configuration information further comprises trunk configuration information.
25. The apparatus of claim 24, wherein the trunk configuration information comprises administratively configured trunk mode information for trunk mode negotiation.
26. The apparatus of claim 24, wherein the trunk configuration information comprises virtual storage area network list information.
27. The apparatus of claim 24, wherein the trunk configuration information comprises port virtual storage area network information.
28. The apparatus, of claim 24, wherein the administratively configured trunk mode information comprises a setting selected from the group consisting of ON, OFF and AUTO.
means for configuring a port on the second network device to be a trunking port for transmitting frames in Extended lnterswitch Link format.
Allowed Claims for the U.S. Appl. No. 10/170,855.
Allowed Claims from related U.S. Appl. No. 7,206,288.
Allowed claims from related U.S. Appl. No. 7,406,034.
Armitage (Jan. 2000), "MPLS: The Magic Behind the Myths," IEEE Communications Magazine, pp. 124-131, (XP000908346).
Armitage, Grenville, "MPLS: The Magic Behind the Myths", Jan. 2000, IEEE Communications Magazine, pp. 124-131.
ATM Forum Committe, Chapter 10, "Flush Message Protocol Procedures and Frame Formats," Lan Emulation Over ATM Version 2 - LUNI Specification, AF-Lane-0084.000, Jul. 1997, pp. 111-115.
AU Office Action (May 23, 2007) from AU Patent Application No. 2003226093.
AU Office Action dated May 30, 2007, from AU Patent Application No. 2003226022.
Australian Examiner's Report dated Mar. 16, 2007, received Apr. 6, 2007, from Australian Patent Application No. 2002364204, Methods and Apparatus for Encapsulating a Frame for Transmission in a Storage Area Network 2pp. (ANDIP001.AU).
Brocade Communication Systems, Inc. (2002-2003), "Optimizing the performance and management of 2Gbit/sec SAN fabrics with ISL trunking," White Paper (webprint).
Brocade Communication Systems, Inc. (Jun. 2001), "Increasing Intelligence with the SAN Fabric," White Paper (webprint), XP002251362.
CA Office Action (Nov. 15, 2006) from CA Application No. 2,487,071.
Chinese Office Action, Chinese Patent Application No. 03807600.4, issued Sep. 8, 2006.
Cisco Systems, "Cisco MDS 9000 Family of Multilayer Directors and Fabric Switches", 1992-2003 Cisco Systems, Inc., pp. 1-4.
Cisco Systems, "Cisco SAN-OS", 1992-2003 Cisco Systems, Inc. pp. 1-7.
Cisco Systems, Inc. (1992-2003), "Cisco SAN-OS," pp. 1-7.
Cisco Systems, Inc. (1992-2004), "Cisco SAN-OS Reference Guide," pp. 1-13.
Claudio DeSanti, "Virtual Fabrics", T11/03-220v0, PowerPoint presentation, Apr. 2003, 11 pages.
CN Office Action (Oct. 19, 2007) from CN Application No. 2004800108260.
CN Office Action (Sep. 8, 2006) from CN Patent Application No. 03807600.4.
Cometto et al. (Apr. 1, 2002), "Methods and Apparatus for Fibre Channel Frame Delivery," U.S. Appl. No. 10/114,568.
Desanti (Apr. 2003), "Virtual Fabrics," T11/03-220v0, 11 pages (Microsoft PowerPoint presentation).
Desanti (Jul. 2004), "Virtual Fabrics N<SUB>-</SUB>Port Support," VF N<SUB>-</SUB>Support, T11/04-49v0, 13 pages.
Desanti (Oct. 2004), "Extended<SUB>-</SUB>Headers," VF N<SUB>-</SUB>Port Model T11/04-627v1, 1 page.
Desanti (Oct. 2004), "Virtual Fabrics Switch Support", VF Switch Support, T11/04-395v3, 15 pages.
Desanti et al. (May 2003), "Tagged Frame Specification," Tagged Frame Spec., T11/03-353v0, 4 pages.
DeSanti et al., "Tagged Frame Specification," Tagged Frame Spec., T11/03-353v0, May 2003, 4 pages.
DeSanti, Claudio, "Virtual Fabrics N<SUB>-</SUB>Port Support", VF N<SUB>-</SUB>Support, T11/04-494v2, Oct. 2004, 14 pp.
EP Examiner's Communication pursuant to Article 96(2) EPC dated Jun. 11, 2007, from European Patent Application No. 03 789 766.7 Methods and Devices for Exchanging Peer Parameters Between Network Devices, 5 pp. (ANDIP025.EP).
EP Office Action (Apr. 5, 2006) from EP Patent Application No. 03739 127.3-2416.
EP Office Action (Mar. 28, 2007) from EP Application No. 03746053.2-2416.
EP Office Action (Oct. 1, 2007) from EP Application No. 03746053.2-2416.
EP Office Action dated Oct. 18, 2005 from related EP Application No. 02799279.1-1525.
EP Search Report (Feb. 10, 2006) from EP Patent Application No. 03746053.2-2416.
EP Search Report (May 19, 2005) from EP Patent Application No. 03746062.3 -249.
EPO Search Report dated May 19, 2005, App.#03 746062.3-1249.
Fabric Shortest Path First (FSPF) Project 1508-D Switch Fabric -3 Rev. 6.5, (http://t11/org/index.htm webprint), pp. 117-140 (Oct. 31, 2003).
Final Office Action mailed Apr. 4, 2007 from related U.S. Appl. No. 10/114,394.
Final Office Action mailed Jan. 29, 2008 from related U.S. Appl. No. 10/034,160.
Guan et al., Inter-fabric FC Architecture, May 30, 2003, Brocade - The Intelligent Platform for Network Storage.
IEEE Std 802.3-2002, Chapter 43.5 Marker Protocol, Part 3: Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications, 2002, pp. 329-345.
International Search Report, mailed Jul. 12, 2004 from corresponding International Application No. PCT/US03/336432, 4 pp. including Notification of Transmittal.
K. White, IBM Corp., RFC 2925, Sep. 2000.
Kiiskilá, Marko, "Implementation of LAN Emulation of ATM in Linux," Tampereen Teknillinen Korkeakoulu, Oct. 1996, 57 pages.
Lee et al. (4/12002), "Label Switching in Fibre Channel Networks," U.S. Appl. No. 10/114,394.
Listanti et al. (Sep. 2000), "Architectural and Technological Issues for Future Optical Internet Networks," Optical Solutions for Next-Generation Internet Networks, IEEE Communication Magazine.
Listanti, et al., "Architectural and Technological Issues for Future Optical Internet Networks", IEEE Communication Magazine, Sep. 2000.
Mearian et al., "What's After Fibre Channel?", Computerworld, Online!, Oct. 15, 2001, XP002246206.
Mills (Jul. 1988), "Network Working Group Request for Comments 1059, Network Time Protocol (Version 1) Specification and Implementation," University of Delaware, pp. 1-50.
Molero et al. (Dec. 7-9, 2000), "On the effect of link failure in fibre channel sotrage area networks," Parallel Architectures, Algorithms and Networks 2000, I-Span 2000 Proceedings, Int'l Symposium.
Monia (Dec. 12, 2000), "iFCP- A Protocol for Internet Fibre Channel Storage Networking" Nishan Systems (webprint), P002246205.
Monia et al., "iFCP - A Protocol for Internet Fibre Channel Storage Networking" Feb. 2002.
NCITS (Feb. 19, 2003), "Fibre Channel Switch Fabric-3 (FC-SW-3)," working draft (XP002300830(A,B,C)).
NCITS (Jun. 26, 2001), "Fibre Channel Switch Fabric-2 (FC-SW-2)," working draft.
NCITS (Nov. 28, 2000), "Fibre Channel Generic Services-3 (FC-GS-3)," working draft.
Non-Final Office Action dated May 13, 2008 from related U.S. Appl. No. 10/979,886.
Non-Final Office Action mailed Feb. 6, 2008 from related U.S. Appl. No. 10/114,394.
Non-Final Office Action mailed Oct. 29, 2007 from related U.S. Appl. No. 11/027,252.
Notice of Allowance mailed Mar. 26, 2008 from related U.S. Appl. No. 10/114,568.
Notice of Allowance mailed May 1, 2008 from related U.S. Appl. No. 10/974,368.
Office Action dated Jan. 18, 2005 from related EP Patent Application No. 02799279.1-1525.
Office Action dated May 22, 2006; App.#10/170,855.
PCT International Preliminary Report on Patentability dated Dec. 29, 2004 from related PCT/US05/044726.
PCT International Search Report dated Oct. 25, 2006; App.#PCT/US05/37763 (CISCP396WO).
PCT International Search Report, PCT/US02/41072, mailed May 23, 2003, five pages.
PCT Written Opinion dated Oct. 25, 2006; App.#PCT/US05/37763 (CISCP396WO).
Rajagopal et al. (Jun. 30, 1999), "IP and ARP Over Fibre Channel," Request for Comments: 2625 (webprint), XP002246207.
Rangan (Sep. 4, 2001), "Re: FCIP/1FCP: Gurantee In-Order delivery for FC N/NL<SUB>-</SUB>ports," IP Storage-Mailing List Archive (http://www.pdl.cmu/edu/mailinglists/ips/mail/msg03069.html webprint).
Rosen et al. (Jan. 2001), "Multiprotocol Label Switching Architecture," Network Working Group, RFC 3031, XP002251364.
U.S. Office Action mailed Sep. 10, 2007, from U.S. Appl. No. 10/974,368.
U.S. Office Action, mailed Aug. 22, 2005, from U.S. Appl. No. 10/034,160.
U.S. Office Action, mailed Dec. 13, 2005, from U.S. Appl. No. 10/034,160.
U.S. Office Action, mailed Feb. 5, 2007, from U.S. Appl. No. 10/034,160.
U.S. Office Action, mailed May 31, 2006, from U.S. Appl. No. 10/034,160.
U.S. Office Action, mailed Sep. 26, 2006, from U.S. Appl. No. 10/034,160.
US Notice of Allowance and Notice of Allowability (Dec. 6, 2006) for U.S. Appl. No. 10/170,855.
US Office Action (Apr. 4, 2007) from U.S. Appl. No. 10/114,394.
US Office Action (Apr. 6, 2007) from U.S. Appl. No. 10/114,568.
US Office Action (Aug. 22, 2005) from U.S. Appl. No. 10/034,160.
US Office Action (Aug. 22, 2007) from U.S. Appl. No. 10/114, 394.
US Office Action (Jul. 30, 2007) from U.S. Appl. No. 10/034,160.
US Office Action (Mar. 28, 2007) from U.S. Appl. No. 10/791,143.
US Office Action (May 22, 2006) from U.S. Appl. No. 10/114,568.
US Office Action (May 31, 2006) from U.S. Appl. No. 10/034,160.
US Office Action (Oct. 17, 2006) from U.S. Appl. No. 10/114,394.
US Office Action (Oct. 23, 2006) from U.S. Appl. No. 10/114,568.
US Office Action (Oct. 29, 2007) from U.S. Appl. No. 11/027,252.
US Office Action (Sep. 20, 2007) from U.S. Appl. No. 10/114,568.
US Office Action (Sep. 5, 2007) from U.S. Appl. No. 10/609,442.
Valdevit (May 23, 2000), "Fabric Shortest Path First Version (FSPF) Rv. 0.2", Fabric Shortest Path (http://t11.org/index.htm webprint), XP002959525.
White (Sep. 2000), RFC 2925, IBM Corp.
White Paper, Link Aggregation According to IEEE Standard 802.3ad, Oct. 10, 2002, vol. 1.10, pp. 1-21.
WO International Search Report (Oct. 17, 2003) from PCT/US03/09328.
WO Search Report (Jul. 12, 2004) from International Patent Application No. PCT/US03/36452.
WO Search Report (Jul. 15, 2003) from International Patent Application No. PCT/US03/09442.
WO Search Report (Nov. 4, 2003) from International Patent Application No. PCT/US03/18765.
WO Search Report (Oct. 25, 2006) from International Patent Application No. PCT/US05/37763.
WO Search Report (Oct. 27, 2007) from International Patent Application No. PCT/US2004/020518.
WO Written Opinion (Oct. 25, 2006) from International Patent Application No. PCT/US05/37763.

References: Application No. 60
 Application No. 2003226093
 Application No. 2003226022
 Application No. 2002364204
 Application No. 2
 Application No. 03807600
 Application No. 2004800108260
 Application No. 03807600
 Application No. 03
 Application No. 03739
 Application No. 03746053
 Application No. 03746053
 Application No. 02799279
 Application No. 03746053
 Application No. 03746062
 Application No. 02799279