Converged fabric for FCoE

Network devices, systems, and methods, including program instructions are disclosed which provide a converged fabric for Fiber Channel over Ethernet (FCoE). A network device includes a Fiber Channel Controller (FCC), located outside of a lossless Ethernet network. The FCC has a processing resource coupled to a memory. The memory includes program instructions executed by the processing resource to terminate Fiber Channel (FC) Initialization Protocol (FIP) frames, generated to and by initiator and target devices, on the FCC.

BACKGROUND

Computing networks can include multiple network devices including network devices such as routers, switches, hubs, and computing devices such as servers, desktop PCs, laptops, workstations, mobile devices and peripheral devices, e.g., printers, facsimile devices, and scanners, networked together across wired and/or wireless local and/or wide area network (LANs/WANs).

In current blade server/switch environments networks are used having Fiber Channels (FCs) and/or Fiber Channels over Ethernet (FCoE) connections according to existing backbone5(BB5) standards and emerging backbone6(BB6) draft standards (currently under development), using compliant converged network adapters and control plane software. Currently FCoE standards involve a FC software stack and look up capability at multiple lossless Ethernet switches along the end to end path from initiator (e.g., host) to target device. This involves network3(L3) FC hops in communications between end points and rigid, hierarchical, network topologies. An example of an L3FC hop is an L3Destination ID (DID) look up, that may occur for example at an FDF, or another example is an FCF lookup, etc.

DETAILED DESCRIPTION

Embodiments of the present disclosure may include network device systems, and methods, including computer executable instructions (CEI), e.g., program instructions, which provide a converged fabric for Fiber Channel over Ethernet (FCoE). One network device example includes a Fiber Channel Controller (FCC), located outside of a lossless Ethernet network. The FCC has a processing resource coupled to a memory. The memory includes computer readable instructions, e.g., program instructions, executed by the processing resource to terminate Fiber Channel (FC) Initialization Protocol (FIP) frames generated to and by initiator and target devices. Embodiments allow a switch to forward frames over a lossless Ethernet fabric end to end without the use of a full FC software stack in the switch or the use of L3look ups, for example.

As used herein, the designators “N,” “M,” “Q”, “R”, “S”, and “W,” particularly with respect to reference numerals in the drawings, indicate that a number of the particular feature so designated can be included with examples of the present disclosure. The designators can represent the same or different numbers of the particular features.

FIG. 1illustrates an example of a number of virtual domains, e.g., virtual domain A (101-1) and virtual domain B (101-2), among various nodes as part of a FC switching fabric100between switches in a network according to an embodiment of the present invention. The virtual domains,101-1and101-2, can form part of a storage area network (SAN) switching fabric (SAN fabric)100. A SAN Fabric100is a collection of fiber channel (FC) switches and/or fiber channel over Ethernet (FCoE) forwarders (FCF) which share a fiber channel fabric mapping (FCF_MAP) and run fabric protocols over their “E” Ports (E_Ports) and/or virtual E_Ports (VE_Ports). E_Ports are a type of port for connecting switches. The FCF_MAP is a unique identifier for the SAN Fabric100. While switches and switching fabric are used in the example ofFIG. 1, embodiments are not limited to switches and switching fabric for network devices. Further, while only two virtual domains101-1and101-2are shown, embodiments are not limited to two virtual domains. A virtual component, e.g., port, or connection, e.g., link, is a logical connection versus an express physical connection.

FIG. 1shows the evolution of FCoE standards from existing Backbone5(BB5) to draft proposal Backbone6(BB6). In BB6, the nodes in a SAN Fabric can include a controlling FCF node102-1and an adjacent controlling FCF node (c/FCF)102-2each with their own principal domain. The nodes in the SAN Fabric can further include a number of FCoE Data Forwarder (FDFs), e.g.,104-1,104-2, . . . ,104-N, associated with an FCF node, e.g.,102-1and102-2, in the SAN Fabric100.

For native FC the FCF node is referred to as FCCF and the FDF node is referred to as FCDF. An FDF can be a top of rack (ToR) switch and an FCF an aggregation or end of row (EoR) switch or director class switch as it is called in FC parlance. FCFs are nodes within a network or in FC terms a so called fabric. FDFs are only visible within a virtual domain or distributed switch. Nodes are connected with links to form the SAN or “fabric”. Links connect ports which are of various types N, F, A, E, etc., and described more below. A fabric is a collection of switches which share a database and a so termed FC_MAP.

Each FDF,104-1,104-2, . . . . ,104-N, includes a number of “F” ports (F_Ports) and/or virtual F_Ports (VF_Ports), e.g.,106-1,106-2, . . . ,106-M, . . . ,106-W. F_Ports are a type of port terminating at a switch and can terminate the SAN fabric to “N” ports (N-Ports) and/or virtual N_Ports (VN_Ports) on a number of host devices, e.g, hosts112-1,112-2, . . . ,112-6, in a network. N_Ports are a type of port designation on a host device for connecting to switches.

As shown inFIG. 1, each FCF node101-1and101-2and each FDF node104-1and104-2is additionally provided with a number of “A” port (A_Ports) and/or virtual A_Ports (VA_Ports), e.g.,108-1,108-2, . . . ,108-M, . . . ,108-W. An A_Port is a new type of port, consisting of hardware, e.g., logic in the form of an application specific integrated circuit (ASIC), provided to the FCF nodes101-1and101-2and the FDF nodes104-1and104-2. A_Ports allow FCF to FDF and FDF to FDF node connectivity within a distributed switch and virtual domain.

In networking parlance, layer one (L1) is considered the physical layer, e.g. physical Ethernet connections. Layer two (L2) is considered the Media Access Controller (MAC) and Ethernet protocol. FC forwarding is considered as a layer three (L3). For example, by way of illustration and not by way of limitation, an L2Datacenter Bridge (DCB) path is a lossless Ethernet technology consisting of IEEE 802.1Qau, 802.1Qbz and 802.1Qbb standards. Further, by way of illustration and not by way of limitation, L3FC may include functionality for both control and forwarding plane including L3Destination ID (DID) based forwarding and possible hard zoning. Hard zoning is a function in L3, performed by executing program instructions to configure filters, e.g., Access Control Lists (ACLs), that are used in the forwarding path to block access by a host to particular target devices even if the host has a valid address for the target device. By contrast, soft zoning is a functionality that may be performed in L3, where a query by a host for address information to particular target devices may be blocked in the control plane, but the same host possessing the valid address information for those same particular target devices will not be blocked from access to those target devices in the forwarding, e.g, data plane.

FIG. 2illustrates an example of an FCoE forwarder (FCF) node. As shown in the embodiment ofFIG. 2, an FCF node, e.g. as part of a virtual switch, can include a processor220, e.g. processing resource coupled to a memory222. As shown in the embodiment ofFIG. 2, an FCF node can include access to a memory222associated with the FCF node202. The memory222may include a fiber channel control stack224, e.g., control plane software (computer executable instructions or program instructions). The memory222associated with an FCF node202may further include an Ethernet control stack226, and control and connection state228, including instructions executable to track and monitor a connection state of a given host, e.g., host112-1, . . . ,112-6inFIG. 1.

Embodiments of the present disclosure support FCoE VN_Port to VN_Port communications over a lossless Ethernet fabric without involving a full set of the FC stack, e.g., only using a subset of the FC stack, illustrated inFIG. 2and without involving and/or performing L3lookups, e.g., FC DID lookups along the path. As used herein, reference to FC control stack functions is intended to reference functions as defined in the BackBone5(BB5) standard and according to BackBone6(BB6) draft standards, currently under development. That is, embodiments may eliminate the FC software stack use from DCB switches in the end to end path, significantly streamlining switches. As described in more detail in connection withFIG. 4, an FCC supports a subset of the FC stack, e.g., there is no Fiber channel Shortest Path Forward (FSPF) protocol. By eliminating L3FC hops in communications between endpoints, the embodiments allow for flexible flat topology connectivity patterns, e.g., flat tree or Clos, in contrast to current rigid, hierarchical network topologies. Flexible flat topology connectivity patterns can remove potential choke points in the network by introducing support for load balancing.

It is noted that the BB6standard does include a VN_Port to VN_Port fabricless capability. That is, there is no FC control stack in the picture. However, this BB6capability involves upgrades to adapters at the network edge and is not compatible with BB5compliant adapters. Furthermore, the BB6VN_to VN approach operates without a fabric and puts all the burden of connection setups, packet filtering, etc., on adapters at end points. Hence, the BB6approach involves a new generation of more complex adapters and is also limited in the number of VN_Ports that can be supported. Embodiments described herein are different in that a network device, e.g., Fiber Channel Controller (FCC) maintains a subset of the FC Control stack and additional FCoE functions. Instructions are executed by the FCC to configure edge switches to enforce hard zoning at Ethernet L2layer, in a manner that is backward compatible with BB5adapters and is more scalable than the proposed VN_to_VN in BB6while maintaining the current operational practices of FC, e.g., the operational practice of using a fabric controller.

FIG. 3illustrates an example of a converged fabric for FCoE according to a native FC SAN fabric300having end to end connectivity between initiators312(e.g., hosts112-1, . . . ,112-6inFIG. 1) and target devices305(e.g., storage devices) according to an embodiment of the present invention. The embodiment ofFIG. 3illustrates a network that supports FCoE interfaces using BB5and BB6compliant converged network adapters (not shown). Frames go over a lossless Ethernet fabric321end to end between initiators, e.g., hosts (H)312and legacy targets (not shown). The example embodiment ofFIG. 3illustrates end to end connectivity between initiators312(hosts) and legacy targets using existing and/or legacy SAN300via a standard FCF302(202inFIGS. 2 and 102inFIG. 1) function as defined, for example, in the T11.03 BB5standards and/or proposed BB6draft standards.

The example embodiment ofFIG. 3illustrates a number of hosts312and a number of target devices, e.g.,305. The number of hosts312and the number of target device305connect to a lossless Ethernet layer321of a network device via L2DCB links328. The lossless Ethernet network321can include a number of edge nodes325-1, . . . ,325-N, e.g., Top of Rack (ToR) switches in the virtual domain, and a number of aggregator nodes327-1,327-2, . . . ,327-M, e.g., End of Rack (EoR) switches in the virtual domain. A host312to target device, e.g., storage device305, storage architecture can be edge to edge, e.g., host312to storage device (305-1) via ToR nodes (325-1, . . . ,325-N), and/or edge to core, e.g., aggregator nodes (327-1,327-2, . . . ,327-M) to storage device (305-2), connected by DCB links323. The lossless Ethernet fabric321operates at an L2level.

As shown in the example embodiment ofFIG. 3, a network device can be a switch having an FCoE Channel Forwarder (FCF)302and a Fiber Channel Controller (FCC)319. The FCC319includes a processing resource and a memory. The memory includes program instructions that are executed by the processing resource to terminate an FC Initialization Protocol (FIP), generated to and by the hosts312and target devices305, carried over the lossless Ethernet network321. The FCC319is not involved in forwarding of data traffic and only terminates control frames. By way of example, the FCC319is connected to one or more aggregators327-1,327-2, . . . ,327-M which may be interconnected using core switch links329. The FCF302can be connected to the native SAN fabric via FC links331. Other topologies are also possible and the embodiment ofFIG. 3is just one example.

The FCC319executes program instructions to terminate the FIP frames, generated to and by the hosts312and target devices305, outside of the lossless Ethernet network321such that all data traffic end to end, e.g., hosts312to target devices305, goes over the lossless Ethernet network321using only L2lookups. That is, there is no FC L3lookup or software stack in any switch within the Ethernet fabric321.

FIG. 4illustrates an example of a Fiber Channel Controller (FCC) network device402according to an embodiment of the present disclosure. As shown in the example embodiment ofFIG. 3, the FCC402is located outside of a lossless Ethernet network, e.g.,321inFIG. 3. In at least one embodiment the FCC402functionality may reside on a network attached server. The FCC402includes a processing resource, e.g., process420coupled to a memory422.

As shown in the example embodiment ofFIG. 4, the memory422includes only a subset of a fiber channel (FC) control stack functions along with an number of additional FCoE capabilities424, e.g., control plane software (computer executable instructions or program instructions). As used herein, reference to FC control stack functions is intended to reference functions as defined in the BackBone5(BB5) standard and according to BackBone6(BB6) draft standards, currently under development. The memory422associated with an FCC node402further includes control and connection state information428, including instructions executable to track and monitor a connection state of a given host, e.g., host112-1, . . . ,112-6inFIG. 1 or 312FIG. 3. The FCC does not involve an Ethernet control stack or forwarding ASIC and switch as does the FCF shown inFIG. 2.

Program instructions are stored in the memory422and executed by the processing resource420to terminate Fiber channel Initialization Protocol (FIP) frames, generated to and by initiator and target devices. The FC control stack subset424of the FCC402includes instructions that are executed to terminate a Fiber channel Login (FLOGI) procedure. The FC control stack subset and additional FCoE capabilities424include control plane instructions to configure edge switches, e.g.,325-1, . . . ,325-N, in the lossless Ethernet321ofFIG. 3. In at least one embodiment, the FC control stack subset and additional FCoE capabilities424include instructions that can be executed for soft zoning; FLOGI and destination ID (DID) assignment; FC Keep_Alive (FKA) generation and maintenance for logged in connections; and management triggers for configuration of hard zoning at edge switches: and the additional FCoE capabilities include a virtual local area network (VLAN) discovery capability; and a FIP discovery capability.

FIG. 5illustrates a flow diagram of one method embodiment according to the present disclosure. The example embodiment ofFIG. 5illustrates a BB5standards or BB6draft standard compatible execution of the FLOGI program instructions stored on the FCC outside of the lossless Ethernet layer. FCC only supports a subset of the full FC software stack, e.g., as that shown in the FCF ofFIG. 2.

As shown at block550in the example embodiment ofFIG. 5, program instructions are stored in a memory, e.g., memory of the FCC (422inFIG. 4), and executed by a processing resource of the FCC (420inFIG. 4), to terminate FC Initialization Protocol (FIP), generated to and by initiator312(hosts) and target305devices, outside of a lossless Ethernet fabric (321inFIG. 3). That is, as shown inFIG. 3, the program instructions are executed by a Fiber Channel Controller (FCC)319outside of the lossless Ethernet layer321. The program instructions of the FCC support only a sub set of functions defined in backbone5(BB5) standard and backbone6(BB6) draft standards (FC SW5/BB5/SW6/BB6) to terminate an FC Login (FLOGI) procedure. According to embodiments, the subset includes: virtual local area network (VLAN) discovery; FIP discovery; soft zoning; FLOGI and destination ID (DID) assignment; FC Keep_Alive (FKA) generation and maintenance for logged in connections; and management triggers for configuration of hard zoning at edge switches, e.g., TORs325-1, . . . ,325-N inFIG. 3.

At block555the program instructions are executed to assign a FC destination identifier (DID) associated with a storage area network (SAN) fabric, to a host VN_Port upon completion of an FC Login (FLOGI) procedure. The program instructions executed by FCC also assign an FC_Map together with the assigned DID as a concatenation of the FC_Map and the DID (FC_Map+DID) as the source MAC address to the VN_Port. The concatenation of FC_Map+DID is used by the VN_Ports as their source MAC addresses.

Through management configuration, not shown in the figures, an edge switch, e.g., ToRs325-1, . . . ,325-N, are configured to perform a MAC destination address re-write for frames with Ethertype set to FCoE payload. The program instructions are executed to cause the edge switch to re-write the MAC destination address to FC_MAP+DID. DID is carried in the frame as part of its FC frame encapsulation. The frames generated by BB5compatible adapters carry the FCC MAC as their destination address. In this example embodiment, the FCC MAC address is replaced with the “FC_MAP+DID” MAC address to ensure inter-operability with BB5compliant converged network adapters (CNAs). A converged network adapter (CAN) is an adapter that can carry both native Ethernet as well as Ethernet encapsulated FC frames.

In a return path, the program instructions are executed to configure edge switches to replace the source MAC address (which is the assigned MAC address) with a MAC address of the FCC.

FIG. 6illustrates flow diagram of another method embodiment according to the present disclosure. The example embodiment ofFIG. 6illustrates a hard zoning functionality that can interface to both BB5and BB6compliant converged network adapters (CNAs) through the execution of the program instructions stored on the FCC outside of the lossless Ethernet network.

As shown at block655in the example embodiment ofFIG. 6, program instructions are stored in a memory, e.g., memory422of the FCC402inFIG. 4), and executed by a processing resource of a network device, e.g., switch, to terminate FC Initialization Protocol (FIP), generated to and by initiator312(hosts) and target305devices, outside of a lossless Ethernet network. Again, the program instructions of the FCC support only a sub set of functions defined in FC (SW5/BB5/SW6/BB6) standards, and additional FCoE capabilities, to terminate an FC Login (FLOGI) procedure, as well as virtual local area network (VLAN) discovery; FIP discovery; soft zoning; destination ID (DID) assignment; FC Keep_Alive (FKA) generation and maintenance; and management trigger for hard zoning.

At block655the program instructions are executed to assign a FC Destination ID (DID) address associated with a storage area network (SAN) fabric, to a host VN_Port upon completion of an FC Login (FLOGI) procedure. The program instructions are executed to cause a CNA to assign and use a (FC_Map+assigned DID) as the source MAC address for the VN_Port.

In this example embodiment, the BB6CNAs used in VN_to_VN port mode can send and receive frames to and from other VN_Ports. Hence, forward path destination address re-writes and return path rewrites are unnecessary if adapters are configured accordingly.

However, as shown at block660, in this example embodiment the program instructions are executed by the FCC319to trigger, through a management plane, hard zoning on an edge switch. In this embodiment, FCC program instructions, stored in a memory on a Fiber Channel Controller (FCC) as control plane instructions outside of a lossless Ethernet network, are executed such that an edge switch, e.g.,325-1,325-2, . . . ,325-N is configured through the management plane, e.g., triggered by FCC, with one of two types of access control lists (ACLs). One type of ACL is used to ensure inter-operability with BB5compliant converged network adapters A second type of ACL is used for BB6compliant adapters.

For both types of adapters in a network, the FCC program instructions are executed such that an edge switch, e.g.,325-1,325-2, . . . ,325-N is configured through the management plane to perform frame filtering based on source and destination MAC addresses. These MAC addresses correspond to the FC DIDs which are used by FC for equivalent hard zoning. Both source IDs (SIDs) and destination IDs (DIDs) may be assigned, but may be collectively referred to as VN_Port IDs.

However, in the example embodiment ofFIG. 6, the program instructions are executed to trigger the edge switch to use a first type of the two ACL types to filter based on the source MAC address and the encapsulated DID in order to ensure inter-operability with BB5compliant CNAs. Whereas for BB6compliant CNAs, used in VN_to_VN port mode, the program instructions are executed to trigger the edge switch to use a second type of the two ACL types which is based on the least significant three bytes of the source and destination MAC address.

Thus, in the embodiments, for communications within the lossless Ethernet fabric, frame forwarding is entirely under control of the lossless Ethernet fabric (L2), e.g.,321inFIG. 3, end to end without L3hops. Hard zoning as described above is performed at the edge switches, e.g.,325-1,325-2, . . . ,325-N, and routing frames through the fabric, including multiple path, load balancing and reroutes on failures, etc., does not involve FSPF and are under the control of Ethernet protocols.

FIG. 7illustrates a comparison of L3FC hops between initiator and target devices according to current BackBone6(BB6) distributed FCF networks, in various scenarios, to embodiments according to the present disclosure. In the example ofFIG. 7, the line between initiators and targets (I/Ts) is an L2DCB path. The bars represent L3DID lookups and hard zoning.

The first scenario inFIG. 7(710) illustrates initiators and targets (I/Ts) on the same FDF according to current BB6distributed FCF networks. in this scenario a frame traveling between the initiator and target would arrive on an F_Port of an FDF which would perform an L3lookup before traveling through another F_Port to the target.

The second scenario inFIG. 7(720) illustrates I/Ts in the same virtual domain according to current BB6distributed FCF networks. In this scenario a frame traveling between the initiator and target would travel arrive through an F_Port to an FDF which would perform an L3lookup before traveling through an A_Port (e.g., A_Port, A_Link) to another FDF which would perform another L3lookup, and then through yet another F_Port to the target.

The third scenario inFIG. 7(730) illustrates I/Ts in the different virtual domains according to current BB6distributed FCF networks. In this scenario a frame traveling between the initiator and target would arrive through an F_Port to an FDF which would perform an L3lookup before traveling through an A_Port (e.g., A_Port, A_Link) to an FCF (principal and/or secondary (p/s)) which would perform another L3lookup, then on through an E_Port (e.g., E_Port, E_Link) to another FCF which would perform an L3lookup, back on through another A_Port (e.g., A_Port, A_Link) to another FDF which would perform another L3lookup, and then through yet another F_Port to the target.

The fourth scenario inFIG. 7(740) illustrates, in contrast, frame forwarding according to embodiments of the present disclosure. As shown in the fourth scenario740, using the embodiments described herein, a frame can be forwarded through a lossless Ethernet, e.g.,321inFIG. 3, entirely under the control of the lossless Ethernet fabric, end to end (e.g., initiator to target). That is, according to the embodiments described herein, frames can pass from edge switches (e.g., ToRs325-1,325-2, . . . ,325-N inFIG. 3) to aggregators (e.g.,327-1, . . . ,327-M inFIG. 3), and on to targets and/or back through edge switches to targets without using FC FSPF, L3lookups or FC L3hard zoning.

Hence, the embodiments described herein provide for complete separation of control and forwarding. That is, a subset of the FC control stack, e.g., control plane software, is moved entirely out of the lossless Ethernet network. The forwarding is performed entirely by the L2, lossless Ethernet network using shortest path bridging (SPB), TRILL (IETF standard), or other mechanisms. The converged fabric for FCoE of the present disclosure further does not require new ASIC capabilities to forward FC frames. Hard zoning, etc. is applied at the edge switches using ACLs. No FC stack is needed in any Ethernet switch. Since a full FC stack is no longer needed in each switch, switch software design may be significantly simplified.

The embodiments are compatible with both BB5and BB6adapters. This allows for fabric convergence within a datacenter as opposed to previous “overlay” approaches in current BB5and BB6standards. This additionally allows for flexible connectivity and topology within datacenter fabrics without current restrictions of BB5and BB6standards. Since FC/FCoE performance is sensitive to latency, the end to end latency can be reduced by removing L3hops in the topology. Further, the embodiments enable load balancing at L2to make more effective use of available bandwidth in the network.

The term “a number of” is meant to be understood as including at least one but not limited to one.