N-port ID virtualization (NPIV) proxy module, NPIV proxy switching system and methods

Embodiments of an N-Port ID virtualization (NPIV) proxy module, NPIV proxy switching system, and methods are generally described herein. Other embodiments may be described and claimed. In some embodiments, login requests are distributed over a plurality of available N-ports to allow servers to be functionally coupled to F-ports of a plurality of fiber-channel (FC) switches. Fiber-channel identifiers (FCIDs) are assigned to the servers in response to the logon requests to provide single end-host operations for each of the servers.

TECHNICAL FIELD

Some embodiments pertain to fiber-channel (FC) networks. Some embodiments pertain to server virtualization in storage area networks (SANs).

BACKGROUND

In data centers, servers may connect to various target devices, such as small-computer system interface (SCSI) discs, through FC switches. Conventionally, each server may connect to an FC switch using a single port. These ports are referred to as N-ports from the server side and F-ports from the FC switch side. The FC switches conventionally allocate a single FC address, sometimes referred to as a fiber-channel identifier (FCID) or N-port ID, to each server.

There are systematic issues with the connection of servers and storage devices through a FC switch, including interoperability issues that arise with the connection of FC switches from different system vendors. Other issues include the increasing difficulty associated with managing the domain ID space in a large FC data center network because each FC switch may need a domain ID.

N-port ID Virtualization (NPIV) is FC facility allowing multiple N-port IDs to share a single physical N-port. If a FC switch supports NPIV, the switch can allocate multiple FCIDs on the same physical port. Some data centers may implement server virtualization to allow servers to acquire multiple N-port IDs over a single N-port. One N-port may be assigned for each distinguishable logical entity or virtual machine (e.g., a virtual server).

Thus, there are general needs for systems and methods for attachment of multiple physical and/or logical endpoints to a FC network. There are also general needs for systems and methods that utilize NPIV for attachment of multiple physical and/or logical endpoints to a FC network.

DETAILED DESCRIPTION

The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them. Other embodiments may incorporate structural, logical, electrical, process, and other changes. Examples merely typify possible variations. Individual components and functions are optional unless explicitly required, and the sequence of operations may vary. Portions and features of some embodiments may be included in, or substituted for those of other embodiments. Embodiments of the invention set forth in the claims encompass all available equivalents of those claims. Embodiments of the invention may be referred to herein, individually or collectively, by the term “invention” merely for convenience and without intending to limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed.

FIG. 1is a functional diagram of an NPIV proxy switching system operating within a FC network in accordance with some embodiments of the present invention. FC network100may include a plurality of servers102, NPIV proxy switching system104, FC switches106, FC networks108and target devices118. Servers102may comprise a storage area network (SAN), and may utilize one or more of target devices118to store data. In some embodiments, servers102may comprise a virtual SAN (VSAN), although the scope of the invention is not limited in this respect.

NPIV proxy switching system104may include server interfaces123to interface with physical or logical host-bus adaptors (HBAs) of servers102, and external interfaces124to interface with FC switches106. NPIV proxy switching system104includes NPIV proxy module122to perform an NPIV proxy process to allow NPIV proxy switching system104to act as a single-end host for servers102. NPIV proxy switching system104may also include other modules126to support the operations of NPIV proxy switching system104. External interfaces124may be associated with one or more N-ports115to couple with F-ports117of FC switches106. These embodiments are described in more detail below.

In some embodiments, NPIV proxy module122may distribute login requests103over available N-ports115to allow servers102to be functionally coupled to F-ports117. In response to login requests103, NPIV proxy module122may assign fiber-channel identifiers (FCIDs) to servers102which may allow NPIV proxy module122to operate as a single end-host for each of the servers102. In some embodiments, the operations performed by NPIV proxy module122may be initiated by one or more state machines, discussed in more detail below. In some of these embodiments, a physical or logical HBA of one of servers102may issue an F-port login (FLOGI) request, which may be intercepted by NPIV Proxy module122. NPIV proxy module122may select one of N-ports115to associate the server with. NPIV Proxy module122may then issue a proxy FLOGI to the selected N-port. The FC switch106associated with the selected N-port may return an LS_ACC to NPIV proxy module122which may relay this to the requesting HBA. Accordingly, servers102may communicate using their assigned FCIDs with target devices118through one of a plurality of FC networks108. In these embodiments, servers102use login requests103to request an FCID to allow servers102to communicate with target devices118. In these embodiments, the FCIDs are provided by NPIV proxy module122. This is unlike some conventional server systems in which severs request FCIDs directly from the FC switches. Although some embodiments of the present invention are applicable to server systems and networks that implement NPIV, the scope of the invention is not limited in this respect. Embodiments of the present invention may also be applicable to server systems and networks, including servers that do not implement NPIV.

In some embodiments, NPIV proxy module122may be configured to uniformly distribute login requests103over N-ports115to achieve a substantially uniform distribution. The substantially uniform distribution may consider login requests previously distributed to the N-ports115. In these embodiments, the number of existing requests may be considered by NPIV proxy module122when distributing a login request103received from one of servers102.

In some embodiments, each N-port115may be associated with one of external FC interfaces124, and NPIV proxy module122may distribute login requests103over available ones of N-ports115by either weighting the available N-ports115based on a number of login requests previously distributed to N-ports115, or by sorting external interfaces124based on the number of login requests previously distributed the N-ports115. In these embodiments, NPIV proxy module122may forward login requests103to a selected external interface based on either the weighting or sorting. In these embodiments, each of N-ports115may be considered a port channel, although the scope of the invention is not limited in this respect.

In some embodiments, NPIV proxy switching system104may operate in an FC end-host mode. In the FC end-host mode, NPIV proxy module122may request multiple FCIDs for NPIV proxy switching system104to allow NPIV proxy switching system104to operate a single end-host for each of servers102. In these embodiments, each FC switch106may see NPIV proxy switching system104as a single end host requesting multiple FCIDs for itself similar to an HBA implementing NPIV functionality.

In some embodiments, one or more of server interfaces123may be configured to intercept some or all login requests103from servers102and to send login requests103to NPIV proxy module122. In some embodiments, login requests103may comprise FLOGI requests that request a FCID associated with F-port117of one of FC switches106, although the scope of the invention is not limited in this respect. In these embodiments, login requests103may be received from HBAs112of servers102. In some embodiments, login requests103may comprise frames configured in accordance with a particular FC standard, although the scope of the invention is not limited in this respect. Servers102and/or their associated HBAs112may be referred to as hosts.

In some embodiments, one or more of HBAs112may implement NPIV functionality by sending more than one login request103to server interfaces123to request more than one FCID. Although some HBAs112may implement NPIV functionality, the scope of the invention is not limited in this respect as there is no requirement that HBAs112implement NPIV functionality.

In some embodiments, NPIV proxy switching system104may operate in either the FC end-host mode discussed above or in FC switch mode. When operating in FC switch mode, the NPIV proxy operations of NPIV proxy module122may be disabled. For example, when NPIV proxy switching system104operates in FC switch mode, the state machines of NPIV proxy module122may configure hardware elements of NPIV proxy switching system104to functionally connect servers102to FC switches106by assigning FCIDs directly to the HBAs112. In FC switch mode, servers102and FC switches106may operate as if they were directly connected with a cable or direct communication link, although the scope of the invention is not limited in this respect. When NPIV proxy switching system104operates in FC switch mode, NPIV proxy module122may refrain from distributing login requests103received from servers102over the N-ports115and may further refrain from operating as the single end-host for each of the servers102. When NPIV proxy switching system104operates in FC switch mode, NPIV proxy switching system104may operate as an FC switch.

In some embodiments, the state machines of NPIV proxy module122may comprise a FLOGI finite state machine (FSM) which may be started for each login request, and an external interface state machine which may be started for each external interface124, although the scope of the invention is not limited in this respect. In some embodiments, the state for each login session may be handled by one of the FSMs. External events, including FLOGI requests, logout (LOGO) requests, interface up/down events, VSAN added, and deleted events, etc., may feed into these state machines to cause actions and state transitions. These embodiments are discussed in more detail below.

In accordance with some embodiments, NPIV proxy module122may capture login/logout associated packets received from servers102through server interfaces123, and/or through external interfaces124which may be enabled for N-Port or NPIV functionality. Examples of login/logout associated packets include login requests103(e.g., FLOGIs), logout requests, responses for login and logout requests (e.g., LS_ACC and LS_RJT responses), although the scope of the invention is not limited in this respect. NPIV proxy module122may perform intelligent load balancing of the login sessions associated with login requests103over external interfaces124to uniformly distribute login requests103over N-ports115. These embodiments are described in more detail below.

In some embodiments, NPIV proxy module122may provide retry functionality for sending a login request103on a different external interface124when a login request on one of external interfaces124fails, although the scope of the invention is not limited in this respect.

In accordance with some embodiments, NPIV proxy module122may provide high-availability (HA) functionality via process restart and switchover, although the scope of the invention is not limited in this respect.

In some embodiments, NPIV proxy module122may communicate and process packets in accordance with one or more FC standards including, for example, the FC-DA-2, FC-FS-2, and/or FC-LS-1.2 standards, although the scope of the invention is not limited in this respect. In some embodiments, NPIV proxy module122may maintain the same external behavior for each of HBAs112and FC switches106in failure scenarios so that HBAs112and FC switch106believe they are directly connected to each other.

In some embodiments, FC switches106may run Switch Fabric Internal Link Services (SW-ILS) to get a domain ID associated with one of FC networks108. HBAs112may be assigned to one of FC networks108allowing each FC network108to serve as an alternate path to target devices118. In some embodiments, some of target devices118may comprise discs to store/backup data for use by servers102.

FIG. 2is a block diagram illustrating functional elements of an NPIV proxy module in accordance with some embodiments of the present invention. NPIV proxy module122may correspond to NPIV proxy module122(FIG. 1), although other configurations may also be suitable. As illustrated inFIG. 2, NPIV proxy module122may include a plurality of modules including event dispatcher202, MTS event handler204, FLOGI/LOGO packet handler206, timer handler208, and system event handler210. As discussed above, NPIV proxy module122may also comprise several state machines. As illustrated inFIG. 2, NPIV proxy module122may comprise FLOGI/LOGO FSM212which may be started for each login request103, and external interface FSM213which may be started for each external interface124(FIG. 1) associated with a login request.

Event dispatcher202may communicate with server interfaces123(FIG. 1) and external interfaces124(FIG. 1) to dispatch packets to one of the other modules. FLOGI/LOGO packet handler206may handle packets comprising login requests103(FIG. 1) and logout requests received from servers102(FIG. 1). MTS event handler204may handle message and transactional services (MTS) events. Timer handler208may be used for scheduling queues and timeout for state machines212&213. System event handler210may handle events associated with external interfaces124(FIG. 1), and may generate proxy logouts when server interfaces124(FIG. 1) and/or VSANs go up and down.

NPIV proxy module122may also include running configuration persistent storage system (PSS) element214to store configuration information for state machines212&213, and runtime data PSS element216to store runtime information for state machines212&213. Elements214and216may also maintain information related to the dynamic FLOGI/LOGO state for each login session from servers102(FIG. 1) and information related to login requests103(FIG. 1) received from servers102(FIG. 1) queued per external interfaces124and/or the VSAN.

Although NPIV proxy switching system104(FIG. 1) and NPIV proxy module122are illustrated as having several separate functional elements (e.g. modules and/or interfaces), one or more of these functional elements may be combined and may be implemented by combinations of software-configured elements, such as processing elements including digital signal processors (DSPs), and/or other hardware elements. For example, some functional elements may comprise one or more microprocessors, DSPs, application specific integrated circuits (ASICs), and combinations of various hardware and logic circuitry for performing at least the functions described herein. In some embodiments, the functional elements of NPIV proxy switching system104(FIG. 1) may refer to one or more processes operating on one or more processing elements. In some embodiments, the modules of NPIV proxy module122may comprise entirely software, although the scope of the invention is not limited in this respect. In some embodiments, NPIV proxy module122may comprise one or more software-implemented processes discussed in more detail below. These software processes may be implemented in DC-OS and may interact with a CPU packet I/O handler in the kernel (PF-FC) and other user space processes, although the scope of the invention is not limited in this respect. In some embodiments, NPIV proxy switching system104may be implemented as part of a top-of-rack switch (TORS) in a data center, although the scope of the invention is not limited in this respect.

FIG. 3is a system-flow diagram illustrating the operations of an NPIV proxy switching system in accordance with some embodiments of the present invention. System flow diagram300is a high level architecture diagram illustrating packet flow and inter-process interaction within NPIV-proxy module122(FIG. 1).

InFIG. 3, paths301illustrate FLOGI/LOGO packet flow and paths303illustrate inter-process interaction via MTS event handler204(FIG. 2). Paths305indicate hardware programming operations. InFIG. 3, the operations performed by NPIV proxy module122are illustrated within region322. Operations302may comprise operations performed by event dispatcher202(FIG. 2), MTS event handler204(FIG. 2), packet handler206(FIG. 2), timer handler208(FIG. 2) and system event handler210(FIG. 2). Operations312may comprise operations performed states machines212(FIG. 2). Operations314may comprise operations for hardware programming. In some embodiments, the operations within region322may be platform independent, although the scope of the invention is not limited in this respect.

In accordance with embodiments, NPIV proxy module122(FIG. 1) may register for login packets with PF-FC I/O packet handler331. Packet handler331may be a CPU handler and may reside in kernel space. NPIV proxy module122may program hardware with CPU redirect module337for login associated packets using an applications programming interface (API). The API also may be configured to provide parameters for configuring FLOGI and/or FDISC processing from the servers.

CLI/SNMP module332may interface with NPIV proxy module122using a CMI interface built on top of MTS. The CMI interface may provide configuration information as discussed below.

In some embodiments, NPIV proxy module122may query FC port manager333for list of server and external interfaces operating in N-port mode when the conditional service starts and may expect MTS notifications for changes in the interface state due to configuration changes or external events.

FC RIB manager334may populate the mapping of an FCID to server interfaces123on which a corresponding FLOGI was received. This mapping may be used to forward an FC data packet coming in on one of external interfaces124to the correct one of server interfaces123(FIG. 1).

FWD manager335may populate the mapping of server interfaces123to one of external interfaces124. This mapping may be used to forward an FC data packet coming in on one of server interfaces123to the correct one of external interfaces124. The mappings may be stored in one or more tables. This may be referred to server interface pinning and may be coordinated by pinning manger336.

System manager338may monitor the health of the NPIV proxy process and may assist in the HA of NPIV proxy module122. Since NPIV proxy module122may comprise FSM122, interaction with system manager338may be transparent.

VSAN manager339may be used by NPIV proxy module122to derive VSAN membership on server interfaces123and external interfaces124via queries and notifications.

FIG. 4illustrates data structures with the NPIV proxy module in accordance with some embodiments of the present invention. As illustrated inFIG. 4, the data structures of NPIV proxy module122(FIG. 1) maybe organized in a hierarchical manner, although the scope of the invention is not limited in this respect. Data structures400may include NPIV globals table402, universal connectivity domain (UCD) table404, server interface table406, external interface table408, UCD VSAN table410, UCD VSAN table412, FLOGI/LOGI state table414, and list of pointers table416. These data structures may be encapsulated in FLOGI/LOGO FSM212(FIG. 2) and external interface FSM213(FIG. 2). In some embodiments, external interface FSM213(FIG. 2) may reside in external interface table408.

NPIV globals table402may holds global variables including global timer values and may include pointers to UCD table404, which may be indexed by a UCD-ID, server interface table406that FLOGI/LOGO requests may be received from, and external interface table408which may comprise a table of external interfaces that connect to FC switches106. UCD VSAN table410may comprise a table of allowed VSANs for server interfaces123(FIG. 1) and UCD VSAN table412may comprise a table of allowed VSANs for external interfaces124(FIG. 1). Each VSAN under an external or server interface may store the FLOGI/LOGO state for logins for the associated VSAN. Each VSAN under a server interface store a table of all FLOGIs received from a server interface. Similarly each VSAN under an external interface may store a list of FLOGIs sent out on an associated external interface and VSAN. The list may be used to send proxy logouts to a server102(FIG. 1) on an external interface124, or to send a VSAN down event.

In some embodiments, global data table402may maintain pointers to UCD VSAN tables410&412, server interface table406, and external interface table408. Server interface table406may comprise a tree of server interface objects. Each server interface123(FIG. 1) may point to a hash table of the UCD VSAN tables410&412.

VSAN table410may comprise a unique identifier (e.g., UCD, VSAN) for which a server VSAN object that is allocated. This structure may point to entries of FLOGI/LOGO state table414that has the FSM for each login session.

External Interface table408may maintains information for external interface124(FIG. 1) and may run a state machine for external interface124(FIG. 1) for handling events such as link-up, link-down, interface-deleted and the initial FLOGI generated by the NPIV Proxy for interface bring up, although the scope of the invention is not limited in this respect.

UCD VSAN table412may comprise an external VSAN object to maintain a per VSAN state on external interface124(FIG. 1). Table412may include a queue for all FLOGIs queued on each external interface and VSAN. This queue may enable the serialization of FLOGI/LOGO requests from NPIV-proxy module122(FIG. 1) to one of FC switches106(FIG. 1). This object also maintains a linked list of successful FLOGIs. This list is used to send proxy logouts on an interface down event.

List of pointers table416may comprise an FLOGI/LOGO object. This structure may run FSM212for every FLOGI/LOGO request. In some embodiments, some runtime information may be stored in table416to drop timed-out packets, to send proxy logouts etc, although the scope of the invention is not limited in this respect.

Referring back toFIG. 2, after one or external interface124is selected, the FLOGI request may be queued on a per VSAN queue on the selected external interface. If the queued entry is the only entry in the queue, then the FLOGI/FDISC is sent to the external interface. If there are other entries in the queue, the queued FLOGI request may wait until it gets to the top of the queue.

The FLOGI request at the top of the queue may be waiting for a response from one of FC switches106. If this response is an LS_ACC, then the received FCID may be used to update the hardware. There are two pieces of information that are updated in hardware. The first piece of information is the mapping of the received FCID to the server interface so that traffic coming in from the FC fabric is routed correctly to the server. This information is programmed in FC RIB334(FIG. 3). The second piece of information that is updated in hardware is the mapping of the server interface to the external interface so that all traffic coming in from the server is routed over the correct external interface. This information also called ‘server interface pinning’ information may be programmed using API's provided by the platform independent portion of the forwarding manager. Once hardware programming is complete, the LS_ACC is forwarded back to the server.

If the response is LS_RJT and the error code is “Unable to perform command request due to insufficient resources”, the NPIV Proxy Manager will re-try the FLOGI on a different interface. The decision to re-try FLOGI on a different interface may be a function of time taken for the LS_RJT response to come back and a configurable retry timer. For some other error codes, the LS_RJT may be forwarded back to the server interface. If there is no response from the FC switch the FLOGI may be re-tried on a different external interface again depending on a configurable timer. In some embodiments, NPIV proxy module122may generate proxy LOGOs when an external interface or a server interface goes down.

In some embodiments, the following algorithm may be used to pick an external interface124used to forward a FLOGI request received from server interfaces123. If the external interface is explicitly configured by the user and if this interface and VSAN are up, pick this interface. If the external interface is not UP or the VSAN does not exist, an error may be generated and the FLOGI request may be dropped. If no external interface is explicitly configured but an external interface was already chosen but the FLOGI request did not succeed, the same external interface for the FLOGI request may be chosen. If no external interface was already chosen, a list the external interfaces that are in this UCD and VSAN may be used to chose. Note that the external interfaces124that are not up are eliminated. External interfaces124may be eliminated which have encountered “out of resources” and “waiting for response timed out” failures. If the external interface124is a port channel, the port channel may be given a weight equal to the number of component links in the port channel and proportionally greater number of FLOGIs are pinned to this port channel. External interfaces124may be sorted to identify the interface which has the least number of FCIDs used. When there are multiple external interfaces124with an equal number of FCIDs, the external interface124may be selected from these at random.

Unless specifically stated otherwise, terms such as processing, computing, calculating, determining, displaying, or the like, may refer to an action and/or process of one or more processing or computing systems or similar devices that may manipulate and transform data represented as physical (e.g., electronic) quantities within a processing system's registers and memory into other data similarly represented as physical quantities within the processing system's registers or memories, or other such information storage, transmission or display devices. Furthermore, as used herein, a computing device includes one or more processing elements coupled with computer-readable memory that may be volatile or non-volatile memory or a combination thereof.

Some embodiments of the invention may be implemented in one or a combination of hardware, firmware, and software. Some embodiments of the invention may also be implemented as instructions stored on a computer-readable medium, which may be read and executed by at least one processor to perform the operations described herein. A computer-readable medium may include any non-transitory mechanism for storing information in a form readable by a machine (e.g., a computer). For example, a computer-readable medium may include read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash-memory devices, and others.