Dynamic network session redirector

A dynamic session redirector provides the ability to receive requests for network accessible storage resources and redirect them to the appropriate resources as needed while providing a single system image to the system requesting access to the resources. The redirector provides an ability to reconfigure the network storage resources without altering the system image presented by the redirector to the clients on the network. This may be used to provide for dynamic reallocation of the resources in order to improve efficiency and reliability of the storage system, as well to provide support for a wide variety of protocols to be redirected, including stateful protocols.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to network transaction handling. More specifically, the invention relates to the redirecting of network transactions under a stateful protocol.

2. Description of the Related Art

With the continued increase in the usage of electronic systems for the storage and distribution of information, it has become desirable for users of such systems to have access to ever larger storage facilities. Particularly when working with graphical and video information, the amount of storage that may be required can become much larger than the amount of storage available in a single disk drive or other storage medium.

In addition to an ever increasing demand for storage, there has also been an increase in demand for remote access and distribution of such electronically stored information. For instance, large database applications may support hundreds or even thousands of concurrent users, all of whom require access to the information which may be within a single database, even though the users may be dispersed across networks which span large geographic ranges.

Various systems have been developed which provide access to an aggregation of storage via an electronic network. Generally such systems involve large numbers of individual storage units arranged in a variety of architectures and accessible via the network. However, simply providing access to a large number of individual resources may present difficulties in configuration, maintenance and performance.

Therefore, there is a continued need for improved systems and methods for allowing client computers on electronic networks to have efficient, reliable access to large storage resources which may be operated and maintained effectively within the structure of existing computer networks.

SUMMARY OF THE INVENTION

One aspect of the system described herein is a dynamic session redirector (DSR) which is used to provide a single system image for access to network resources which may be distributed among many individual devices. By redirecting network requests received by the DSR to storage resources on the network, network resources being shared through the DSR can be presented using a single system image to network clients.

Another aspect of the DSR system presented is that the DSR examines the incoming requests and passes them on to the appropriate resources and then forwards the results of any request to the requesting client. Requests are sent to the DSR from client systems and the DSR responds as though the storage resources were part of the DSR. In this way, the DSR can present a single system image for network storage resources to clients on the network.

Further aspects of the system involve the use of individual network attachable storage (NAS) devices connected to the DSR in order to provide resources that the DSR is able to share with clients on the network. These network attachable storage devices may comprise either stand-alone devices or a storage area network (SAN) which provides servers that provide access to individual resources.

Another aspect of the system involves integrating the DSR with a network switch or other network routing apparatus. By providing a network switching function in the DSR, the requests that the DSR makes of the storage resources may be performed in hardware rather than software, improving the efficiency of the redirection function.

In another aspect of the invention, a technique to redirect network storage requests which are made using both stateful and stateless protocols is provided. The DSR maintains a set of tables which store the state of the various sessions initiated between the clients and the DSR, as well as the status of individual connections established by clients with particular resources. Incoming requests are modified and redirected to the appropriate systems on the storage side of the DSR in order to provide transparent access to the resources available to the clients through a single DSR interface.

Another aspect of the system provides for allowing a single session between the DSR and a client to support multiple connections to resources which may be accessed through different storage systems on the storage side of the DSR. This operation occurs transparently to the client, and provides for a single interface between the client and the resources, even when the resources are spread among a number of storage devices or servers.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description and Figures describing the preferred embodiments are made to demonstrate various configurations of possible systems in accordance with the current invention. It is not intended to limit the disclosed concepts to the specified embodiments. In addition, various systems will be described in the context of a networked computer system for carrying out the described techniques and methods. Those of skill in the art will recognize that the techniques described are neither limited to any particular type of computer or network, nor to the use of any particular hardware for every described aspect herein.

To facilitate a complete understanding of the invention, the remainder of the detailed description describes the invention with reference to the Figures, wherein like elements are referenced with like numerals throughout. Reference numbers in parentheses, for example “(600)”, are used to designate steps or processes.

Overview

One aspect of the present system involves a system to redirect network requests for resources to systems providing these resources in a manner which is transparent to the client requesting access to the resources. A system demonstrating this capability is shown inFIG. 1and described herein. A network100is shown to which a client system110is connected. The network100may be any standard network for connecting electronic systems as is known in the art, for example an Ethernet network. Although only a single client system110is shown inFIG. 1, the described system is not limited to a single client system; indeed, it is contemplated that multiple client systems110can connect to the network100.

The client110may use the network to request access to information which is stored on other systems. In the embodiment shown inFIG. 1, these requests are sent from the client110to the dynamic session redirector (DSR)120. The DSR is connected to the same network100as the client110, and receives requests for access to resources135via the network100. Although the network as shown inFIG. 1includes a network backbone with the client110and DSR120connected to it, the network100, however, may include additional devices to facilitate network communication. For instance, routers, hubs, switches, and other networking components may be used to provide for network connections between the client110and the DSR120without altering the nature of the system described herein. The words “hub” and “switch” are used broadly and interchangeably herein to include hubs switches, routers, gateways and such other networking hardware providing similar functions.

The DSR120is connected to one or more network storage devices130. Each network storage device provides storage resources135which may be made available to the DSR120. These resources135are in turn made available by the DSR120to clients110via the network100.

In one embodiment, the network storage devices130may comprise network attachable storage (NAS) devices which are directly connected to the DSR120, as shown inFIG. 1. In such configurations, it is desirable for the DSR120to include a network switch140or similar component in order to allow the NAS devices130to be connected to the DSR. In an alternate embodiment, storage devices130comprising network attachable storage may simply be connected to the same network100that is used for communication between the client110and the DSR120.

In operation, the DSR120is visible to a client110or other machine on the network100as providing the resources135of the various network storage devices130directly. In one embodiment, any client110on the network is unable to access the storage devices130directly, but rather accesses the DSR120. Conversely, the storage devices130do not make their resources135available to any clients110directly, but rather only to the DSR120. In this way, the DSR120may serve as a bi-directional proxy between the resources135which are made available by the storage devices130, and the client110which requests access to the devices. In alternate embodiments, storage devices may be made accessible to both the DSR and to the clients directly.

When the client110wishes to access a resource135, such as a particular volume of disk storage, the client initiates a communications session with the system providing the desired resource. Because the resource is visible to the client as being part of the DSR120, the client110initiates a session with the DSR120, and the DSR responds, producing a communications session between the DSR and client. This may take place using any of a variety of standard networking protocols, such as Server Message Block (SMB), Common Internet File System (CIFS), Network File Service (NFS), i-SCSI or other similar file or block level protocols as are known in the art. These protocols may include state information related to the communications session, and will generally be transported within another networking protocol such as TCP or NetBIOS.

Once the session is established between the client110and the DSR120, the client may request a connection to one of the resources135being made available through the DSR120. When such a connection is requested, the DSR120receives the request for the connection from the client110, and then proceeds to relay the request to the appropriate storage device130which is handling the particular resource135for which access is requested. In order to do this, the DSR120keeps a table of the resources provided by each network storage device. Throughout this document, the word “table” is defined broadly to refer to any data structure for later access, including without limitation a table, a list, a database, executable program logic to store data, object oriented code, arrays, variables or any other structure for storing and retrieving data. The DSR determines which of the network storage devices130will handle the resource135requested, and then the DSR modifies the connection request it received from the client110and passes the request on to the appropriate network storage device130.

The request is modified so that rather than appearing to be a request from the client110directly to the network storage device130, the request appears to the storage device to be originating from the DSR120. When the DSR120receives the reply from the storage device130, a response to the client110is prepared by taking the response received by the DSR from the storage device and making it appear to originate from the DSR, and then sending it to the client110. This provides the client with the impression that the resource135is part of the DSR and the storage device with the impression that the request was made from the DSR.

In this way, the burden may be reduced on the network storage device associated with establishing sessions, authenticating, maintaining status, broadcasting resource availability, or any of the other functions that might otherwise be required if the storage device were making its resources135directly accessible to the clients110on the network100. The storage device130interacts with the DSR120, and the DSR handles interaction and coordination with the clients110. In this way, the DSR120acts as a front end for the storage devices130and their resources135, handling and brokering requests, storing session related data such as client authorizations, while isolating the data storage devices130and the client110from one another.

One advantage of allowing the DSR to act as a front end for the resources135provided by the storage devices is that a single system image of the resources135available via any number of network storage devices130may be provided to a client110on the network100. This provides for a simpler view of the available resources to the client110and also means that a single session between the client110and the DSR120may provide the basis for connections to any of the resources135on any of the storage devices130. If the storage devices were directly accessible by the client110, then the client could alternatively initiate a session with each storage device having a resource to which the client wanted to connect.

As noted above, in one embodiment it may be advantageous to integrate the functionality of a network switch140within the DSR120in order to improve the efficiency of the connections made among the various storage devices130and any clients110on the network100. Because the connection between each storage device130and the DSR120may be made from a separate port of the switch140, the DSR can more efficiently send and receive messages from each storage device130without having a single network port handle data from more than one storage device. By moving this switching function into the network switch140, the DSR120can process more requests and connections more efficiently.

Alternate Embodiments

In addition to the embodiments described above, the DSR may also be used in alternate implementations that replace the network storage devices130shown inFIG. 1with a different configuration of storage devices. One such configuration is shown inFIG. 2and described below. As can be seen inFIG. 2, the network100is still used as a communications medium between the clients110and the DSR120. The clients110initiate communication sessions and request connections to network resources from the DSR in the same way they would in the embodiment shown inFIG. 1and described above.

Instead of connecting directly to storage devices, the storage side of the DSR120is connected to another network200. This network is used to connect the DSR to one or more storage servers210that are used to handle access to a storage area network (SAN). Note that as discussed above, it may also be possible to include NAS devices within the storage are network itself. As used herein, the term “server” is defined broadly to include any hardware or software which provides and controls access to resources for remote systems. For example, a server may be a standalone computer running any of a variety of operating systems, a dedicated piece of hardware which performs only server functions, or a process running on a general purpose computer.

The storage area network may preferably comprise a hub220, such as a fiber channel hub to which one or more storage resources230are connected. The resources230of this storage area network are made available to the clients110in much the same way as is described for the configuration ofFIG. 1above. However, because each storage server210has access via the hub to each of the storage resources230, bottle-necks in performance may be avoided by allowing access to each resource230through more than one server210. In alternate embodiments, the SAN may comprise a network which does not use a fiber channel architecture. For instance, the SAN may use any of the following architectures in place of fiber channel without limitation: ethernet, Virtual Interface (VI), Infiniband, SCSI architectures including SCSI over ethernet (I-SCSI), and other similar architectures.

When requests for connections to resources230are received by the DSR120, the DSR prepares an appropriate request to whichever server210which is capable of fulfilling the request for the resource230. Because the storage area network may provide access to a particular resource through more than one of the servers210, the DSR120may choose which server210to contact based upon other criteria. These may include criteria such as response time, throughput, or such other criteria as are known to those of skill in the art. When the request is received by the appropriate server, it passes the request through the hub220to the appropriate resource and receives a response. The response is forwarded back to the DSR by the server210. The DSR120then sends a response to the requesting client110as if the resource230were a resource directly connected to the DSR itself.

As discussed with reference toFIG. 1above, this embodiment provides a single system image of the resources being available directly from the DSR120when viewed by a client110, and provides a source of requests (i.e. the DSR) for the storage area network. The use of a storage area network, as opposed to a group of network attached storage devices also may provide benefits in terms of scalability and reliability.

Another feature shown inFIG. 2is a second DSR240operating in parallel with the primary DSR120. The second DSR240may provide additional resources in order to improve throughput of the system, and may also serve as a redundant backup system to take over in the event that the first DSR120becomes inoperable. The second DSR240is connected to the client network100in the same manner as the first DSR120, and also has access to the storage side network200. In this way the second DSR240is capable of processing the same requests and redirecting the same resources230as the first DSR120.

In one operating mode, both DSRs,120,240are operating at the same time and each receives requests from clients110and processes them as described herein in order to provide the functions of a DSR. In this mode, each DSR is providing access to the clients110to the storage resources230independently and without communication with the other DSR. In such a mode, each client110message received is processed by one or the other DSR, but not both.

If the first DSR120is to fail in such a mode, any attempt to communicate with any resources through the DSR120will also fail, and the clients110will recognize that their connections and sessions have been terminated. Because the second DSR240will still be operative, requests for sessions and connections to resources from that point forward will be made through the second DSR. The second DSR240will still be providing access to the same resources230, because the second DSR240has the same access to the SAN as the first DSR120did. As a result, access to no resources230are lost when the first DSR120fails, as long as the second continues to run.

However, because particular sessions and connections to resources made with the first DSR120have been lost, any client working with a connection or session from the first DSR will have to attempt to reconnect to the appropriate resources. New requests for connections and sessions will be handled by the second DSR240. As the second DSR240receives the appropriate information with each session or connection request from a client110, this information will be added to the resource and state tables of the second DSR240. This will allow the second DSR to handle the redirection that was handled by the first DSR120.

In another operating mode, each DSR120,140maintains a connection to the other DSR specifically for the purpose of exchanging information related to the state of the various sessions and connections that each is processing. This “heartbeat” connection is used to provide an indication to each DSR that the other is still operational, as well as to allow each to keep a redundant table of any state information that may be necessary in order to process requests from clients110. In this way, each DSR may operate as a real-time backup of the other, and any incoming message may be handled by either DSR. This also means that a request for a connection within a session initiated with the first DSR120may be handled by the second DSR240transparently, since each DSR is aware of the state information associated with each client110and session and connection at all times.

If the first DSR120fails in such a mode, the second DSR240may take over any requests made of the first DSR120without terminating existing sessions that were made with the first DSR120and without requiring the clients110to reconnect or reestablish their credentials with the second DSR240. By using the heartbeat connection to maintain redundant state information between the two DSRs, any data that is identified as being relevant to the state of any session or connection is updated to the other DSR, allowing each to process requests initially intended for the other.

In such cases, when the second DSR240detects that the first DSR120has failed, it may simply begin responding to any requests which are made of the first DSR as if it were the first DSR itself. The clients110need never know that the first DSR120has failed, or that their requests are being handled by a different system. This allows existing sessions and connection to continue to operate in a manner transparent to the client110and without interrupting any of the data transactions which may be in process between the client110and any resources230on the storage area network.

In order to facilitate either of these redundant operating modes, the DSR's may be configured to operate using redundant network interfaces and network switches. Additionally, as will be apparent to those of skill in the art, the first DSR120is capable of taking over in the event that the second DSR240fails in the manner as described above for the second DSR240taking over from the first. It should also be noted that the number of DSRs may be extended to more than two in circumstances where even greater redundancy and reliability is desired. The described techniques may be extended as necessary to handle any number of redundant DSR's connected between the client network100and the storage area network200.

An additional embodiment using a storage area network is shown inFIG. 3and described below. As with the embodiment shown inFIG. 2, the client110and the DSR120communicate via a network100, and the DSR120communicates with one or more storage servers210via a second network200. The storage servers210are connected to a hub220on a storage area network but the individual storage units320are not accessed directly by the servers210through the hub220. Instead, one or more RAID controllers310are used to organize the individual storage units320into volumes. These volumes may be made available as resources by the RAID controllers310to the DSR120across the storage area network.

As in the previous embodiments, the individual volumes that are logically created by the RAID controllers310may be visible to clients110on the first network100as though those volumes were part of the DSR120itself. This configuration may provide advantages in efficiency, failover operation and such other benefits of RAID configurations as are known to those in the art.

Redirection Technique

Although several embodiments of systems including a DSR120for use in managing access to a set of network resources are described herein, the following discussion will be made with particular reference toFIG. 2and the embodiment shown therein. However, the description, including the techniques described can also apply to embodiments other than the exemplary embodiment ofFIG. 2. In particular, the various embodiments illustrated and described herein may also advantageously make use of such techniques.

In the discussion which follows, reference may be made to the exemplary DSR120illustrated inFIG. 4and described below. As shown inFIG. 4, the DSR may comprise electronic hardware configured to perform a variety of functions. In particular, the DSR may comprise a client side network port410or interface for connecting to the client side network100ofFIG. 2, as well as one or more storage side network ports420for connecting to the storage network attached200to the DSR.

In embodiments as described above where it is advantageous to include a network switch140within the DSR120, the switch140will generally be connected to the storage side network ports420. Such an arrangement may be particularly advantageous when the storage side of the system comprises a number of NAS devices130such as are shown inFIG. 1. Connecting different NAS devices130to separate ports of the switch140may allow for greater throughput of the DSR120and fewer collisions on the storage side of the network. In configurations, such as those shown inFIGS. 2 and 3, where the storage side of the DSR120is connected to a second storage area network200, it may be less advantageous to have multiple storage side network ports420, and a single network port to connect to the SAN200may be used. Those of skill in the art will recognize that the DSR120may be made to operate with either SAN or NAS components regardless of the number of storage side network ports420included. The exemplary embodiment of the DSR120illustrated inFIG. 4includes an integral network switch140supporting four storage side network ports420.

In addition to the client side port410and storage side port(s)420, the DSR120may also include one or more heartbeat connection ports430. These connections may be used as described above to communicate with other DSRs in order to exchange state information and provide information related to the status of each DSR.

The operation of the DSR120and the processing of the various information which is received and sent via the ports described above410,420,430is handled by the processor440of the DSR120. The processor may comprise a computer, program logic, or other substrate configurations representing data and instructions which operate as described herein. In alternate embodiments, the processor can comprise controller circuitry, processor circuitry, processors, general purpose single-chip or multi-chip microprocessors, digital signal processors, embedded microprocessors, microcontrollers and the like. The processor440as defined herein may include any such hardware, software, firmware, memory, storage, and input/output and peripheral components as are needed to carry out the functions described herein.

For example, the DSR120may operate on any of a variety of computer hardware architectures and operating systems, including Intel/Windows architectures, RISC architectures, Macintosh architectures and other similar architectures. The operating systems may similarly include without limitation: Microsoft Windows, UNIX, Linux, Macintosh OS, and embedded and real-time operating systems. A particular embodiment of the DSR120may be based, for example, upon a Linux operating system running on a variety of hardware platforms.

In addition to the processor440described above, the DSR120maintains a storage area450to track a number of tables as are described herein. These tables may be used to store information related to the state of various sessions and connections made by clients110to the resources230provided. The tables may also store information related to the operation of redundant DSRs connected via heartbeat connections and credential information for any users that are authenticated through the clients110. This information may be maintained with the DSR in the storage450area and accessed for both reading and writing by the processor440as necessary during the operation of the DSR120.

The DSR120as shown inFIG. 4operates as an intermediary between the client110and the resources which are being made available to the client. In particular, the DSR provides techniques for requests being made using both stateless and stateful communications protocols to be redirected from the DSR itself to the appropriate resources transparently to the client.

In a stateless communications protocol, for instance NFS V.2 or NFS V.3, individual connections between a client110and a network resource135are made with no need to track information from one request to the next. This operation is normally advantageous in that very little overhead is required for the processing of each individual request, because no status has to be tracked from one request from a client110for a particular resource135to the next. However, when many connections are being made between the same clients and resources, such a stateless protocol results in redundant processing of many equivalent requests. For instance, because there is no tracking of status from one request to the next, requests from the same client110for different information from the same resource135may result in repeated and duplicative authentication, identification and lookup operations as the system determines whether or not the client110is to be given access to the particular resource.

The DSR120described herein provides a particular advantage even when redirecting stateless protocols in that it may track certain state information related to the connections being made by clients110to the various resources135and use this information to streamline future requests made by this client.

In addition to tracking state information for the communications protocols, the DSR may also provide a table which stores the authentication credentials of the clients110that establish sessions with the DSR. By maintaining a table of this information, the DSR120may provide credentials as needed for a particular client to any server210to which it may connect in order to provide access to a resource to a client. This avoids a need to get credentials from the client110every time a new connection is established.

For instance, when a client110makes a request for data from a particular resource230, the DSR120determines which of the servers210are responsible for providing access to that resource. In a stateless protocol this information is not stored because the individual server may not be the same the next time that particular resource is accessed by a client. However, by the use of a DSR120capable of tracking state of the mappings between particular resources and the appropriate server responsible for those resources, repeated access to that resource is streamlined.

After an initial determination of the mapping between a particular file handle and the appropriate server, the DSR may add that particular mapping to a table of correspondence between resources, such as file-handles or volume labels, and the server210responsible for that resource. This persistent mapping allows for the redirection of stateless protocols to be handled more efficiently through the use of a DSR120than would be possible if each client were responsible for directly connecting to resources230, or if the DSR were forced to determine the appropriate server for the resource independently each time.

Multiplexing

Another aspect of the benefits available through the use of a DSR120as described above is to allow for the effective multiplexing of an individual session between a client110and the DSR120to produce a plurality of individual connections to various servers210from the DSR when providing resources230to the client. For instance, in a stateful network communications protocol such as SMB or NFS V.4, it is normal for a client to first establish a communications session with a server providing resources, and then to make connections to individual resources from that server. Requests for various types of access to that resource are then made within the context of that connection between the client and the resource.

However, because the DSR120serves as a sort of proxy between the client110and the resources230desired, there is not a one-to-one correspondence between the communications made between the client110and the DSR120on the network100side, and the communications made between the DSR120and the servers210on the storage side. The particular nature of the correspondence will depend upon the type of operation being performed.

For instance, when setting up an initial session between the client110and the DSR120, there is no need for the DSR to make any communications with any of the servers210at all. As shown inFIG. 5, the client110will send a message to the DSR120to establish a session (500). The client110contacts the DSR120because the DSR is the only “server” that the client110can see on the network100, and from the client side, the DSR provides access to the resources230directly. After the DSR receives the message (505), the DSR examines and stores any credentials forwarded by the client110(510) in the appropriate table450within the DSR (510) and then the processor440of the DSR120generates (515) and stores (520) a session identification for this particular client's session with the DSR. This session ID is then send back to the client110(525) where it is received by the client (530) and used for any further communications with the DSR related to this particular session.

As can be seen inFIG. 5, establishing this session does not require that the DSR120exchange any information with the servers210, so a message to set up a session between the client110and the server handling any particular resource230does not actually generate any messages on the storage network200between the DSR120and the servers210.

Note that this circumstance assumes that the DSR120has already established an appropriate registration of the particular resource230from whichever server210is managing that particular resource. This resource table is stored in the DSR120and is used to track what resources230are available for the DSR120to provide to clients110. In one embodiment, the servers210are configured to automatically broadcast what resources230they are providing to the DSR120automatically upon startup. The DSR stores this information in the table.

A resource management process may be running on the DSR in order to maintain and update the resource table during the operation of the DSR. In addition to populating the resource table as information is received from servers210as they come online, this process also may update the table as needed if resources or servers become unavailable, or if control of a particular resource230is transferred from one server210to another for any reason, such as a failure of a single server210.

In the above circumstances, when the message is received by the DSR120from the client110initiating either a session or a connection to a particular resource230, the DSR is able to respond based upon the information it already has in the resource table, and need not make an additional communication with any of the servers210on the storage side.

By contrast to the situation shown inFIG. 5, when an actual connection to a particular resource230is request by a client110, it is necessary to authenticate that client110for that resource230. Similarly, when actually reading or writing data from a resource230, it is clearly necessary for there to be some data exchange between the DSR120and the resource230through the appropriate server210. The data flow associated with such operations is shown inFIG. 6.

When a client110requests a connection to a specific resource230(600), the message is received by the DSR120(605) and the server210associated with that particular resource230is located in the DSR's tables450(610). Once the appropriate server210is located, the DSR takes the request from the client and forms an equivalent request that is sent to the server210responsible for the resource230(615). The DSR120includes any appropriate credential or other identifying data that the server210would need to process this connection request. This information is also looked up from within the storage area450of the DSR.

The server210receives the message requesting a connection to the resource230(620) and proceeds to process this request based upon the credentials other information sent by the DSR120(625). If the credentials presented allow access to the resource230, then the server210will establish the connection to the resource230, generate an appropriate connection identifier (630) and return this identifier to the DSR120(635). If the connection is refused, then the rejection message is generated and returned to the DSR.

The DSR120receives the connection identifier (640) and stores this ID in the appropriate tables within the storage area450of the DSR (645). Once stored, a message is sent back to the client110(650) indicating the connection identifier or the rejection of the connection. The client110receives this message from the DSR (655) and is unaware that the resource230is actually handled by the server210and not the DSR120.

This arrangement allows for multiplexing of connections between the DSR120and the servers210based upon a single session or connection between the client110and the DSR120. In one configuration, a single session may be used to establish connections with resources230spread across multiple servers210. This helps cut down on unneeded overhead associated with multiple connections. In addition to avoiding unneeded messaging between the DSR120and servers210, the single system image provided by the DSR allows for the multiplexing of a single connection between the client110and DSR120to more than one server210on the storage side. For example, after establishing a session between the client110and the DSR120, the client110may connect to a particular resource230made available by the DSR120. This resource230may be handled by the first server210on the storage network. Any request related to a file or other transaction associated with that particular resource will be accepted by the DSR120and redirected to the appropriate server210. The response from the server210will then be redirected back to the client110as if it originated from the DSR120itself.

However, a request for a different resource may be made by the same client to the DSR and handled properly without initiating a new session, even if this second resource is handled by a different server on the storage area network. Because the client110sees the resources as being provided by the DSR, a single session between the DSR120and the client110provides the ability to connect to any of the resources230available on the network200that the DSR is connected to. Therefore, establishing a single session to the DSR120enables the client to connect to the resources from any of the servers210without having to establish separate sessions to each. In this way, it can be said that the single session between the client and DSR is multiplexed into a plurality of connections to separate servers, all through the one session.

This multiplexing is accomplished by maintaining a session table in the DSR120that tracks the sessions that have been established between the DSR120and any clients110, as well as information that is related to those sessions. This information may vary somewhat depending upon the particular protocols used but may include such information such as a session identifier, the connection identifier (the Tree ID in SMB), and a user identification which identifies the particular client110requesting any connection.

Because it is possible that a particular logical volume which is made accessible to the clients110as a single resource230may actually consist of multiple logical units320grouped together as a single volume330, working with that volume330may actually require information from multiple logical units330. These units may represent separate disks within a RAID structure (as will be discussed in greater detail below). Because the client110expects to make only a single request to access a single volume330(or other resource230), if there are multiple logical units320which will be accessed to obtain that data, then the DSR120will make multiple connections to various resources in order to fill the request.

For instance, in the example shown inFIG. 6, the resource for which a connection was requested could be a logical volume330which was spread across three separate physical resources320. In such a case, when the DSR120looks up the location for the appropriate volume, it may discover that the resources320are spread among different servers210. In such instances, it may be necessary for the single request made between the client110and the DSR120to generate a separate request and response between the DSR120and each server210responsible for one of the resources320of the appropriate volume330. In such cases, steps615to645as described above are repeated as needed for each individual server210with which the DSR120must establish a connection. Once the appropriate connections are made and responses are received, the DSR may then complete the transaction with the client110by responding with a connection ID.

Load Balancing and Dynamic Volume Management

Certain embodiments of the DSR perform certain types of optimizations and management functions regarding the operation of the storage side of the system transparently to any client. Particular aspects of this functionality will now be discussed with regard to the embodiment shown and described inFIG. 3, above. Although reference will be made to the specific configuration of the system inFIG. 3, those of skill in the art will recognize that such techniques may be applied to other configurations, such as those shown inFIGS. 1 and 2, as well as to equivalent structures known to those in the art.

One type of storage management function that may be provided by the DSR120is to provide fault tolerance for any failures of servers210located on the storage side network200. An example is shown inFIG. 7. Because each server210shown inFIG. 7is connected via the storage network hub220shown inFIG. 3and is therefore capable of communicating with each of the RAID controllers310, any of the servers210may provide access to the resource320controlled by the RAID controllers310. Each individual resource320may be assigned to a server210, and this mapping between resources320and servers210is stored within a table in the storage area450of the DSR120.

As illustrated inFIG. 7, the first two logical units are initially assigned to the first server210, the next two logical units are assigned to the second server210, and the remaining two are assigned to the third server210. Although the complete multiplexing of connections is not shown inFIG. 7, as noted in the discussion ofFIG. 3, each of the resources320is accessible through the storage area network from any of the servers210. However, for purposes of efficiency, normally a server will handle access to a single resource at a time.

During normal operation, any requests involving the first two resources320will be handled by the first server210, the second two resources320are handled by the second server210and the last two resources320are handled by the third server210.FIG. 7illustrates a circumstance where the first server710is no longer in proper communication with the DSR120. This may be due to a failure in the first server210itself, or a failure of the communications link between the DSR120and the server210, or a failure of the storage port420of the DSR120to which the particular server210is connected.

When such a failure occurs, any requests made by the DSR120of this failed server710will fail. When this occurs, the DSR120will become aware that it is unable to receive responses from this server710, and the resource management process will find that messages sent to this server are unanswered. If each server210were directly connected to only those resources directly under the server's control, there would be no way to access the resource720belonging to the failed server710, even if the corresponding RAID controller310and resources320themselves were still operational.

When requests to the failed server710are not responded to, the DSR120will recognize that no access to resources720can be made through that particular server710. However, because the connections on the storage side are multiplexed through the hub220, the DSR can send commands to the remaining servers210, requesting them to handle the operations of the resources720previously assigned to the failed server710. In this instance, the DSR120can request that the second server210take responsibility for any connections to the second resource720in addition to the two resources320it already handles. The DSR may also request that the third server210take responsibility for the first resource720. In this way, the resources720which were being handled by the failed server710now remain accessible through the remaining two operational servers210.

After the DSR receives confirmation that the remaining servers210can provide access to the remapped resources720, the DSR120may store an updated resource table in its storage area450. Once updated, any future traffic bound for those resources720will be directed to the operational servers210which are still running, rather than the failed first server710. However, the client-side traffic related to those resources is unaffected. The same resources are provided by the DSR120to the client110, and the client110need not be aware that any change in configuration of the resource table has occurred. Because the entire failure and remapping has taken place on the storage side of the DSR120, the entire fail-over process is transparent to the client110and does not effect any connections made between the client110and the DSR120.

Although the example shown inFIG. 7and described above relates to a failure of a server710, such automated failover may be provided by the DSR120for any of the systems which are connected to its storage side interfaces. Such a technique may be used to provide for fault tolerance by allowing healthy servers to take over for failed servers. This also allows for the replacement or upgrade of individual servers210without the need for taking the entire storage system off-line. By reassigning the control of resources to different servers, any individual server may be taken off-line without effecting the overall access to resources by clients. Additionally, by adding additional servers to provide additional capability, a single server can be kept available but unused and may take over from any server that fails. This allows a single server to provide a redundant backup function to any number of servers on the storage side network200without a diminishing of performance, regardless of the number of servers210on the network200.

Similarly, failure capability may be provided at the level of the RAID controllers310themselves. Through the use of multiple RAID controllers, each of which have access to all of the storage units320, the failure of a single RAID controller need not result in a loss of access to the volumes managed through that particular controller310. When this failover feature at the RAID level is combined with the single system image capabilities of the DSR120, it is possible to handle a failed controller310without the need to disturb any client sessions and connections.

The failure of a RAID controller310results in the control of the resources of that controller being handled by a different controller. If the two controllers were themselves being managed through different servers210, then the failover results in the control of the sources of the failed RAID controller switching over to the other server210as well as to the other controller310. If sessions and connections were being handled directly between the servers210and the clients110that were requesting access, this would result in broken connections and dropped sessions for the remapped resources.

However, by having the DSR120in a position to present a single system interface for the resources320to the client110, the failover can be handled transparently to any clients110even when the servers or RAID controllers associated with a particular resource are changed. When the failover of a controller310occurs, the volumes now available through the remaining controller310will appear as resources to the server210handling that controller. The server210will broadcast these newly available resources the DSR120. The DSR can update the resource table with the new mapping between the proper server and the resource, and simply proceed to redirect any further requests related to the shifted resources320to the new server210rather than the previous server. The DSR need not inform the client110of any of this change in operation.

In addition to providing a transparent means for fault tolerance and failover at both the server and controller level, the DSR may also be used to perform load balancing and related management tasks for the storage area network transparently to any connected clients. One example of this is the reassignment of storage resources320from one server to another based upon the current loading of the individual servers, or based upon any other operational metric or status of the servers.

For instance, as shown inFIG. 3, there may be three servers210available on the storage side network200, the servers being used to access a number of resources320being managed by a pair of RAID controllers310. Because each resource is associated with a particular server210in the resource table of the DSR120, any request for that resource will go through that server. As the system runs, demand for particular resources320may increase while demand for other resources320decrease. If the resources320in demand are accessed through the same server210, the performance associated with access to those resources320will decrease because the server210becomes a bottleneck in the smooth access to the resources from the DSR.

However, because the mapping between the resources320and the servers210is stored in the DSR120and can be modified dynamically, the DSR can be configured to automatically remap resources320in demand to servers210with excess capacity at that time. In this way if two of the resources accessed through one server210are in high demand, one of the two resources may be automatically remapped to be accessed through a different server210with more available capacity. As discussed above with reference toFIG. 7, by rewriting the resource table, this remapping may be done without interrupting client sessions and connections, while still providing the benefits of a more efficient distribution of bandwidth among the servers. Furthermore, if the demand for resources changes, the mapping may be reassigned again to rebalance the load.

In general, as the DSR120operates, it monitors the amount of activity and usage of each of the servers to which it is connected. The monitored parameters of the servers210may include such information as the hit rate on that server, the network activity experienced by the server, the CPU utilization of the server, the memory utilization of the server, or such other parameters as are known in the art to related to the load placed upon the server210. An overall load for each server may be calculated based upon these parameters. In addition, the amount of load generated by requests associated with each of the resources320handled by that server may be monitored.

As the DSR runs, the load on each of the servers is periodically compared. If the load between each of the servers is approximately balanced, then there is no need for the DSR120to remap any of the assignments between the resources320and the servers210. However, if the DSR120detects that one or more servers is significantly differently loaded than the others, the DSR may choose to reassign one or more of the resources320to a different server210.

By analyzing the amount of load associated with each of the resources320on each server, the DSR120may attempt to determine whether there is a more efficient distribution of the resources among the server that will result in a more balanced load among the servers210. If there is such a more balanced configuration, the DSR120may remap the resources320among the servers210in order to produce this more balanced load, update the mapping table in the storage450of the DSR120, and then continue with its operation.

For instance, as shown inFIG. 8A, the operation of the system at a given time may indicate that for each of the six resources320being handled by the three servers210, the respective loading due to each resource is as shown in the FIGURE: 100, 100, 150, 200, 300, 400. This results in a loading upon the first server of200, a loading on the second of350, and a loading on the third of700. Note that these units are arbitrary and are intended for illustrative purposes only. This loading is clearly unbalanced as the third server has twice the load of either of the other servers.

In order to rectify this situation and unbalance, the DSR120may ask the first server to take on the responsibilities of the fourth resource320from the second server, and ask the second server to take over handling the fifth resource for the third server. This configuration is shown inFIG. 8B. Note that although the load on the individual resources320have not changed, the overall distribution of the load among the three servers210is now more evenly distributed at a load of400for the first and third server, and a load of450for the second. This is true even though the third server210is handling a single resource320and the first is handling three resources320.

In one embodiment, this load balancing process may be repeated periodically by the DSR120in order to maintain a balanced configuration. In an alternate embodiment, the load balancing may be triggered based upon the load conditions themselves, for instance, when the load on any one server exceeds the load on any other server by more than 50%. In other embodiments, predictive or adaptive algorithms may be used to estimate future load levels based upon current and historical operational patterns, and trigger load balancing based upon such projected levels of usage.

As above with the fail-over of servers, this remapping takes place without any impact on the connections or sessions maintained between the clients110and the DSR120and as a result, such remapping and balancing may be transparent to any user, even while there are open connections.

The various embodiments of dynamic session redirector and associated systems and techniques described above in accordance with present invention thus provide a variety of ways to improve the performance, reliability or scalability of storage on electronic networks while retaining effective access to that storage from remotely located clients. In addition, the techniques described may be broadly applied across a variety of protocols, both stateful and stateless, and system architectures utilizing a variety of different subsystems.

Furthermore, the skilled artisan will recognize the interchangeability of various features from different embodiments. For example, the direct connection of network attachable storage devices130to a DSR including an integral network switch may be combined with the multiplexing of stateful sessions through the DSR. Similarly, the various techniques and architectures discussed above, as well as other equivalents for each such feature, can be mixed and matched by one of ordinary skill in this art to construct storage systems in accordance with principles of the invention.

Although this invention has been disclosed in the context of certain embodiments and examples, it therefore will be understood by those skilled in the art that the invention extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses of the invention and obvious modifications and equivalents thereof. Thus, it is intended that the scope of the invention herein disclosed should not be limited by the particular disclosed embodiments described above, but should be determined by the scope of the claims that follow.