Abstract:
A method is described comprising: maintaining a plurality of data relating to client streaming connections across a plurality of servers; and assigning a particular client streaming connection to a first server upon detecting that a second server previously serving a streaming connection to the client has become inoperative.

Description:
BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   This invention relates generally to the field of network services. More particularly, the invention relates to an improved architecture for providing fault tolerant data communication. 
   2. Description of the Related Art 
   As is known in the art, streaming is a mechanism for playing back audio and/or video content over a network in-real-time, typically used in situations where network bandwidth is limited. The basic streaming concept is that the destination (e.g., a client) begins to play back the underlying streaming file from a buffer before the entire file has been received from its source. 
   A traditional network streaming system is illustrated in  FIG. 1 . As shown, one or more clients  150 ,  160 , configured with streaming application software such as RealPlayer® from RealNetworks® or Windows Media® Player from Microsoft® Corporation, communicate with one or more streaming servers  110 ,  111 , . . . N, over a network  100  (e.g., the Internet). The group of streaming servers  110 ,  111 , . . . N, are located together at a point of presence (“POP”) site. Each of the streaming servers  110 ,  111 , . . . N, may store a copy of the same streaming data or, alternatively, may store different streaming data, depending on the configuration at the POP site  130 . 
   In operation, when a client  150  requests a particular streaming file from a server at the POP site  130 , the request is received by a load balancer module  120 , which routes the request to an appropriate streaming server  111 . Which server is “appropriate” may depend on where the requested file is stored, the load on each server  110 ,  111 , . . . N, and/or the type of streaming file requested by the client (e.g., Windows Media format or RealPlayer format). Once the file has been identified by the load balancer  120  on an appropriate server—server  111  in the illustrated example—it is streamed to the requesting client  150  (represented by stream  140 ) through the network  100 . 
   One problem with current systems for streaming multimedia content, however, is that when delivery of a stream to a client/player  150  is interrupted, there is no automated mechanism for correcting the interruption (e.g., providing another source for the stream) without some type of manual intervention. For example, if server  111  in  FIG. 1  were to crash while streaming content to client  150 , the client&#39;s multimedia stream would be interrupted. The client  150  (or, rather, the streaming application) would then manually attempt to reconnect to the server  111  through the load balancer  120 . 
   The problem is even more serious if the client  150  is receiving a stream of a live event or a scheduled event (e.g., such as a “Web-cast”). In this case, by the time the client  150  reestablished a connection, a significant portion of the event would simply be unavailable to the client  150 . In sum, current streaming applications do not provide any mechanism for automatically and seamlessly reestablishing a streaming session with a client, once the initial session has been interrupted. 
   Accordingly, what is needed is a system and method for providing fault tolerance and/or automatic fail-over techniques with respect to streaming multimedia content over a network. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which: 
       FIG. 1  illustrates a prior art system and method for streaming content over a network. 
       FIG. 2  illustrates an exemplary network architecture including elements of the invention. 
       FIG. 3  illustrates an exemplary computer architecture including elements of the invention. 
       FIGS. 4   a  and  4   b  illustrate embodiments of a system and method for streaming content over a network. 
       FIG. 5  illustrates client streaming information stored in accordance with one embodiment of the invention. 
       FIG. 6  illustrates coordination between various different point of presence sites according to one embodiment of the invention. 
   

   DETAILED DESCRIPTION 
   In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the invention. 
   An Exemplary Network Architecture 
   Elements of the present invention may be included within a multi-tiered networking architecture  200  such as that illustrated in  FIG. 2 , which includes one or more data centers  220 - 222 , a plurality of “intermediate” Point of Presence (“POP”) nodes  230 - 234  (also referred to herein as “Private Network Access Points,” or “P-NAPs”), and a plurality of “edge” POP nodes  240 - 245  (also referred to herein as “Internet Service Provider Co-Location” sites or “ISP Co-Lo” sites). 
   According to the embodiment depicted in  FIG. 2 , each of the data centers  220 - 222 , intermediate POPs  230 - 234  and/or edge POPs  240 - 245  are comprised of groups of network servers on which various types of network content may be stored and transmitted to end users  250 , including, for example, Web pages, network news data, e-mail data, File Transfer Protocol (“FTP”) files, and live &amp; on-demand multimedia streaming files. It should be noted, however, that the underlying principles of the invention may be practiced using a variety of different types of network content. 
   The servers located at the data centers  220 - 222  and POPs  230 - 234 ;  240 - 245  may communicate with one another and with end users  150  using a variety of communication channels, including, for example, Digital Signal (“DS”) channels (e.g., DS-3/T-3, DS-1/T1), Synchronous Optical Network (“SONET”) channels (e.g., OC-3/STS-3), Integrated Services Digital Network (“ISDN”) channels, Digital Subscriber Line (“DSL”) channels, cable modem channels and a variety of wireless communication channels including satellite broadcast and cellular. 
   In addition, various networking protocols may be used to implement aspects of the system including, for example, the Asynchronous Transfer Mode (“ATM”), Ethernet, and Token Ring (at the data-link level); as well as Transmission Control Protocol/Internet Protocol (“TCP/IP”), Internetwork Packet Exchange (“IPX”), AppleTalk and DECnet (at the network/transport level). It should be noted, however, that the principles of the invention are not limited to any particular communication channel or protocol. 
   In one embodiment, a database for storing information relating to distributed network content is maintained on servers at the data centers  220 - 222  (and possibly also at the POP nodes  230 - 234 ;  240 - 245 ). The database in one embodiment is a distributed database (i.e., spread across multiple servers) and may run an instance of a Relational Database Management System (RDBMS), such as Microsoft™ SQL-Server, Oracle™ or the like. 
   An Exemplary Computer Architecture 
   Having briefly described an exemplary network architecture which employs various elements of the present invention, a computer system  300  representing exemplary clients and servers for implementing elements of the present invention will now be described with reference to  FIG. 3 . 
   One embodiment of computer system  300  comprises a system bus  320  for communicating information, and a processor  310  coupled to bus  320  for processing information. The computer system  300  further comprises a random access memory (RAM) or other dynamic storage device  325  (referred to herein as “main memory”), coupled to bus  320  for storing information and instructions to be executed by processor  310 . Main memory  325  also may be used for storing temporary variables or other intermediate information during execution of instructions by processor  310 . Computer system  300  also may include a read only memory (“ROM”) and/or other static storage device  326  coupled to bus  320  for storing static information and instructions used by processor  310 . 
   A data storage device  327  such as a magnetic disk or optical disc and its corresponding drive may also be coupled to computer system  300  for storing information and instructions. The computer system  300  can also be coupled to a second I/O bus  350  via an I/O interface  330 . A plurality of I/O devices may be coupled to I/O bus  350 , including a display device  343 , and/or an input device (e.g., an alphanumeric input device  342  and/or a cursor control device  341 ). 
   The communication device  340  is used for accessing other computers (servers or clients) via a network  100 . The communication device  340  may comprise a modem, a network interface card, or other well known interface device, such as those used for coupling to Ethernet, token ring, or other types of computer networks. 
   EMBODIMENTS OF THE INVENTION 
   One embodiment of the invention, illustrated in  FIG. 4   a , provides an automatic fail-over solution for content streaming. The cluster of streaming servers  400 , . . . N, in this embodiment actively share client data between one another relating to current client streaming connections. Accordingly, when a client-server streaming connection is lost (e.g., due to a server crash), a different server reestablishes a connection to the client at a point in the streaming file where the previous server left off. 
   In one embodiment, a cluster manager  410 ,  415  maintains up-to-date connection data for each client. The cluster manager  410 ,  415  may reside on one server at the POP site  430  or multiple servers, depending on the particular configuration (although illustrated in  FIG. 4   a  in a multiple server configuration). Moreover, in one embodiment, the cluster manager  410 ,  415  may be distributed across multiple POP sites (see, e.g.,  FIG. 6  described below). 
   In one embodiment, the cluster manager  410 ,  415  receives client connection updates from cluster agents  425 ,  420  running on various streaming servers  400 ,  405  at the POP site  430 . For example, when a client  150  initially connects to a particular streaming server  405  (e.g., through the load balancer  405 ), a cluster agent  425  running on that server  405  transmits connection data pertaining to that client to the cluster manager  410 ,  415 . In addition, in one embodiment, the cluster agent  425  regularly transmits client connection updates to the cluster manager  410 ,  415  at predetermined intervals. Thus, the client data maintained by the cluster manager  410 ,  415  is kept up-to-date. 
   As illustrated in  FIG. 5 , various types of client connection data may be collected and transmitted by the cluster manager  410 ,  415 , including, for example, an identification code assigned by the cluster agent  425 , the client&#39;s Internet Protocol (“IP”) address, the current state of the streaming connection (e.g., active, paused, ended, . . . etc), a media identification code indicating the particular streaming content requested by the client (including a network path), the type of streaming protocol used to stream the content (e.g., real time streaming protocol, active streaming format, . . . etc), a media offset indicating the point in time to which the multimedia stream has progressed, and various networking data such as the Transmission Control Protocol (“TCP”) port, Network Address Translation data (if any), the maximum supported client bit-rate and/or the actual bit-rate used in the stream delivered to the client. In one embodiment, copies of the foregoing client data are maintained on multiple streaming servers by the cluster manager  410 ,  415 . 
   As described above, in one embodiment, cluster agents  420 , 425 , running on the streaming servers  400 ,  405  send client connection updates to the cluster manager  410 , 415  at predetermined intervals. Accordingly, in one embodiment, the cluster manager  410 ,  415  uses this periodic update as a “heartbeat” to detect when a particular streaming server has become inoperative (or, alternatively, to detect other problems with the server such as an unmanageable server load or a server communications/network problem). If the cluster manager  410 ,  415  does not receive an update from a server after one or more update periods have elapsed, it may conclude that the server is inoperative. Alternatively, before arriving at this conclusion, the cluster manager  410 ,  415  may first attempt to communicate with the server to verify that the server is, in fact, inoperative. 
   Regardless of how the cluster manager  410 ,  415  determines that a server is inoperative, once it does, it attempts to reassign each of the client connections supported by the inoperative server to an operative server. Thus, as illustrated in  FIG. 4   b , if server  405  crashes while serving a stream  440  to a client  150 , the cluster manager  410  will reestablish a new stream  450  with the client  150  from a different streaming server  400 . Moreover, because client streaming data (such as the data illustrated in  FIG. 5 ) is continually updated at the client manager  410 , the new streaming server  400  will begin serving the client  150  at the same point in the stream (i.e., at the same media offset) at which the original server  405  became inoperative. In a live stream embodiment, there may be a slight loss of stream data while the connection is re-established. In addition, the new server  400  will know the exact streaming configuration (e.g., bit-rate, streaming format, . . . etc) required to stream data to the client  150 . 
   The foregoing reallocation mechanism may be employed regardless of whether the original stream was of a live/scheduled event  460  (e.g., a Webcast) or a previously-stored audio/video file. In one embodiment, each of the streaming servers  400 ,  405 , may be configured to buffer a predetermined portion of the streaming data for the live event  460 . Accordingly, if a server  405  crashes, the new server  400  assigned by the cluster manager  410  can begin streaming the event at the exact point (stored in the buffer) where the original server  405  became inoperative. 
   The cluster manager  410  may assign servers to handle the streams of the inoperative server based on a variety of factors. For example, in one embodiment, the assignment may be based on the relative load on each of the servers at a given point in time. The assignment may also be based on the type of content supported by the servers. For example, some servers may be configured to support only real time streaming protocol (“RTSP”) requests, while the other may be configured to support only active streaming format (“ASF”) requests. Various additional factors may be evaluated for the purpose of making an assignment while still complying with the underlying principles of the invention. 
   In one embodiment, after the various client streams are reassigned to new servers, the load balancer module  405  is notified of the new server assignments so that it can properly distribute new client requests based on server load. In one embodiment, the load balancer module  405  is a layer  4  switch, capable of directing client requests to a particular server based on the type of streaming service being requested by the client (e.g., based on a virtual IP address associated with the service) and/or the server. 
   As illustrated in  FIG. 6 , in one embodiment, cluster managers  610 - 612  and/or agents  620 - 622  from different POP sites  600 - 602  may communicate with one another. Accordingly, in this embodiment, if the server capacity at a particular POP site  602  has been reached when one of the servers at that site becomes inoperative, the client manager  612  may assign a server from a different site  600  to handle the client streams previously supported by the inoperative server. In this embodiment, cluster managers  610 - 612  and/or cluster agents  620 - 622  at different sites continually exchange data relating to client streaming connections. As such, a server from a completely different site  600  will begin streaming data to a client at the same point in the stream at which the previous server became inoperative, using the same set of streaming variables. 
   Embodiments of the present invention include various steps, which have been described above. The steps may be embodied in machine-executable instructions. The instructions can be used to cause a general-purpose or special-purpose processor to perform certain steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components. 
   Elements of the invention may also be provided as a machine-readable medium for storing the machine-executable instructions. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, propagation media or other type of media/machine-readable medium suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program which may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection). 
   Throughout the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. Accordingly, the scope and spirit of the invention should be judged in terms of the claims which follow.