Abstract:
A system, method, and apparatus for active-active 1+1 redundancy for a plurality of bi-directional communication modules of a distributed point to multi-point communications network includes pairing active modules into redundancy groups for providing backup in the event of a failure. Each module is active during normal operation and may also act as a backup for at least one other module in the event of a failure. In the event of a failure, the paired operational module takes over the failed module&#39;s service domain, and the operational module continues to meet the needs of its own service domain, thus eliminating the need for passive backup modules.

Description:
CROSS REFERENCE TO RELATED APPLICATION  
       [0001]    This application is related to the following additional commonly assigned, co-pending application: application Ser. No. 09/632,649, entitled, “System And Method For Active-Active Redundant Cable Modem Service At Head End,” filed Aug. 4, 2000. 
     
    
     
       TECHNICAL FIELD  
         [0002]    The present invention is related to a bi-directional point to multi-point distributed communications system, and more particularly to a method and apparatus for providing redundancy for cable modem termination system modules used in a system for delivering data service to end-users via cable, fiber optic or hybrid cable fiber networks.  
         BACKGROUND INFORMATION  
         [0003]    Cable operators today are deploying cable modem technology that allows subscribers to access the Internet over the same wires that deliver television signals, at speeds 100 times faster than standard V.90 telephone modem technology and without waiting for a dial-up connection.  
           [0004]    In 1996, several cable operators commissioned the development of the data over cable service interface specification (DOCSIS) with the objective of establishing a single specification for equipment. DOCSIS covers all operational elements used in delivering dat service to end-users, including service provisioning, security, data interfaces and radio frequency interfaces (RFI).  
           [0005]    The architecture of the DOCSIS RFI consists of three major components: the cable modem termination system (CMTS), installed at the head end, or main facility of the cable operator, the hybrid fiber coaxial (HFC) cable network wiring infrastructures; and the customer cable modems, installed at the customers&#39; premises.  
           [0006]    Cable modems translate digital data packets into radio frequency signals that are mapped into an unused 6 MHz television channel slot and broadcast to all homes by the CMTS modules at the HFC node. The signal is received in homes by any cable modems on the local area network segment. The downstream signal can be mapped anywhere in the downstream cable spectrum, from 91 MHz 857 MHz.  
           [0007]    The CMTS modules of the cable operator&#39;s facility receive signals from the downstream cable modems on a different set of upstream frequencies in the 5 MHz to 42 MHz band. The throughput of this channel is variable, based on the quality of the upstream channel. Throughput varies from 160 kbps 10 Mbps. The DOCSIS architecture provides for one downstream channel to send signals to all cable modems, which may broadcast return signals on several different, nonoverlapping frequencies.  
           [0008]    The cable modem translates the downstream radio frequency signal into packets, determines if the packets are destined for that particular cable modem and, if so, it sends the packets along to the computer or a local area network on the client side of the cable modem. This network connection may be {fraction (10/100)} Mbps Ethernet, universal serial bus (USB) or PCI.  
           [0009]    A cable operator facility typically includes one or more head end switches, which include a number of CMTS modules, typically 12. Reliable operation of the head end CMTS modules is essential to providing uninterrupted service to customers. Redundancy or backup is a key component to providing highly reliable service. In the event that one or more of the CMTS cards does fail, then the outage time to the customers should be minimized.  
           [0010]    Perhaps the most important components to redundancy at the headend are the CMTS modules because of their role in communicating with cable modems at the customer premise and vice versa. Pure one-to-one, active-passive redundancy allows a hot standby “passive” CMTS to take over if the active CMTS fails. This type of redundancy has been used for DOCSIS 1.0-based cable modem systems because DOCSIS 1.0 provides no direct support for CMTS failure. DOCSIS 1.1 permits cable modems to be aware of a backup CMTS and gives instructions about how to locate the backup if necessary. Thus, a passive backup may serve more than one active CMTS. The problem with passive redundancy however is that it requires passive backup CMTS modules, i.e., CMTS modules that function only to provide backup to active online modules. The use of passive backup CMTS modules adds significantly to the cost of the headend switch.  
           [0011]    The present invention addresses the foregoing problems, at least in part, as well as other problems, which will be understood by reading and studying the following specification. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0012]    [0012]FIG. 1 is a block diagram of downstream operation of a CMTS redundant system in normal operation, according to an example of the present invention.  
         [0013]    [0013]FIG. 2 is a block diagram of upstream operation of a CMTS redundant system operating in response to a CMTS failure condition, according to an example of the invention.  
         [0014]    [0014]FIG. 3 is a block diagram of upstream port configuration of a CMTS redundant system in normal operation according to an example of the invention.  
         [0015]    [0015]FIG. 4 is a block diagram of upstream port configuration of a CMTS redundant system in response to a CMTS failure condition according to an example of the invention.  
         [0016]    [0016]FIG. 5 is a block diagram of downstream port configuration of a CMTS redundant system in response to a CMTS failure condition according to an example of the invention.  
         [0017]    [0017]FIG. 6 is a block diagram of an alternative upstream port configuration of a CMTS redundant system in normal operation according to an example of the invention.  
         [0018]    [0018]FIG. 7 is a block diagram of an alternative upstream port configuration of a CMTS redundant system in response to a CMTS failure condition according to an example of the invention.  
     
    
     DETAILED DESCRIPTION  
       [0019]    In the following detailed description of the preferred embodiments, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific preferred embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that logical, mechanical and electrical changes may be made without departing from the spirit and scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the claims.  
         [0020]    [0020]FIG. 1 shows a downstream configuration of a representative CMTS redundancy group or pair  100  according to the present invention. Although only two CMTS modules are shown, a typical headend switch may include 5 or more pairs of CMTS modules as well as other related hardware, depending on design requirements. CMTS modules A and B are identical from a hardware standpoint. CMTS modules A and B each include two outputs,  102   a  and  104   a  of CMTS module A and  102   b  and  104   b  of CMTS module B. Outputs  102   a  and  104   a  are downstream RF outputs of the transmitter of module A and outputs  102   b  and  104   b  are downstream RF outputs of the transmitter of module B. Output  104   a  functions as the primary output of the transmitter of module A and output  102   a  serves as the secondary output of module A when module B fails. Output  104   b  functions as the primary RF output of the transmitter of module B and output  102   b  serves as the secondary output of module B when module A fails. The primary output  104   a  of module A is merged with the secondary output  102   b  of module B through RF combiner  120 . RF combiner  120  feeds the downstream signals to service areas  122 ,  124  and  126  of the cable modem system. The primary output  104   b  of module B is merged with the secondary output  102   a  of module A through RF combiner  118 . RF combiner  118  feeds the downstream signals to service areas  128 ,  130  and  132  of the cable modem system. During normal operation, modules A and B use their primary outputs only. When one module in a redundancy pair fails, the other module enables its secondary output so that its output signal is provided to both primary and secondary service areas.  
         [0021]    [0021]FIG. 2 shows an upstream configuration of a CMTS redundant system  100  according to the present invention. Upstream redundancy for module A is provided as follows. An upstream link from service area  122  passes through splitter  134  before it is provided to input port  116   a  of module A. Splitter  134  also feeds an upstream link from service area  122  to input port  110   b  of module B. An upstream link from service area  124  passes through splitter  136  before being provided to input port  114   a  of module A. Splitter  136  also feeds an upstream link from service area  124  to input port  108   b  of module B. An upstream link from service area  126  passes through splitter  138  before it is provided to input port  112   a  of module A. Splitter  138  also feeds an upstream link from service area  126  to input port  106   b  of module B.  
         [0022]    Upstream redundancy for module B is provided as follows. The upstream link from service area  128  passes through splitter  140  before it is provided to input port  116   b  of module B. The upstream link from service area  128  is also provided by splitter  140  to input port  110   a  of module A. The upstream link from service area  130  passes through splitter  142  before it is provided to input port  114   b  of module B. The upstream link from service area  130  is also provided by splitter  142  to input port  108   a  of module A. The upstream link from service area  132  passes through splitter  144  before it is provided to input port  112   b  of module B. The upstream link from service area  132  is also provided by splitter  144  to input port  106   a  of module A.  
         [0023]    Each one of the upstream receivers of CMTS modules A and B may be configured to receive signals from any one or more of the input ports. Thus, modules A and B may be configured to accept signals from primary service areas in normal operating mode and both primary and secondary service areas in failover mode of operation. For example, module A may accepts signals from its own service areas  122 ,  124  and  126  and also accepts the signals from CMTS module B&#39;s service areas  128 ,  130  and  132 .  
         [0024]    In a normal operation mode, either multiple upstream channels or a single upstream channel may be mapped to an input port. If multiple upstream channels are mapped to a single input port, each channel can be configured to use a different frequency. For example, on input port  116   a , an upstream channel from service area  122  may be configured to use a frequency of 20 MHz and another upstream channel from service area  122  may be configured to use a frequency of 31 MHz, for both normal operation and failover operation.  
         [0025]    [0025]FIG. 3 shows one example of upstream port configuration in a normal mode of operation. Each upstream receiver is configured to receive a different upstream channel. In this example, the channels are designated as channels  1  through  6 . Upstream receivers  156   a  (channel  1 ) and  154   a  (channel  2 ) are mapped to port  116   a , upstream receivers  152   a  (channel  3 ) and  150   a  (channel  4 ) are mapped to port  114   a  and upstream receivers  148   a  (channel  5 ) and  146   a  (channel  6 ) are mapped to port  112   a  of CMTS module A. Similarly, with regard to module B, upstream receivers  156   b  (channel  1 ) and  154   b  (channel  2 ) are mapped to input port  116   b , upstream receivers  152   b  (channel  3 ) and  150   b  (channel  4 ) are mapped to input port  114   b  and upstream receivers  146   b  (channel  5 ) and  148   b  (channel  6 ) are mapped to input port  112   b . This approach to port mapping maximizes the capacity of the modules, since all upstream receivers are used.  
         [0026]    [0026]FIG. 4 shows one example of upstream port configuration in a failover mode of operation. In this example, module A has failed and all upstream traffic has been switched over to module B. Module B must now provide coverage as follows. Upstream Channel  1 , which used port  116   b  under normal operation, continues to use port  116   b . Cable modems in service area  128  that used upstream Channel  1  under normal conditions are unaffected. Upstream Channel  2 , which used port  116   b  under normal operation, now uses port  114   b . Cable modems in service area  128  that normally used upstream channel  2  switch over to upstream Channel  1 . Upstream Channel  3 , which used to port  114   b  under normal operation, now uses port  112   b . Cable modems in service area  130  that normally use upstream Channel  3  switch over to upstream Channel  2 . Upstream Channel  4 , which used port  114   b  under normal operation, now uses port  110   b , providing service to cable modems in service area  122  that were previously served by the failed module. Cable modems in service area  130  that normally use upstream Channel  4  switch over to upstream Channel  2  on port  114   b . Upstream Channel  5 , which used port  112   b  under normal operation, now uses port  108   b  providing service to cable modems and service area  124  that were previously served by the failed module. Cable modems and service area  132  that normally use upstream Channel  5  switch over to upstream Channel  3  on port  112   b . Upstream Channel  6 , which used upstream physical port  112   b  under normal operation, now uses upstream physical port  106   b , providing service to cable modems and service area  126  that were previously served by the failed module. Cable modems in service area  132  that normally use upstream channel  6  switch over to upstream Channel  3  on port  112   b.    
         [0027]    Unlike upstream ports, downstream port mappings are not reconfigured for a failover. FIG. 5 shows a sample redundant downstream configuration in the event of a failover. Downstream port  104   b  of module B provides service to service areas  128   130  and  132 . Downstream port  102   b  provides service to service areas  122 ,  124  and  126  which were previously served by the failed module.  
         [0028]    Various other redundant port configurations are possible depending on system requirements. FIG. 6 shows one example of a redundant upstream port configuration designed to meet the needs of a high priority data or voice over IP application. The configuration of FIG. 6 maps and single upstream channel to each port. While this approach to port mapping does not maximize the capacity of the module under normal conditions, since not all upstream channels are used, it provides for at least some of module capacity (50 percent) in the event that the other module in the group fails. It also assures that cable modems on upstream channels and  1   2  and  3  will continue to operate when the other module fails. FIG. 7 shows a sample redundant upstream configuration for a high-priority data or voice over Internet application in the event of a failover. Upstream receiver  156   b  (channel  1 ), which used upstream port  116   b  under normal operation continues the use port  116   b . Cable modems and service area  128  the used upstream Channel  1  under normal conditions are unaffected. Upstream receiver  154   b  (channel  2 ), which used upstream physical port  114   b  under normal operation continues to use port  114   b  cable modems and service area  130  that used upstream channel  2  under normal conditions are unaffected. Upstream receiver  152   b  (channel  3 ), which used upstream physical port  112   b  under normal operation continues to use port  112   b . Cable modems in service area  130  that used upstream Channel  3  under normal conditions are unaffected. Upstream receiver  150   b  (channel  4 ) on port  110   b  provides service to cable modems in service area  122  that were previously served by the failed module. Upstream receiver  148   b  (channel  5 ) on port  108   b  provides service to cable modems and service area of  124  there were previously served by the failed module. Upstream receiver  146   b  (channel  6 ) on port  106   b  provides service to cable modems and service area  126  set were previously served by the failed module.  
         [0029]    In order to determine whether a CMTS module is working properly, in one example, the CMTS modules may communicate directly with each other by sending a signal that indicates the unit is functioning normally to the other unit in the redundancy pair. This signal, may be a simple “heartbeat” that is sent periodically, for example, every second, to let the other card know that everything is normal and no backup services are required. Alternatively, the signal may include telemetry or failure codes to more particularly identify the nature and extent of the failure. In the event that the signal ceases or indicates a condition other than normal, the backup module may take over immediately for the failed module (failover), perform further diagnostics such as sending a heartbeat request, and/or attempt to bring the failed module back online by rebooting, for example.  
         [0030]    Another way to determine whether a failover event has occurred is for each module to provide a signal to a controller which will supervise the failover process to ensure that there is an appropriate response sequence to minimize downtime.  
         [0031]    In order to provide an optimal backup operation according to the present invention that is compatible with the DOCSIS protocol, the cable modems in the affected service areas need to have a downstream and an upstream signal to maintain connectivity with the CMTS. The problem of maintaining connectivity is presented both during the failover/takeover process and during the recovery/giveback process.  
         [0032]    Timing for substituting the downstream signal for a failed module must consider requirements of the DOCSIS protocol. When a cable modem loses downstream synchronization with the CMTS, the DOCSIS protocol specifies that the cable modem should reset and attempt to establish connectivity. The exact way this occurs depends on the version of DOCSIS. DOCSIS 1.00 cable modems were originally specified to immediately reset and attempt to re-establish connectivity is downstream synchronization were lost. Changes to the DOCSIS specification, to support more reliable system operation overall, now specify that cable modems should wait between 30 and 35 seconds before reset. This added delay is further complicated by variability in transmission time between the CMTS and each cable modem due, for example, to distance, ambient conditions, wiring, differences in the design and manufacture of cable modems, and other like considerations. Several approaches are available to minimize the loss of connectivity.  
         [0033]    Whenever a CMTS module loses upstream signals from a cable modem, the CMTS is specified, by the DOCSIS protocol, to terminate the ranging process. The ranging process is the mechanism used to ensure correct power and frequencies are selected when the cable modem transmits data upstream. Loss of the ranging opportunities is interpreted by the cable modem as a loss in connectivity. Thus, the cable modem will reset when it detects that ranging opportunities are no longer available.  
         [0034]    The ability to internally combine and split upstream signals from the Cable Modem network into the CMTS greatly reduces expense, effort and cabling. However, it is possible to configure normal and failover port maps such that more than one CMTS module&#39;s cable modems are affected by a CMTS failure and recovery. For example, port maps may be defined such that a failed module results in almost all the cable modems losing upstream connectivity and resetting. Only Cable Modems on upstream channel  1  and port  1  will remain connected during the takeover (failover) process. The problem is not in the configuration, rather, the problem is in the change between normal and failover port maps. A mechanism is needed to instruct cable Modems connected to the good (takeover) module to move upstream channels before a change is made to the port maps. This “Upstream Channel Change” or “Dynamic Channel Change” is a standard mechanism in the DOCSIS protocol and permits the cable modem to remain connected during a change to the upstream port mappings.  
         [0035]    One approach to address control of downstream transmission and upstream reception when the CMTS recovers or fails over is to create an independent state machine or controller within the CMTS to handle the timing whenever a failover or recovery takes place. A first such state machine may be used to determine the health of a CMTS module&#39;s redundancy peer. Such a redundancy protection switch (RPS) state machine maintains information about the status of the module and its peer. As such, each module knows what and how the other module is doing. Timing of takeover and giveback operations may be configured by an operator to account for differences in modems so that a takeover or recovery does not conflict with a change in the upstream port mappings.  
         [0036]    A second state machine (IntfPortMgr) may be used to control the status of the downstream transmitter and upstream port mappings. The RPS and IntfPortMgr are designed to be independent and cooperative. This cooperation permits user configurable control of the downstream transmitter and upstream receivers. As such, downstream transmission during a failover (or recovery) is governed by a user configurable parameter to provide optimal interoperability with different DOCSIS Cable Modem types. Similarly, upstream channel change is provided during the failover/takeover and recovery/giveback operations. This permits the good CMTS module to maintain connectivity with the cable modems during a change in port mappings. For example, when upstream channel  1  and  2  are mapped to physical port  1 , any change (i.e., takeover or giveback) that forces channel  2  to a different port will result in Cable Modems on channel  2  losing connectivity. This can be avoided by first instructing the Cable Modems to move to channel  1  and then changing the port mapping for channel  2  to use a port other than number  1 .  
       Conclusion  
       [0037]    A system, method, and apparatus for active-active 1+1 redundancy for a plurality of CMTS modules has been detailed. Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that any arrangement, which is calculated to achieve the same purpose, may be substituted for the specific embodiment shown. This application is intended to cover any adaptations or variations of the present invention. Therefore, it is manifestly intended that this invention be limited only by the claims and the equivalents thereof.