Method and apparatus for improving data integrity during a router recovery process

An apparatus and method for enhancing data integrity during router recovery using dual-homed host configuration are disclosed. A process of routing resumption, in one embodiment, is able to recover or reset a network element (“NE”) such as a primary router from system failure. A first link configured to transmit data packets between the NE and a network device is reestablished. Upon reestablishing a second link configured to transmit data packets between the NE and other NEs, a network discovery process utilizing network reachability protocol is initiated to identify routing paths associated with the NE. A routing table in the NE is updated in accordance with the routing paths. A ready message is issued from the NE to the network device when the routing table is at least partially completed.

FIELD

The exemplary embodiment(s) of the present invention relates to communications network. More specifically, the exemplary embodiment(s) of the present invention relates to improve data integrity during a router recovery process.

BACKGROUND

A high-speed network environment typically includes network devices such as routers and bridges used for facilitating delivery of information packets and/or data traffic from source devices to destination devices. Information pertaining to the transfer of packet(s) through the network is usually embedded within the packet itself. Each packet traveling through one or more communications networks such as Internet and/or Ethernet can typically be handled independently from other packets in a packet stream or traffic. For example, each router which may include routing, switching, and/or bridging engines processes incoming packets and determines where the packet(s) should be forwarded.

A problem associated with a high-speed computing network is data (or packet) loss due to data connection(s) (or data link) and/or device failure. For example, when data packets are sent to a recipient, the recipient router may drop the data packets because the router is not ready to process and route the data packets. As such, data loss can occur when the link and router are not fully functional at the same time.

SUMMARY

A communication network capable of providing PWE to VRF network application using dual-homing protection for improving data integrity is disclosed. After a primary router is recovered from an earlier device failure, a first link configured to transmit data packets between the primary router and an access switch is reestablished. Upon reestablishing a second link which facilitates data transmission between the primary router and other network elements, a network discovery process using a network reachability protocol is initiated to identify routing paths associated with the primary router. A routing table in the primary router is subsequently updated in accordance with the routing paths. When the routing table, according to a predefined condition, is completed or at least partially completed, a ready message is issued by the primary router to the access switch. The ready message indicates that a switchover or a reversion of network service between primary router and backup router should take place.

DETAILED DESCRIPTION

Exemplary embodiment(s) of the present invention is described herein in the context of a method, device, and apparatus of improving network performance during a switchover from a backup router to a primary router using a delay-switching circuit.

Those of ordinary skills in the art will realize that the following detailed description of the exemplary embodiment(s) is illustrative only and is not intended to be in any way limiting. Other embodiments will readily suggest themselves to such skilled persons having the benefit of this disclosure. Reference will now be made in detail to implementations of the exemplary embodiment(s) as illustrated in the accompanying drawings. The same reference indicators will be used throughout the drawings and the following detailed description to refer to the same or like parts.

In the interest of clarity, not all of the routine features of the implementations described herein are shown and described. It will, of course, be understood that in the development of any such actual implementation, numerous implementation-specific decisions may be made in order to achieve the developer's specific goals, such as compliance with application- and business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be understood that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skills in the art having the benefit of embodiment(s) of this disclosure.

Various embodiments of the present invention illustrated in the drawings may not be drawn to scale. Rather, the dimensions of the various features may be expanded or reduced for clarity. In addition, some of the drawings may be simplified for clarity. Thus, the drawings may not depict all of the components of a given apparatus (e.g., device) or method.

The term “system” is used generically herein to describe any number of components, elements, sub-systems, devices, packet switch elements, packet switches, access switches, routers, networks, computer and/or communication devices or mechanisms, or combinations of components thereof. The term “computer” includes a processor, memory, and buses capable of executing instruction wherein the computer refers to one or a cluster of computers, personal computers, workstations, mainframes, or combinations of computers thereof.

IP communication network, IP network, or communication network means any type of network having an access network able to transmit data in the form of packets or cells, for example of ATM (Asynchronous Transfer Mode) type, on a transport medium, for example, the TCP/IP or UDP/IP type. ATM cells are the result of decomposition (or segmentation) of packets of data, for example, IP type, and those packets (here IP packets) comprise an IP header, a header specific to the transport medium (for example UDP or TCP) and payload data. The IP network may also include a satellite network, for example a DVB-RCS (Digital Video Broadcasting-Return Channel System) network, providing Internet access via satellite, or an SDMB (Satellite Digital Multimedia Broadcast) network, or a terrestrial network, for example a cable (xDSL) network or a mobile or cellular network (GPRS/EDGE, or UMTS (where applicable of the MBMS (Multimedia Broadcast/Multicast Services) type, or the evolution of the UMTS known as LTE (Long Term Evolution), or DVB-H (Digital Video Broadcasting-Handhelds)), or a hybrid (satellite and terrestrial) network.

Embodiments of the present invention illustrate a network system using dual-homed hosts to reduce packet-drop or black holing during a switchover of network service between a backup router and a primary router. After a primary router resumes its routing functionalities from an earlier system failure, a first link configured to transmit data packets between the primary router and an access switch is reestablished. Upon reestablishing a second link able to route data packets, a network discovery process using a network reachability protocol is initiated to identify routing paths associated with the primary router. A routing table in the primary router is subsequently updated in accordance with the routing paths. When the routing table is completed or at least partially completed in accordance with a predefined condition, a ready message is issued by the primary router to the access switch. Upon receipt of the ready message, the access switch switches network services from the backup router to the primary router.

FIG. 1is a block diagram100illustrating a computer network having a primary router and backup router organized in a dual-homing configuration in accordance with one embodiment of the present invention. Diagram100includes multiple cell sites102-103, a switching network104, multiple routers, and a Radio Network Controller (“RNC”)110. RNC110is further coupled with a Wide Area Network (“WAN”) and/or Internet170. Depending on the applications, RNC110may be coupled with other RNC or RNCs to enhance network management and capacities. In an alternative configuration, RNC110may be replaced with other network element(s) such as gateway(s) and router(s). It should be noted that the underlying concept of the exemplary embodiment(s) of the present invention would not change if one or more blocks (or elements) were added to or removed from diagram100.

Switching network104includes an access switch (“AS”)148, a primary router150, and a backup router152wherein AS148and routers150-152are configured to form a dual-homed or dual-homing redundancy network configuration. Note that AS148can be configured to be at edge and/or outside of switching network104and is coupled to one or more cell sites102-103via connections116. Switching network104may include additional network elements (“NEs”) and/or network management system (“NMS”) depending on the applications. Routers106-108, in one embodiment, can be edge routers and/or routers inside of switching network104. Switching network104, in one example, can be an IP and/or Multi Protocol Label Switching (“MPLS”) based circuit network which may operate at a layer of Open Systems Interconnection Basic Reference Model (“OSI model”). Network104may include a circuit switch block and a backhaul block for transferring information and/or various data traffic to and from network clients.

AS148, in one embodiment, is a managed edge system and/or management system capable of managing a network, connections, ports, and switching services. For example, AS148allows service providers' access networks at traffic aggregation points or cell sites. An advantage of employing AS148is that it provides scalable network solution between customer equipment (“CE”) and provider edge (“PE”) routers for data transfer. Virtual Private LAN Service (“VPLS”) provides Ethernet based multipoint to multipoint communication over the IP/MPLS network. The protocols, such as interior border gateway protocol (“iBGP”), MPLS, OSPF, and RSVP (resource reservation protocol), may be used as Layer 2 (L2) VPN-related applications. AS148, in one example, is capable of performing both access switching functions and router functions.

A router, for example, is an NE or network device capable of forwarding data packets across one or more communication networks in accordance with its routing mechanism such as a routing table. A router may be a microprocessor-controlled computing system which may be coupled to two or more data lines configured to direct data traffic through one or more communication networks. NE or network client, in one example, can include one or more routers, hubs, switches, hosts, base stations, and the like. A NMS, in one aspect, is a computer system or server including hardware and/or software used to monitor and control the network including various NEs. Diagram100, for example, includes router106-108which are capable of routing information between cell sites102-103and RNC110via switching network104.

A dual-homed redundancy host, network or gateway is, for example, situated between two interfaces to enhance data integrity or prevent data drop. Dual-homed redundancy, also known as dual-homing, provides two independent data paths for each dual-attached device. AS148, in one embodiment, is structured in a dual-homed redundancy configuration wherein a primary path142is used to connect AS148to primary router150and a secondary path144is used to connect AS148to a backup router152. Under normal conditions, AS148transmits data packets to and from primary router150via primary path142. In the event that path142or primary router fails, AS148switches its connection from primary router150to backup router152whereby AS148can continue network services via a backup route. When primary router150recovers from an earlier crash or failure, primary router150, in one embodiment, informs AS148to switch back from backup router152to primary router150in accordance with delay-switching circuit160and content in IP routing table162.

Primary and backup routers150-152, for example, are also interconnected by Interior Gateway Protocol (“IGP”)146for redundancy purposes. Similarly, routers106-108are also interconnected by IGP118. Each router, for example, includes functions of IP routing. Connections130-132are used to couple RNC110with routers106-108wherein connections130-132can be land line connections, wireless connections, or a combination of wired and wireless connections.

Cell site102, also known as a base station, includes a radio tower112, a computer126, and a server128, wherein radio tower112further includes a cellular phone120and a handheld device124connected via wireless communications. Base station or cell site102is capable of communicating with mobile devices such as cellular phone120and handheld device124via radio tower112. It should be noted that cell site102, not shown inFIG. 1, may include additional radio towers as well as other land switching circuitry. The cell stations such as cell sites102-103can be configured to support wireless communications as well as wired communications. Each cell site such as cell site102can be considered as a host and it is capable of maintaining a connectivity session such as a bidirectional forwarding detection (“BFD”) session with a destination router for continuously verifying the connectivity between the host and the router.

BFD is a network connectivity protocol used to authenticate or detect failures between two endpoints (i.e., a host and a master router). BFD is a short-duration for failure detection for path(s) between forwarding network elements including interfaces, data links, forwarding planes, and forwarding engines. A session is down if a BFD packet(s) is failed to receive. It should be noted that, instead of using BFD sessions, other connectivity protocols can also be used. For example, Open Shortest Path First (“OSPF”), Intermediate System to Intermediate System (“IS-IS”), and/or any other protocols complying IEEE 802.1ag can be used.

During an operation, after activating backup router152for routing services, primary router150begins a recovery process to recover itself from inactive status to active status. Once primary router150is up and is able to resume network service, primary router150reestablishes label-switched path (“LSP”) using RSVP via connection142between router150and AS148. Similarly, LSPs are also reestablished for other connections such as connections146-147. After LSPs are reestablished, primary router150begins to learn routing paths by activating a network reachability protocol such as BGP or iBGP. Primary router150activates delay-switching circuit160which monitors BGP sessions and status of routing paths. IP routing table162within primary router150is updated as additional IP paths are learned or discovered. Once the IP routing table162is completed or partially completed in accordance with a predefined condition, an Ethernet Pseudo Wire Emulation (“PWE”) between primary router150and AS148is activated and PWE failure signal is cleared. After receiving a ready message from primary router150, AS148reverts network services from backup router152to primary router150.

It should be noted that the predefined condition(s) can be set for users, providers, or network administrator(s). For example, the predefined condition may set to a minimal number of routing paths which is sufficient for a router to process and route an incoming packet. The predefined condition can also identify which condition to use. For example, the predefined condition may indicate use IP routing table or use IP routing table plus BGP session(s).

In addition to Multi-protocol tunnels, Multi-Protocol Label Switching, Generic Routing Encapsulation (“GRE”), and IPSec (Internet Protocol Security), PWE is also able to emulate VC (Virtual Circuit) channels inside tunnels with multicast features. For example, PWE is able to emulate Ethernet, Frame Relay, ATM, TDM (Time Division Multiplexing), SONET/SDH (Synchronous Optical NETworks/Synchronous Data Hierarchy) and other services across an MPLS/IP network.

An advantage of providing a redundant dual-homing networking solution for L2 PWE is to improve reliability of IP/IP-VPN services. For example, monitoring PWE status, route paths status, and/or status of routing protocols by delay-switching circuit160can reduce packet drop or data loss during a switchover or revision of network service.

FIG. 2is a block diagram200illustrating an exemplary computer network having multiple network elements and access switches (“AS”) in accordance with one embodiment of the present invention. Diagram200includes ASs202-204, routers206-212, RNC214, and connections220-238, wherein connection220-238are used to interconnect ASs, routers, and RNC. AS202is further coupled to a network device216such as a base station via a wired or wireless communication network. In one embodiment, AS202and router206and212are formed and/or connected in a dual-homing network configuration wherein router206is the primary router and router212is the backup router. It should be noted that the underlying concept of the exemplary embodiment(s) of the present invention would not change if one or more blocks (or circuits) were added to or removed from diagram200.

The dual-homing redundancy network includes an AS202, a primary router206, and a backup router212. In one embodiment, AS202, which may be edge router or switch, is configured to control connections of network systems situated at edge of a communication network. For example, AS202controls connections or ports coupled to router206, router212, and AS204. Backup router212is configured to provide an alternative (or backup) routing service for the network device when primary routing service is not available. For example, primary routing service may not be available when the primary router is down. Alternatively, primary routing service is out of service because connection224is down. Primary router206, in one embodiment, includes a delay-switching circuit which provides a delay of switchover to the primary router because the primary router is not ready even though the primary link is up. For example, a router is not able to route any incoming packets when it is still learning the routing paths and updating its routing table.

The delay-switching circuit is also capable of delaying a switchover of network service in response to media access control (“MAC”) addresses in the routing table. Furthermore, the delay-switching circuit is further capable of delaying a switchover of network service in response to selected LSPs. It should be noted that PWE240is established between AS202and primary router206to provide primary routing service and PWE242is established between AS202and backup router212to provide the backup or alternative routing service.

L2 PWE, as shown in diagram200, connects to IP-VPN network providing network service for PWE to VRF (virtual routing and forwarding) network applications and services. Routers206-212, in one embodiment, use iBGP to learn and identify neighboring NEs or peers. iBGP is a network discovery or reachability tool to identify or map its peer(s) connections and/or paths. While primary path224facilitates Ethernet PWE activity240, backup path238facilitates data transmission through Ethernet PWE activity242between AS202and backup router212. It should be noted that the backup path can also be formed through connections220-222via AS204.

Under the normal condition, primary router206handles and transmits data, data packets, or data stream via data path244-246between AS202and router208via PWE240and connection228. If the destination is RNC214, data or packets travel via data path252from router208to RNC214via connection234. If primary router206or PWE240fails, backup router212takes over the data transmission between AS202and router208via PWE242and connections230-232. If the destination is RNC214, packets travel from router208to RNC214via connection234. Alternatively, packets can also travel from router210to RNC214via connection236.

Diagram200illustrates a circuit network having attachment circuits for providing IP/IP-VPN service which includes L2 interface(s), such as VLAN, PPP, ATM VC, Frame Relay DLCI, and the like. To reduce packet black holing, the delay-switching circuit, in one embodiment, monitors status such as PWE end-to-end up-or-down status and attachment circuit (“AC”) status, and delays a switchover(s) in accordance with the observed status. Table 1, shown below, illustrates an exemplary list of conditions and/or status to be monitored before a switchover takes place.

TABLE 1No.Statusa)route or a set of IP routes within global or local VRF (present ornot present)b)status of selected routing protocols (iBGP up/down)c)presence of MAC addresses in the routing tables (present/notpresent)d)selected LSP (example up or down)e)status of selected standard connectivity verification protocolsdefined for various technologies (i.e., BFD, 802.1ag) that canperform functions on interface(s) that is not necessary AC of PWE

During an operation, the delay-switching circuit, in one embodiment, is able to issue a switching or reverting notice to AS202based on the observed status as shown in Table 1. For instance, when the routing table contains a complete set or a partially complete set of IP route paths, the delay-switching circuit sends a ready message to AS202informing AS202to perform a switchover. Note that the partially complete set of IP route paths is a predefined or predetermined portion of IP route paths that is sufficient to route incoming data to its destination based on the available route paths. Upon receipt of the ready message, AS202reverts network service from backup router212to primary router206. It should be noted that when a switch or reversion of network service is based on status of PWE and status of selected routes (and/or iBGP session(s)), primary router such as router206is ready to route coming data as soon as the reversion takes place.

An advantage of using PWE using dual-homing or dual-homed network protection can enhance reliability of the communication network.

FIG. 3is a block diagram300illustrating an exemplary process of rerouting data through a backup path upon identifying primary link failure in accordance with one embodiment of the present invention. Diagram300, which is similar to diagram200shown inFIG. 2, includes ASs202-204, routers206-212, RNC214, and connections220-238, wherein connection220-238are used to interconnect ASs, routers, and RNC. AS202is further coupled to a network device216such as a base station via a wired or wireless communication network. In one embodiment, AS202and router206and212are formed and/or connected in a dual-homing network configuration wherein router206is the primary router and router212is the backup router. It should be noted that the underlying concept of the exemplary embodiment(s) of the present invention would not change if one or more blocks (or circuits) were added to or removed from diagram300.

When the primary link or link224fails as numeral302indicated during PWE to VRF network operation, a backup router such as router212automatically takes over the responsibility of network service or routing service between AS202and router206. For example, upon identifying link failure302, backup router212activates Ethernet PWE or PWE242and takes over the network routing task. After PWE242is activated, a new data path304is established via links220-222and226. Alternatively, new data path304can also be formed through links238and226. While primary router206is able to continue receiving and processing packets or data streams from AS202through backup router212and link226, primary router206begins to recover or repair the down link such as link224.

FIG. 4is a diagram illustrating an exemplary process of rerouting data through a backup path upon identifying node or primary node failure in accordance with one embodiment of the present invention. Diagram400, which is similar to diagram200shown inFIG. 2, includes ASs202-204, routers206-212, RNC214, and connections220-238, wherein connection220-238are used to interconnect ASs, routers, and RNC. In one embodiment, AS202and router206and212are formed and/or connected in a dual-homing network configuration wherein router206is the primary router and router212is the backup router.

When the primary node or primary router206fails as numeral402indicated during a PWE to VRF network operation, a backup router such as router212automatically takes over the responsibility of network service or routing service between AS202and router208. When primary router206fails, internal iBGP session goes down and primary node crashes or goes down. After primary node goes down, RSVP LSPs declare down over links224-228and link failures404-408occur. PWE240subsequently goes down and data paths244-246are removed upon detecting iBGP session time out and/or multiprotocol (mp)-BGP session ends.

Upon identifying router failure402and link failures404-408, backup router212activates Ethernet PWE or PWE242and takes over the network routing task for AS202. New data paths248and450are established via links238and230-232. The routing service or network service between AS202and RNC210, for example, is rerouted through data paths248,450, and252. Alternatively, the new data path between AS202and RNC210can also be established via links238,230, and236through routers212-214. While backup router212routes data packets or data streams between AS202and RNC210, primary router206begins to recover from router failure402and link failure404-408.

FIG. 5is a diagram illustrating a recovery process of a node failure using a delay-switching circuit to prevent black holing traffic in accordance with one embodiment of the present invention. Diagram500, which is similar to diagram400shown inFIG. 4, includes ASs202-204, routers206-212, RNC214, and connections220-238, wherein connection220-238are used to interconnect ASs, routers, and RNC. While diagram400shown inFIG. 4illustrates a process of performing a switchover from primary router206to backup router212due to device failure, diagram500illustrates a process of performing a switchback or a reversion from backup router212to primary router206once primary router206is recovered from the device failure.

When a primary node or primary router206in the PWE to VRF network operation recovers or resumes from an earlier failure502, a delay-switching circuit160of primary router206delays a switchback or reversion of network routing from backup router212in accordance with one or more predefined conditions. The predefined conditions, as discussed earlier, includes, but not limited to, routing paths listed in routing table162, iBGP session/route status, mp-BGP status, and the like. To prevent black holing traffic during the switchback or reversion of network routing, delay-switching circuit160is able to reduce and/or prevent scenario of black hole traffic. The black holing traffic or black hole means when data traffic or data packet is discarded or dropped during a transmission to a recipient or destination without informing the source or destination that the data traffic did not reach its intended recipient.

Upon resumption of primary router206from failure502, LSPs are reestablished over links224-228in a sequentially or substantially simultaneous depending on the number of routers. After link228is up, iBGP session and/or monitored route is up and running. Mp-iBGP is subsequently reestablished between routers206-208. In one embodiment, delay-switching circuit160activates network discovery process using mp-iBGP to identify paths and/or alternative paths such as the paths between router206and RNC210. The identified paths, for example, are stored in routing table162. When routing table162is completed or partially completed depending on the predefined conditions, PWE240is reestablished and PWE failure signal is cleared. Once Ethernet PWE240is activated, delay-switching circuit160sends a ready message to AS202indicating that PWE240and primary router206are ready to resume network routing services. The network service is reverted or switched from backup router212to primary router206. Upon reestablishing data paths244-246, Ethernet PWE242is deactivated and backup data paths248-450are removed.

For a circuit switching network, a link or a channel can be established between multiple nodes and/or terminals to facilitate network communication. It should be noted that a link or a circuit or a channel acts as an electrical circuit physically connecting two nodes, two NEs, or two network devices. During a network discovery process, every NE as well as network circuit needs to be discovered and/or initialized by NMS or NMSs before each NE can access the network. Network discovery process can take time and resources for each NMS to complete a network configuration and/or a network discovery process. In another example, a network element discovery may be required when a primary router switches over to a backup router or vice verse.

An advantage of using the delay-switching circuit is that PWE does not revert until the primary router is ready whereby black holing traffic can be reduced. It should be noted that primary router206is able to select peer(s) or route(s) status as AC status in response to iBGP sessions. Primary router206, in one embodiment, is able to map PWE status message such as up-or-down in order to prevent potential black holing scenarios.

FIG. 6is a logic diagram600illustrating an exemplary process of control flow during a PWE to VRF network operation having dual-homing protection in accordance with one embodiment of the present invention. The process detects an error at the primary router at block602. The error, which can be an operational error or system error, causes the system or the primary router to crash. When the primary router is shutting down due to the error, the network service is switched from the primary router to the backup router at block604. After the primary router is down, PWE between AS and primary router is torn down at block606. Once LSP is down, the primary router is logically removed. The Ethernet PWE between the backup router and AS is activated and new routing path(s) is established at block608. While the backup router provides network service such as routing data to and from AS, the primary router begins to repair and/or remove the error(s).

When the primary router including its control card are activated at block610, LSP between primary router and other network devices including neighboring routers or peers are reestablished at block612. The process proceeds to block616if the BGP session is not up yet. Otherwise, the process moves to block622. LSPs between the primary router and the secondary routers are reestablished at block616. While BGP session is activated and monitored at block618, the routing table, at block620, may be completed or at least partially completed in response to a network discovery process. The process subsequently proceeds to block614to exam the status of BGP session. At block622, PWE between primary router and AS is reestablished and PWE failure signal is cleared. Upon advertizing routing paths at block624, the network service is reverted from backup router to primary router at block626.

The exemplary aspect of the present invention includes various processing steps, which will be described below. The steps of the aspect may be embodied in machine or computer executable instructions. The instructions can be used to cause a general purpose or special purpose system, which is programmed with the instructions, to perform the steps of the exemplary aspect of the present invention. Alternatively, the steps of the exemplary aspect of the present invention may be performed by specific hardware components that contain hard-wired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

FIG. 7is a flowchart700illustrating a router recovery process which reduces data loss during a switchover between a backup router and a primary router in accordance with one embodiment of the present invention. At block702, a recovery process using dual-homing protection is capable of recovering a first NE from an earlier device failure. The first NE, in one aspect, is primary router. Upon detecting an error associated with the first NE, the first NE is shutting down for error recovery. The network service is switched from the first NE to a backup NE wherein the backup NE takes over the routing task from the first NE. Upon removing the error, the first NE resumes routing functions and changes its status from inactive to active.

At block704, a first link configured to transmit data packets between the first NE and a network device is reestablished. In one embodiment, the process restores a connection for data transfer between a first router and an access switch. While an NE can be a router, the access switch may be an edge router or switching hub.

At block706, a second link configured to transmit data packets is reestablished between the first NE and a second NE. The process, in one example, is able to repair a logical network connection between the first router and a second router using RSVP LSP.

At block708, a discovery process using network reachability protocol such as iBGP is initiated to identify routing paths associated with the first NE. For example, the first NE learns alternative routing paths between the first NE and other nodes.

At block710, a routing table in the first NE is updated in accordance with the routing paths discovered from the network discovery process. For example, the discovered or identified routing paths are recorded in the routing table. The content of routing table will be used for routing.

At block712, a ready message from the first NE is issued to the network device when the routing table is at least partially completed. In one embodiment, the process informs the access switch to revert from a backup NE to the first NE for the subsequent network services. A data packet such as an incoming data stream is subsequently sent from the network device to the first NE. The data packet is routed or forwarded to its destination in accordance with the routing table.

FIG. 8is a flowchart800illustrating an alternative embodiment of a recovery from a device failure during a switchover between a backup router and a primary router in accordance with one embodiment of the present invention. At block802, a recovery process reboots a control card of a primary NE from an earlier system failure. Before system failure, the process is able to identify an error during the network communication between the primary NE and an access switch and subsequently, the primary NE is temporary inactivated whereby an error recovery process can be executed. The network service is switched from the primary NE to a backup NE. Once the primary NE is up, a first link capable of transmitting data between the primary NE and the access switch is reestablished. A second link configured to transmit data packets between the primary NE and a secondary NE is also resumed.

At block804, a network discovery process is activated to identify routing paths between the primary NE and a destination node. In one example, a routing map is created during the discovery process.

At block806, a routing table resided in the primary NE is reconstructed or reloaded in accordance with discovered or identified routing paths. The routing table will be used for subsequent packet routing.

At block808, the process reverts routing service from a backup NE to the primary NE when the routing table is at least partially completed. Upon receiving a data packet from the access switch to the primary NE, the process routes the data packet to the RNC in accordance with the routing table. Before reversion of the routing service, a ready message is sent to an access switch in accordance with IP routes of virtual routing and forwarding (“VRF”). In one embodiment, the process is capable of sending a ready message to an access switch in accordance with MAC addresses in the routing table. Alternatively, the process sends a ready message to an access switch in accordance with selected LSPs in the routing table.