Redundant gateway system for device level ring networks

Multiple gateway devices communicating between a device level ring (DLR) network and a spanning tree (ST) network may be provided a gateway protocol that cooperatively ensures that only a single gateway is active at a given time. This cooperation may be effected by the transmission of advertise messages by gateways, the advertise messages holding precedence values so that only a single gateway having a highest precedence value is active at a given time. Loss of the advertise messages may trigger a gateway held in a backup state to assume an active gateway role.

BACKGROUND OF THE INVENTION

The present invention relates to data networks suitable for industrial control and in particular to gateways communicating between a spanning tree (ST) network and device level ring (DLR) network.

Networks used for communication among industrial controllers differ from standard networks in that they must operate to communicate data reliably and within predefined time limits among network devices that control equipment. A bounded response time may be provided by communication protocols that reserve network bandwidth and schedule messages. Network reliability may be provided by the introduction of redundant network components.

Many computer networks provide for automatic “repair” of the network in the event of network device failure by switching between redundant components. These protocols can take a relatively long time to reconnect the network (as much as 30 seconds) and thus are unacceptable for industrial control networks where the controlled process often cannot be undirected during this period without serious consequences.

High-speed correction for network failure in an industrial control environment can be obtained by connecting network devices in a device level ring (DLR) where the ring network topology presents redundant paths (along the ring in two opposite directions) between any two devices. Normally the ring is “open” at a supervisor device for all standard data and thus operates in a normal linear topology for most data messages. The supervisor may send out “beacon” frames in both directions on the ring on different ports which are received back at the opposite port to indicate the integrity of the ring. If the ring is broken by device or media failure outside of the supervisor, the supervisor rejoins the ends of the ring at the supervisor to restore a continuous linear topology with the ring now separated by the failed component rather than the supervisor. Changes in the state of operation of the supervisor from “separated” to “joined” may be transmitted to the other nodes using notification frames so that these nodes can rebuild their MAC address routing tables used to associate a port with a destination address.

The error detection time of such ring systems can be quite fast, limited principally by the transmission rate of the beacons (every several microseconds). This rate defines the maximum time before which an error is detected and the ring may be reconfigured.

It is often desired to connect a DLR network with other networks, for example, those associated with devices that do not require the benefits of the DLR network topology. Such networks may permit a more flexible device interconnection, facilitated by a “spanning tree protocol” (STP) that detects and eliminates possible “loops” in connections between devices, such loops which otherwise might permit messages to pass indefinitely in circles through the network. As is understood in the art, spanning tree protocols identify loops in a network built with infrastructure devices called bridges, and provide instructions to bridges to block certain ports to eliminate these network loops. These instructions are transmitted as “bridge protocol data units” (BPDUs) to the various bridges in the network.

The loop structure of a DLR can be incompatible with a spanning tree (ST) type network, which attempts to eliminate loops. This incompatibility may be accommodated when providing a gateway between a DLR and ST type network by ensuring that each given DLR network has only a single gateway. This limitation to the number of gateways, however, increases the risk that a single gateway failure will prevent communication between the two networks.

SUMMARY OF THE INVENTION

The present invention provides a system to permit redundant gateways between DLR and ST networks through a protocol ensuring that only one of multiple gateways in a DLR network may be active at one time. Generally, the gateways operate in either an active or a backup state, operating in the active state to permit connection of the DLR and ST networks through the gateway, and operating in the backup state to largely separate DLR traffic on one side of the gateway and ST traffic on the other side. Broadcast “advertise” messages, containing a precedence value, may be used to communicate among gateways to hold one gateway in an active state (having the highest precedence) and the other gateways in a backup state. Loss of the “advertise” message or explicit fault transmission causes a switchover between devices.

Specifically, the present invention provides, in one embodiment, a gateway for connecting a device level ring (DLR) network to a spanning tree (ST) network. The gateway includes a first and second port connectable to devices in a DLR network to communicate DLR topology messages and general messages with other devices connected in the DLR network, the DLR topology messages controlling reconfiguration of the DLR in the event of a break in the DLR ring. The gateway also includes at least one third port connectable to devices in the ST network to communicate general messages with other devices connected in the ST network. A controller in the gateway communicates with the first, second, and third ports to operate in at least two states including an active state and a backup state where the gateway: (i) exchanges DLR topology messages only with other devices in the DLR network in the active and the backup states; (ii) exchanges general messages among devices in the ST network in the active and the backup states; and (iii) exchanges general messages between devices in the DLR network and devices the ST network only in the active state.

It is thus a feature of at least one embodiment of the invention to provide redundancy in gateways between DLR and ST type networks without the conflicts that can occur in these two different network types, for example, spanning tree algorithms attempting to break the loop of the DLR network.

The gateway may further monitor the DLR network to switch between the active and backup state based on messages from another gateway in the DLR network so that there is only one active gateway in the DLR network.

It is thus a feature of at least one embodiment of the invention to provide an automatic configuration of redundant gateways between a DLR network and ST type network. By monitoring messages on the DLR network, the redundant gateways may self-organize so that only one gateway is active at a time and so that in the event of failure of a gateway, another gateway activates itself.

The gateway may transmit advertise messages providing a precedence value unique to the gateway in the active state and listen for advertise messages in the backup state, and may switch from the backup state to the active state if advertise messages are not received during a predetermined time or are received with a lower precedence than the precedence value unique to the gateway.

It is thus a feature of at least one embodiment of the invention to make use of the DLR network itself for organizing the multiple gateways.

The precedence value may include a stored value set by a user and a MAC address of the gateway.

It is a feature of at least one embodiment of the invention to eliminate the possibility of precedence “ties” when identifying a new gateway by using the unique MAC address as a tie-breaker.

The gateway states may include a listen state and a backup state, and the gateway may switch from the listen state to the backup state if advertise messages are received with a higher precedence than the precedence value unique to the gateway and may switch from the backup state to the listen state if advertise messages are not received during a predetermined time or are received with a fault indication; and wherein the gateway transmits advertise messages in the listen state and not in the backup state.

It is thus an object of at least one embodiment of the invention to permit the transmission of advertise messages from gateways performing a backup function without transmitting unnecessary advertise messages at all times during the backup function.

The gateway may transmit a broadcast flush tables message to other devices on the DLR network causing them to relearn associations between addresses and ports when the gateway switches to the active state. The flush tables message may trigger transmission of a broadcast learning update message by the devices on the DLR network to the bridges on the ST network and to the other devices on the DLR network causing them to rapidly relearn associations between addresses and ports when the gateway switches to the active state.

It is thus an object of a least one embodiment of the invention to provide a state change that may be used to trigger a learning update message to rapidly relearn the network topology on both sides of the gateway.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now toFIG. 1, an industrial control network10may include, for example, multiple control devices12a-dsuch as may exchange signals directly or indirectly with an industrial process14for real-time control of that process. Real-time control, in this context, means control that is subject to well-defined maximum delay periods between an output signal generated by the control device12and an electrical signal sent to an actuator in the industrial process14, and similarly well-defined maximum delay periods between the generation of a signal by a sensor in the industrial process14and its receipt and processing by a control device12.

The control devices12may communicate with each other and with first and second gateway devices16aand16bby means of network media18supporting an industrial control network protocol to implement a full duplex IEEE 802.3 Ethernet network.

The control devices12may provide the functionality of the programmable logic controller, a motor drive, an I/O module or the like and may include an electronic computer executing a stored program held in memory and providing logic for the necessary control. Often the stored program is generated uniquely for the particular industrial process14. The control devices12will also include standard network communication interfaces compatible with the protocol described above.

Improved tolerance to network fault (being either the loss of a device12or network media18) may be obtained by arranging the devices12a-12dand gateways16a-16bin a device level ring (DLR) network19wherein each device12and gateway16communicates with two other devices12or gateways16that flank it in the ring. Thus, for example, each given device12or gateway16may have a first and second DLR port20aand20bcommunicating via network media18with one device in a clockwise direction (e.g., port20aof the given device communicating with port20bof a clockwise flanking device) and with a one device in the counterclockwise direction (port20bof the given device communicating with port20aof the counterclockwise flanking device). A DLR network19suitable for use with the present invention is described, for example, in U.S. patent application Ser. No. 12/493,838 filed Jun. 29, 2009, assigned to the assignee of the present invention and hereby incorporated by reference.

Each gateway16aand16bmay also include multiple standard ports22that may connect to a standard, spanning tree (ST) network24. The ST network24may include multiple bridges26connected by network media18to permissibly create multiple physical loops and redundant interconnections between the gateways16and bridges26. The particular spanning tree protocol implemented by the spanning tree network24may follow IEEE 802.1D “rapid spanning tree protocol” (RSTP) or IEEE 802.1Q “multiple spanning tree protocol” (MSTP) or other similar standards.

Referring now toFIGS. 2 and 3, each of the gateways16aand16bmay operate in an active state50as shown inFIG. 2or a backup state48as shown inFIG. 3. In the active state50, general messages28may be transmitted between ports20aand/or20band all other ports including ports22, for example, using common Ethernet protocols. Such general messages28exclude only DLR topology messages30which relate to reconfiguration of the topology of the DLR network19as may be transmitted from a supervisory node, as will be discussed below, and as is disclosed in the prior application cited above. In the active state50, DLR topology messages30may be transmitted only between ports20aand20b. General messages28may also include “bridge protocol data units” (BPDUs) from the spanning tree network24which may be communicated into the DLR network19which in normal operation will have no bridging loops because the logical loop of the DLR network19is broken either by a supervisory node or by a failure in the loop.

Referring toFIG. 3, in the backup state48, general messages28from either of ports20aor20bmay be transmitted only to the other of the ports20aand20bin the manner of the DLR topology messages30. Likewise general messages and other messages from the spanning tree ports22may only be transmitted to other spanning tree ports22. General messages are blocked from transmission between ports20and ports22.

Referring now toFIGS. 4 and 5, each gateway16may have software or firmware providing a stored program36providing coordination between the gateways16on the DLR network19. When power is first applied to a given gateway16a, for example, the given gateway16astarts in a startup state40where traffic forwarding between ports20and22is blocked. If redundant gateway operation for the given gateway16ais disabled, the program proceeds to single gateway node44per state transition arrow45and traffic is enabled between ports20and22.

Alternatively, if redundant gateway operation is enabled for the given gateway16a, for example by user command programmed into the gateway16a, the program36proceeds to listen state42per state transition arrow47while continuing to block transmissions between ports20and22.

At listen state42, the gateway16awill transmit an advertise message46on the DLR network19(advertise messages shown for only gateway16ainFIG. 4for clarity). The gateway16awill also listen for advertise messages from other gateways (e.g. gateway16b). The advertise messages46incorporate a precedence value that may be programmed into the gateway16aby the user and also incorporate elements of the MAC address of the gateway16aon the DLR network19. The advertise message46will also provide the state of the transmitting device, for example, as indicated by the listen state42or the active state50to be described below.

If, at the listen state42, an advertise message46is received from another gateway16bhaving a higher precedence value than that of the gateway16areceiving the advertise message, the gateway16amoves to a backup state48per state transition arrow49. As noted above, the precedence value includes a programmed portion and the MAC address of the gateway16. If the programmed portion of the precedence value of the received advertise message46is identical to the programmed portion of the precedence value of the receiving gateway16a(for example by erroneous duplicate programming), the MAC address of the two gateways16aand16bare compared and the MAC address is used as a tiebreaker. That is, if the MAC address of the transmitting gateway16bis numerically greater than the MAC address of the receiving gateway16a, the receiving gateway16awill transition to the backup state48per state transition arrow49.

In the backup state48traffic forwarding from ports20to22is blocked and the gateway16awill stop transmission of advertise messages46preventing unnecessary use of network bandwidth.

If at the listen state42no advertise message46is received within a predetermined time out period or if an advertise message is received with a lower precedence value, the program36will proceed to the active state50per state transition arrow51and will become an active gateway device with traffic forwarding from ports20to22and vice versa.

Upon transition to the active state50, the gateway16awill transmit an advertise message46and will continue to transmit advertise messages46on a regular basis while in the active state50. In addition, immediately upon transition to the active state50, the gateway16awill transmit a broadcast “flush tables” message to all DLR devices12and will flush its own unicast and multicast address learning filter tables (routing tables). The gateway16aat this time will send a broadcast learning update frame to the non-DLR bridges26and to other DLR devices12to accelerate their learning. Upon receiving “flush tables” message, a DLR device12will flush its own unicast and multicast address learning filter tables (routing tables) and will send a broadcast learning update frame to the non-DLR bridges26and to other DLR devices12to accelerate their learning.

While the gateway16ais in the active state50, it continues to monitor the DLR network19for advertise messages46. If an advertise message46is received from another gateway16bwith a higher precedence (as described above) the given receiving gateway16awill transition to the backup state48per state transition arrow53.

While the program36is in the backup state48, if physical connection is lost on all the uplink ports22of the gateway16aor higher level connection fault is detected on the ports22, the gateway16awill transition to the fault state56as indicated by state transition arrow58. In the fault state56, forwarding of traffic between ports20and22will be blocked and no advertise messages46will be transmitted; however, the fault will continue to be monitored.

Alternatively, while the gateway16ais in the active state50or listen state42, if physical connection is lost on all the uplink ports22or higher level connection fault is detected on the ports22, the gateway16awill transmit an advertise message46denoting a fault state56and will transition to the fault state56per state transition arrow59or state transition arrow61as appropriate, again blocking traffic between the ports20and the ports22and ceasing transmission of the advertise message46in the fault state56.

While in the backup state48, if an advertise message46is received from a gateway16bdenoting a fault state or if advertise messages46are not received from an active gateway16bfor predetermined time out period, the gateway16awill move to the listen state42as indicated by state transition arrow57. As before in this listen state42, traffic is blocked between ports20and22.

While the gateway16ais in the fault state56, it continues to monitor the advertise messages46and if the connection on ports22is restored and the advertise messages46indicate an active gateway16bhaving a precedence greater than the receiving gateway16a, the program36transitions to the backup state48as indicated by state transition arrow55. Alternatively, if the connection on ports22is restored and the received advertise messages46are from a gateway16bhaving a lower precedence than the receiving gateway16a, the program36transitions from the fault state56to the listen state42as indicated by state transition arrow60. The gateway16astays in the fault state56per state transition arrow62, if the advertise messages46are not received from an active gateway16bfor predetermined time out period and the connection on ports22is still not restored.

It is possible for partial network fault to occur so that data traffic is lost in only one direction on a given section of network medium18suggesting to a gateway16athat an active higher precedence gateway16bdevice is lost when in fact it is simply a failure of the network medium18. To prevent multiple gateways16from being enabled in this situation a gateway16ain active state50that receives advertise messages46from a gateway16in the active state50but having a lower precedence will block traffic forwarding from ports20to22until this condition is cleared by the user.

References to a controller, computer or processor or its equivalent can be understood to include one or more computational devices including microprocessors, field programmable gate arrays, and application specific integrated circuits that can implement state aware logic and that can communicate in a stand-alone and/or a distributed environment(s), and can thus be configured to communicate via wired or wireless communications with other processors, where such one or more processor can be configured to operate on one or more processor-controlled devices that can be similar or different devices. Furthermore, references to memory, unless otherwise specified, can include one or more processor-readable and accessible memory elements and/or components that can be internal to the processor-controlled device, external to the processor-controlled device, and can be accessed via a wired or wireless network.