# Patent

## Title
Fault-tolerant, frame-based communication system

## Abstract

A packet-switched, fault-tolerant, vehicle communication internetwork (100, 400, 500) comprising port-based VLANs. Two or more VLANs are embodied where a source node (110, 410, 510, 610) comprises two or more network interface circuits (130,140, 415,425, 515,525, 630,640), and where looping is precluded via specific VLAN tagging and switch ports (131-134, 200, 300, 420, 430, 435, 445, 455, 465, 535, 540, 545, 560, 575, 585, associated with at least one specific VLAN. A destination node (120, 440, 450, 460, 570, 580, 590, 620) may feedback packets to the source node via a general VLAN tag along pathways associated with the two or more specific outgoing VLAN tags.

## Background
CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a continuation of U.S. patent application Ser. No. 12/889,284, filed Sep. 23, 2010, which is a continuation application of International Application No. PCT/US2010/49893 filed Sep. 22, 2010, which claims the benefit of U.S. Provisional Application No. 61/245,207 filed Sep. 23, 2009, the disclosures of which are incorporated by reference herein for all purposes.


TECHNICAL FIELD
Embodiments include a packet-switched network comprising a frame-based communication system, and particularly pertain to port-based VLANs in a fault-tolerant communication system.
BACKGROUND
The Internet Protocol may use Ethernet frames where an Ethernet frame format may include a media access control (MAC) header comprising a destination MAC address, a source MAC address and a data payload.
SUMMARY
A frame-internal field may be used for tagging the frame with virtual local area network (VLAN) information. Embodiments may include a system, or internetwork, comprising three or more switching nodes disposed between a source node and a destination node wherein: (a) a first switching node of the three or more switching nodes may be configured to tag a frame received from the source node with a first virtual local area network (VLAN) designator and output via at least one port the frame tagged with the first VLAN designator; (b) a second switching node of the three or more switching nodes may be configured to tag a frame received from the source node with a second VLAN designator, and output via at least one port the frame tagged with the second VLAN designator; and (c) a third switching node (FIG. 3) of the three or more switching nodes may be configured to exclude from output, via at least one port, a received frame having the first V-LAN designator; and wherein the destination node may be configured to receive: (i) at least one of: a frame having the first VLAN designator and a frame having the second VLAN designator; or (ii) at least one of: a first frame stripped of its first VLAN designator and/or a second frame stripped of its second VLAN designator.
The source node may comprise a near real-time controller and the destination node may comprise a gateway and at least one of: (1) an effector responsive to payload information of the received frames; (2) a radio frequency transmitter; and (3) a flight sensor.



BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, and in which:
FIG. 1 is an exemplary top-level system diagram;
FIG. 2 is top-level functional block diagram of an exemplary switch;
FIG. 3 is top-level functional block diagram of an exemplary switch;
FIG. 4 is an exemplary top-level system diagram of an embodiment;
FIG. 5 is an exemplary top-level system diagram of an embodiment;
FIG. 6 is another exemplary top-level system diagram;
FIG. 7A is a top-level functional block diagram of an exemplary switch; and
FIG. 7B is a top-level functional block diagram of an exemplary switch.



DETAILED DESCRIPTION
Reference is made to the drawings that depict exemplary embodiments. Embodiments include deterministic, non-looping, packet routing systems and networks that may employ aspects of virtual local area networking. A virtual local area network (VLAN) may be embodied as a level two of the Open Systems Interconnection (OSI) model, i.e., the data link layer, where data may be portioned according to frame, and reference may be made via media access control (MAC). A VLAN may exploit a tag or indicator available in the Ethernet packet or a frame to discern the VLAN to which a packet may be associated. Exemplary VLAN embodiments may be used to filter data traffic from specific ports of switches. Exemplary VLAN embodiments may perform the filtering function independent of communication type such as unicast and multicast, and transport protocols such as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). VLAN routing configurations may be programmed into the non-volatile memory of each of a plurality of network switches. A network controller, a network master, or an OSI Layer 3 device, may be a node having a processor configured to dynamically adjust the port routing via VLAN management tables, e.g., based on the ping responses of network elements.
FIG. 1 is an exemplary top-level system diagram 100. A processor is defined herein as a computer or computing device where the computer device includes a central processing unit (CPU) and addressable memory, and where the computing device may be configured by way of executable machine-readable instructions, circuitry, or combinations of both, to function as a special computing device. A network may comprise a source node 110 comprising a processor, a destination node 120 comprising a processor, and one or more network links interposed between the nodes, and shown by example in FIG. 1 as directional arrows or pairs of directional arrows. The source node 110 may generate a message such as a command intended for a destination node 120. The exemplary message may be prepared and transmitted by a plurality of network interface cards 130, 140. In the exemplary diagram of FIG. 1, the source node 110 interfaces with the network via the two network interface devices 130, 140 or computer circuit boards that may be network interface cards (NICs). Each NIC may then connect to a routing switch 131-134, e.g., a switch having a table for directed port routing. In the exemplary network of FIG. 1, the first NIC 130 may transmit to a first switch (SW_1) 131 and the first switch may transmit to a second switch (SW_2) 132. Also in the exemplary network of FIG. 1, the second NIC 140 may transmit the same messages as the first NIC 130 to a fourth switch (SW_4) 134 and the fourth switch (SW_4) may transmit to a third switch (SW_3) 133. The second switch (SW_2) 132 and the third switch (SW_3) 133 may transmit packets/datagrams to the destination node. Accordingly, the destination node may receive redundant messages from one source node via two or more channels. In some embodiments, a switch may receive redundant messages from one source node 110 via two or more channels, e.g., VLAN_1 and VLAN_2, remove the VLAN tags, reconcile the redundancy in the packets, and provide the reconciled and untagged packets to the destination node 120.
FIG. 2 is top-level functional block diagram of an exemplary switch 200. The switching node 210 of FIG. 2 is depicted as configured to tag 215 a frame received via port_1 211 from the source node 110 with a second VLAN (e.g., VLAN_2) designator, and, with reference to a routing table 216, output via at least one port 211-213 the frame tagged with the second VLAN designator. Referring to FIGS. 1 and 2, port-based VLANs include nodes on the network that may send standard packets without VLAN tags, e.g., as untagged Ethernet packets 145,147. When the untagged packet 145 arrives at its first network switch port (SW_1) 131, it may be examined. If the examining switch finds no tag on the packet, it may attach the tag assigned to the particular port 151 receiving the untagged packet 145, e.g., VLAN_1 may be the tag assigned for packets received via port number one 211. The examining switch may determine via processing 215 and by referring to the port-routing table 216, which of its other outgoing ports, i.e., one or more ports other than the instant receiving port, are also configured for VLAN_1. For each switch port of the examining port that is configured for VLAN_L the now tagged packet is forwarded, e.g., to port_2 212 and port_3 213. Upon arrival at a next, in this example the second, network switch (SW_2) 132, the packet may be examined. If the second network switch 132 discerns the VLAN_1 tag, then for each switch port of the second examining port that is configured for VLAN_1, the VLAN_1 tagged packet is forwarded. In some embodiments, the destination node comprises a receive port and circuitry configured to remove the VLAN tags of the received packets.
FIG. 3 is top-level functional block diagram of an exemplary switch 300. The exemplary switching node 310 of FIG. 3 is shown configured to exclude from output, via at least one port 311-314, a received frame having the first VLAN designator (e.g., VLAN_1), received via port_2 312. For example, switching node 310 may determine via processing 315 and with reference to a port-routing table 316, that the frame having the first VLAN designator may be transmitted via port_4 314, but not via port_3 313. Referring to both FIGS. 1 and 3, port_1 may receive a packet tagged as VLAN_2 from SW_4 and a packet tagged as VLAN_1 from SW_1. The switching node 310 may be configured to only output VLAN_2 tagged packets via port_4 314. Switching node 310 may output the VLAN2 tagged frames to a destination node where the tags may be removed and the packets further processed. Accordingly, embodiments of the networked system may include one or more destination switch nodes where a destination node switch is a node having a direct link to a destination node 120, i.e., no additional switch node is interposed between a destination switch node and the destination node. Upon arrival at a destination switch node, in some embodiments, the VLAN_1 or VLAN_2 tag may be removed from the packet and the packet may then be forwarded to the destination node 120, also shown in FIG. 1 as untagged packets 146,148.
FIG. 4 is an exemplary top-level block diagram of an internetwork 400 where a source node 410 may have two NIC cards 415, 425 to interface with respective network segments. Each NIC interfaces with a respective switch: NIC_1 415 with SW_1 420, and NIC_2 425 with SW_6 430. Each switch is shown having four ports and may have more or less to support various embodiments. By this illustrative network: a first destination node (Destination Node_1) 440 may receive packets from both SW_1 420 and SW_6 430; a second destination node (Destination Node_2) 450 may receive packets from both SW_2 435 and SW_5 445; and a third destination node (Destination Node_3) 460 may receive packets from SW_3 455, for example. In this embodiment, SW_6 430 is configured to not output packets having the VLAN_1 tag and SW_1 420 is configured to not output packets having the VLAN_2 tag. SW_2 435 is shown configured to have only one port 436 that may output packets having the VLAN_2 tag, and SW_5 445 is depicted as having only one port 446 that may output packets having the VLAN_1 tag. Accordingly, the exemplary network does not loop packets of either VLAN tag for this example. That is in some embodiments of the internetwork, at least one of the switching nodes (SW_1 to SW_6) may be configured to block transmission of a received packet comprising a frame tagged with a first VLAN designator, i.e., the VLAN_1 tag, and thereby foreclose the possibility of a looping counter clockwise transmission (of FIG. 4). Likewise in some embodiments of the internetwork, at least one of the switching nodes (SW_1 to SW_6) may be configured to block transmission of a received packet comprising a frame tagged with a second VLAN designator, i.e., the VLAN_2 tag, and thereby foreclose the possibility of a looping clockwise transmission (of FIG. 4, the dashed arrow pathways).
FIG. 5 is an exemplary top-level block diagram of an internetwork 500 where a first controller node 510 may have two NIC cards 515, 525 to interface with respective network segments. Each NIC interfaces with a respective switch: NIC_1 515 with SW_1 535 and NIC_2 525 with SW_6 545. In addition, a second controller node 550 may have two NIC cards 555, 565 to interface with respective network segments. Each NIC interfaces with a respective switch: NIC_3 555 with SW_3 540 and NIC_4 565 with SW_4 560. Each switch is shown having four ports and may have more or less to support various embodiments. By this illustrative network: a first destination node (Destination Node_1) 570 may receive packets from both SW_1 535 and SW_6 545; a second destination node (Destination Node_2) 580 may receive packets from both SW_2 575 and SW_5 585; and a third destination node (Destination Node_3) 590 may receive packets from both SW_3 540 and SW_4 560. In this embodiment, SW_6 545 is configured to not output packets having the VLAN_1 tag and SW_1 535 is configured to not output packets having the VLAN_2 tag. SW_2 575 is shown configured to have only one port 536 that may output packets having the VLAN_2 tag, and SW_5 585 is shown having only one port 586 that may output packets having the VLAN_1 tag. Accordingly, the exemplary network does not loop packets of either VLAN tag for this example. In addition, the first controller node 510 and the second controller node 550 may be preset or arbitrate the role of a master network controller with the other controller node tracking the network activity, monitoring network component/node health signals and/or metrics, and configured to replace the current network controller as the master network controller based on an external input and/or an event exceeding a pre-defined network health status threshold.
FIG. 6 is an exemplary top-level system diagram 600 similar to FIG. 1. FIG. 6 depicts a communication to controller node 610 comprising a processor, a destination node 620 comprising a processor, and a network link interposed between the nodes. The destination node 620 may generate a message such as a feedback signal for the controller node 610. The exemplary message may be prepared and tagged 650 with a third VLAN tag, e.g., VLAN_3, by circuitry of the destination node 620 or, in this example, by the proximate switch, SW_2 132, functioning in this example as a destination node switch. The tagged message may be transmitted 651-653 via routing switches, e.g., SW_1 131, SW_3 133, and/or SW_4 134, and received by a plurality of network interface cards 630, 640 of the controller node 610. Accordingly, the controller node may receive redundant messages from one destination node 620 via two or more channels, i.e., channels that permit the routing of messages tagged with VLAN_3. In some embodiments, a switch may receive redundant messages from one destination node via two or more channels, remove the VLAN_3 tags, reconcile the redundancy in the packets, and provide the reconciled and untagged packets to the controller node 610. Accordingly, network interface_1 630 may receive untagged packets 645 from SW_1 131, and network interface_2 640 may receive untagged packets 647 from SW_4 134.
FIG. 6 also depicts a network structure supportive of redundant paths for destination nodes, such as actuator or sensor nodes having a single physical connection to the network, e.g., an untagged packet path 646. As disclosed in FIG. 1, the network configuration supports a message path from each of two interfaces (NICs) 130,140 on the controller node 110 that may be established using two one-directional VLANs, i.e., two unidirectional of the asymmetrical VLAN configuration—VLAN_1 and VLAN_2. The configuration of FIG. 1 allows for directing the message packets from the two interfaces (NICs) of the controller node 110 over different paths through the network of switches 131-134 between the controller node 110 and the actuator/sensor 620. The two paths selected for the packets may not be optimized for number of hops or speed, but rather may be selected to achieve maximum probability of a path remaining functional in the event the other path experiences a disruption. This configuration uses two VLANs to achieve performance continuity and transmitted data integrity in the face of a single point failure, e.g., a failure in SW_1 131, SW_3 133, or SW_4 134.
FIG. 6 depicts the return paths from the actuator/sensor 620 to the two physical interfaces (NICs) 630,640 on the controller 610 that may be established by use of a third VLAN, e.g., VLAN_3. The return packets, from the destination node 620 to the controller node 610, may be broadcast from the actuator/sensor 620, and directed along two or more paths back to the two interfaces (NICs) 630, 640 of the controller 610 by use of the third uni-directional VLAN—in a direction opposite that of the asymmetrical VLAN configuration. The two or more paths selected for the packets may not be optimized for number of hops or speed, but rather may be selected to achieve maximum probability of a path remaining functional in the event the other path experiences a disruption. Accordingly, this return path embodiment uses at least one additional VLAN. The path from the first NIC 630 to the destination node 620 is determined by VLAN_1. The path from the second NIC 640 to the destination node 620 is determined by VLAN_2. The return paths from destination node 620 to both the first NIC 630 and the second NIC 640 are determined by VLAN_3. The tagging of the packets with the VLAN_3 tag can be performed by the actuator/sensor 620 or, in this example, by SW_2 132 upon arrival at the switch port. The learning and subsequent lookup of which physical port the switch may send a packet of a return message in order to reach the Ethernet addresses of the NICs may be independent of the VLAN tag. Return routing independent of the original outgoing VLAN tag allows return packets of either outgoing VLAN to be routed back to the correct port, e.g., via the VLAN_3 tag.
FIG. 7A is top-level functional block diagram of an exemplary switch 700. The exemplary switching node 710 of FIG. 7A is shown configured as a destination node so that upon arrival at a destination switch node, in some embodiments, the VLAN_1 or VLAN_2 tag may be removed from the packet and the packet may then be forwarded to the destination node. In this example and also with reference to FIG. 1, the switching node 710 may receive at port_1 711 a packet having a VLAN_1 tag from SW_1 131, may receive at port_2 712 a packet having a VLAN_2 tag from SW_3 133, and optionally may receive at a port_3 713 a packet having a VLAN_2 tag from SW_4 134. By processing and untagging 715 and by making reference to the port-routing table 716, the switching node may output the untagged frames via port_4 714 to the destination node.
FIG. 7B is top-level functional block diagram of an exemplary switch 700 of FIG. 7A where the switching node 710 is depicted as receiving an untagged packet from the destination node at port_4 714. By processing and tagging 715, the switching node 710 is depicted as outputting a packet via two or more ports: a VLAN_3 tagged packet to SW_1 via port_1 711; a VLAN_3 tagged packet to SW_3 via port_2 712; and optionally a VLAN_3 tagged packet to SW_4 via port_3 713.
The exemplary networks may employ a lightweight transmission protocol such as the User Datagram Protocol (UDP). A destination node may interface with a gateway allowing for communication with devices such as sensors (e.g., attitude, altitudes, temperature, pressure, Global Positioning Satellite receiver outputs, and/or strap-down navigation sensors), radio transceivers, engine controllers, and actuating devices. The exemplary network may be embodied as a real-time communication network as part of a vehicle such as a land vehicle, an air vehicle, and/or a sea-going vehicle.
Embodiments may readily scale up where a controller node may comprise three or more network interface circuits and accordingly three or more VLAN outgoing pathways may be configured. The return path may remain as a single VLAN that functions as a return superset of the outgoing VLANs, or additional return VLAN pathways may be configured.
It is contemplated that various combinations and/or sub-combinations of the specific features and aspects of the above embodiments may be made and still fall within the scope of the invention. Accordingly, it should be understood that various features and aspects of the disclosed embodiments may be combined with or substituted for one another in order to form varying modes of the disclosed invention. Further it is intended that the scope of the present invention herein disclosed by way of examples should not be limited by the particular disclosed embodiments described above.

## Claims

1. A system comprising:
a source node;
a first routing switch of four or more routing switches configured to output, via at least one port, a frame received from the source node and tagged with a first virtual local area network (VLAN) designator, said first routing switch having a first table for directed port routing, wherein said first routing switch is configured to exclude from output a received frame having a second VLAN designator;
a second routing switch of the four or more routing switches configured to output, via at least one port, a frame received from the source node and tagged with the second VLAN designator, said second routing switch having a second table for directed port routing, wherein said second routing switch is configured to exclude from output a received frame having the first VLAN designator; and
a first destination node of two or more destination nodes configured to receive at least one of: the frame output by the first routing switch and the frame output by the second routing switch.

2. The system of claim 1 further comprising:
a third routing switch of the four or more routing switches configured to output, via only one port, a frame received from at least one routing switch of the four or more routing switches and tagged with the second VLAN designator to the first routing switch, said third routing switch having a third table for directed port routing;
a fourth routing switch of the four or more routing switches configured to output, via only one port, a frame received from at least one routing switch of the four or more routing switches and tagged with the first VLAN designator to the second routing switch, said fourth routing switch having a fourth table for directed port routing; and
a second destination node of the two or more destination nodes configured to receive at least one of: the frame output by the third routing switch and the frame output by the fourth routing switch.

3. The system of claim 2 wherein:
the third routing switch of the four or more routing switches is further configured to output, via at least one port, a frame received from at least one routing switch of the four or more routing switches and tagged with the first VLAN designator; and
the fourth routing switch of the four or more routing switches is further configured to output, via at least one port, a frame received from at least one routing switch of the four or more routing switches and tagged with the second VLAN designator.

4. The system of claim 2 wherein the frame received by the first routing switch from the source node is tagged with the first VLAN designator by the first routing switch, and wherein the frame received by the second routing switch from the source node is tagged with the second VLAN designator by the second routing switch.

5. The system of claim 2 wherein the frame received by the first routing switch from the source node is tagged with the first VLAN designator by a first network interface circuit in communicative link with the first routing switch, and wherein the frame received by the second routing switch is tagged with the second VLAN designator by a second network interface circuit in communicative link with the second routing switch.

6. The system of claim 2 wherein at least one of: the first routing switch, the second routing switch, the third routing switch, and the fourth routing switch, is configured to block transmission of a received frame tagged with the first VLAN designator.

7. The system of claim 2 wherein at least one of: the first routing switch, the second routing switch, the third routing switch, and the fourth routing switch, is configured to block transmission of a received frame tagged with the second VLAN designator.

8. The system of claim 2 wherein at least one of: the first routing switch and the second routing switch, is configured to remove at least one of: the first VLAN designator and the second VLAN designator, prior to outputting the frame to the first destination node.

9. The system of claim 2 wherein at least one of: the third routing switch and the fourth routing switch, is configured to remove at least one of: the first VLAN designator and the second VLAN designator, prior to outputting the frame to the second destination node.

10. The system of claim 2 further comprising:
a fifth routing switch of the four or more routing switches configured to output, via at least one port, a frame received from at least one routing switch of the four or more routing switches and tagged with at least one of: the first VLAN designator and the second VLAN designator, said fifth routing switch having a fifth table for directed port routing;
a sixth routing switch of the four or more routing switches configured to output, via at least one port, a frame received from at least one routing switch of the four or more routing switches and tagged with at least one of: the first VLAN designator and the second VLAN designator, said sixth routing switch having a sixth table for directed port routing; and
a third destination node of the two or more destination nodes configured to receive the frame output by the fifth routing switch.

11. An internetwork comprising three or more switching nodes disposed between a source node and a destination node wherein:
the source node comprises:
a first network interface circuit in communicative link with a first switching node of the three or more switching nodes, said first switching node having a first table for directed port routing; and
a second network interface circuit in communicative link with a second switching node of the three or more switching nodes, said second switching node having a second table for directed port routing;

a third switching node of the three or more switching nodes, said third switching node having a third table for directed port routing; and
a frame sent from the destination node and received by said third switching node, said frame tagged with a third virtual local area network (VLAN) designator;
wherein the internetwork routes the frame from the destination node to the source node by two paths through the three or more switching nodes, the first network interface card, and the second network interface card, by at least one of: the first table for directed port routing, the second table for directed port routing, and the third table for directed port routing.

12. The internetwork of claim 11 wherein the two paths routing the frame from the destination node to the source node achieve maximum probability of one path of the two paths remaining functional in the event the other path of the two paths experiences a disruption.

13. The internetwork of claim 12 wherein the disruption is a failure in at least one of the three or more switching nodes.

14. The internetwork of claim 12 wherein the two paths are based on ping responses of one or more internetwork elements.

15. The internetwork of claim 11 wherein the frame received by the third switching node from the destination node is tagged with the third VLAN designator by the third switching node.

16. The internetwork of claim 11 wherein the frame received by the third switching node from the destination node is tagged with the third VLAN designator by the destination node.

17. The internetwork of claim 11 wherein the third VLAN designator is removed from the received frame tagged with the third VLAN designator by at least one of: the first switching node and the second switching node, prior to outputting the frame to the source node.

18. The internetwork of claim 11 wherein the destination node is at least one of: an actuator and a sensor.

19. The internetwork of claim 18 wherein the frame sent from the destination node includes a feedback signal.