Abstract:
A method for routing a data flow from a source node to a destination node among a plurality of nodes mutually linked in a ring by segments of a communication network, configured so that the data flow can traverse the ring in either of first and second, mutually-opposing directions of communication. The method includes maintaining at each of the nodes a record of respective levels of use of a network resource on the segments in the ring for both the first and second directions of communication. Responsive to the respective levels of use, one of the directions is selected at the source node, in which to route the data flow to the destination node. A notification is sent from the source node to the other nodes in the ring of an additional allocation of the resource required by the source node in order to convey the data flow to the destination node in the selected direction on the segments in the ring. The record of the levels of use of the resource at each of the nodes is updated, responsive to the notification from the source node.

Description:
FIELD OF THE INVENTION  
         [0001]    The present invention relates generally to communication networks, and specifically to methods and systems for efficient allocation of resources for data flow in such networks.  
         BACKGROUND OF THE INVENTION  
         [0002]    Network ring topologies are gaining in popularity, particularly in Internet Protocol (IP) networks. Such networks enable carriers to offer large bandwidth to users in a cost-effective manner. In order to gain these benefits, however, IP needs appropriate support at the Media Access Control (MAC-protocol layer  2 ) level, to provide functions such as load balancing, protection and clock synchronization.  
           [0003]    One solution that has been proposed to meet these needs is the Spatial Reuse Protocol (SRP), which is described by Tsiang et al., in Request for Comments (RFC) 2892 of the Internet Engineering Task Force (IETF). This document, which is available at www.ietf.org/rfc.html, is incorporated herein by reference. SRP relates to the ring network as two overlapping local area networks (LANs), identified arbitrarily as an inner ring and an outer ring. In one of the rings, communication flows clockwise, while in the other it flows counterclockwise. Each node in the ring can communicate directly with all other nodes through either of the rings, using the appropriate MAC addresses of the nodes. Spatial reuse enables different nodes to use different, non-overlapping spans of the same ring simultaneously (unlike earlier ring protocols), thus increasing the overall aggregate bandwidth that is available. SRP allows nodes to choose whether to route their packets on the inner or the outer ring, but it does not provide any method for nodes to use in deciding which ring to choose.  
           [0004]    SRP also defines a mechanism to be used by nodes on the ring in learning the ring topology. In the topology discovery phase of network start-up, described in section 4.6 of RFC 2892, each node can send out topology packets on one or both rings. The packet hops around the ring from node to node. Each node appends to the packet its own MAC address binding and other information. Eventually the packet comes back to the originating node, which uses the information that has been appended by the other nodes to build a topology map of the ring.  
           [0005]    There are routing protocols known in the art for choosing an optimal path from a source node to a destination node through a network when multiple paths are available. These protocols have generally been designed with mesh networks in mind, although they can also be applied to bi-directional ring networks. For example, the Open Shortest First Protocol (OSFP) is a link-state routing protocol that is used to identify and select the path that has the lowest overall “cost.” OSFP is described by Moy in RFCs 1583 and 2328 of the IETF, which are available at the above-mentioned Web address and are likewise incorporated herein by reference. This protocol enables a system administrator to assign a cost to each link in the network, so that low-cost links are the ones most likely to be selected for routing. OSFP does not specify, however, how the costs are to be determined. Moreover, since the costs are assigned statically by the system administrator, the protocol does not provide any means or basis for updating the costs automatically, while the network is running, in response to network resource constraints.  
           [0006]    After learning the network topology and routing information, the source node must still confirm that it has sufficient resources (such as bandwidth) available to it over a desired route before it can transmit a data flow over the route. The resource requirements for a given transmission depend, inter alia, on the Quality of Service (QoS) level at which the transmission is to be sent. Typically, one of the nodes is appointed to centrally manage resource allocation among all of the nodes in the network or in a given subnet of tne network. The manager may be a fixed entity, or it may be chosen from among the nodes using a suitable network management protocol. An example of such a protocol is the Subnet Bandwidth Manager (SBM) protocol, described by Yavatkar et al., in IETF RFC 2814, which is available at the above-mentioned Web address and is incorporated herein by reference. The source node requests the resources that it needs from the manager, typically using another standard protocol, such as the Resource Reservation Protocol (RSVP), described by Braden et al., in IETF RFC 2205, which is likewise available at the above-mentioned Web address and is incorporated herein by reference.  
         SUMMARY OF THE INVENTION  
         [0007]    It is an object of the present invention to provide improved methods and apparatus for managing resource allocation in a network, and particularly in a network that is arranged in a ring topology.  
           [0008]    It is a further object of some aspects of the present invention to provide a method for routing data flows that makes optimal use of bandwidth and other resources that are available on different links of a ring network.  
           [0009]    In preferred embodiments of the present invention, a communication network comprises a plurality of nodes arranged in a ring topology. The nodes are capable of transmitting data around the ring in either a clockwise or a counterclockwise direction, or in other words, following SRP convention, over an inner or an outer ring within the network. Each of the nodes maintains a load table of resource use on each of the segments of each of the directional rings. Each node can update its load table for a given ring by sending a load update request packet to the next node on the ring. The packet is passed around the ring from node to node. Each node that receives the packet adds its own status information to the packet. The originating node receives the packet back after it has made a complete circuit and analyzes the packet contents to determine current resource use. The node preferably sends the load update request in preparation for making a new resource reservation on the ring. Alternatively, the node may send the request periodically, independent of specific resource reservations, and it preferably also updates its load tables based on resource reservation packets sent by other nodes on the ring, as described below.  
           [0010]    Whenever a node needs to request network resources to carry a data flow to another node, it preferably chooses to send the flow on the less heavily loaded of the directional ring. Most preferably, the node first finds the most heavily loaded segment in each of the rings, and then chooses the ring that has the lower level of load on this segment. The node verifies that the desired flow is within its maximum permitted allocation on the chosen ring. If so, the node notes the allocation in its own load table and sends a load update advertisement packet around the ring, to advertise that the allocation has been made. Preferably, before sending this packet, the node sends a lock packet to the other nodes on the ring, to maintain synchronization among the nodes and prevent situations in which two or more nodes attempt to make reservations at once. Each node in turn receives this load update advertising packet, and updates its own load tables accordingly. When the packet returns to the originating node, the transaction is finalized.  
           [0011]    Thus, preferred embodiments of the present invention provide a decentralized mechanism for managing resource allocation in a ring network. The two phases of the transaction—using the lock packet followed by the load update advertising packet—ensure that synchronization and consistency are maintained among the nodes. Decentralized resource management is an efficient means for controlling data flow allocations, particularly in ring networks, and is more robust than centralized management protocols, such as SBM, in the face of node failures.  
           [0012]    Although preferred embodiments are described herein mainly with reference to controlling bandwidth allocation, the principles of the present invention are similarly applicable to management of other resources and data flow allocations based on these resources. Such resources may include, but are not limited to, processing power used to transmit, receive, encrypt or decrypt data, as well as substantially any other allocable factor that may affect the ability of a network link to convey a data flow between nodes.  
           [0013]    There is therefore provided, in accordance with a preferred embodiment of the present invention, a method for routing a data flow from a source node to a destination node among a plurality of nodes mutually linked in a ring by segments of a communication network, configured so that the data flow can traverse the ring in either of first and second, mutually-opposing directions of communication, the method including:  
           [0014]    maintaining at each of the nodes a record of respective levels of use of a network resource on the segments in the ring for both the first and second directions of communication;  
           [0015]    selecting at the source node, responsive to the respective levels of use, one of the directions in which to route the data flow to the destination node;  
           [0016]    sending a notification from the source node to the other nodes in the ring of an additional allocation of the resource required by the source node in order to convey the data flow to the destination node in the selected direction on the segments in the ring; and  
           [0017]    updating the record of the levels of use of the resource at each of the nodes, responsive to the notification from the source node.  
           [0018]    Preferably, the network includes an Internet Protocol (IP) network, and sending the notification includes sending a User Datagram Protocol (UDP) packet over the IP network. Alternatively, sending the notification comprises sending a media access control (MAC) frame over the network. In a preferred embodiment, selecting one of the directions includes determining whether to convey the data flow over an inner or outer data link ring provided by a Spatial Reuse Protocol (SRP).  
           [0019]    Preferably, the network resource includes a segment bandwidth. Further preferably, selecting one of the directions includes verifying that a sufficient amount of the network resource is available to carry the data flow on every one of the segments included in a path from the source node to the destination node in the selected direction. Additionally or alternatively, selecting one of the directions includes selecting the direction having the lowest level of load with. respect to the resource. Most preferably, selecting the direction includes finding a maximum of the respective levels of use of the resource, taken over all of the segments included in a path from the source node to the destination node in each of the directions, and selecting the direction having the lower maximum level of use.  
           [0020]    Preferably, sending the notification includes sending a data packet from node to node in succession around the ring in the selected direction, and receiving the data packet back at the source node after the packet has traversed the ring. Further preferably, receiving the data packet back at the source node includes receiving an account of the allocation of the resource on each of the segments in the ring, and verifying, responsive to the account, that the additional allocation can be made. Additionally or alternatively, sending the data packet includes adding a unique identifier to the packet, and wherein receiving the data packet includes identifying the packet as having been sent by the source node responsive to the identifier.  
           [0021]    Further additionally or alternatively, updating the record includes, after receiving the data packet back at the source node, sending an additional data packet from the source node around the ring in the selected direction so as to notify the other nodes that the additional allocation of the resource has been made. Preferably, sending the data packet includes sending a lock packet, so as to prevent the nodes other than the source node from allocating the resource, and sending the additional data packet includes sending an unlock packet, so that the nodes other than the source node can resume allocating the resource after the additional allocation by the source node has been completed.  
           [0022]    There is also provided, in accordance with a preferred embodiment of the present invention, a communication network, including:  
           [0023]    a plurality of link segments for carrying communication traffic;  
           [0024]    a plurality of nodes, mutually connected by the segments in a ring so that the communication traffic can traverse the ring in either of first and second, mutually-opposing directions of communication,  
           [0025]    wherein each of the nodes is configured to maintain a record of respective levels of use of a network resource on the segments in the ring for both the first and second directions of communication, such that in order to route a data flow from a source node to a destination node among the plurality of nodes in the ring, the source node selects one of the directions in which to route the data flow to the destination node responsive to the recorded levels of use, and sends a notification to the other nodes in the ring of an additional allocation of the resource required by the source node to convey the data flow to the destination node in the selected direction on the segments in the ring, whereupon the record of the levels of use of the resource is updated at each of the nodes responsive to the notification from the source node.  
           [0026]    The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings in which: 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0027]    [0027]FIG. 1 is a block diagram that schematically illustrates a ring network, in accordance with a preferred embodiment of the present invention; and  
         [0028]    [0028]FIG. 2 is a flow chart that schematically illustrates a method for assigning network resources to network nodes, in accordance with a preferred embodiment of the present invention.  
     
    
     DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS  
       [0029]    [0029]FIG. 1 is a block diagram that schematically illustrates an IP ring network  20 , in accordance with a preferred embodiment of the present invention. Network  20  comprises nodes  22 , which are connected by links  24  labeled S 1  through S 4 . Each node can communicate with every other node over either a clockwise ring  26  or a counterclockwise ring  28  around the ring, indicated respectively by arrows adjacent to node A. Following SRP convention, these two paths are identified respectively as an outer ring and an inner ring, each of which is made up of ring segments corresponding to the physical links between the nodes. Typically, each of tne nodes also links the ring to a respective subnet, which may be of substantially any topology known in the art. When node A, for example, receives a data flow from its respective subnet, which is destined for the subnet of node C, node A determines whether to transmit the flow over ring  26  or ring  28 , using the method described hereinbelow.  
         [0030]    [0030]FIG. 2 is a flow chart that schematically illustrates a method for assigning resources to nodes  22  and updating resource assignments, in accordance with a preferred embodiment of the present invention. At start-up of the network, each of the nodes learns the topology of ring network  20 , at a topology learning step  30 . This step is preferably carried out using the mechanism provided for this purpose by SRP, as described in the above-mentioned RFC 2892. The step is preferably repeated periodically in order to track changes in the ring. In addition, the nodes build routing tables for the overall network (including the subnets that are outside the ring network), at a route learning step  32 . The routing tables are preferably built using the above-mentioned OSPF protocol, or alternatively using other suitable protocols known in the art, such as the Routing Information Protocol (RIP), described by Malkin in IETF RFC 2453. This RFC document, which is available at www.ietf.org/rfc.html, is incorporated herein by reference. At this stage, nodes  22  also learn forwarding information within ring network  20 , relating the MAC addresses of the nodes to their IP addresses, typically using local area network (LAN) protocols known in the art. Examples of such protocols include the Address Resolution Protocol (ARP), described by Plummer in RFC 826, and the Reverse Address Resolution Protocol (RARP), described by Finlayson et al., in RFC 903. These documents, available at the above-mentioned Web address, are also incorporated herein by reference.  
         [0031]    In preparation for allocating bandwidth to data flows within ring network  20 , nodes  22  build load tables, at a load table building step  34 . Each node maintains two tables, one for ring  26  and the other for ring  28 . Each table has entries for all of segments  24  in the respective ring, preferably arranged in topological order according to the direction of the ring, with the node maintaining the table at the root. Each entry in the table holds a record of the current load on the respective segment, i.e., the amount of bandwidth and/or other relevant resources that is allocated and/or available for use on the segment.  
         [0032]    In order to build the load table for each ring, the nodes send load update request packets around the ring. The load update request packet is passed around the ring, hop-by-hop, from one node to the next. Each of the nodes in turn reads the information in the packet and adds or alters the information as appropriate before passing the packet on to the next node. Preferably, the load update request packer comprises a User Datagram Protocol (UDP/IP) packet, although other transport layer protocols known in the art may also be used. A similar packet format is used by the nodes to make and advertise resource reservations, as described below. The packet preferably includes the following field values:  
         [0033]    IP headers:  
         [0034]    Source IP address of the sending node on each hop.  
         [0035]    Destination IP address (unicast address) of the next node on the hop.  
         [0036]    UDP headers:  
         [0037]    Source port—arbitrary.  
         [0038]    Destination port—value reserved for this resource assignment method.  
         [0039]    UDP payload:  
         [0040]    OPCODE (operation code), indicating the type of message being sent—load update request, or a lock/unlock or load update advertisement, as described below.  
         [0041]    STATUS, enabling each node to confirm or deny a requested transaction (when the packet is sent by a node in preparation for making a resource reservation, as described below). The node will deny the transaction if the information in the packet does not conform to information stored in the node&#39;s own tables or when the node cannot find itself listed in the packet. When one of the node enters a STATUS denial, a synchronization process is preferably invoked to resolve the discrepancy in the tables of the different nodes.  
         [0042]    TID (transaction identifier) is a unique number assigned by the originating node of the resource request (node A in the present example). The TID is generated, for example, by applying a random function to an identifying address of the node. It must not be changed by the other nodes as the packet is passed from hop to hop.  
         [0043]    The load update request or other packet makes a complete circuit of the ring, until it returns to the originating node. The originating node recognizes that this was the packet that it originally sent out on the basis of the TID. Use of these TIDs allows multiple transactions to take place on network  20  simultaneously without the risk of confusion among the transactions.  
         [0044]    In an alternative embodiment, the load update request and other packets used in managing resource allocation comprise Layer 2 (media access control—MAC) frames. MAC protocols that can be used for this purpose include Ethernet-II and IEEE 802.3, for example. Nodes  22  are connected by point-to-point Layer 2 links, as described in the SRP specification. Each node on the ring that receives the Layer 2 frame checks whether the source address is equal to its own address. If not, the node extracts the frame, processes it, and forwards it to the next node. If the source address is equal to the node&#39;s own address (indicating that the frame has made a complete circuit of the ring), the node extracts and terminates the frame.  
         [0045]    ETHERNET-II frames used in this context preferably comprise the following fields:  
         [0046]    Ethernet header:  
         [0047]    Destination-Address: multicast address or a unique address (e.g., zeros).  
         [0048]    Source-Address: the address of the source node.  
         [0049]    Ethernet-Type: a unique value dedicated to the resource allocation protocol (a single value for all types of messages).  
         [0050]    Ethernet payload:  
         [0051]    OPCODE.  
         [0052]    STATUS.  
         [0053]    TID.  
         [0054]    FCS (Frame Check Sequence).  
         [0055]    Alternatively, IEEE802.3 frames for this purpose preferably comprise the following fields:  
         [0056]    Header:  
         [0057]    Destination-Address: multicast address or a unique address (e.g., zeros).  
         [0058]    Source-Address: the address of the source node.  
         [0059]    Length: length of the frame.  
         [0060]    DSAP/SSAP (Source Service Access Point/Destination Service Access Point): a unique value dedicated to the resource allocation protocol (a single value for all types of messages).  
         [0061]    Payload:  
         [0062]    OPCODE.  
         [0063]    STATUS.  
         [0064]    TID.  
         [0065]    FCS (Frame Check Sequence).  
         [0066]    A resource request step  36  is invoked whenever one of nodes  22  receives a request to allocate resources to a data flow coming from its respective subnet. A corresponding resource release step (not shown) is invoked when a data flow terminates, so that the resources that were allocated for the data flow can be torn down and returned to general use. For the sake of example, it will be assumed that node A receives the data flow allocation request, and that the destination address of the data flow is in the subnet of node C. Node A checks its load table to find the most heavily loaded segment on each of rings  26  and  28  between node A and node C, at a load checking step  38 . (Optionally, node A may send a load update request packet at this stage, as well, in order to update its load tables before making the allocation.) For example, it may be in this case that segment S 2  is the most heavily loaded between node A and node C on ring  26 , while segment S 4  is the most heavily loaded between node A and node C on ring  28 . Because of the spatial reuse properties of SRP, node A is indifferent (with respect to this particular data flow) to the load levels on segments S 3  and S 4  of ring  26  and on segments S 1  and S 2  of ring  28 .  
         [0067]    Node A chooses to send the data flow over ring  26  or ring  28  depending on which of the respective segments S 2  and S 4  is the less heavily loaded, at a path selection step  40 . The purpose of this step is to maintain, insofar as possible, a balance between the maximum segment loads on the two rings. Before requesting bandwidth for the data flow, however, node A verifies that there is sufficient additional bandwidth available to it on the selected segment and path, at a verification step  42 . Assuming ring  26  is selected, it is possible that segment S 2  does not have sufficient additional capacity free to carry the desired data flow. It also may be that the data flow will put node A over a pre-allocated resource budget that is available to it on ring  26 . In such cases, node A cannot pass on the data flow through network  20  and must return an error message to the originator of the data flow in its subnet.  
         [0068]    Having chosen the ring over which to send the data flow, and ascertained that there are sufficient resources available there, node A must now reserve the bandwidth that it requires. Preferably, in order to maintain synchronization between the nodes on the same ring, node A sends a lock packet (identified by its OPCODE) around the ring, at a locking step  44 . The lock packet informs all of the nodes that a reservation is about to be made. No additional reservation can be requested until an unlock packet is received (identified by its OPCODE) or sufficient time has passed to invoke a lock-timeout.  
         [0069]    The lock packet makes a complete circuit of ring  26 , returning to node A at a response step  46 . If node A does not receive the lock packet back within a predetermined timeout period, it preferably resends the packet. If the packet still does not return after multiple resends, node A typically invokes the synchronization or topology learning process mentioned above.  
         [0070]    When the lock packet returns to node A indicating that none of the other nodes has denied the request, node A is able to commit the bandwidth needed to carry the requested data flow, at a commitment step  48 . (Alternatively, steps  44  and  46  may be omitted, and node A can go directly to step  48  when it determines that there are sufficient resources available on the chosen ring.) Accordingly, node A sends another packet around ring  26 , hop-by-hop, to advertise the new load it is placing on the ring. This packet is also preferably a UDP/IP packet of the form described above, but with an OPCODE indicating that it is a load update advertisement packet, rather than a load update request, as noted above. If node A sent a lock packet at step  44 , the update advertisement packet also serves as an unlock packet, allowing the other nodes to make further resource reservations. The advertisement packet preferably carries in its payload the entire updated load table for the ring, including the new bandwidth commitment  
         [0071]    As the advertisement packet is passed around the ring, each of the nodes in turn records that the transaction announced by the preceding load update request packet has taken place, and updates its own load tables accordingly, at an update step  50 . When this packet returns to the originating node, that node is able to finalize the transaction and to convey the data flow using the newly-allocated bandwidth.  
         [0072]    Although preferred embodiments are described hereinabove with reference to a ring topology and to certain specific protocols, such as SRP and IP, used in conjunction with this topology, the principles of the present invention may similarly be applied,  mutatis mutandis,  to networks of other topologies, such as mesh topologies, and using other protocols. It will thus be appreciated that the preferred embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.