Patent Publication Number: US-7898957-B2

Title: Non-blocking destination-based routing networks

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application claims the benefit of priority from previously filed provisional application entitled “Non-Blocking Destination-Based Routing Networks,” filed on Dec. 8, 2005, with Ser. No. 60/748,282, and the entire disclosure of which is herein incorporated by reference. 
     This application is a continuation-in-part application of U.S. application entitled “Non-Blocking Internet Backbone Network,” filed on Oct. 3, 2005, with Ser. No. 11/243,117, and the entire disclosure of which is herein incorporated by reference. 
    
    
     BACKGROUND 
     1. Field of the Invention 
     This invention relates to techniques for maintaining the hard quality of service (QoS) of the developing Internet. In particular, it relates to a method and system for controlling the admissible ingress and egress traffic of edge routers of the Internet and for balancing loads among equal-distance shortest paths for achieving a non-blocking Internet. 
     2. Description of Related Art 
     Internet Protocol (IP) networks are traditionally designed to support a best-effort service, with no guarantees on the reliability and the timely delivery of the packets. In interconnected packet networks, such as the Internet, users establish a connection between a source and a destination with a stream of data packet transferred through the network over a network path. The network path is defined by a set of nodes interconnected by a set of links through which packets of the connection are transferred. The network path can be generated by a variety of algorithms, such as the Dijkstra&#39;s shortest path algorithm or the like. Packet networks may have a hierarchical structure in which smaller networks are interconnected by larger networks. A packet network connects to one or more other packet networks through ingress and egress points (routers) of the network. 
     Interior routing protocols are employed by network routers to determine a path through the internal routers of the network along which packets between the ingress and the egress routers are forwarded. Packets received by a router are forwarded to other routers of the network based on a forwarding table constructed in accordance with the interior routing protocol, but may also be forwarded through routes installed with explicit route provisioning. Interior routing protocols may also specify network topology, link capacity/usage, and link-state information (“network information”) that is exchanged between the network routers. Network information allows the routers to construct the corresponding forwarding table. Examples of widely used interior routing protocols, and most relevant to this invention, are the Open Shortest Path First (OSPF) and IS-IS protocols. Common routing protocols such as OSPF and IS-IS choose least-cost paths using link weights, so inferred weights provide a simple, concise, and useful model of intra-domain routing. In this model, every link is labeled with a number called the weight or cost; conventionally this link weight may be a function of delay time, average traffic, and sometimes simply the number of hops between nodes. 
     As IP networks mature and are increasingly being used to support real-time applications, such as voice onto IP-based platforms, the existing IP networks need to provide a new level of QoS for such new applications. Differentiated Services (DiffServ) have become the main QoS architecture for the Internet. DiffServ avoids per-flow bandwidth reservation inside the network. It classifies flows into aggregates (classes), and provides appropriate QoS for the aggregates. A small bit-pattern in each packet—the ToS octet of IPv4 or the Traffic Class octet of IPv6—is used to mark a packet for receiving a particular forwarding treatment at each network node. A service level agreement (SLA) is signed between a service provider and customers to specify the type of services and the amount of traffic required for each type. An SLA codifies what a provider promises to deliver in terms of what, how, and associated penalties for failures. 
     QoS requirements of the premium class of traffic need to be achieved with components in both the data-plane and the control-plane. Data-plane components include traffic shaping and policing, traffic classification, scheduling and buffer management. Control-plane components include signaling and flow admission control (FAC) and network provisioning/traffic engineering. Effective implementations of data-plane components are well understood and available; only local state information in a router or switch is required. 
     In contrast, control-plane components, such as FAC and network dimensioning, remain open issues. When link-state information or link-bandwidth information (e.g., connectivity or available bandwidth) is exchanged between routers, each router in the network has a complete description of the network&#39;s topology. The challenges of the control-plane design arise from the fact that the implementations of control-plane components need the state information of the entire network. Typically, there are millions of flows traversing through a high-speed link, and therefore maintaining the state information of all links of the entire network is simply not practical. 
     There are several proposals for FAC. The general concerns about these proposals include the following. 
     (a) Scalability and Effectiveness: referring to  FIG. 1 , the network equipment that performs provisioning, resource management and FAC is called bandwidth broker (BB). BB architecture implies that admission control decisions are made at a central location for each administrative domain, such as ISP A  101  and ISP B  105 . Although the cost of handling service requests is significantly reduced, it is unlikely that this approach can be scaled upward for large networks. In order to cope with scalability, most relevant studies adopt distributed admission control schemes, which are further distinguished into model-based and measurement-based approaches. Both approaches assess QoS deterioration probability upon service request arrivals; model-based approaches maintain state information for active services and employ mathematical models, whereas measurement-based approaches rely on either passive or active aggregate measurements. The main concern is the effectiveness of the schemes. The centralized FAC, although not scalable, can provide better QoS than the distributed admission control schemes. 
     (b) Applicability to Inter-domain QoS: All FAC schemes must fully address the inter-domain QoS issues. It is anticipated that there will be significant variation in the implementations and resource management strategies from one ISP to another. It is unlikely that we will find a unified approach across the Internet. Cascading different QoS approaches will work only if they cooperate with each other, which is difficult to achieve. For example, if one network uses measurement-based FAC and the other uses model-based FAC, it is unlikely that the end-to-end QoS can be achieved as anticipated for a flow path passing through the two networks. 
     While some QoS capabilities in an isolated environment have been demonstrated, providing end-to-end QoS at a large scale and across domain boundaries remains a challenging and unsolved problem. A need exists for designing a new and practical FAC scheme to maintain the QoS in the future Internet. 
     In another patent application filed on Oct. 3, 2005 with Ser. No. 11/243,117, commonly assigned to the Hong Kong University of Science &amp; Technology, we proposed the concept of non-blocking networks to solve the QoS problem and simplify the FAC design of an MPLS Internet backbone network. A network is called non-blocking if it can always accommodate a new flow (or a dynamic service-level-agreement) as long as the ingress and the egress nodes or routers have capacity to set up the flow. One major advantage of the invention is that, if a network is non-blocking, its FAC will be greatly simplified as we do not need to check the capacity utilization of all internal links. Those methods described in U.S. patent application Ser. No. 11/243,117 are for MPLS-type networks that use explicit routing. The Internet uses destination-based routing where each node uses the destination of the packet to route the packet. Destination-based routing is also called hop-by-hop routing. Therefore, there is a need for a method, apparatus, and system for designing a non-blocking hop-by-hop backbone network. 
     SUMMARY 
     In this patent, we present methods, apparatus, and system to build non-blocking destination-based routing networks, such as the Internet. According to the methods and system of this invention, only the ingress and egress routers need to be verified for accepting a new flow. As demonstrated in  FIG. 2 , we will assume a new flow from router A  201  to router D  203  needs to be added. If the network is non-blocking, we only need to check if the ingress node or router (i.e., router A  201 ) and the egress node or router (i.e., router D  203 ) have the required capacity to accommodate the new flow. The complex task of monitoring the status of each link inside the network, such as links  210 ,  215 , and  220 , can be avoided. 
     Given a network topology and its link capacities, the present invention provides a way to determine the optimal link weights and the resulting paths that can maximize the amount of admissible ingress and egress traffic of the network and still maintain the non-blocking property. As long as the ingress and egress traffic is within this specified amount and the routing follows the determined paths and the associated load-distribution scheme, the traffic load on any internal link will never exceed the link capacity. Thus, the network is non-blocking internally. 
     Another aspect of this invention includes a blocking mechanism with two thresholds for on-off admission control of each direction. A router has a meter inside. When the total ingress (or egress) traffic exceeds t_high, flow control is turned on and ingress (or egress) flows will be blocked. When the total amount of traffic drops below t_low, flow control will be turned off and ingress (or egress) flows are allowed to enter again. Detecting flows is based on both flow-setup packets and flow-setup ACK packets. 
     In accordance with another aspect of the invention, a FAC scheme is invented which decides whether a new TCP or UDP flow can be added to the network. The FAC scheme only requires the ingress and egress edge routers to monitor the amount of ingress and egress traffic and there is no need to check the state information of the internal links of the network. The FAC scheme also includes a load-distribution scheme to balance the loads among the equal-distance shortest paths between any two edge routers. 
     In accordance with still another aspect of the invention, we can set an arbitrary amount as the admissible ingress and egress traffic constraints for each node and compute the capacities of the internal links of the network such that the network is non-blocking. Along with the internal link capacities are the paths for each source-destination pair and the associated load-distribution ratios among these paths. 
     Further features of the invention, its nature and various advantages will be apparent from the accompanying drawings and the following detailed description of the invention. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a schematic representation of the traditional bandwidth broker infrastructure. 
         FIG. 2  is a schematic representation of the internal links connecting the various edge routers within a network. 
         FIG. 3(   a ) is a schematic representation of router architecture. 
         FIG. 3(   b ) is logical view of the blocking mechanism implemented in a router. 
         FIG. 3(   c ) is a diagram of the blocking and unblocking of the setup packets according to the blocking mechanism implemented in a router. 
         FIG. 4(   a ) is a schematic representation of a protocol processor. 
         FIG. 4(   b ) is a schematic representation of a sample balancing scheme. 
         FIG. 5  is a typical hardware system which can be used to implement the method of this invention. 
     
    
    
     Like reference numbers and designations in the different figures indicate like elements. 
     DETAILED DESCRIPTION 
     This invention involves the novel design of a non-blocking network. In this invention, a method and a system for determining traffic constraints, routes, and load-distribution among the equal-distance shortest paths to achieve non-blocking in each router are implemented. 
     The invention will be illustrated in conjunction with illustrative embodiments of a network and an associated route computation and load-distribution scheme. It should be understood, however, that the invention is not limited to use with the particular network system or node implementation described, but is instead more generally applicable to any route computation algorithm that intends to make network non-blocking. 
       FIG. 3(   a ) is a diagram illustrating a typical router  300  according to one embodiment of the invention. The router  300  includes a routing processor  310 , and the switching fabric  320 . The switch fabric  320  is an interconnected network of switching devices connecting multiple numbers of input ports, such as input port  312 , input  315 , etc., and output ports, such as output port  322 , output port  325 , etc. The switch fabric  320  transports data from the input ports to the output ports. 
     The router  300  may also include a flow blocking mechanism. Referring to  FIG. 3(   b ), there are two thresholds for on-off admission control of each direction. Router  300  has a meter inside, the Admission Control Block  330 . When the total ingress or egress traffic exceeds t_high, flow control is turned on and ingress or egress flow setup packets and flow setup acknowledgement packets will be blocked ( 335 ). When the total amount of traffic drops below t_low, flow control will be turned off and ingress or egress flow setup packets and flow setup acknowledgement packets are allowed to enter or leave again ( 332 ). 
     As well known for persons in the art, a flow is a sequence of packets transmitted along a defined route. Flow setup packets and acknowledgement of flow setup packets are distinguished from other data packets of the flow which are transmitted after a flow is set up. Note that detecting flows is based on both flow-setup packets and flow-setup ACK packets. For example, referring to  FIG. 3(   c ), a TCP connection setup packet arrives at edge router A  201 , and leaves the network from edge router B  202 . This means that the ingress flow control in A and the egress flow control in B have not been turned on. Assume the setup packet reaches the destination host and a TCP flow ACK packet is sent back (SYN=1, ACK=1). This packet will enter the network from edge router B, and leave the network from edge router A. This ACK packet can be blocked by B if B&#39;s ingress flow control is turned on. The same discussions apply to session layer flow control packets (like SIP). 
     Assume a network has n edge routers. The techniques of this invention will produce the following outputs. 
     The link weights and the resulting routes between any pair of edge routers. 
     The ingress and egress admissible traffic constraints, denoted by (α i ,β i ), 1≦i≦n, of all edge routers. This determines how much traffic each edge router can inject into or receive from the network without causing overflow on any link. 
     Referring to  FIG. 2 , which illustrates a network  200  with, for example, six routers. Four routers within the network are edge routers: router A  201 , router B  202 , router D  203 , and router E  204 . All edge routers  201 - 204  are connected to the outside world. All links, such as link AB  210 , link BC  215 , and link CD  220 , have the same capacity of 1 unit on both directions and all edge routers have the same admissible ingress and egress traffic constraints—that is, α i =β i =γ, 1≦i≦n. 
     Using our techniques, we find the following: 
     1. γ=2, where γ is the maximum amount of traffic that any edge router A  201 , B  202 , D  203 , E  204  can inject into or receive from the network. 
     2. A set of paths from an edge router to another edge router. We can use (A, B) and (A, D) as an example. The rest of the paths can be deduced accordingly. 
     
         
         
           
             (a) (A, B): two paths: A-B  210  and A-F-B. The load-balancing ratios are 0.5 and 0.5, respectively. 
             (b) (A, D): A-D, A-F-D. Each path gets 0.5 of traffic from A to D. 
           
         
       
    
     Consider the traffic injected by all edge routers into the network. As long as the total amount of traffic injected by an edge router does not exceed 2 and as long as the edge routers will not receive more than 2 from all the routers, it can be verified that the traversing traffic on any internal link will never exceed 1 regardless of the distribution of the traffic between each edge router pair. This makes the FAC decision an easy task because we only need to check the ingress and the egress routers to see if adding a flow will violate the ingress and egress admissible constraints. For example, suppose we want to set up a new flow with rate=0.5 from A  201  to B  202 . We first check if the current load of both A ( 201 ) and B ( 202 ) has exceeded 1.5. If not, the flow is accepted; otherwise the flow is rejected. It is not necessary to check the status of each link inside the network. 
     The techniques can also answer the reversed question, namely the internal link bandwidth that can make the network non-blocking for γ=3. The answer is 1.5, which is easily derived from the example by just scaling up the link capacity accordingly. 
     To implement the scheme, the lookup engine in a router needs to do load-distribution according to the computed load-balancing ratios. It also needs to do this without causing out-of-sequence transmissions. There are many ways to conduct load balancing. One example is shown in  FIG. 4 .  FIG. 4(   a ) shows a standard protocol processor  400 . It has a Protocol Processing unit  410 , which are fed by data through Data RX  412 , Data RX  415 , etc. It also has Table Lookup unit  420 , which feeds data into Egress Queue  422 , Egress Queue  425 , etc.  FIG. 4(   b ) is a schematic representation of a load-balancing unit  450  with the Table Lookup Engine  438  conducting load balancing according to the computed load-balancing ratios. The hashing performed by Hashing unit  430  maps the host number field of the IP packet into a random number between [0, M]. Based on the result we select a path. For example, if the range is [0, 99] and there are three pre-determined paths with load-distribution ratios: 0.2, 0.4, and 0.4. If the output of the hashing unit  430  is between [0, 19], we use the first path. If it is in the range [20, 59], we use the second path; if in the range [60, 99], the third path. Alternatively, we can also use (destination host+TCP channel) as the input of the hashing unit. This can do load-balancing with a finer granularity. 
     Weight Setting Algorithms for Non-blocking Destination-Based Routing Networks 
     We assume the network has n edge routers and the ingress and egress bandwidth constraints of each router are given in the following H vectors
 
 H =[(θ{tilde over (α)} 1 ,θ{tilde over (β)} 1 ), . . . (θ{tilde over (α)} n ,θ{tilde over (β)} n )]  (1)
 
where (θ{tilde over (α)} i ,θ{tilde over (β)} i ) represent the maximum amount of ingress and egress traffic allowed to enter the network at the edge router i, where {tilde over (α)} 1  and {tilde over (β)} 1  are constants describing the degree of unevenness of traffic patterns in the network and θ is a parameter to be maximized in our design. For example, suppose ({tilde over (α)} 1 =5,{tilde over (β)} 1 =5) and ({tilde over (α)} 2 =15,{tilde over (β)} 2 =15), then the traffic allowed at edge router 2 is three times that of router 1. Note that only the relative—not absolute—magnitudes of {tilde over (α)} i  and {tilde over (β)} i  have significance as the real amount of admissible traffic is determined by θ.
 
     Compared with a conventional traffic matrix T={d ij }, where d ij  represents the traffic rate from node i to node j, we must have 
                 ∑   j     ⁢     d     i   ⁢           ⁢   j         ≤     θ   ⁢           ⁢       α   ∼     i     ⁢           ⁢   and   ⁢           ⁢       ∑   i     ⁢     d     i   ⁢           ⁢   j           ≤     θ   ⁢           ⁢         β   ∼     j     .             
Traffic pattern of this kind was also called “hose model” pattern. For a given H, there are many traffic matrices T s that satisfy the constraints imposed by H. A traffic matrix T={d ij } that does not violate the constraints imposed by H is called a valid traffic matrix. Let D be the set of all valid T s.
 
     The network is described as a directed graph G(V,E) where V is the set of vertices (nodes) and E is the set of links. Let Q ⊂ V be the set of edge routers through which traffic is admitted into the network. 
     Define: 
     C e : the capacity for link eεE, 
     w e : the weight of link e, and {1, . . . , K} be the set of possible weights. 
     x ij   e : the portion of traffic from node iεQ to jεQ routed through link e. 
     Note that if link weights are given, routing (and x ij   e ) will be determined as long as we use the shortest path routing and even load-balancing among equal-distance paths. 
     The patent describes several methods to compute the routes and the maximum θ such that the backbone network is non-blocking. Define link congestion ratio as the ratio of the amount of traffic routed through a link over the link&#39;s capacity. The network congestion ratio r refers to the maximum value of all link congestion ratios. If r&lt;1, the network has no congestion as traffic routed through any link is below the link capacity. We use the concept of minimizing congestion ratio to find the optimal weights. θ and r are related. To maximize θ, we first assume the ingress and egress traffic constraints of node i are ({tilde over (α)} i ,{tilde over (β)} i )—instead of (θ{tilde over (α)} i ,θ{tilde over (β)} i )—and compute the network congestion ratio r. If we change the admissible amount at node i to 
               (           α   i     ∼     r     ,         β   i     ∼     r       )     ,         
the new congestion ratio of the network is guaranteed to be ≦1. Thus
 
             (           α   i     ∼     r     ,         β   i     ∼     r       )         
will be the maximum admissible amount at edge router i. The problem of maximizing θ is now converted to the minimization of r as θ=1/r.
 
     To put it differently, consider the following: a network has five links,  11 ,  12 ,  13 ,  14 , and  15 . The capacity of each link is given below:
       11 ,  12 ,  13 : 10 Mbps     14 ,  15 : 20 Mbps
 
Traffic routed through each link:
     11 : 15 Mbps,  12 : 15 Mbps,  13 : 10 Mbps     14 : 40 Mbps, and  15 : 10 Mbps
 
The congestion ratio of all links are: 11:1.5, 12:1.5, 13:1 14:2, 15:0.5
   

     Accordingly, the network congestion ratio (the largest value among all) is 2.0, which means that if we lower the ingress and egress admissible traffic by a factor of 2.0, none of the links will experience congestion (all link congestion ratios will be ≦1.0). As traffic is routed to minimize the congestion ratio of the network, the admissible amount of traffic is maximized. Note that the congestion ratio is not always to lower the admissible ingress and egress traffic; it can be used to increase the traffic. 
     Let o(e) and t(e) denote the originating and terminating nodes of link e. Let W(i,j) be the length of the shortest path from i to j. Let δ(e,j) be a set of binary variables such that δ(e,j)=1 if link e is on a shortest path to node j, and 0 otherwise. Let
 
 f   ij ( v )=(the portion of traffic from  i  to  j  that arrives at  v )/ m,  
 
where there are m shortest paths incident from v. In hop-by-hop routing, once link weights are determined, paths will be known and the amount of admissible QoS traffic can be computed accordingly. So the key question in maximizing QoS traffic of a DiffServ Internet will be the setting of link weights.
 
     Method 1 
     We can use the following mixed-integer programming technique to find the optimal link weights that can minimize r, and consequently maximize θ. 
                     min   ⁢           ⁢   r     ⁢     
     ⁢     subject   ⁢           ⁢   to                         (     2   ⁢           ⁢   a     )                           ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x     i   ⁢           ⁢   j     e       -       ∑     e   ⁢           ∈           ⁢       Γ   -     ⁢     (   v   )           ⁢     x             ⁢     i   ⁢           ⁢   j                 ⁢   e           =   0           i   ,     j   ∈   Q     ,     v   ≠   i     ,   j                             (     2   ⁢           ⁢   b     )                           ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x     i   ⁢           ⁢   j     e       -       ∑     e   ⁢           ∈           ⁢       Γ   -     ⁢     (   v   )           ⁢     x             ⁢     i   ⁢           ⁢   j                 ⁢   e           =   1           i   ,     j   ∈   Q     ,     v   =   i                               (     2   ⁢           ⁢   c     )                           ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x     i   ⁢           ⁢   j     e       -       ∑     e   ⁢           ∈           ⁢       Γ   -     ⁢     (   v   )           ⁢     x             ⁢     i   ⁢           ⁢   j                 ⁢   e           =     -   1             i   ,     j   ∈   Q     ,     v   =   j                               (     2   ⁢           ⁢   d     )                           ∑     i   ⁢           ∈           ⁢   Q       ⁢                 ⁢     α   ∼       i     ·       π   e     ⁡     (   i   )           +       ∑     i   ⁢           ∈           ⁢   Q       ⁢         β   ∼     i     ·       λ   e     ⁡     (   i   )             ≤       c   e     ·   r             e   ∈   E                             (     2   ⁢           ⁢   e     )                       x     i   ⁢           ⁢   j     e     ≤         π   e     ⁡     (   i   )       +       λ   e     ⁡     (   j   )                 i   ,     j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   f     )                     0   ≤         f     i   ⁢           ⁢   j       ⁡     (     o   ⁡     (   e   )       )       -     x     i   ⁢           ⁢   j     e       ≤     1   -     δ   ⁡     (     e   ,   j     )                 i   ,     j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   g     )                       x     i   ⁢           ⁢   j     e     ≤     δ   ⁡     (     e   ,   j     )               i   ,     j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   h     )                     0   ≤       W   ⁡     (       t   ⁡     (   e   )       ,   j     )       +     w   e     -     W   ⁡     (       o   ⁡     (   e   )       ,   j     )         ≤       (     1   -     δ   ⁡     (     e   ,   j     )         )     ⁢   M               j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   i     )                       1   -     δ   ⁡     (     e   ,   j     )         ≤       W   ⁡     (       t   ⁡     (   e   )       ,   j     )       +     w   e     -     W   ⁡     (       o   ⁡     (   e   )       ,   j     )                   j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   j     )                     0   ≤     x     i   ⁢           ⁢   j     e     ≤   1           i   ,     j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   k     )                         π   e     ⁡     (   i   )       ,         λ   e     ⁡     (   i   )       ≥   0               i   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   l     )                         f     i   ⁢           ⁢   j       ⁡     (   t   )       ≥   0           i   ,     j   ∈   Q     ,     t   ∈   V                               (     2   ⁢           ⁢   m     )                       W   ⁡     (     t   ,   j     )       ≥   0             j   ∈   Q     ,     t   ∈   V                               (     2   ⁢           ⁢   n     )                       δ   ⁡     (     e   ,   j     )       ∈     {     0   ,   1     }               j   ∈   Q     ,     e   ∈   E                               (     2   ⁢           ⁢   o     )                     1   ≤     w   e     ≤   K           e   ∈   E                             (     2   ⁢           ⁢   p     )               
Constraints (2g) and (2h) are the flow splitting constraints such that traffic is split to the shortest paths according to the even distribution rule. Constraints (2i) and (2j) are the shortest path constraints. If link e does not lie on any shortest path to node j (i.e., δ(e,j)=0), W(t(e),j)+w e −W(o(e),j)≧1 must hold because w e ≧1. This is stated by (2j). On the other hand, constraint (2i) implies that W(t(e),j)+w e −W(o(e),j)=0 if link e is on one of the shortest paths to node j. In addition, when δ(e,j)=0, (2i) becomes redundant if M (an artificial constant) is sufficiently large.
 
     Method 2 
     The above method is based on mixed-integer programming and its complexity is high. Another method is given below that takes a heuristic approach and has a much lower complexity. The algorithm uses iterative search and there are two stages in the search. At the first stage, we assume link weights are given. We find the worst-case traffic matrix T, in terms of congestion ratio minimization. At the second stage, the newly generated T will be used to search for a new set of link weights. The search will be guided by the minimization of an objective function that will play a critical role in the efficiency of the search algorithm. 
     Stage 1: The first stage is based on the fact that once link weights are known, routing x ij   e  is determined. Then we can compute a new worst-case traffic matrix T that leads to the largest congestion ratio r by solving the following linear programming (LP) problem for each link. 
                     max   ⁢           ⁢       ∑     i   ⁢           ⁢   j       ⁢       x     i   ⁢           ⁢   j     e     ⁢     d     i   ⁢           ⁢   j             ⁢     
     ⁢     subject   ⁢           ⁢   to             (     3   ⁢   a     )                         ∑     j   ∈   Q       ⁢     d     i   ⁢           ⁢   j         ≤       α   ∼     i             i   ∈   Q                 (     3   ⁢           ⁢   b     )                         ∑     i   ∈   Q       ⁢     d     i   ⁢           ⁢   j         ≤       β   ∼     j             j   ∈   Q                 (     3   ⁢           ⁢   c     )                       d     i   ⁢           ⁢   j       ≥   0           i   ,     j   ∈   Q                   (     3   ⁢           ⁢   d     )               
where constraints (3b) and (3c) are the ingress and egress bandwidth constraints. We add the newly generated T to the set {tilde over (D)} that stores the worst-case traffic matrices Ts computed from the previous steps. The worst-case traffic matrices in {tilde over (D)} will serve as the representative traffic matrices in our search for the optimal link weights at the second stage.
 
     Stage 2: We now have a new set {tilde over (D)} which will be used as the representative traffic matrices in the search of new link weights. We use simulated annealing, a typical local search method, to find a new set of link weights that can achieve a lower objective function. Our objective is to minimize congestion ratio r. But r cannot be directly used as the objective function for the local search. Note that given a routing, we can compute the maximum link utilization for traffic matrix T, which is denoted as r T . Obviously the network congestion ratio r=max{r T }, for all Tε{tilde over (D)}. Changing link weights can reduce a particular r T , but may increase another r T . Thus, the network congestion ratio r may not be improved, and this makes r a poor objective function for the search. 
     Therefore, the objective function should include r T  for all traffic matrices in {tilde over (D)}. We use the following objective function to guide our search. 
                       F   (     D   ∼     )       ❘     for   ⁢           ⁢   given   ⁢           ⁢   weights         =       ∑     T   ∈     D   ∼         ⁢     ϕ   ⁡     (     r   T     )                 (   4   )               
where φ(r T ) is the cost function for r T  and is defined as a convex piece-wise linear cost function given below:
 
     
       
         
           
             
               
                 
                   
                     ϕ 
                     ⁡ 
                     
                       ( 
                       
                         r 
                         T 
                       
                       ) 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                               
                             ⁢ 
                             
                               r 
                               T 
                             
                           
                         
                         
                           
                             0 
                             ≤ 
                             
                               r 
                               T 
                             
                             &lt; 
                             
                               1 
                               3 
                             
                           
                         
                       
                       
                         
                           
                               
                             ⁢ 
                             
                               
                                 3 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   r 
                                   T 
                                 
                               
                               - 
                               
                                 2 
                                 3 
                               
                             
                           
                         
                         
                           
                             
                               1 
                               3 
                             
                             ≤ 
                             
                               r 
                               T 
                             
                             &lt; 
                             
                               2 
                               3 
                             
                           
                         
                       
                       
                         
                           
                               
                             ⁢ 
                             
                               
                                 10 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   r 
                                   T 
                                 
                               
                               - 
                               
                                 16 
                                 3 
                               
                             
                           
                         
                         
                           
                             
                               2 
                               3 
                             
                             ≤ 
                             
                               r 
                               T 
                             
                             &lt; 
                             
                               9 
                               10 
                             
                           
                         
                       
                       
                         
                           
                               
                             ⁢ 
                             
                               
                                 70 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   r 
                                   T 
                                 
                               
                               - 
                               
                                 178 
                                 3 
                               
                             
                           
                         
                         
                           
                             
                               9 
                               10 
                             
                             ≤ 
                             
                               r 
                               T 
                             
                             &lt; 
                             1 
                           
                         
                       
                       
                         
                           
                               
                             ⁢ 
                             
                               
                                 500 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   r 
                                   T 
                                 
                               
                               - 
                               
                                 1468 
                                 3 
                               
                             
                           
                         
                         
                           
                             1 
                             ≤ 
                             
                               r 
                               T 
                             
                             &lt; 
                             
                               11 
                               10 
                             
                           
                         
                       
                       
                         
                           
                               
                             ⁢ 
                             
                               
                                 5000 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   r 
                                   T 
                                 
                               
                               - 
                               
                                 16318 
                                 3 
                               
                             
                           
                         
                         
                           
                             
                               11 
                               10 
                             
                             ≤ 
                             
                               r 
                               T 
                             
                             &lt; 
                             ∞ 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
     Eq. (5) implies that cost φ(r T ) grows drastically as r T  increases. Note that to our search problem, the exact definition of the cost function is not so important as its convexity. 
     At the end of stage 2, based on the newly obtained weights, we re-compute the congestion ratio r for the hose-model vector H. The program terminates if the new congestion ratio differs from the old one by an amount smaller than a pre-defined ε. The framework of the heuristic algorithm is presented in CHART 1. 
     
       
         
           
               
             
               
                 CHART 1 
               
               
                   
               
               
                 Framework of the Heuristic Algorithm 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
            
               
                 Step 0. Initialization. 
               
               
                  Initialize a weight setting w , setting w e  = 1 , for e ∈ E . Compute the 
               
               
                  traffic distribution x e   ij , based on the weight setting w . Solve Eq. (3) to 
               
               
                  determine the congestion ratio r . Set the set of traffic matrices {tilde over (D)} = Ø . 
               
               
                  Set optimal weight setting as w* = w . 
               
               
                 Step 1. Determine the set of traffic matrices. 
               
               
                  For each link, determine the traffic matrix T that achieves the congestion 
               
               
                  ratio r. If T ∉ {tilde over (D)} , put T into {tilde over (D)} . 
               
               
                 Step 2. Search the optimal weight setting for the set of traffic matrices in {tilde over (D)} 
               
               
                  Use local search algorithms (such as simulated annealing) to find the 
               
               
                  optimal weight setting for the set of traffic matrices in {tilde over (D)} . The best weight 
               
               
                  setting obtained is denoted as w best  .  
               
               
                 Step 3. Determine the new congestion ratio r for weight setting w best  . 
               
               
                  If r differs from the old one by an amount smaller than a defined ε , set 
               
               
                  w* = w best  and terminate. Otherwise, return to Step 1. 
               
               
                   
               
            
           
         
       
     
     The key issue of the proposed heuristic algorithm is to search the optimal link weight for a given set of traffic matrices at the second stage. In our work, we use simulated annealing method to achieve this goal. Important issues of the method include the space for which the search is to be carried out and an effective neighborhood structure. In our algorithm, the search space is the link weight space which is defined by w e ε{1, 2, . . . , K} for all eεE. We use a neighborhood structure that is based on modifying the link weight of one single link. Note that randomly changing a link weight value does not always result in a different set of shortest paths (and, thereby, a different traffic distribution). We develop a neighborhood structure where a neighbor of the current weight always implies a different traffic distribution. Suppose link e is selected and its current weight is w e . We randomly select a source-destination pair (i,j). Then based on the load distribution variable x ij   e  (i.e., x ij   e =0, 0&lt;x ij   e &lt;1, or x ij   e =1), we can easily compute the minimum amount that w e  needs to be changed to obtain a different load distribution x ij   e . 
     At each step, the algorithm randomly selects a link and modifies the link weight (increase or decrease randomly) to obtain a new routing and a new traffic distribution. The objective function is calculated according to the new traffic distribution and compared to the old one. The algorithm then uses the simulated annealing scheme to decide if the new weight setting will be accepted or not. Since the initial link weights play an important role in the effectiveness of the method, we randomly sample 10 independent initial weighs and pass them to the simulated annealing algorithm at the second stage. 
     The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. The invention, for example, can be implemented in a computing system of  FIG. 5  including a processor  505 , an I/O bus  510 , system memories  515  and  520 , a CPU bus  525 , and interface slots to connect to a variety of I/O devices, including storage devices, such as a removal disk drive  530 . The processor represents a central processing unit of any type of architecture, such as complex instruction set computers (CISC), reduced instruction set computers (RISC), very long instruction word (VLIW), or hybrid architecture. The invention could be implemented in a multi-processor or single processor computer system. The system memories may include flash memory, read only memory (ROM)  520 , or random access memory (RAM)  515 . The system memories may include a program and/or data. The program may contain traffic management programs such as hop-by-hop selector. The data may contain databases such as resource bandwidth constraint, link weights, other routing protocol databases, and the like. 
     When implemented in software, the elements of the present invention are essentially the code segments to perform the necessary tasks. The program or code segments can be stored in a processor readable medium or transmitted by a computer data signal embodied in a carrier wave over a transmission medium. The “processor readable medium” may include any medium that can store or transfer information. Examples of the processor readable medium include an electronic circuit, a semiconductor memory device, a ROM, a flash memory, an erasable ROM (EROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, a fiber optic medium, a radio frequency (RF) link, etc. The computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic, RF links, etc. The code segments may be downloaded via computer networks such as the Internet, Intranet, etc. 
     Those skilled in the art will readily observe that numerous modifications and alterations of the device may be made without departing from the scope, spirit or teachings of the invention. The invention is defined by the following claims and their equivalents.