Abstract:
A method and system for computing the maximum amount of admissible ingress and egress traffic of each edge router of the MPLS core network is presented. As long as the ingress and egress traffic amounts are below the admissible amount limit, traffic routes through any link in the network will never exceeds the link capacity. A calculation scheme and a load-distribution scheme make an Internet MPLS backbone network non-blocking. The output of the route computation algorithm includes a set of paths for any source-destination pair and the load distribution ratios among the paths. The routing algorithm and the associated distribution scheme will determine how to balance the loads that meet the distribution requirements specified by the ratios computed from the route algorithm. This greatly simplifies call admission control and allows hard QoS to be supported in a large scale and cross domains in the Internet.

Description:
CROSS-REFERENCE TO RELATED APPLICATION  
       [0001]     This application claims the benefit of previously filed provisional application entitled “QoS Routing Methods and the Design of a Non-Blocking Internet Backbone Network,” filed on Feb. 7, 2005, with serial number 60/650,341, and the entire disclosure of which is herein incorporated by reference. 
     
    
     BACKGROUND  
       [0002]     1. Field of the Invention  
         [0003]     This invention relates to techniques for maintaining the hard quality of service (QoS) of the developing Internet. In particular, it relates to a method and system for controlling the admissible ingress and egress traffic of edge routers of the Multiprotocol Label Switching (MPLS) core network for achieving a non-blocking Internet backbone network.  
         [0004]     2. Description of Related Art  
         [0005]     Internet Protocol (IP) networks are traditionally designed to support a best-effort service, with no guarantees on the reliability and the timely delivery of the packets. As IP networks mature and are increasingly been used to support real-time applications, such as voice onto IP-based platforms, the existing IP networks need to provide a new level of QoS for such new applications. Differentiated Services (DiffServ) and MPLS have become the main QoS architecture for the Internet. DiffServ avoids per-flow bandwidth reservation inside the network. It classifies flows into aggregates (classes), and provides appropriate QoS for the aggregates. A small bit-pattern in each packet—the ToS octet of Ipv4 or the Traffic Class octet of Ipv6—is used to mark a packet for receiving a particular forwarding treatment at each network node. A service level agreement (SLA), is signed between a service providers and customers to specify the type of services and the amount of traffic required for each type. An SLA codifies what a provider promises to deliver in terms of what, how, and associated penalties for failures.  
         [0006]     QoS requirements of SLAs need to be achieved with components in both the data-plane and the control-plane. Data-plane components include traffic shaping and policing, traffic classification, scheduling and buffer management. Control-plane components include SLA creation and configuration, signaling and SLA admission control (SAC) and network provisioning/traffic engineering. Effective implementations of data-plane components are well understood and available; only local state information in a router or switch is required. In contrast, control-plane components, such as SAC and network dimensioning, remain open issues. The challenges of the control-plane design arise from the fact that the implementations of control-plane components need the state information of the entire network. Typically there are millions of flows traveling through a high-speed link, and therefore maintaining the state information of all links of the entire network is simply not practical.  
         [0007]     There are several proposals for SAC. The general concerns about these proposals include the following.  
         [0008]     (a) Scalability and Effectiveness: referring to  FIG. 1 , the network equipment that performs provisioning, resource management and SAC is called bandwidth broker (BB). BB architecture implies that admission control decisions are made at a central location for each administrative domain, such as ISP A 101 and ISP B 105. Although the cost of handling service requests is significantly reduced, it is unlikely that this approach can be scaled upward for large networks. In order to cope with scalability, most relevant studies adopt distributed admission control schemes, which are further distinguished into model-based and measurement-based approaches. Both approaches assess QoS deterioration probability upon service request arrivals; model based approaches maintain state information for active services and employ mathematical models, whereas measurement-based approaches rely on either passive or active aggregate measurements. The main concern is the effectiveness of the schemes. The centralized SAC, although not scalable, can provide better QoS than the distributed admission control scheme.  
         [0009]     (b) Applicabilityy to Inter-domain QoS All SAC schemes must fully address the inter-domain QoS issues. It is anticipated that there will be significant variation in the implementations and resource management strategies from one ISP to another. It is unlikely that we will find a unified approach across the Internet. Cascading different QoS approaches will work only if they cooperate with each other. That is difficult to achieve. For example, if one network uses measurement-based SAC and the other uses model-based SAC, it is unlikely that the end-to-end QoS can be achieved as anticipated for an SLA path passing through the two networks  
         [0010]     While some QoS capabilities based on isolating voice traffic over IP are currently evolving (e.g., DiffServ, MPLS), providing end-to-end QoS at a large scale and across domain boundaries remains a challenging and unsolved problem. Thus a need exists for designing a new and practical SAC scheme to maintain the QoS in the future Internet.  
       SUMMARY  
       [0011]     This invention uses the concept of non-blocking network to solve the QoS problem and simplify the SAC design of the fuiture Internet. Among the advantages of the invention comprise one or more of the following: one advantage of the claimed invention is to provide a non-blocking network. A network is called non-blocking if it can always accommodate a new SLA as long as the source and destination routers have capacity to set up the flow. One advantage of the invention is that, if a network is non-blocking, its SAC will be greatly simplified as we do not need to check the capacity utilization of all internal links. According to the method and system of this invention, only the ingress and egress points need to be verified for accepting a new SLA. As demonstrated in  FIG. 2 , assuming a new SLA from router A  201  to router D  205  needs to be added. If the network is non-blocking, we only need to check if the ingress router (i.e. router A  201 ) and the egress router (i.e. router D  205 ) have capacity to accommodate the new SLA. The complex task of monitoring each link inside the network, such as links  210 ,  215 ,  220 ,  225 , and  230 , can be avoided.  
         [0012]     Given anetwork topology and its link capacities, the present invention provides a way to determine a set of paths for any given source-destination pair in an MPLS network and the maximum admissible ingress and egress traffic. As long as the ingress and egress traffic is within this specified amount and the routing follows the determined paths and the associated load-distribution scheme, none of the internal links will ever exceed the link capacity. Thus, the network is non-blocking internally. As an added advantage, the techniques are well suited for the class of traffic demanding the best QoS, but can also be used with other types of data traffic.  
         [0013]     Yet another advantage of this invention is the flexibility in the “turning-on” and “turning-off” of the method and system of the invention according to other relevant considerations in the overall bus utilization or overall performance.  
         [0014]     A SAC scheme is invented which decides whether a new SLA can be added to the network. The SAC scheme only requires the source and destination edge routers to monitor their ingress and egress traffic situations and there is no need to check the state information of internal links of the network. The SAC scheme also includes a load-distribution scheme to balance the loads among the paths assigned for each source-destination pair.  
         [0015]     Additional constraints can be added to the paths generation scheme. For example, we can specify that a maximum of k routes will be used for any given source-destination pair, or we can specify the hop-count limit for all the paths. We then use the algorithm described in the patent to compute the maximum admissible ingress and egress traffic. As long as the total ingress and egress traffic is within the admissible constraints, the network is still non-blocking.  
         [0016]     Furthermore, we can set an arbitrary amount as the admissible ingress and egress traffic constraints and compute the capacities of internal links of the network such that the network is non-blocking. Along with the internal link capacities are the paths for each source-destination pair and the associated load-distribution ratios among these paths.  
         [0017]     In yet another aspect of the invention deals with link failure. When a link fails, the techniques in the patent can determine the amount of admissible traffic we need to reduce to keep the non-blocking properties of the network. Except the paths affected by the link failure, all the other paths remain unchanged (Prof. Lea, this aspect is not discussed anywhere else. Do you want to keep this still?).  
         [0018]     Further features of the invention, its nature and various advantages will be more apparent from the accompanying drawings and the following detailed description of the invention. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0019]      FIG. 1  is a schematic representation of the traditional bandwidth broker infrastructure.  
         [0020]      FIG. 2  is a schematic representation of internal links connecting the various edge routers within a network.  
         [0021]      FIG. 3 ( a ) is a schematic representation of a protocol processor.  
         [0022]      FIG. 3 ( b ) is a schematic representation of a sample balancing scheme. 
     
    
       [0023]     Like reference numbers and designations in the different figures indicate like elements.  
       DETAILED DESCRIPTION  
       [0024]     This invention involves the novel design of a non-blocking network. In this invention, a method and a system for determining traffic constraints, routes, and load-distribution among the routes to achieve non-blocking inside an MPLS Internet backbone network are implemented.  
         [0025]     The invention will be illustrated in conjunction with illustrative embodiments of a network and an associated route computation and load-distribution scheme. It should be understood, however, that the invention is not limited to use with the particular network system or node implementation described, but is instead more generally applicable to any route computation algorithm that intends to make network non-blocking.  
         [0026]     Assume a network has n edge routers. The techniques of this invention will produce the following outputs.  
         [0027]     The ingress and egress admissible traffic constraints, denoted by (α i , β i ), 1≦ i≦n, of all edge routers. This determines how much traffic each edge router can inject into or receive from the network without causing overflow on any link    
         [0028]     The routing method will generate a fixed number of paths, say 4 in our example, between any pair of nodes. The four paths from edge router i to edge router j are denoted by p y   1 , p y   2 , p y   3 , p y   4  and the associated load-balancing ratios v y   1 , v y   2 , v y   3 , v y   4 . For all traffic from edge router i to edge router j, the network will split the load among the paths according to the ratios v BC   1 , v BC   2 , v BC   3 , v BC   4 .  
         [0029]     Referring to  FIG. 2 , which illustrates a network  200  with, for example, six routers which are all edge routers: router A  201 , router B  203 , router C  204 , router D  205 , router E  207 , router F  209 . All routers  201 - 207  are connected to the outside world. All links, such as link AB  201 , link BC  216 , link CD  220 , link DF  225 , and link EA  230 , have the same capacity of 1 unit on both directions and all edge routers have the same admissible ingress and egress traffic constraints—that is, α i =β i =γ, 1≦i≦n. Using our techniques, we find the following:  
         [0030]     1. γ=2, where γ is the amount of traffic that any router A  201 , B  203 , C  204 , D  205 , E  207 , F  209  can only inject into or receive from the network.  
         [0031]     2. A set of paths for each (source, destination) pair. We can use (A, B) and (A,D) as an example. The rest of the paths can be deduced accordingly 
        (i) (A, B): three paths: A-B  210 , A-C-B, A-F-B. The load-balancing ratios are 0.5, 0.25, 0.25.     (ii) (A,D): A-B-D, A-C-D, A-E-D, A-F-D. Each path gets 0.25 of traffic from A to D.     (iii) Because of symmetry of the network, we can derive the paths and split ratios for other pairs as well. For example, for (A, C) we use three paths: A-C (0.5), A-E-C (0.25) and A-B-C (0.25).        
 
         [0035]     Consider the traffic injected by all routers into the network. As long as the total amount does not exceed 2 and as long as the destination routers will not receive more than 2 from all the routers, it can be verified that the traversing traffic on any internal link will never exceed 1 regardless of the distribution of the traffic inside the network. This make the SAC decision of a new SLA easy because we only need to check the source and destination routers and see if adding the new SLA will violating the ingress and egress admissible constraints. For example, suppose we want to set up a new SLA with rate=0.5 from A  201  to B  203 . We first check if B ( 203 )&#39;s current load has exceeded 1.5. If not, accept the SLA. Otherwise, reject it. We don&#39;t need to check the status of each link inside.  
         [0036]     The techniques can also answer the reversed question: What is the internal link bandwidth that can make the network non-blocking for γ=3? The answer is 1.5. It is easily derived from the example by just scaling up the link capacity accordingly. To implement the scheme, the lookup engine in a router needs to do load-balancing according to the computed load-balancing ratios. It also needs to do this without causing out-of-sequence transmissions. There are many ways to do the load balancing. One example is shown in  FIG. 3 .  FIG. 3 (a) shows a standard protocol processor  300 .  FIG. 3 (b) is a schematic representation of a load-balancing unit with the lookup table engine  310  conducting load balancing according to computed load-balancing ratios. The hashing performed by the unit  305  maps the hose number field of the IP address into a random number between [0, M]. Bases on the result we select a path. For example, if the range is [0,99 ] and there are three pre-determined paths with load-distribution ratios: 0.2, 0.4, 0.4. If the output of the hashing unit  305  is between [0, 19], we use the first path. If it is in the range [20,59], the second path; if in the range [60,99], the third path. Alternatively, We can also use (destination host+TCP channel) as the input of the hashing unit. This can do load-balancing with a finer granularity.  
         [0000]     Path Setup Algorithns for Non-blocking Backbone Networks  
         [0037]     Let (θ{tilde over (α)} i , θ{tilde over (β)} i ) represent the maximum amount of ingress and egress traffic allowed to enter the network at the edge router i, where {tilde over (α)} i  and {tilde over (β)} i  are constants describing the degree of unevenness of traffic patterns in the network and θ is a parameter to be maximized in our design. For example, suppose ({tilde over (α)} 1 =5, {tilde over (β)} 1 =5) and ({tilde over (α)} 2 =15, {tilde over (β)} 2 =15), then the traffic allowed at edge router  2  is three times that of router  1 . Note that only the relative—not absolute—magnitudes of {tilde over (α)} i  and {tilde over (β)} i  have significance as the real amount of admissible traffic is determined by θ.  
         [0038]     We assume the network has n edge routers and the ingress and egress bandwidth constraints of each router are given in the following H vectors 
 
 H= [(θ{tilde over (α)} 1 , θ {tilde over (β)} 1 ), . . . (θ {tilde over (β)} n )]   (1) 
 
         [0039]     Compared with a conventional traffic matrix T={d if }, where d if  represents the traffic rate from node i to node j, we must have  
           ∑   j     ⁢     d   ij       ≤     θ   ⁢           ⁢       α   ~     i     ⁢           ⁢   and   ⁢           ⁢       ∑   i     ⁢     d   ij         ≤     θ   ⁢           ⁢         β   ~     j     .           
 
 For a given H, there are many traffic matrices Ts that satisfy the constraints imposed by H. A traffic matrix T={d if } that does not violate the constraints imposed by H is called a valid traffic matrix. Let D be the set of all valid Ts. 
 
         [0040]     The backbone network can be described as a directed graph G (V, E) with capacity Ce for link eε E. Let Q ⊂ V be the set of edge routers of the backbone network with n=||Q||. So Q is the set of edge nodes, or provider edge (PE) routers defined in a MPLS network, that to the outside world. Let x if   e  represent the portion of traffic from node i ∈ Q to j ∈ Q that is routed through link e and 0≦x if   e ≦1. Once we have x if   e , we have determined the paths from edge router i to j. Our routing techniques determine two things:  
         [0041]     The traffic allocation variables x if   e .  
         [0042]     The maximum ingress and egress bandwidth (θ{tilde over (α)} i , θ{tilde over (Note that {tilde over (α)} 1 , {tilde over (β)} 1  are given) that can be admitted to the network at edge router i without destroying the non-blocking property of the backbone network.  
         [0043]     The patent describes several methods to compute the routes and the maximum θ such that the backbone network is non-blocking. Define link utilization for a link as the ratio of total traffic going though the link over the link capacity. The congestion ratio, denoted by r, of a network is defined as the maximum link utilization of all links. When r &gt;1 for a link, it means that overflow will occur on that link and its value indicates the amount of excessive traffic on that link. Our first method will compute the maximum admissible ingress and egress traffic at each node based on the concept of congestion ratio. By computing the minimum r, we thus maximize the amount of admissible ingress and egress traffic. The problem is formulated as a linear programming problem. The results also include the routes for every ingress and egress pair.  
       METHOD 1  
       [0044]     The link congestion ratio is defined as the ratio of the amount of traffic routed through a link over the link&#39;s capacity. The network congestion ratio, denoted by r, is defined as the maximum value of all its link congestion ratios. We first assume the ingress and the egress traffic constraints at edge router i are ({tilde over (α)} i , {tilde over (β)} i ). We then compute the congestion ratio r. If we change the ingress and the egress traffic to  
         (           α   ~     i     r     ,         β   ~     i     r       )     ,       
 
 then the congestion ratio of the network will be ≦1. The maximum admissible amount of traffic at edge router i is thus  
         (           α   ~     i     r     ,         β   ~     i     r       )     .       
 
 The problem of maximizing θ is now converted to the minimization of r as θ=1/r. Method 1 shows how to use linear programming to find a routing that minimizes r. 
 
 Master LP  
               Master   ⁢           ⁢   LP     ⁢     
     ⁢     min   ⁢           ⁢   r                                   s   .   t   .           ⁢       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x   ij   e         -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     x   ij   e         =     0   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   ≠   i     ,   j           (     2   ⁢   a     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x   ij   e       -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     x   ij   e         =     1   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   i             (     2   ⁢   b     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x   ij   e       -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     x   ij   e         =       -   1     ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   j             (     2   ⁢   c     )                       ∑     i   ,     j   ∈   Q         ⁢       x   ij   e     ⁢     d   ij         ≤         c   e     ·   r     ⁢           ⁢   e       ∈   E     ,     T   ∈   D             (     2   ⁢   d     )                 0   ≤     x   ij   e     ≤     1   ⁢           ⁢   i       ,     j   ∈   Q     ,     e   ∈   E             (     2   ⁢   e     )             
 
         [0045]     where ( 2 d) is the bandwidth constraint. Constraint ( 2 d) needs to include all valid traffic  5  matrixes. We can use a slave program to find the T that should be included in constraint ( 2 d). The process works iteratively. 
 
 Slave LP  
       Slave   ⁢           ⁢   LP       
       max   ⁢       ∑     i   ,     j   ∈   Q         ⁢       x   ij   e     ⁢     d   ij             
                 s   .   t   .       ∑     j   ∈   Q       ⁢     d   ij         ≤         α   ~     i     ⁢           ⁢   i       ∈   Q           (     3   ⁢   a     )                     ∑     i   ∈   Q       ⁢     d   ij       ≤         β   ∼     j     ⁢           ⁢   j       ∈   Q           (     3   ⁢   b     )             
 
         [0046]     Method 1 is summarized in the following steps below.  
                                                                 begin       Solve Eq. (2) without constraint (2d) and get the solution x ij   e  and r.       repeat                D = Ø;                (a)   for e := 1 to E do                                   2.   ⁢           ⁢   Solve   ⁢           ⁢     Eq   .           ⁢     (   3   )       ⁢           ⁢   and   ⁢           ⁢   get   ⁢           ⁢   the   ⁢           ⁢   solution   ⁢           ⁢     d   ij     ⁢           ⁢   and   ⁢           ⁢       ∑     i   ,     j   ∈   Q                 ⁢       x   ij   e     ⁢       d   ij     .                                                   3.   ⁢           ⁢   if   ⁢           ⁢       ∑     i   ,     j   ∈   Q                 ⁢       x   ij   e     ⁢     d   ij           ≥         c   e     ·   r     ⁢           ⁢   then                                     4. Put traffic matrix T = {d ij } to D               5. end if           (b)   end for           (c)   Add new capacity constraints (2d) to Eq. (2), solve it, and get               the new solution x ij   e  and r.            until no traffic matrix is in D       end                  
 
       METHOD 2  
       [0047]     We can combine Eqs. (2)-(3) into one linear programming formulation given in Method 2.  
       min   ⁢           ⁢   r       
                   s   .   t   .       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x   ij   e         -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     x   ij   e         =     0   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   ≠   i     ,   j           (     4   ⁢   a     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x   ij   e       -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     x   ij   e         =     1   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   i             (     4   ⁢   b     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     x   ij   e       -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     x   ij   e         =       -   1     ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   j             (     4   ⁢   c     )                       ∑     i   ∈   Q       ⁢         α   ~     i     ⁢       π   e     ⁡     (   i   )           +       ∑     i   ∈   Q       ⁢         β   ~     i     ⁢       λ   e     ⁡     (   i   )             ≤         c   e     ·   r     ⁢           ⁢   e       ∈   E           (     4   ⁢   d     )                   x   ij   e     ≤         π   e     ⁡     (   i   )       +         λ   e     ⁡     (   j   )       ⁢           ⁢   i         ,     j   ∈   Q     ,     e   ∈   E             (     4   ⁢   e     )                 0   ≤     x   ij   e     ≤     1   ⁢           ⁢   i       ,     j   ∈   Q     ,     e   ∈   E             (     4   ⁢   f     )                   π   e     ⁡     (   i   )       ,           λ   e     ⁡     (   i   )       ≥     0   ⁢           ⁢   i       ∈   Q     ,     e   ∈   E             (     4   ⁢   g     )             
 
       METHOD 3  
       [0048]     We can transform Eq. (4) into a different master-slave LP formulation that renders a very fast implementation.  
         [0049]     Let f if   e =x if   e θ. Since θ=(1/r), Eq. (4) can be transformed into Eq. (5) given below.  
       max   ⁢           ⁢   θ       
                   s   .   t   .       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     f   ij   e         -       ∑     f   ij   e         e   ∈       Γ   -     ⁡     (   v   )             =     0   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   ≠   i     ,   j           (     5   ⁢   a     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     f   ij   e       -       ∑     f   ij   e         e   ∈       Γ   -     ⁡     (   v   )             =     θ   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   i             (     5   ⁢   b     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     f   ij   e       -       ∑     f   ij   e         e   ∈       Γ   -     ⁡     (   v   )             =       -   θ     ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   j             (     5   ⁢   c     )                       ∑     i   ∈   Q       ⁢         α   ~     i     ⁢       π   e     ⁡     (   i   )           +       ∑     i   ∈   Q       ⁢         β   ~     i     ⁢       λ   e     ⁡     (   i   )             ≤       c   e     ⁢           ⁢   e       ∈   E           (     5   ⁢   d     )                   f   ij   e     ≤         π   e     ⁡     (   i   )       +         λ   e     ⁡     (   j   )       ⁢           ⁢   i         ,     j   ∈   Q     ,     e   ∈   E             (     5   ⁢   e     )               f   ,   θ   ,   π   ,     λ   ≥   0             (     5   ⁢   f     )             
 
         [0050]     Eq. (5a)-(5c) and (5e) are derived by multiplying both sides of Eq. (4a)-(4c), and (4c) by θ, and Eq. (5d) is derived by moving r to the left. Note that in Eq. (5), π and λ are scaled by θ if we compare them with those in Eq. (4). The important fact about Eq. (5) is that it can be easily decomposed into a master-slave problem which renders a fast implementation. 
 
 Master Problem  
       Master   ⁢           ⁢   Problem       
       max   ⁢           ⁢   θ       
                   s   .   t   .       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     f   ij   e         -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     f   ij   e         =     0   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   ≠   i     ,   j           (     6   ⁢   a     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     f   ij   e       -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     f   ij   e         =     θ   ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   i             (     6   ⁢   b     )                       ∑     e   ∈       Γ   +     ⁡     (   v   )           ⁢     f   ij   e       -       ∑     e   ∈       Γ   -     ⁡     (   v   )           ⁢     f   ij   e         =       -   θ     ⁢           ⁢   i       ,     j   ∈   Q     ,     v   =   j             (     6   ⁢   c     )                   f   ij   e     ≤         π   e     ⁡     (   i   )       +         λ   e     ⁡     (   j   )       ⁢           ⁢   i         ,     j   ∈   Q     ,     e   ∈   E             (     6   ⁢   d     )               f   ,     θ   ≥   0             (     6   ⁢   e     )             
 
 Slave Problem  
       Slave   ⁢           ⁢   Problem       
       max   ⁢           ⁢     θ   ⁡     (   ϕ   )           
                   s   .   t   .       ∑     i   ∈   Q       ⁢         α   ~     i     ⁢       π   e     ⁡     (   i   )             +       ∑     i   ∈   Q       ⁢         β   ~     i     ⁢       λ   e     ⁡     (   i   )             ≤       c   e     ⁢           ⁢   e       ∈   E           (     7   ⁢   a     )               π   ,     λ   ≥   0             (     7   ⁢   b     )             
 
 where ø is the vector consisting of all variables π e  (i) and λ e  (i). 
 
         [0051]     For a link e, we define 
 
(a) φ e =[π e (1), . . . , π e (n), λ e (1), . . . , λ e  (n)].   (8) 
 
         [0052]     where n is the number of edge nodes (assuming the set of edge nodes is labeled from 1 to n). Then 
 
(a) ø=]φ 1 , φ 2 , . . . , φ m ].   (9) 
 
         [0053]     where m is the number of links in the network. We call ø the resource-allocation vector and its dimension is 2 mn . If ø is known, the master program produces an optimal θ, denoted by θ (ø), for the given ø. The slave program then searches a new ø that can increase the function θ (ø). The method is summarized in the following steps.  
                                       Step 0   Initialization.           Select a sufficiently small value ε &gt; 0 denoting the minimum           value of the step size. Set the optimal value z* = 0 and the           iteration counter k = 1.       Step 1   Determine an initial feasible allocation φ 1 .           Set π e (i) = c e /(2n · α i ), λ e (i) = c e /(2n · β i ) for i ∈ Q.       Step 2   Solve the Eq. (6).           Solve the Eq. (6) based on the current resource allocation φ k .           If θ(φ k ) &gt; z*, then save the flows, set z* = θ(φ k ).       Step 3   Determine a new allocation.           Compute the subgradient γ k . Determine a new allocation           φ k+1             φ k  + t k  · γ k . If φ k+1  is feasible move to this point.           Otherwise, project φ k+1  onto the feasible region.       Step 4   Check the convergence.           If the step size t k  &lt; ε, terminate the algorithm. Otherwise,           set k = k + 1 and return to Step 2.                  
 
       METHOD 4: SETTING PATH LIMIT  
       [0054]     We can add a path limit to the algorithms above. Let L be the number of paths allowed for each source-destination pair. We first use Method 3 to produce a set of paths for each source-destination pair. We then select the most loaded L paths among those generated by Method 3 (or Method 2). Given the set of paths, we can re-compute θ by solving a linear programming problem similar to that given in Eq. (5). The difference is that path-flow representation, instead of link-flow representation, must be used in the formulation. Let P if  be the set of paths between the node pair (i, f), x (p) the amount of flow sent on path p, and δ e  (p) a link-path indicator variable, that is, δ e  (p) equals 1 if link e is contained in the path p, and 0 otherwise. Following a similar formulation of Eq. (5), we can use the following linear programming formulation to find the maximum θ. Method 4 can be easily solved with a typical LP solver.  
       max   ⁢           ⁢   θ       
                 s   .   t   .       ∑     p   ∈     P   ij         ⁢     x   ⁡     (   p   )           =     θ   ⁢           ⁢   i       ,     j   ∈   Q             (     10   ⁢   a     )                       ∑     i   ∈   Q       ⁢         α   ~     i     ⁢       π   e     ⁡     (   i   )           +       ∑     i   ∈   Q       ⁢         β   ~     i     ⁢       λ   e     ⁡     (   i   )             ≤       c   e     ⁢           ⁢   e       ∈   E           (     10   ⁢   b     )                     ∑     p   ∈     P   ij         ⁢         δ   e     ⁡     (   p   )       ⁢     x   ⁡     (   p   )           ≤         π   e     ⁡     (   i   )       +         λ   e     ⁡     (   j   )       ⁢           ⁢   i         ,     j   ∈   Q     ,     e   ∈   E             (     10   ⁢   c     )               x   ,   θ   ,   π   ,     λ   ≥   0             (     10   ⁢   d     )             
 
         [0055]     The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. The invention, for example, can be implemented in a computing system with one or more programmable processors executing a computer program to perform functions of the invention. Method can also be performed by, for example, logic circuitry, ASIC, and the like. The processors suitable for execution of a computer program include, for example, general and specific microprocessors of various kinds of digital computers. The computer system may include, for example, a display device, a pointing device, an input device, in addition to the processor.  
         [0056]     Those skilled in the art will readily observe that numerous modifications and alterations of the device may be made without departing from the scope, spirit or teachings of the invention. The invention is defined by the following claims and their equivalents: