Abstract:
Aspects of the invention pertain to transmitting packet data across a computer network. The packets may be sent via one or more distinct routes from a source to a destination. Each route may employ multiple routers disposed along the network. Non-colliding routes are determined by transmitting pairs of probe packets along the routes. A first probe packet has a maximal length, and a second probe packet has a minimal length. Depending on the order of arrival of the probe packets, the system determines whether two transport layer ports at the destination device collide. If there is a collision, then the system searches for a set of non-colliding ports. Once the non-colliding ports are determined, application data may be sent as packets along the different routes to those ports.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     The present application is a continuation of U.S. patent application Ser. No. 12/533,187, filed Jul. 31, 2009, the disclosure of which is incorporated herein by reference. 
    
    
     BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The invention relates generally to routing data in computer networks. More particularly, the invention pertains to identifying multiple paths between source and destination nodes in computer networks. 
     2. Description of Related Art 
     With a networked computer, it is possible to increase the communication bandwidth or the availability of network connectivity by using multiple interfaces concurrently to aggregate bandwidth. One strategy of bandwidth aggregation is known as Link Aggregation (“LAG”). Another is known as Equal Cost Multiple Path (“ECMP”). 
     Such strategies allow router or switch nodes in the network to load balance traffic across multiple outgoing links. The outgoing link of a packet is often determined based on the hash value of information (source IP address, destination IP address, source port, destination port) in the packet header. A node may maximize its application throughput to a destination by identifying a set of port pairs (source port, destination port) to send traffic across multiple paths available to the destination. 
     However, specific implementations of such bandwidth allocation, e.g., the use of hashing functions on switches/routers, are often proprietary. Furthermore, the results of such hashing functions may depend on the seed value of individual switches. It is not feasible for a source node to determine a priori if port pairs are to be hashed to different paths to a destination node, especially in wide area network where end hosts have little knowledge of network topologies and router/switch configurations in between them. 
     In contrast, the invention provides non-proprietary systems and methods that identify and employ multiple paths between network nodes. 
     SUMMARY OF THE INVENTION 
     In one embodiment, a method for identifying ports to support data packet traffic between a source device and a destination device in a computer network is provided. The method comprises setting a set S D  of destination ports p the destination device is configured to listen to; selecting a subset S k   D  from the set S D  for serving the data packet traffic between the source device and the destination device; choosing a port p in the set S D  for analysis; determining whether the chosen port p collides with any of the ports in the subset S k   D ; and if there is a collision involving the chosen port p, eliminating the chosen port p from the set S D . 
     In one example, the chosen port p is removed from the set S D  prior to determining whether there is a collision. In another example, if there is no collision, the method further comprises modifying the subset S K   D  according to the following equation: S K   D   modified =S K   D  U {p}. In this case, the method may further comprise repeating the selecting, choosing, determining and eliminating operations for k rounds, wherein a new port p is chosen in each round. In one alternative, after each port p is identified that does not collide with any of the ports in the subset S D   K , the method further comprises sending the data packet traffic from the source device to the destination device via at least two of the non-colliding ports. 
     In another example, determining whether the chosen port p collides with any of the ports in the subset S k   D  includes transmitting a pair of probe packets from the source device to the destination device. In one alternative, a first one of the pair of probe packets is of a maximum packet transmission size. In this case, a second one of the pair of probe packets may be of a minimum packet transmission size. And in a further example, the port p is randomly selected from the set S D . 
     In accordance with another embodiment, a method of determining port collisions when transmitting traffic between a source device and a destination device in a computer network is provided. The method comprises setting a set S D  of destination ports p the destination device is configured to listen to; selecting a subset S k   D  from a set S D  for serving the data packet traffic between the source device and the destination device; initializing a test subset of ports S Y  to be equal to S K   D ; choosing a port p′ from the set S Y  and choosing a port p from the set S D  but not in S k   D  for analysis; sending a pair of probe packets from the source device to the destination device via p and p′; evaluating arrivals of the pair of the probe packets at the destination device to determine if there is a collision at ports p and p′; and if there is no collision, then identifying port p as a selected port for transmitting data packets between the source device and the destination device. 
     In one example, the destination device sends an acknowledgement to the source device upon the arrival of the pair of probe packets. Here, the acknowledgement may identify whether there is a collision between p and p′. 
     In another example, the port p′ is removed from S Y  upon choosing. In this case, the method may further comprise repeating the choosing, sending, evaluating and identifying operations for each port in S Y  until all of the ports have been removed from S Y . 
     In yet another example, a first one of the pair of probe packets is of a maximum packet transmission size and a second one of the pair of probe packets is of a minimum packet transmission size. In this case, a collision may be determined if the pair of probe packets are received in order of transmission. 
     In accordance with another embodiment, a processing system for transmission of data packets between a source device and a destination device in a computer network is provided. Here, the computer network includes a plurality of nodes for routing the data packets between the source and destination devices. The processing system comprises a memory for storing data and a processor operatively coupled to the memory for reading the data from and writing the data to the memory. The processor is configured to select a subset S k   D  from a set S D  of ports p the destination device is configured to listen to. The subset S k   D  is configured to serve the data packet traffic between the source device and the destination device. The processor is further configured to choose a port p in the set S D  for analysis, to determine whether the chosen port p collides with any of the ports in the subset S k   D , and, if there is a collision involving the chosen port p, to eliminate the chosen port p from the set S D . 
     In one example, the processor determines whether the chosen port p collides with any of the ports in the subset S k   D  by transmitting a pair of probe packets to the destination device. In this case, the processor may determine whether there is a collision based upon an order of arrival of the transmitted probe packets. In an alternative, the processor comprises the source device. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates a computer network for use with aspects of the invention. 
         FIG. 2  illustrates aspects of a network having multiple routes in accordance with the invention. 
         FIG. 3  is a flow diagram for identifying suitable paths to a given destination port according to aspects of the invention. 
         FIG. 4  is a flow diagram for determining collisions among ports of a destination node according to aspects of the invention. 
         FIG. 5  illustrates cross traffic increasing dispersion in a network. 
       And  FIG. 6  illustrates cross traffic decreasing dispersion in a network. 
     
    
    
     DETAILED DESCRIPTION 
     Aspects, features and advantages of the invention will be appreciated when considered with reference to the following description of preferred embodiments and accompanying figures. The same reference numbers in different drawings may identify the same or similar elements. Furthermore, the following description is not limiting; the scope of the invention is defined by the appended claims and equivalents. 
       FIG. 1  presents a schematic diagram of a computer network  100  depicting various computing devices that can be used in a networked configuration in accordance with aspects of the invention. For example, computer network  100  may have a plurality of computers  102 ,  104 ,  106  and  108  as well as other types of devices such as portable electronic devices such as a mobile phone  110  and a PDA  112 . Such computing devices may be interconnected via a local or direct connection  114  and/or may be coupled via a communications network  116  such as a LAN, WAN, the Internet, etc. The communications network  116  may include multiple nodes comprising switches or routers, as will be discussed below. 
     Each computing device may include, for example, one or more processing devices (e.g., a CPU) and have user inputs such as a keyboard  118  and mouse  120  and/or various other types of input devices such as pen-inputs, joysticks, buttons, touch screens, etc., as well as a display  122 , which could include, for instance, a CRT, LCD, plasma screen monitor, TV, projector, etc. Each computer  102 ,  104 ,  106  and  108  may be a personal computer, server, etc. By way of example only, computers  102  and  106  may be personal computers while computer  104  may be a server and computer  108  may be a laptop. 
     Each computer such as computers  102  and  104  contains a processor, memory/storage and other components typically present in a computer. For instance, memory/storage stores information accessible by processor, including instructions that may be executed by the processor and data that may be retrieved, manipulated or stored by the processor. The memory/storage may be of any type or any device capable of storing information accessible by the processor, such as a hard-drive, ROM, RAM, CD-ROM, flash memories, write-capable or read-only memories. The processor may comprise any number processing elements, such as sub-processing units operating in a parallel-processing configuration. Alternatively, the processor may be a dedicated controller for executing operations, such as an ASIC. 
     The communications network  116  is preferably configured to handle data packets (e.g., packets P 1  and P 2 ) using one or more nodes.  FIG. 2  illustrates a network configuration  200 , where a source device  202  (e.g., computer  102 ) may send data to a destination device  204  (e.g., computer  104 ) via one or more nodes  206  in the network. As shown, each node may comprise a router/switch such as router R 0 , R I , etc. 
     The routers may include a processor such as a CPU, as well as memory for storing/queuing buffered data packets. The packets are preferably queued in a first-in, first-out (“FIFO”) order. The routers may be arranged in the network so that packets from the source device  202  may be passed to the destination device  204  via multiple alternative routes. Identifying such routes enables the system to efficiently route the packets. By way of example,  FIG. 2  illustrates a first path  208  along the route R 0 , R L , R L+1   1 , R M   1  and R M+1 , as well as a second path  210  along the route R 0 , R L , R L+1   2 , R M   2  and R M+1 . 
     In accordance with aspects of the invention, several propositions or requirements regarding analyzing packet routing through the network are set forth. The first proposition is that there is no out of order delivery of packets sent from the source device to the destination device if these packets traverse the same route, unless a link failure occurs and causes a route change at a router in between its transmissions of these packets. 
     A second proposition is that if two consecutive packets traverse different routes, they may arrive out of order at the destination. The probability of out of order delivery is maximized when the size of first probe packet is maximized and that of second probe packet is minimized. 
     And a third proposition is that if two consecutive probe packets P 1  and P 2  of respective lengths L max  and L min  (e.g., the largest and smallest packet lengths supported by node R 0 ) traverse different routes yet arrive in order at the destination, the path traversed by P 2  experiences much larger queuing and/or transmission delay than P 1 . In this case, it may be of little gain to split the data traffic across the two paths traversed by P 1  and P 2 , since the throughput improvement is marginal. 
     Assume source device  202  is configured to initiate a data intensive application (such as a file transfer) that sends a large amount of traffic (e.g., 10 Mbytes) from its port X to destination device  204 . According to one aspect, S D  is defined as a set of destination ports p the destination device  204  is listening to and can receive data from the source device  202 . In this embodiment, the invention allows the source device  202  or other associated device to discover N paths each served by a destination port in S N   D  (S N   D ⊂S D ) of the destination device  204 . The source device  202  thus can split its traffic across these N paths, for example by establishing N TCP/UDP sessions, one to each destination port in S N   D . 
     In one embodiment, a process is executed iteratively to arrive at a solution.  FIG. 3  illustrates a flow diagram  300  for identifying multiple paths to a given destination device and a set of ports on the destination serving these paths. The process desirably executes iteratively for up to N−1 rounds. As shown in block  302 , a counter k of the rounds is incremented (and which may be initialized to 0). In the k th  round, the process starts with S k   D , which is a subset of k ports in S D  to serve traffic from the source device to the destination device over k unique paths. This is done to discover the (k+1) th  path from the source to the destination, as well as a port p in S D  to serve packets traversing this path. This process is desirably done by exchanging at least one probe pair or acknowledgement (“ACK”) packet between the source device and the destination device. 
     In one example, two destination ports, namely p 1 , and p 2 , “collide” if packets destined to p 1  traverse the same path as those destined to p 2 . The source device (e.g., router R 0  or source  202 ) searches for a port p in S D  that does not collide with any ports in S k   D , and eliminates ports from S D  that collide with ports in S k   D . First, in block  304  a port p is removed from S D . Preferably, the port p is randomly selected for removal in block  304 . Then, in block  306 , it is determined if the port p just removed from S D  collides with any port in S k   D . If so, the removed port p is not added back to S D  such that it is skipped in the next round k+1, and the process returns to block  302 . If there is no collision, the process proceeds to block  308 , where S k+1   D  is assigned to be S k+1   D =S k   D ∪p. Here, port p is added to S k+1   D . The “∪” symbol in this equation is a union or addition operator. 
     As shown in block  310 , if the counter k is less than N−1, then the process returns to block  302 ; otherwise it ends at block  312 . 
     To determine if a given port p collides with any port in S k   D  without modifying S k   D , the process  400  shown in  FIG. 4  may be employed. First, as shown in block  401 , a set of ports S Y  is initialized to be the same as the set of ports in S K   D . As shown in block  402 , a port p′ is removed from the set S Y . The port p′ is preferably randomly selected for removal from S Y . 
     Next, as shown in block  404 , for each port p′ in S k   D , a pair of probe packets is sent back to back via ports p′ and p respectively. The size of the first probe packet is preferably a maximum transmission unit (L MAX ) supportable by the network between the source  202  and the destination  204 , while the second probe is preferably of a minimum packet length (L MIN ) supported by the network between the source  202  and destination  204 . The destination device  204  may send an acknowledgement ACK back to the source device  202 , as shown in block  406 . 
     As shown in block  408 , the ACK may indicate whether the ports p and p′ collide. In particular, if the destination device  204  received the two probe packets out of order, it is determined that ports p′ and p do not collide at block  410 . If the packets are received in order, it is determined that p′ and p collide (e.g., port p collides with port p′ in S K   D ) as shown in block  412 . 
     As shown in block  414  if S Y  is not empty, the process preferably returns to block  402  and repeats until a port p′ is found in S Y  that collides with p, or until it is determined that no ports in S Y  collides with p. If S Y  is empty or a collision is found, the process terminates at bock  416 . 
     One of the propositions discussed above was that if two consecutive packets traverse different routes, they could arrive out of order at the destination. The probability of out of order delivery is maximized when the size of first probe is maximized and that of second probe is minimized. 
     To prove this observation, consider the following network model depicting two routes between a source node and a destination node. In this example, two probe packets, P 1  of length L 1 , and P 2  of length L 2 , are sent back to back from the source node to the destination node. The probe packets each traverses one of two routes, which may initially share certain routers, e.g., R 0 , R 1 , . . . R 1 . In this example, the two routes branch off after R 1  and re-merge at R m . The transmission rate of a router R (Rε{R i |0≦i≦l}∪{R i   1 |l&lt;i≦m}∪{R i   2 |l&lt;i≦m}) is r(R), and Q 2 (R) and Q 1 (R) represent the queuing delays experienced by P 1  and P 2  at R, respectively. 
     Assume there is no cross traffic and the queuing delay is zero for P 1  and P 2  at R 0  to R 1 . It is straightforward to prove that: 
     
       
         
           
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
             
             = 
             
               
                 L 
                 2 
               
               
                 min 
                 ⁢ 
                 
                   { 
                   
                     
                       r 
                       ⁡ 
                       
                         ( 
                         
                           R 
                           i 
                         
                         ) 
                       
                     
                     | 
                     
                       0 
                       ≤ 
                       i 
                       ≤ 
                       L 
                     
                   
                   } 
                 
               
             
           
         
       
     
     T l   2  and T l   2  are the arrival times of P 1  and P 2  at R 1 , and min{r(R i )|0≦i≦l} is the bandwidth of bottleneck link between R 0  and R 1 . When there is cross traffic, T l   2 −T l   1 , the dispersion of P 1  and P er  can expand or compress. To express this phenomena, we let: 
     
       
         
           
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
             
             = 
             
               
                 
                   L 
                   2 
                 
                 
                   min 
                   ⁢ 
                   
                     { 
                     
                       
                         r 
                         ⁡ 
                         
                           ( 
                           
                             R 
                             i 
                           
                           ) 
                         
                       
                       | 
                       
                         0 
                         ≤ 
                         i 
                         ≤ 
                         l 
                       
                     
                     } 
                   
                 
               
               + 
               
                 δ 
                 c 
               
             
           
         
       
     
     Cross traffic, such as a packet P 3  may be serviced in between P 1  and P 2 . As shown in  FIG. 5 , if this occurs, then the dispersion between P 1  and P 2  increases. The probability of such cross traffic expanding the dispersion of P 1  and P 2  decreases with the length L 2  of packet P 2 . Thus, if the length L 2  of packet P 2  is minimized, the probability of cross traffic packet P 3  increasing the dispersion between P 1  and P 2  is minimized. 
     If cross traffic P 3  is serviced before P 1  as shown in  FIG. 6 , the dispersion of P 1  and P 2  is decreased. The probability of such cross-traffic P 3  decreasing the dispersion of P 1  and P 2  increases with the length L 1  of packet P 1 . Furthermore, it can be proved that: 
     
       
         
           
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
             
             ≥ 
             
               
                 L 
                 2 
               
               
                 max 
                 ⁢ 
                 
                   { 
                   
                     
                       r 
                       ⁡ 
                       
                         ( 
                         
                           R 
                           i 
                         
                         ) 
                       
                     
                     | 
                     
                       0 
                       ≤ 
                       i 
                       ≤ 
                       l 
                     
                   
                   } 
                 
               
             
           
         
       
       
         
           
             Hence 
             ⁢ 
             
               : 
             
           
         
       
       
         
           
             
               
                 L 
                 2 
               
               
                 max 
                 ⁢ 
                 
                   { 
                   
                     
                       r 
                       ⁡ 
                       
                         ( 
                         
                           R 
                           i 
                         
                         ) 
                       
                     
                     | 
                     
                       0 
                       ≤ 
                       i 
                       ≤ 
                       l 
                     
                   
                   } 
                 
               
             
             ≤ 
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
             
             ≤ 
             
               
                 L 
                 2 
               
               
                 min 
                 ⁢ 
                 
                   { 
                   
                     
                       r 
                       ⁡ 
                       
                         ( 
                         
                           R 
                           i 
                         
                         ) 
                       
                     
                     | 
                     
                       0 
                       ≤ 
                       i 
                       ≤ 
                       l 
                     
                   
                   } 
                 
               
             
           
         
       
     
     Thus T l   2 −T l   1  is minimized if L 2  is minimized and L 1  is maximized. Next, T m   1  and T m   2 , the arrival times of P 1  and P 2  at R m , respectively, and the condition for T m   1 &gt;T m   2 , that is P 2  arrives at R m  and eventually at the destination node before P 1  are derived. In particular: 
     
       
         
           
             
               T 
               m 
               1 
             
             = 
             
               
                 T 
                 l 
                 1 
               
               + 
               
                 
                   Q 
                   1 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               + 
               
                 
                   L 
                   1 
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   ∑ 
                   
                     
                       l 
                       + 
                       1 
                     
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     1 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       1 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   L 
                   1 
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       1 
                     
                     ) 
                   
                 
               
             
           
         
       
       
         
           
             
               T 
               m 
               2 
             
             = 
             
               
                 T 
                 l 
                 2 
               
               + 
               
                 
                   Q 
                   2 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               + 
               
                 
                   L 
                   2 
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   ∑ 
                   
                     
                       l 
                       + 
                       1 
                     
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     2 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       2 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   L 
                   2 
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       2 
                     
                     ) 
                   
                 
               
             
           
         
       
     
     T m   2 &lt;T m   1  if and only iff: 
     
       
         
           
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
             
             &lt; 
             
               
                 
                   Q 
                   1 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               - 
               
                 
                   Q 
                   2 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               + 
               
                 
                   
                     L 
                     1 
                   
                   - 
                   
                     L 
                     2 
                   
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   ∑ 
                   
                     
                       l 
                       + 
                       1 
                     
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     1 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       1 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   ∑ 
                   
                     
                       l 
                       + 
                       1 
                     
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     1 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         1 
                       
                       ) 
                     
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     
                       l 
                       + 
                       1 
                     
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     2 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       2 
                     
                     ) 
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     
                       l 
                       + 
                       1 
                     
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     2 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         2 
                       
                       ) 
                     
                   
                 
               
             
           
         
       
     
     In the above formula, when L 1  is maximized and L 2  is minimized, T l   2 −T l   1  is minimized, and 
                   L   1     -     L   2         r   ⁡     (     R   l     )         +       ∑       l   +   1     ≤   i   ≤   m       ⁢       L   1       r   ⁡     (     R   i   1     )           -       ∑       l   +   1     ≤   i   ≤   m       ⁢       L   2       r   ⁡     (     R   i   2     )                 
is maximized. Assume the sum of queuing delays experienced by P 1  and P 2 ,
 
                   Q   1     ⁡     (     R   l     )       +       ∑       l   +   1     ≤   i   ≤   m       ⁢       Q   1     ⁡     (     R   i   1     )           ,       and   ⁢           ⁢       Q   2     ⁡     (     R   l     )         +       ∑       l   +   1     ≤   i   ≤   m       ⁢       Q   2     ⁡     (     R   i   2     )           ,         
are independent of L 1  and L 2 . The probably of T m   2 &lt;T m   1  is maximized when L 1  is maximized and L 2  is minimized.
 
     It can be further proved that if between R m  and the destination node, P 1  and P 2  traverse on different routes, the lead time of P 2  over P 1  is also maximized when is L 1  maximized and L 2  is minimized. 
     With regard to the third proposition, if two consecutive probe packets P 1  and P 2  of length L max  and L min  (denoting the largest and smallest packet length supported by the network&#39;s nodes) traverse different routes yet arrive in order at the destination, the path traversed by P 2  experience much larger queuing and/or transmission delay than P 1 . It is thus of little gain to split the application traffic across the two paths traversed by P 1  and P 2 , since the throughput improvement is marginal. 
     When L 1 =L max  and L 2 =L min , the probability of cross traffic in between P 1  and P 2  is close to zero, and it may be assumed that: 
     
       
         
           
             
               
                 L 
                 2 
               
               
                 max 
                 ⁢ 
                 
                   { 
                   
                     
                       r 
                       ⁡ 
                       
                         ( 
                         
                           R 
                           i 
                         
                         ) 
                       
                     
                     | 
                     
                       0 
                       ≤ 
                       i 
                       ≤ 
                       l 
                     
                   
                   } 
                 
               
             
             ≤ 
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
             
             ≤ 
             
               
                 L 
                 2 
               
               
                 min 
                 ⁢ 
                 
                   { 
                   
                     
                       r 
                       ⁡ 
                       
                         ( 
                         
                           R 
                           i 
                         
                         ) 
                       
                     
                     | 
                     
                       0 
                       ≤ 
                       i 
                       ≤ 
                       l 
                     
                   
                   } 
                 
               
             
           
         
       
       
         
           
             
               
                 T 
                 l 
                 2 
               
               - 
               
                 T 
                 l 
                 1 
               
               + 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     2 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       2 
                     
                     ) 
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     1 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       1 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   Q 
                   2 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               - 
               
                 
                   Q 
                   1 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               + 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     2 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         2 
                       
                       ) 
                     
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     1 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         1 
                       
                       ) 
                     
                   
                 
               
               + 
               
                 
                   
                     L 
                     2 
                   
                   - 
                   
                     L 
                     1 
                   
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
             
             &gt; 
             0 
           
         
       
       
         
           
             
               
                 
                   L 
                   2 
                 
                 
                   min 
                   ⁢ 
                   
                     { 
                     
                       
                         r 
                         ⁡ 
                         
                           ( 
                           
                             R 
                             i 
                           
                           ) 
                         
                       
                       | 
                       
                         0 
                         ≤ 
                         i 
                         ≤ 
                         l 
                       
                     
                     } 
                   
                 
               
               + 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     2 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       2 
                     
                     ) 
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     Q 
                     1 
                   
                   ⁡ 
                   
                     ( 
                     
                       R 
                       i 
                       1 
                     
                     ) 
                   
                 
               
               + 
               
                 
                   Q 
                   2 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               - 
               
                 
                   Q 
                   1 
                 
                 ⁡ 
                 
                   ( 
                   
                     R 
                     l 
                   
                   ) 
                 
               
               + 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     2 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         2 
                       
                       ) 
                     
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     1 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         1 
                       
                       ) 
                     
                   
                 
               
               + 
               
                 
                   
                     L 
                     2 
                   
                   - 
                   
                     L 
                     1 
                   
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
             
             &gt; 
             0 
           
         
       
     
     If the queuing delays of P 1  and P 2  are equal, 
     
       
         
           
             
               
                 
                   L 
                   2 
                 
                 
                   min 
                   ⁢ 
                   
                     { 
                     
                       
                         r 
                         ⁡ 
                         
                           ( 
                           
                             R 
                             i 
                           
                           ) 
                         
                       
                       | 
                       
                         0 
                         ≤ 
                         i 
                         ≤ 
                         l 
                       
                     
                     } 
                   
                 
               
               + 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     2 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         2 
                       
                       ) 
                     
                   
                 
               
               + 
               
                 
                   L 
                   2 
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
               - 
               
                 
                   ∑ 
                   
                     l 
                     ≤ 
                     i 
                     ≤ 
                     m 
                   
                 
                 ⁢ 
                 
                   
                     L 
                     1 
                   
                   
                     r 
                     ⁡ 
                     
                       ( 
                       
                         R 
                         i 
                         1 
                       
                       ) 
                     
                   
                 
               
               - 
               
                 
                   L 
                   1 
                 
                 
                   r 
                   ⁡ 
                   
                     ( 
                     
                       R 
                       l 
                     
                     ) 
                   
                 
               
             
             &gt; 
             0 
           
         
       
     
     Once a pair of non-colliding ports has been found, the system may configure packet transmission to send one or more packets along the different routes from the source node to the destination node. In one alternative, the system evaluates the network to determine whether more than two non-colliding destination ports are present. If so, the packet traffic may be split among all non-colliding routes. 
     Furthermore, the processes discussed herein, such as the operations discussed with regard to  FIGS. 3 and 4 , may be performed by one or more processors in the system. By way of example only, processing may be performed by the source device  202 , destination device  204  or one of the routers  206  of  FIG. 2 . The processor(s) may execute a program recorded/stored on a computer-readable recording medium, such as ROM, RAM, flash memory, CD-ROM, DVD-ROM or the like. 
     Although aspects of the invention herein have been described with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications may be made to the illustrative embodiments and that other arrangements may be devised without departing from the spirit and scope of the invention as defined by the appended claims.