# Patent

## Title
Flexible bandwidth allocation in high-capacity grooming switches

## Abstract

A method and apparatus for flexible sharing of bandwidth in switches with input buffering by dividing time into a plurality of frames of time slots, wherein each frame has a specified integer value of time slots. Counters associated with the input-output queues of the switches are loaded with a negotiated integer value once per frame. The inputs sequentially select available outputs to which the inputs send packets in specified future time slots. Priority is given to input-output queues with counters having positive values. The selection of outputs by the inputs is done using a pipeline technique and a schedule is calculated within multiple time slots. The counters for selected queues are then decremented by 1.

## Background
REFERENCE TO RELATED APPLICATIONS
The present application claims priority to Provisional Patent Applications Nos. 60/211,474 filed Jun. 15, 2000; 60/213,250 file Jun. 22, 2000 and 60/260,526 filed Jan. 10, 2001.


BACKGROUND OF THE INVENTION
The present invention generally relates to the field of computer packet-switched networks and more particularly, is directed to a method and apparatus for flexible bandwidth allocation in a high capacity packet switch.
As known in the prior art, packet switches transfer packets from their inputs to the specified outputs. It is important to be able to flexibly share an output bandwidth among the inputs. In other words, inputs should be guaranteed to get the negotiated bandwidth even if some other inputs are overloaded. A switch with output buffers is usually a set of statistical multiplexers. Packets coming from different inputs are stored in the output buffer and transmitted according to some scheduling policy. For example, a weighted round-robin (WRR) policy would provide to the inputs their reserved bandwidth shares. However, the capacity of a switch with output buffers is limited by the speed of the output buffers.
In contrast, the capacity of a switch with input buffers is not limited similarly because packets are stored at the line bit-rate. Switches with input buffers can provide a much higher switching capacity which is why they have attracted much interest recently in the published literature. In a switch with input buffers, a packet competes not only with the packets of other inputs bound for the same output, but also with the packets of the same input bound for other outputs.
Several proposed protocols calculate the maximal matching between inputs and outputs that does not leave input-output pair unmatched if there is traffic between them. However, these protocols do not provide flexible sharing of the output bandwidth among the inputs in a switch with input buffers. Accordingly, there is a need in the art for an improved method and apparatus for such purposes.
BRIEF SUMMARY OF THE INVENTION
Accordingly, it is an overall object of the present invention to obviate the above-noted shortcomings and disadvantages of bandwidth allocation protocols in packet switches known in the prior art.
A further object of the present invention is to provide an improved method and apparatus for bandwidth allocation in packet switches.
Another object of the present invention is to an provide an improved method and apparatus for bandwidth allocation in packet switches which can be easily and inexpensively implemented.
Another object of the present invention is to an provide an improved method and apparatus for fast bandwidth allocation in circuit switches.
These another objectives of the present invention are achieved by the present invention as described below.



BRIEF DESCRIPTION OF THE DRAWINGS
The novel features of the present invention are set out with particularity in the appended claims, but the invention will be understood more fully and clearly from the following detailed description of the invention as set forth in the accompanying drawings in which:
FIG. 1 is a block diagram for a central controller for the PIM protocol;
FIG. 2 is a block diagram for a central controller for the RRGS protocol;
FIG. 3 is a time diagram for a switch controller in accordance with the present invention;
FIG. 4 is a time diagram for a the switch controller in accordance with another embodiment of the present invention; and
FIG. 5 is a time diagram for a switch controller in accordance with a further embodiment of the present invention.



DESCRIPTION OF THE PREFERRED EMBODIMENT
In accordance with the present invention, a new protocol is proposed which is simpler than the previously proposed ones, and can, consequently support packet switching of higher capacity. The maximal matching of inputs and outputs not only removes the head-of-line (HOL) blocking, but also simplifies the flexible bandwidth sharing in a switch with input buffers.
The simplest way to share the bandwidth in a switch with input buffering is to precompute a schedule in advance based on the reservations made in a connection setup phase. Time is divided into frames that consist of time slots. The schedule determines input-output pairs that will be connected in each time slot of a frame. Each input-output pair is assigned a certain number of time slots within a frame, which ensures the requested bandwidth share. It can be shown that requests can be accommodated as long as




  
    
      
        
          
            
              ∑
              m
            
            ⁢
            
              a
              im
            
          
          ≤
          F
        
        ⁢
        
          

        
        ⁢
        
          
            
              ∑
              m
            
            ⁢
            
              a
              mj
            
          
          ≤
          F
        
        ⁢
        
          

        
        ⁢
        
          
            0
            ≤
            i
          
          ,
          
            j
            ≤
            
              N
              -
              1
            
          
          ,
        
      
    
    
      
        (
        1
        )
      
    
  




Where aij is the number of time slots requested by input-output pair (i,j), F is the frame length, and N is the number of input and output ports. As a result, the bandwidth reserved for input-output pair (i,j) is pij=aij/F times the line bit-rate. However, computing the schedule has a complexity on the order of O(FN2), and may become impracticable for the fast varying traffic. For this reason, a statistical matching algorithm has been proposed to arbitrarily share the switch capacity. In the statistical matching algorithm, output j grants input i with probability pij=aij/F. Each input chooses one output from which it received a grant in a specified probabilistic way. It has been shown that the statistical matching uses 63% of the total switch capacity, or 72% if two iterations are performed. Weighted probabilistic iterative matching (WPIM) has also been proposed instead of statistical matching. They argue that the computing of several distribution functions within one time slot, as in statistical matching, becomes impractical in high-capacity switches. In WPIM, time is divided into frames, and input-output pair (i,j) is assigned aij credits within each frame. Namely, a counter associated to input-output air (i,j) is set to cij=aij at the beginning of a frame, and is decremented whenever this queue is served. Queues with positive counters compete for transmission with higher priority. They are rewarded according to the parallel iterative matching (PIM) algorithm. Remaining queues compete for the rest of the bandwidth, and they are again rewarded according to the PIM algorithm. The performance of the WPIM protocol has been assessed only through simulations.

A scheduling algorithm for flexible bandwidth reservations in a WDMA optical network with input buffering has also been proposed. If the number of wavelengths equals the number of users, such WDMA network is equivalent to a switch with input buffering. In the proposed scheduling algorithm, each input-output queue is associated with a counter which is increased in each time slot by pij, and decreased by 1 if this queue has been served. Queues with positive counters compete for service, and they are served according to some efficient maximal weighted matching algorithm. For example, queues are considered for service in the order in which their counters decrease. Since it processes N2 input-output pairs, this algorithm can also become a bottleneck in high-capacity switches. It was shown in that this algorithm guarantees 50% of the switch capacity.
In accordance with the present invention, new protocol, the weighted round-robin greedy scheduling (WRRGS), that provides flexible bandwidth sharing in switches with terabit capacity. Terabit switches involve more than 100 ports, line bit-rates as high as 10 Gb/s, and processing times (equal to packet transmission times) of 10–100 ns. Our approach is similar to the WPIM, only it is based on the round robin greedy scheduling (RRGS) protocol instead of the PIM. In this way, the WRRGS implementation is further simplified in a comparison to the WPIM. The PIM algorithm performs 2 log2 N+2 selections on average in order to find maximal matching, and involves the full interconnection between input and output modules of the central controller.
On the other side, the RRGS algorithm performs only one selection per time slot, and involves simple structure of the central controller. So, WRRGS can potentially be used in a switch with a larger number of ports and/or higher line bit-rate, i.e. in a switch with a higher capacity. Applicant has shown that WRRGS can flexibly allocate at least 50% of the total switch capacity.
Different architectures for optical packet-switching fabrics have been proposed. Optical cross-connects capable to reconfigure on the nanosecond time scale seem to be the best candidates for a switch core due to their simplicity. Namely, the complexity and cost of the optical technology are very high, so that simplicity of the switch core is essential. Key fast switching optical devices that can be used in packet switches are semiconductor optical amplifiers (SOA) and rapidly tunable lasers.
In the most straightforward design, a packet switch with N inputs and N outputs require N2 SOAs which are playing the role of gates. However, by combining WDM with space division multiplexing, the overall switch complexity measured in the number of SOAs is significantly reduced: the number of SOAs in a switch is decreased to 2N √N while √N √N×√N waveguide grating routers (WGR) are added. The 256×256 switch with the switching time of 5 ns has been demonstrated. If line bitrate is 10 Gbps, short packets of 64 bytes last 64 ns and could be successfully switched in the proposed architecture. The total switching capacity in that case would be 256×10Gb/s=2.56 Tb/s.
Alternatively, each input of a packet switch is equipped with the fast tunable laser which is connected to the outputs through large WGR. Fast tunable laser tunes to the wavelength that will be routed by WGR to the packet designated destination. The 80×80 switch with the switching time of 100 ns has been demonstrated. Thus, the proposed architecture would switch only longer packets. But, the long switching time is the result of the driver design and not the laser limitation. It has been shown in that the same laser can tune among wavelengths within less than 15 ns.
A. Protocol Description
The WPIM and WRRGS protocols compare similarly as the PIM and RRGS protocols. The PIM protocol consists of several iterations: all inputs send requests to the outputs for which they have packets to send, requested outputs send acknowledgements to their selected inputs, and selected inputs choose one output each. Inputs and outputs that have not been selected in the previous iterations compete in the next iteration in the same way.
It has been shown that the PIM algorithm finds a maximal matching after in log2 N+3⁄4 iterations on average. Each iteration involves two selections, and all iterations have to be completed one after another within one packet transmission time. The planar and two-dimensional designs of the central controller that execute the PIM algorithm are shown in FIG. 1(a) or (b), respectively. Each input module (IM) sends a request to each output module (OM) and each OM sends an acknowledgement to each IM. There should be 2N2 wires connecting input and output modules. Such central controllers may become difficult for implementation as N grows. On the other side, the RRGS protocol consists of N steps. In the first step, some particular input chooses one of the outputs for which it has packets to send. In each following step, the next input chooses one of the remaining outputs for which it has packets to send. Clearly, RRGS can be implemented by using a pipeline technique. Each step of the algorithm is completed within a separate time slot, and the algorithm is completed within N time slots. But, in each time slot, all inputs choose outputs for different time slots in future, so, the central controller is calculating in parallel schedules for N future time slots. As a result, only one selection has to be performed within one time slot (other N−1 simultaneous selections are done in parallel).
A simple structure of the central controller that executes the RRGS algorithm is shown in FIG. 2. A round-robin (RR) arbiter associated to each input module communicates only with the RR arbiters associated to adjacent input modules, and the complex interconnection between input and output modules is avoided. It stores addresses of the reserved outputs into the memory (M). Price that RRGS pays for its simplicity is the additional pipeline delay, which is on average equal to N/2 time slots. This pipeline delay is not critical for assumed very short packet transmission time.
The RRGS protocol needs to be further modified in order to provide flexible sharing of the total switch capacity. We propose that time is divided into frames, and counter associated with input-output queues are set to their negotiated values at the beginning of each frame, as in WPIM. Queues with positive counters compete with higher priority according to RRGS. Then, the remaining queues contend according to RRGS for the available bandwidth.
Consider an N×N cross-bar switch, where each input port i. i ε {0, 1, . . . , N−1}, has N logical queues, corresponding to each of the N outputs. All packets are fixed size cells. The output of the protocol is a schedule or a set S={(i,j) | packet will be sent from input i to output j}. Note that in each time slot, an input can only transmit one packet, and an output can receive only one packet. The schedule for the kth time slot is determined as follows:

    
    
        Step 1: If k=1 mod F then cij=aij;
        Step 2: Ik=Ok={0, . . . , N−1}; i=0;
        Step 3: Input i chooses in a round-robin fashion output, if any, j from Ok such that cij>0, and i there are unscheduled packets in queue (i,j); if there is no such j go to Step 5.
        Step 4: Remove j from Ok and i from Ik; Add (i,j) to Sk; cij=Cij−1;
        Step 5: If i<N−1 choose i=i+1 and go to Step 3;
        Step 6: i=0;
        Step 7: If i ε Ik choose j from Ok for which it has unscheduled packets to send; if there is no such j go to Step 9;
        Step 8: Remove j from Ok and i from Ik; Add (i,j) to schedule Sk;
        Step 9: If i<N−1 choose i=i+1 and go to Step 7;

In steps 1–5, prioritized packets compete for a service according to RRGS. Then, in steps 6–9, the remaining packets compete once again for the given time slot according to RRGS. Steps 6–9 are optional, they will increase the efficiency of WRRGS, but introduce an additional average pipeline delay of N/2 time slots. They, actually, represent a service for the best-effort traffic. Note that in RRGS input 0 is always the first to pick up an output, while in originally proposed RRGS all inputs get chance to be the first to choose an output. In the latter case an input might reserve an output in the earlier time slot for the later time slot in future, in other words, it might interchangeably reserve outputs for different frames. So, each queue should be assigned multiple counters related to different frames.

    
    


B. Pipelined WRRGS
First consider steps 1–5 of the pipelined WRRGS. WRRGS as outlined in the previous section is easy to implement by using a pipeline technique. In time slot k, input i reserves an output for time slot k+N−i within frame ┌(k+N−i)/F┐, where ┌x┐ is the smallest integer not smaller than x. Also, input i resets its counters cij, 0≦j≦N−1, in time slots mF+1−N+i, where m≧÷┌N/F┐. Time diagram for this first case of WRRGS applied in a 5×5 switch is shown in FIG. 3. This figure shows the relation between inputs and the time slots for which they are choosing their outputs. For example, in time slot T5input I1 is scheduling or choosing an output for transmission during time slot T9, while I3 is scheduling for time slot T7 and so on. After it chooses an output, e.g., input I1 forwards the control information (about available outputs) to input I2 which reserves an output for time slot T9 in the next time slot T6. Bold vertical line denotes that input I0 starts a new schedule choosing any of the outputs, i.e. it does not receive the control information from input I4.
Pipelining proposed for RRGS in might be applied to WRRGS in order to equalize inputs. Time diagram for this case of WRRGS applied in a 5×5 switch is shown in FIG. 4. Here, in each time slot another input starts a schedule. But, an input might interchangeably reserve outputs for different frames. For example, input I0 reserves an output for time slot T11 in time slot T6, and it reserves an output for time slot T9 in the next time slot T7. If the frame length is F=5, then input I0 interchangeably reserves outputs for frames F3 and F2. For a reasonable assumption that F≧N, an input might interchangeably reserve outputs for at most two consecutive frames. So, each queue should be assigned multiple counters related to different frames. Depending on the future time slot for which an input reserves an output, a specified counter of the chosen queue will be decremented by one. Counters are reset every F time slots.
Let us now consider all 1–9 steps of the pipelined WRRGS, including service of the best effort traffic. In any time slot k, each input chooses outputs for two different time slots in future, k+N−i and k+2·N−i within frames ┌(k+N−i)/F┐ and ┌(k+2·N−i)/F┐. First, an input reserves an output with the positive counter for time slot k+2·N−i, then, it reserves any output for time slot k+N−i. Also, input i resets its counters cij, 0≦j≦N−1, in time slots mF+1−2·N+i, where m≧┌2·N/F┐. FIG. 5 shows the time diagram for all 1–9 steps of WRRGS applied in a 3–3 switch. For example, in time slot T7, input I1 chooses one of the available prioritized outputs for the time slot T12, and then it chooses any of the available outputs for time slot T9. This is because input I1 uses its first chance to schedule for time slot T12 in time slot T7, and, therefore, it considers only queues with positive counters. On the other side, input I1 uses the second chance to schedule for time slot T9 in time slot T7, and, therefore, it considers all queues for service. It is possible to equalize inputs assuming service of the best-effort traffic as well.
C. Protocol Performance
It is essential to determine the portion of the switch capacity that a scheduling algorithm can share among the inputs. More precisely, we want to determine the maximum admissible utilization, p, of any input or output line:




  
    
      
        ∑
        m
      
      ⁢
      
        p
        im
      
    
    =
    
      
        
          1
          F
        
        ⁢
        
          
            ∑
            m
          
          ⁢
          
            a
            im
          
        
      
      ≤
      p
    
  
  ,
  
    

  
  ⁢
  
    
      
        ∑
        m
      
      ⁢
      
        p
        mj
      
    
    =
    
      
        
          1
          F
        
        ⁢
        
          
            ∑
            m
          
          ⁢
          
            a
            mj
          
        
      
      ≤
      p
    
  
  ,
  
    

  
  ⁢
  
    0
    ≤
    i
  
  ,
  
    j
    ≤
    
      N
      -
      1
    
  
  ,




which can be guaranteed to the input-output pairs. So, if input-output pair (i,j) requests a new portion of bandwidth, Δpij, it is accepted if:





  
    
      
        
          ∑
          m
        
        ⁢
        
          p
          im
        
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          p
          ij
        
      
    
    ≤
    p
  
  ,
  
    

  
  ⁢
  
    
      
        
          ∑
          m
        
        ⁢
        
          p
          mj
        
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          p
          ij
        
      
    
    ≤
    p
  
  ,




and input-output pair (i,j) is assigned Δaij=┌Δpij·F┐ new time slots per frame. We will prove that p=1⁄2 for the WRRGS, due to the fact that the RRGS finds a maximal matching between inputs and outputs.

Lemma 1: The WRRGS protocol ensures aij time slots per frame to input-output pair (i, j), 0≦i,j≦N−1, if the following condition holds:




  
    
      
        
          
            
              
                ∑
                m
              
              ⁢
              
                a
                im
              
            
            +
            
              
                ∑
                m
              
              ⁢
              
                a
                mj
              
            
            -
            
              a
              ij
            
          
          ≤
          F
        
        ,
      
    
    
      
        (
        2
        )
      
    
  




Proof: Only prioritized packets are being viewed as if WRRGS consists only of steps 1–5. Observe time slots within a frame in which either input i or output j are connected, but they are not connected to each other. In each of these time slots, sum




  
    s
    ij
  
  =
  
    
      
        ∑
        
          m
          ≠
          j
        
      
      ⁢
      
        c
        im
      
    
    +
    
      
        ∑
        
          m
          ≠
          i
        
      
      ⁢
      
        c
        mj
      
    
  




is greater than 0, and then it is decremented by at least 1. Sum Sij is the largest at the beginning of a frame and from (2), it fulfills:





  
    
      
        
          
            s
            ij
          
          =
          
            
              
                
                  ∑
                  
                    m
                    ≠
                    j
                  
                
                ⁢
                
                  a
                  im
                
              
              +
              
                
                  ∑
                  
                    m
                    ≠
                    i
                  
                
                ⁢
                
                  a
                  mj
                
              
            
            ≤
            
              F
              -
              
                a
                ij
              
            
          
        
        ,
      
    
    
      
        (
        3
        )
      
    
  




As a conclusion, in at least aij time slots per frame neither input i is connected to some output other than j, nor output j is connected to some input other than i. In these time slots, input i reserves output j if there are packets in queue (i,j) and unused credits cij>0. This is because none of the inputs have chosen output j before input i, and input i is not choosing any other output. Therefore, input i will choose output j as supposed by RRGS, and by any other algorithm that finds a maximal matching between inputs and outputs. In summary, if condition (2) is fulfilled then aij time slots per frame are guaranteed to input-output pair (i,j).

Lemma 2: The WRRGS protocol ensures aij time slots per frame to input-output pair (i, j), 0≦i,j≦N−1, if the following condition holds:




  
    
      
        
          
            
              ∑
              m
            
            ⁢
            
              a
              im
            
          
          ≤
          
            
              F
              +
              1
            
            2
          
        
        ,
        
          

        
        ⁢
        
          
            
              ∑
              m
            
            ⁢
            
              a
              mj
            
          
          ≤
          
            
              F
              +
              1
            
            2
          
        
        ,
      
    
    
      
        (
        4
        )
      
    
  




Proof: From inequality (4), it follows that:




  
    
      
        
          
            ∑
            m
          
          ⁢
          
            a
            im
          
        
        +
        
          
            ∑
            m
          
          ⁢
          
            a
            mj
          
        
      
      ≤
      
        F
        +
        1
      
    
    ⇒
    
      

    
    ⁢
    
      
        
          
            ∑
            m
          
          ⁢
          
            a
            im
          
        
        +
        
          
            ∑
            m
          
          ⁢
          
            a
            mj
          
        
        -
        
          a
          ij
        
      
      ≤
      F
    
  
  ,




since aij 1. Because inequality (4) implies inequality (2), Lemma 1 directly follows from Lemma 2.

Theorem: The WRRGS protocol ensures pij of the line bit-rate to input-output pair (i,j), 0≦i,j ≦N−1, if the following condition holds:




  
    
      
        
          
            
              ∑
              m
            
            ⁢
            
              p
              im
            
          
          ≤
          
            1
            2
          
        
        ,
        
          

        
        ⁢
        
          
            
              ∑
              m
            
            ⁢
            
              p
              mj
            
          
          ≤
          
            1
            2
          
        
        ,
      
    
    
      
        (
        5
        )
      
    
  




Proof: Condition (5) implies (4), so Theorem follows from Lemma 2.

The above theorem holds for the WPIM as well, considering the fact that PIM finds a maximal matching between inputs and outputs.

Admission control in WRRGS is simple, new Δaij time slots are assigned to input-output pair (i,j) if:




  
    
      
        
          
            
              
                ∑
                m
              
              ⁢
              
                a
                im
              
            
            +
            
              Δ
              ⁢
              
                
              
              ⁢
              
                a
                ij
              
            
          
          ≤
          
            
              F
              +
              1
            
            2
          
        
        ,
        
          

        
        ⁢
        
          
            
              
                ∑
                m
              
              ⁢
              
                a
                mj
              
            
            +
            
              Δ
              ⁢
              
                
              
              ⁢
              
                a
                ij
              
            
          
          ≤
          
            
              F
              +
              1
            
            2
          
        
        ,
      
    
    
      
        (
        6
        )
      
    
  




Central controller does not have to precompute schedule when a new request is admitted. Only input i has to update the value of aij←aij+Δaij, 0≦j≦N−1, in order to set the correct counter value cij=aij at the beginning of each frame. Consequently, WRRGS can follow fast changes of traffic pattern.

Let us assume that N is the number of inputs and outputs, F is the frame length in time slots, vb is the line bit-rate, and Tp is the packet transmission time. The maximum switch throughput is:

C=N·vb.  (7)

The session admission time equals the frame duration which is equal to:

A=F·Tp.  (8)

If some input-output pair is assigned one time slot per frame, it is guaranteed the bandwidth of:

G=vb·Tp/(F·Tp)=vb/F.  (9)

So, G is the traffic granularity. The line bit-rate vb and the packet transmission time Tp are limited by the technology, and the frame length F can be chosen arbitrarily. There is an apparent trade-off between the admission time and the traffic granularity: by increasing F the granularity is refined and the admission time is prolonged and vice versa. For some realistic parameters vb=10 Gbps, Tp=50 ns and chosen F=104, the admission time is A=104·50 ns=500 μs, and the traffic granularity is G=10 Gbps/104=1 Mbps. So, the proposed switch can rapidly allocate bandwidth with fine granularity.

Packets generated by some source for the given destination may have to pass through multiple switches. Therefore, the bandwidth should be reserved at each of these switches. With WRRGS, the bandwidth reservation at a particular switch is equivalent to the bandwidth reservation through the input and output lines in question. A switch stores the information about the bandwidth assigned to any input or output line, and would advertise this information to the other switches in the network by using interior gateway protocols (IGP). As has been shown, half of each link bit-rate can be reserved. Accordingly, the procedure for bandwidth reservation in wide area network becomes very simple. Namely, links with that do not have enough of the spare capacity are removed, and, then, the route is found by using the shortest path algorithm, for example. The small number of high-capacity switches would allow the fast bandwidth reservation in the wide area network.
The scheduling technique of the present invention can also be used for fast bandwidth reservations in high-capacity circuit switches. Instead of reserving an output for a single time slot in the future, an input reserves an output for a certain time slot of each frame. Afterwards, input also terminates the reservation. Unlike prior approaches, input modules must store the information about outputs reserved in each time slot of a frame. An input module also stores a table about its reserved output in each time slot of a frame. Moreover, the time slot duration can be very short in circuit switches, so that a selection takes multiple, e.g. r, time slots to be calculated. It follows that the bandwidth allocation can be changed in every block of r frames. Both bandwidth reservation and release are based on credits. At the beginning of a block of frames, each counter is loaded to the difference of the number of time slots newly assigned to the input-output pair, and the number of time slots released by this pair.
If the counter value is negative, an input-output pair releases its previously assigned time slot and increments the counter by one, until it becomes zero. Otherwise, if the counter value is positive, an input-output pair reserves time slots in a frame, and decrements the counter until its value is zero. As before, new bandwidth is allocated to some input-output pair if inequalities are fulfilled. Inputs sequentially release previously assigned time slots and then sequentially reserve admitted time slots, one after another. Pipelining can be applied. For example, input i releases an output r(2N−i+1) time slots in advance, and reserves an output r (N−i+1) time slots in advance. Input picks up output that has not been reserved in some of the previous blocks of frames, or by some of the previous inputs which reserve the outputs for the same time slot in the current block of frames. Note that each node would learn about all released and reserved outputs for some future time slot exactly r·N time slots after it releases or reserves an output for that time slot. So, the node can store the information in its tables before the next block of frames as long as rN≦rF, which is the case of interest. In a conclusion, in arbitrary block of frames the scheduler accepts new bandwidth requests, in the next block of frames it calculates new time slot assignment, and finally in the third block of frames the circuits are switched according to the new schedule. Of course, this process is also pipelined so that the switch time slot assignment can be changed at the beginning of each block of frames.
In accordance with the present invention, a simple way to flexibility share bandwidth in switches with input buffering has been described. The simplicity of the proposed protocol makes it attractive for switching of several Tb/s, assuming the current technology. It has also been shown that the proposed WRRGS can share at least 50% of the total switch capacity.
WRRGS has several desirable features. First, WRRGS algorithm can serve traffic with fast varying bandwidth requirements typical in data networks. Second, WRRGS requires simple. processing: only two round-robin choices are to be performed within one time slot. So, it can switch short cells transmitted at high bit-rates. In addition, a linear structure of the central controller easily scales to accommodate a large number of input and output ports, and provide high switching capacity.
While there are given above certain specific examples of this invention and its application in practical use, it should be understood that they are not intended to be exhaustive or to be limiting of the invention. On the contrary, these illustrations and explanations herein are given in order to acquaint others skilled in the art with this invention and the principles thereof and a suitable manner of its application in practical use, so that others skilled in the art may be enabled to modify the invention and to adapt and apply it in numerous forms each as may be best suited to the requirement of a particular use.

## Claims

1. A method of allocating bandwidth for use at a packet switch having a plurality of inputs and a plurality of outputs for switching to specified ones of said outputs, and a plurality of corresponding input buffers having input-output queues, and associated counters, said method comprising the steps of
dividing time into a plurality of frames of time slots, each frame having a specified integer value of time slots;
loading said counters associated with said input-output queues to a negotiated integer value once per said frame;
said inputs sequentially selecting said outputs that are available to which said inputs send packets in specified future time slots, wherein priority is given to those of input-output queues whose associated ones of said counters having positive values;
said selection of outputs by said inputs being done using a pipeline technique where input i, i=0,1, . . . , N−1 chooses output for time slot k in time slot k−┌(N−i)/p┐, where p is the number of selections made in one time slot, N is the number of inputs of said switch, and ┌(N−i)/p┐ is the smallest integer greater or equal to * (N−i)/p, and wherein a schedule is calculated within multiple time slots, wherein schedules for multiple future time slots being calculated in each time slot; and
decrementing said counters of selected queues by 1.

2. The method of claim 1, comprising a pipeline technique where input i, i=0, 1, . . . , N−1, chooses an output from a group of outputs l, l=0, 1, . . . , N−1, for time slot k in time slot k−((N−i)*p−l), where p is the number of time slots required for one selection and N is an integer corresponding to the number of inputs of the switch and also to the number of outputs of the switch.

3. The method of claim 1 implemented for packet transmission time T wherein new bandwidth can be allocated by using methods of claims 12 and 13 within time FT, wherein F is an integer corresponding to the number of time slots in said frame and wherein one time slot per frame guarantees bandwidth of B/F by using methods of claims 12 or 13, where B is the line bit-rate and a minimum guaranteed bandwidth can be made low by increasing F.

4. A method of allocating bandwidth in a circuit switch having a plurality of inputs for switching to outputs, input tables and input-output counters, said method comprising the steps of:
dividing time into a plurality of data frames comprising data time slots, and reservation frames comprising reservation time slots, each data frame having a specified number of data time slots;
loading said counters to a difference between a current total number of negotiated credits and a total number of released credits in the previous said reservation frames of said reservation frame;
said inputs sequentially releasing the time slot reservations for outputs if said inputs corresponding counters have negative values;
said inputs sequentially reserving time slots for unreserved outputs if said inputs corresponding counters have positive values;
whenever one of said inputs releases an output, the corresponding counter is incremented by 1, and whenever one of said inputs reserves an output the corresponding counter is decremented by 1;
said input tables being updated after all inputs have had a chance to release and reserve outputs for future time slots, storing outputs reserved in each time slot of a frame, and outputs reserved by the corresponding input in each time slot of a frame; and
said release of time slot reservation, reserving of time slots, and said updating of input tables being done using a pipeline technique, wherein a schedule is calculated within multiple reservation time slots, and schedules for multiple future time slots are calculated in each reservation time slot.

5. The method of claim 4, comprising a pipeline technique where input i releases output for time slot k+mF, m=1, 2, . . . in time slot ┌k−(2N−i)/p┐, and input i chooses output for time slot k+mF, m=1, 2, . . . in time slot ┌k−(N−i)/p┐, where ┌k−(N−i)/p┐ is the smallest integer greater or equal to k−(N−i)/p, and p is the integer ratio of the time slot duration and the reservation time slot duration, F is an integer corresponding to the number of time slots in said frame.

6. The method of claim 4, comprising a pipeline technique where input i, i=0, 1, . . . , N−1 releases an output from group of outputs l, l=0, 1, . . . , N−1, for time slot k+mF, m=1, 2, . . . in time slot k−((N−i)*p−l)−Np, and input i chooses an output from group of outputs l for time slot k+mF, m=1, 2, . . . in time slot k−((N−i)*p−l), where p is the ratio of the durations of the reservation time slot and data time slot, and F is an integer corresponding to the number of time slots in said frame.

7. The method of claim 4, comprising a pipeline technique wherein input table i is finally updated N reservation time slots after it reserved the output.

8. The method of claims 1 or 5, wherein input i loads its counters in time slot k*F+1−┌(N−i)/p┐, where F is an integer corresponding to the number of time slots in said frame.

9. The method of claims 2 or 6, wherein input i loads its counters for group of outputs l in time slot k*F+1−((N−i)*p−l), where F is an integer corresponding to the number of time slots in said frame.

10. The method of claim 4, comprising a pipeline technique where input i releases output for time slot k+nF+mpF, k<F, m, n=1, 2, . . . in reservation time slot (m−1)F+k−(2N−i), and input i chooses output for time slot k+nF+mpF, m, n=1, 2, . . . in reservation time slot (m−1)F+k−(N−i) where p is the integer ratio of the reservation time slot duration and the time slot duration, and F is an integer corresponding to the number of time slots in said frame.

11. The method of claim 10, wherein input i loads its counters in reservation time slot (m−1)F+1−(2N−i).

12. The method of claim 1 or 4 further comprising the step of negotiating values of time slots per frame associated with input-output pairs, wherein input-output pair (i,j) can be allocated additional Δaij time slots per frame of length F if it holds that:




  
    
      
        
          ∑
          m
        
        ⁢
        
          a
          im
        
      
      +
      
        
          ∑
          m
        
        ⁢
        
          a
          mj
        
      
      -
      
        a
        ij
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          a
          ij
        
      
    
    ≤
    F
  
  ,




where ak,l, 0≦k, l≦N−1, is the number of time slots previously assigned to input-output pair (k,l), and N is the number of inputs and outputs.

13. The method of claim 1 or 4 further comprising the step of negotiating values of time slots per frame associated with input-output pairs, wherein input-output pair (i,j) can be allocated additional Δaij time slots per frame of length F if it holds that:




  
    
      
        
          ∑
          m
        
        ⁢
        
          a
          im
        
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          a
          ij
        
      
    
    ≤
    
      
        (
        
          F
          +
          1
        
        )
      
      /
      2
    
  
  ,
  
    
      
        
          ∑
          m
        
        ⁢
        
          a
          mj
        
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          a
          ij
        
      
    
    ≤
    
      
        (
        
          F
          +
          1
        
        )
      
      /
      2
    
  
  ,




where ak,l, 0≦k,l≦N−1, is the number of time slots previously assigned to input-output pair (k,l).

14. The method of claim 1 or 4 further comprising the step of updating the number of credits per frame assigned to input-output pair, when its bandwidth request is accepted, or when it releases previously reserved bandwidth:


aij←aij+Δaij.

15. The method of claim 12 wherein steps 3 and 4 of bandwidth allocation the claim 1 method are replaced with any maximal matching algorithm, wherein the maximal matching algorithm is one in which either input will transmit packets or receiver will receive packets in some time slot if there is at least one packet from said input to the said output.

16. The methods of claim 12 wherein steps 3 and 4 of the claim 1 are replaced with parallel iterative matching algorithm (PIM).

17. The method of claim 13 wherein switches in a wide area network advertise to other switches reserved bandwidth on all input and output links:




  
    
      B
      i
    
    ⁢
    
      
        ∑
        m
      
      ⁢
      
        
          a
          im
        
        /
        F
      
    
  
  ,
  
    
      B
      j
    
    ⁢
    
      
        ∑
        m
      
      ⁢
      
        
          a
          mj
        
        /
        F
      
    
  
  ,
  
    0
    ≤
    i
  
  ,
  
    j
    ≤
    
      N
      -
      1
    
  
  ,




where Bi, Bj, are bit-rates of the corresponding links, and when a new bandwidth ΔB is requested between two nodes in the network, then all links for which:
ΔB>Bj/2−Bl′, are excluded from the network if Bl is the link bit-rate, and Bl′ is the link reserved bit-rate (bandwidth), and bandwidth is reserved according to any routing protocol using remaining links from the source to the destination.

18. An apparatus for allocating bandwidth for use at a packet switch having a plurality of inputs for switching to specified outputs and a plurality of corresponding input buffers having input-output queues, said buffers being coupled to said input-output queues having associated counters, said apparatus comprising:
means for loading said counters associated with said input-output queues to said negotiated integer value once per said a frame of a preselected number of time slots; and
means for negotiating values of time slots per frame associated with input-output pairs, wherein input-output pair (i,j) can be allocated additional Δaij, time slots per frame of length F if it holds that:





  
    
      
        
          ∑
          m
        
        ⁢
        
          a
          im
        
      
      +
      
        
          ∑
          m
        
        ⁢
        
          a
          mj
        
      
      -
      
        a
        ij
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          a
          ij
        
      
    
    ≤
    F
  
  ,



 where ak,l, 0≦k, l≦N−1, is the number of time slots previously assigned to input-output pair (k,l), and N is the number of inputs and outputs, and F is an integer corresponding to the number of time slots in said frame;
wherein said inputs sequentially selecting said outputs that are available to which said inputs send packets in specified future time slots, wherein priority is given to input-output queues with counters having positive values;
and wherein selection of outputs by said inputs is done using a pipeline technique, and wherein not all inputs select outputs for a specified time slot in the future while other inputs choose outputs for some other future time slots.

19. The apparatus of claim 18 further including negotiating means for negotiating values of time slots per frame associated with input-output pairs, wherein input-output pair (i,j) can be allocated additional Δaij time slots per frame of length F if it holds that:




  
    
      
        
          ∑
          m
        
        ⁢
        
          a
          im
        
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          a
          ij
        
      
    
    ≤
    
      
        (
        
          F
          +
          1
        
        )
      
      /
      2
    
  
  ,
  
    
      
        
          ∑
          m
        
        ⁢
        
          a
          mj
        
      
      +
      
        Δ
        ⁢
        
          
        
        ⁢
        
          a
          ij
        
      
    
    ≤
    
      
        (
        
          F
          +
          1
        
        )
      
      /
      2
    
  
  ,




where ak,l, 0≦k, l≦N−1, is the number of time slots previously assigned to input-output pair (k,l).