Abstract:
Apparatus and a method for transmitting a plurality of streams of data packets through a computer network assigns queued packets from each of the plurality of data streams to respective slots in scheduling windows such that queued packets from respective ones of the plurality of data streams occupy different slots in respectively different ones of the scheduling windows. The packets are transmitted through the network in the order defined by the scheduling windows.

Description:
FIELD OF THE INVENTION 
     The present invention concerns the streaming of multiple packet streams over a local area network and in particular to balancing the network behavior of multiple real time protocol (RTP) streams. 
     BACKGROUND OF THE INVENTION 
     When an RTP scheduler selects which packets to send, it typically processes the streams in one-by-one order, according to either a round-robin scheme, in which each stream has equal priority, or some prioritized scheme in which streams having higher priority are scheduled with greater frequency than those with low priority. When the network becomes congested, however, these types of scheduling may present problems, in particular, the last-scheduled stream may experience a larger number of dropped packets than earlier transmitted streams. 
     To understand how this may happen, consider the transmission of three streams from a home server to two audio/video (A/V) clients using real-time protocol/real time streaming protocol (RTP/RTSP) in a network that is composed of an Ethernet® portion and a wireless portion. In this example, the wireless portion has a limited bandwidth and is accessed from a wireless access point (AP) on the Ethernet network. In this example, congestion may occur at the bridge between the Ethernet portion and the wireless portion of the network. 
     In the exemplary system, it is assumed that three RTP streams having the same constant bit-rate are to be sent from the wired network to respective wireless AV devices coupled to the network via the AP. In this simplified example, it is also assumed that these packets are the only traffic on the network between the wired and wireless portions. The scheduler is coupled to all three of the RTP sources and schedules the packets for transmission over the network. The scheduling of the packets occurs at known time intervals with a granularity that is determined by clock parameters of the scheduler. In an exemplary operating system (e.g. Microsoft® Windows,®) the granularity of the transmission interval is, in general, larger than one millisecond, which is much larger than a typical Ethernet packet. For example, a 100 Mbps Ethernet network typically transmits packets having a maximum size of about 1.5 KB. This packet may be transmitted in less than one-tenth of a millisecond. Consequently, the sequence of packets may appear as shown in  FIG. 1 . 
       FIG. 1  shows N scheduling windows ( 100   1  through  100   N ). Packets for stream  1  ( 110 ), stream  2  ( 112 ) and stream  3  ( 114 ) are sequentially scheduled in respective slots at the start of each scheduling window. In this example, following the stream  3  packet, the scheduling window is idle until the next scheduling period ( 116 ). The server transmits the scheduled packets during the next transmission interval. This example assumes that a backlog of queued packets exists at all times. In other words, the scheduler schedules “expired” packets, that is to say packets that should have been sent prior to the current moment and are now queued for transmission. While this scheduling algorithm typically works well for maintaining streaming rates for multiple streams, it may not operate well in a congested network. This is because different streams may experience different levels of dropped packets when, for example, the streams are sent at a higher rate than can be absorbed by the wireless portion of the network. In particular, the third stream may have more gaps because its packets are more likely to experience a full buffer at the wireless bridge than packets of the first or second streams. Although the exemplary embodiments show only one expired packet for each stream being scheduled during a scheduling window, depending on the maximum data rate of each stream, it is contemplated that multiple packets from a data stream may be scheduled during each scheduling window. 
     Another way to view the problem is to consider a congested network in having a router with a limited buffer queue for incoming packets. In this network if batches of packets are sent in order by first stream, second stream and third stream, then the first stream will be the least likely to experience packet loss and the third stream will be the most likely to experience packet loss. This may be seen, in the exemplary buffer of  FIGS. 2A through 2F . These Figures shows a buffer  200  at various points in time after receiving packets as transmitted using the scheduling algorithm shown in  FIG. 1 . This exemplary buffer  200  is a first-in-first-out (FIFO) buffer that is filled from the top and emptied from the bottom. In this example, the buffer is temporarily congested such that, during the time interval covered by the buffer diagrams of  FIGS. 2A through 2F , two packets are fetched from the buffer  200  in the time that one batch of three packets is received. The operation of this buffer is simplified in order to illustrate the problem addressed by the subject invention. It does not, for example, take into consideration the transfer of packets from other sources to the wireless network via the network bridge. It also assumes that the three data streams have equal data rates. These simplifications are assumed to clarify the explanation of the problem. The illustrated level of congestion is extreme, as one of every three packets is dropped. In a typical congested network the drop rate may be much less, for example one packet in 10 or one packet in 100. 
       FIG. 2A  shows the buffer  200  with one available slot  210 . In  FIG. 2B , two packets have been removed from buffer slots  214  and  216  and the remaining packets have been shifted down, leaving space to store three new packets in slots  213 ,  212  and  210 . As shown in the Figure, the stream  1  packet is received first, followed by the stream  2  and stream  3  packets.  FIG. 2C  illustrates the buffer  200  after the next batch of packets has been received. As before, two packets are removed from slots  214  and  216  of the buffer  200  and the remaining packets have been shifted down. This leaves only two slots  210  and  212  to receive the next batch of three packets. As shown in  FIG. 2C , the stream  1  packet is stored into slot  212  and the stream  2  packet is stored in slot  210 . The stream  3  packet (not shown) can not be received because the buffer  200  is full. Thus, the stream  3  packet is dropped.  FIGS. 2D ,  2 E and  2 F show the buffer at subsequent time intervals. During each time interval, two packets have been fetched from the buffer while a batch of three packets arrived to be stored. In each of these Figures, one packet must be dropped and in all cases, it is the packet for stream  3 . Thus, this scheduling scheme applied to the exemplary simplified buffer system effectively drops stream  3  entirely. 
     SUMMARY OF THE INVENTION 
     The present invention is embodied in apparatus and a method for transmitting a plurality of streams of data packets through a computer network. According to this method, queued packets from each of the plurality of data streams are assigned to respective slots in scheduling windows such that queued packets from respective ones of the plurality of data streams occupy different slots in respectively different ones of the scheduling windows. The packets are transmitted through the network in the order defined by the scheduling windows. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention is best understood from the following detailed description when read in connection with the accompanying drawings, with like elements having the same reference numerals. When a plurality of similar elements are present, a single reference numeral may be assigned to the plurality of similar elements with a small letter designation referring to specific elements. When referring to the elements collectively or to a non-specific one or more of the elements, the small letter designation may be dropped. The letter “n” may represent a non-specific number of elements. Included in the drawings are the following figures. 
         FIG. 1  is a scheduling diagram of a prior art scheduling algorithm. 
         FIGS. 2A ,  2 B,  2 C,  2 D,  2 E and  2 F are memory diagrams showing the state of a router buffer in response to the prior art scheduling algorithm shown in  FIG. 1 . 
         FIG. 3  is a functional block diagram of a system according to the present invention. 
         FIG. 4  is a scheduling diagram of an exemplary scheduling algorithm according to the present invention. 
         FIGS. 5A ,  5 B,  5 C,  5 D,  5 E and  5 F are memory diagrams showing the state of a router buffer in response to the exemplary scheduling algorithm shown in  FIG. 4 . 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       FIG. 3  is a functional block diagram of an exemplary home network in accordance with the present invention. As shown in  FIG. 3 , three streaming sources  310 ,  312  and  314  provide RTP packet streams to the network via the network interface  317  of a server  300 . The RTP packets are streamed onto the network. Because multiple streams are provided concurrently and because the home network has a limited bandwidth each stream includes packets that are currently being provided by the streaming source  310 ,  312  and  314  as well as queued packets that are overdue. These queued packets are labeled as “packets due” for the sources  310 ,  312  and  314 , shown in  FIG. 3 . 
     The queued packets provided by the various streams are applied to block  316  which schedules the packets for the streams in varying order before they are sent onto the network  318 . The packets are provided to the network  318  via the network interface  317  which, in turn, provides them to the router queue of the AP  320 . The AP, in turn, provides the packets for streams  1 ,  2  and  3  to the receivers  324 ,  326  and  328  respectively. Also applied to the router  320  are packets from other network nodes and, in a multi-network environment the AP  320  may also receive packets from other wireless networks. Although the router is shown as being a part of the AP, it is contemplated that it may be separate from the AP and that the router may serve several APs (not shown). 
     In one exemplary embodiment of the invention, the order of the packets in each scheduling window is randomly assigned by the packet scheduler  316 , responsive to an optional random number generator  317  (shown in phantom). In another exemplary embodiment, the order of the packets is changed according to a fixed algorithm. Because the packets for any stream may occur in various slots in the scheduling window, dropped packets are shared among the streams, resulting in better overall performance. 
     The operation of the algorithm is shown in  FIGS. 4-5F . The scheduling algorithm shown in  FIG. 4  changes the order of the packet streams according to a round-robin algorithm. As shown in  FIG. 4 , the scheduler  316  changes the order of the packets from stream  1 , stream  2 , stream  3  in respective slots of the first scheduling window, to stream  2 , stream  3 , stream  1  in the corresponding slots for the second scheduling window, to stream  3 , stream  1 , stream  2  for the third scheduling window (not shown) and so on. The sequence repeats every three scheduling windows as shown for the nth scheduling window. Although  FIG. 4  shows the scheduler  316  scheduling one packet per slot, it is contemplated that the scheduler may schedule more than one packet for a particular stream in each scheduling slot. Thus, for example, instead of scheduling one packet from stream  1  in the first slot  410 , the scheduler  316  may schedule multiple packets from stream  1  in the slot. In the exemplary embodiment of the invention, however, packets from only one stream are scheduled in any one scheduling slot. 
     The router buffer produced by the scheduling algorithm shown in  FIG. 4  is illustrated by  FIGS. 5A through 5F . For the sake of simplicity, the exemplary buffer shown in  FIGS. 5A through 5F  operates in the same way as the exemplary buffer described above with reference to  FIGS. 2A through 2F ; it is temporarily congested such that it provides two packets to the wireless network in the time that it receives three packets from the wired network.  FIG. 5A  shows the buffer  500  with one available slot  510 . As can be seen from the packets in the buffer  500 , the packets have arrived at the buffer in the round-robin changing order, described above with reference to  FIG. 4 . In the packet transmission time corresponding to each scheduling interval, one packet from each of the three streams is received and stored into the buffer. In  FIG. 5B , two packets have been removed from buffer slots  514  and  516  and the remaining packets have been shifted down, leaving space to store three new packets in slots  513 ,  512  and  510 . As shown in the Figure, the stream  2  packet is received first, followed by the stream  3  and stream  1  packets.  FIG. 5C  illustrates the buffer  500  after the next batch of packets has been received. As before, two packets are removed from slots  514  and  516  of the buffer  500  and the remaining packets have been shifted down. This leaves only two slots  510  and  512  to receive the next batch of three packets. As shown in  FIG. 5C , the stream  3  packet is stored into slot  512  and the stream  1  packet is stored in slot  510 . The stream  2  packet (not shown) can not be received because the buffer  500  is full. Thus, the stream  2  packet is dropped. 
     In  FIG. 5D , two packets have been removed from the bottom of the buffer and three packets have arrived. According to the round-robin schedule varying algorithm, these packets are from stream  1 , stream  2  and stream  3 . Because the buffer only has two empty slots, however, only the packets from stream  1  and stream  2  are stored in slots  512  and  510  of the buffer  500 .  FIG. 5E  shows the buffer after the next batch of packets has been received. Again, only two packets have been fetched from the buffer while three packets have been provided for storage. According to the round-robin algorithm, these packets are from steam  2 , stream  3  and stream  1 . In this instance, as shown in  FIG. 5E , it is the packet from stream  1  that is dropped. Finally,  FIG. 5F  shows the state of the buffer after the next time interval. As shown in this Figure, packets from stream  3 , stream  1  and stream  2  are received but only the packets from stream  3  and stream  1  are stored. The packet from stream  2  is dropped. 
     As can be seen from the buffer diagrams of  FIGS. 5A through 5F , the round-robin scheduling varying scheme is more fair than the prior art scheduling algorithm, described above with reference to  FIGS. 1 and 2A  through  2 F. Using the round-robin schedule changing algorithm, each stream experiences dropped packets but all streams continue to be received. For streaming media such as voice over Internet protocol (VoIP), which transmit data using UDP/IP, in which there is no provision for recovering dropped packets, the exemplary algorithm described above with reference to  FIGS. 4 and 5A  through  5 F may be preferred as the signals received at each of the receivers  324 ,  326  and  328  (shown in  FIG. 3 ) may still be intelligible even though each stream experiences dropped packets. Using the prior art fixed scheduling algorithm, by contrast, stream  3  is lost entirely so receiver  328  does not receive any signal. 
     As an alternative to the round-robin schedule changing algorithm, described above, other deterministic schemes may be used as long as the packets in scheduling windows are evenly distributed among the available slots. Furthermore, it is contemplated that the order of the packets in each scheduling window may be changed randomly or pseudo-randomly. In this embodiment of the invention, the random number generator  317  controls the packet scheduler  316  to randomly change the order of the packets in each scheduling window. If the random number generator controls the packet scheduler to equally distribute packets from the various streams among the slots in the scheduling window, the dropped packets resulting from network congestion will be equally distributed among the streams. It is contemplated that the random number generator may be a pseudo-random number generator implemented using a linear feedback shift register as is well known or it may be a true random number generator that may be implemented, for example, using circuitry that amplifies electrical noise to generate random data values. 
     Furthermore, it is contemplated that the scheme outlined above may be modified to implement a priority scheme while still ensuring that at least some packets from each stream are received. To implement a priority scheme using round-robin schedule changing algorithm described above, for example, packet schedules may be modified by inserting fixed packet schedules among the changing schedules. For example, the round robin algorithm  1 , 2 , 3 ;  2 , 3 , 1 ;  3 , 2 , 1 ;  1 , 2 , 3 ;  2 , 3 , 1 ;  3 , 1 , 2  may be changed by adding, for example, one or more  1 , 2 , 3  schedules. This would give added priority to streams  1  and  2  over stream  3 . 
     The pseudo-random or random algorithms may also be modified to implement a priority scheme by having the random number generator assign packets from one stream to the first slot in the scheduling window with greater probability. A typical random number generator, for example, generates random numbers ranging between zero and one. If this range is divided equally among the various combinations of packets, each combination should be sent with equal probability. For example, in the exemplary system described having three streaming sources labeled  1 ,  2  and  3 , there are six possible packet combinations:  1 , 2 , 3 ;  1 , 3 , 2 ;  2 , 3 , 1 ;  2 , 1 , 3 ;  3 , 1 , 2 ; and  3 , 2 , 1 . If the random number generator provides numbers in a range that is equally divided among these six combinations, for example,  0 - 1 / 6 .  1 / 6 - 2 / 6 ,  2 / 6 - 3 / 6 ,  3 / 6 - 4 / 6 ,  4 / 6 - 5 / 6  and  5 / 6  to  1 , then packets from each source have an equal probability of being dropped. If, however, the combinations that start with stream  1  are given a larger portion of the range then they would experience fewer dropped packets. One exemplary assignment for the set of combinations described above may be  0 - 1 / 4 ,  1 / 4 - 1 / 2 ,  1 / 2 - 5 / 8 ,  5 / 8 - 6 / 8 ,  6 / 8 - 7 / 8  and  7 / 8 - 1 . This assignment would give packets from stream  1  a lower likelihood of experiencing dropped packets while still ensuring that at least some packets from streams  2  and  3  are received. 
     It is contemplated that the subject invention may be implemented as computer software running on a general purpose computer. This computer software may be embodied in a computer readable carrier such as a magnetic or semiconductor memory, a magnetic or optical disc or in an audio, radio-frequency or optical carrier wave. 
     While the invention has been described in terms of exemplary embodiments, it is contemplated that it may be practiced as outlined above with modifications within the scope of the following claims.