An Improved Algorithm for Lossless Smoothing of MPEG Video with Delay Constraints ? John Lauderdale , Danny H. K. Tsang feelauder,eetsangg@ee.ust.hk Department of Electrical and Electronic Engineering George Baciu gbaciu@cs.ust.hk Department of Computer Science Hong Kong University of Science and Technology Clear Water Bay , Kowloon , Hong Kong Abstract This paper improves upon a lossless smoothing algorithm designed to be used with a real-time Variable Bit Rate -LRB- VBR -RRB- MPEG video coding and transmission system . The original algorithm -LSB- 1 -RSB- is characterized by three parameters : D -LRB- delay bound -RRB- , K -LRB- number of frames with known sizes -RRB- , and H -LRB- lookahead interval -RRB- . The algorithm computes a lower bound for the transmission rate during each frame period such that the delay bound D is satisfied . An upper bound for the rate is computed to ensure continuous transmission . The rate assigned during each time interval should be between the upper and lower bounds . The improved algorithm introduces two additional parameters , and , which make network scheduling easier by reducing the number of rate changes . 1 Introduction With the large scale deployment of Broadband ISDN over the next few years , digital video is foreseen to become an increasingly important type of network traffic . It is expected that much of this digital video traffic will comply to the MPEG standard -LSB- 3 -RSB- . An important part of the current research work involving the transmission of multimedia over high speed computer networks is finding a practical solution for the efficient transport of MPEG video such that delay requirements are met and network management demands remain reasonable . Most video coder-decoders -LRB- codecs -RRB- for MPEG compliant bit streams produce either Constant Bit Rate -LRB- CBR -RRB- or Variable Bit Rate -LRB- VBR -RRB- streams . Most CBR codecs are considered \ lossy '' because they keep the output bit rate constant by changing the amount of discarded video information in each frame .1 An undesireable characteristic of the video produced by CBR ? Supported by Hongkong Telecom Institute of Information Technology grant HKTIIT93/94 . EG01 1In order to reduce the video bitstream rate , either the number of quantization levels of the DCT coefficients is reduced , or the high frequency DCT coefficients are not coded . See -LSB- 1 , 2 , 3 -RSB- for details about MPEG rate control . streams is that the video quality may change from frame to frame . However , the advantage of using CBR video is that it is very easy for a network provider to allocate a fixed bandwidth . VBR video codecs keep the amount of discarded video information in each frame constant , which results in the desireable characteristic of maintaining the quality of the video produced by the VBR stream decoder . However , VBR streams may require the output bit rate to change from frame to frame . These changes in rate from one frame to another may be very high , up to an order of magnitude , due to video content as well as the type of frame in an MPEG sequence , as decribed below . These frequent rate changes are more difficult for a network provider to effectively deal with . In order to avoid loss of data , the network provider can allocate bandwidth equivalent to the peak rate , which is the largest number of bits required to represent a frame divided by the frame period . This approach is inefficient because it wastes bandwidth . Alternatively , the network provider can allocate bandwidth dynamically during each frame interval . This saves bandwidth but is more difficult to schedule . Adaptive rate controlling techniques have been proposed by other researchers -LSB- 6 , 7 -RSB- , many of which change the rate of the codec based on the status of the network . These adaptive techniques are lossy , so the quality of the resulting video changes based on both the content of the video as well as the condition of the network . The algorithm for lossless smoothing of MPEG video detailed in -LSB- 1 -RSB- smooths the frame-toframe rate fluctuations in a video sequence , resulting in better network performance but slightly longer delay . Reducing the variance of the rate fluctuations results in improved statistical multiplexing performance when used with packet switching networks such as ATM . Lossless video smoothing is a lossless technique since it uses a buffer to smooth the output bit rate instead changing the amount of discarded video information . Assuming that the buffer never overflows and that no data is lost in the network , the quality of the resulting video remains constant . The algorithm is designed for realtime encoding so it is assumed that not all frame sizes within a sequence are known a priori . A lookahead strategy is used to estimate future frame sizes in order to adjust the transmission rate . The algorithm is characterized by three parameters : D -LRB- delay bound -RRB- , K -LRB- number of frames with known sizes -RRB- , and H -LRB- lookahead interval -RRB- . It attempts to transmit all frames within the pattern at approximately the same rate while guaranteeing that a delay bound D is satisfied for each frame . The algorithm computes a lower bound for the transmission rate during each frame period such that the delay bound D is satisfied . An upper bound for the rate is computed to ensure continuous transmission . The rate assigned during each time interval should be between the upper and lower bounds . If the rate during the previous period is between the upper and lower bounds of the current frame , the rate remains the same for the current frame . This reduces the number rate changes required . Otherwise , the rate during the current frame period is assigned to be the same as either the upper or lower bound of the current frame . The resulting bit rate profile is relatively smooth and changes gradually from one rate to another over a period of several frames . In many cases there is a small rate change during each consecutive frame . This type of smoothing , in which there are many small rate changes , works well if several sources are multiplexed onto a channel and no scheduling attempt is made to ensure that the traffic from all sources does not exceed the network capacity . However , the small but frequent rate changes in this algorithm can make network scheduling difficult . The main contribution of this paper is to demonstrate that the number of rate changes can be reduced by introducing two new parameters , and . When the previous rate is not between the current bounds , these parameters set the current rate between the current upper and lower bounds . Using a long , statistically significant MPEG sequence , the introduction of and is experimentally shown to reduce the number of rate changes required by up to a factor of four when compared with the original algorithm . Additionally , and can be selected such that minor improvement is shown based on several other performance measures including the rate difference from an \ ideal '' smoother , the maximum rate , and the standard deviation of the rate . The remainder of this paper is organized as follows . In Section 2 , the size and structure of MPEG frames are briefly reviewed . This is necessary to understand the size difference among different types of MPEG frames as well as the repeating structure found in MPEG 's Group of Pictures -LRB- GOP -RRB- . The reader is refered to -LSB- 1 , 2 , 3 -RSB- for a more complete discussion about MPEG . Section 3 presents the improved algorithm . The reader is refered to -LSB- 1 -RSB- for thorough details of the original algorithm as well as the mathematical justification of the rate bounds . Section 4 presents details of the MPEG test sequences used for the simulations . Section 5 uses diagrams to show the difference between the original and improved algorithms . It also shows how changing the delay constraint affects the bounds and rates . The algorithm performance is evaluated in Section 6 , where performance measures are defined which are used to compare the original and improved algorithms . Section 7 evaluates the performance of the algorithm as a function of parameters , , D , H , and K. Given a particular set of D ; K ; H parameters , a method is suggested for selecting good values and . Section 8 concludes the paper and presents suggestions for future research . 2 MPEG Video Frame types , sizes , and sequences Digital video is a set of sequentially displayed frames . Each frame is made up of an array of picture elements -LRB- pels -RRB- which are each represented by a certain number of bits2 . A video codec can be used to produce a bit stream which represents the video using less bits . Compressed video is more efficient to store and to transmit over networks . Intraframe coding can be performed on a single frame of video to reduce spatial redundancies , while interframe coding exploits the temporal redundancies between frames in a sequence of frames . Both interframe and intraframe compression techniques can be used to produce a MPEG compliant bit stream -LSB- 2 , 3 -RSB- . MPEG specifies three types of encoded frames : I -LRB- intracoded -RRB- , P -LRB- predicted -RRB- , and B -LRB- bi-directionally interpolated -RRB- . I-frames are coded with reference to only a single frame which results in a low 2Usually 24 bits are used to specify both luminance and color information . compression ratio -LRB- about 10:1 -RRB- . P-frames use motion compensation to predict frames from previous I-frames or P-frames , resulting in a higher compression ratio of approximately 20:1 . B - frames use bi-directional interpolation between the previous and successive I-frames or P-frames resulting in the highest compression ratio of approximately 40:1 . The actual compression ratios are specified by a parameter3 at the encoder . Due to different types of compression used for each frame type , an I-frame is , on average , two times larger than a P-frame and four times larger than a B-frame . However , a particular I-frame may be an order of magnitude larger than a particular B-frame . Figure 4 shows the frame sizes for a sample MPEG sequence . Successive frames with large size differences produce a very \ bursty '' bit stream when transmitted over a network on a frame-by-frame basis . A Group of Pictures -LRB- GOP -RRB- consists of a sequence of encoded frames which are specified by M , the distance , in number of frames , between I-frames or P-frames , and N , the distance between I-frames . For example , if M = 3 and N = 12 , the resulting GOP sequence is IBBPBBPBBPBBIBBP ... The sequence repeats indefinitely . Since B frames use both previous and successive frames as a basis for bidirectional interpolation , the order in which frames are displayed may not necessarily be the same order in which they are transmitted . For example , for a GOP sequence of IBBPBBI ... , the frames are displayed in sequential order -LRB- 1,2,3,4,5,6,7 ... -RRB- , while the order in which the frames are transmitted would be 1,4,2,3,7,5,6 ... resulting in the transmission sequence IPBBIBB ... . This is because frame 2 and 3 are B-frames which require that the previous I-frame -LRB- frame 1 -RRB- and the successive P-frame -LRB- frame 4 -RRB- be decoded before the B-frame can be decoded at the receiver . For our experiments , we assume that the transmission sequence is the same as the display sequence . 3 Algorithm Design The system model consists of a video encoder , a FIFO queue , and a network . The video encoder ri MPEG Encoder To network QueueFIFO Host Figure 1 : Rate smoothing system model . encodes frame number i , where i = 1,2,3 ... , at a rate of 1 = o frames per second , where o is the frame period . Let Si be the size , in bits , of the ith encoded frame . During the interval -LSB- -LRB- i ? 1 -RRB- o ; io -RSB- , Si 3The parameter most responsible for the compression ratio is the quantization parameter Q. See -LSB- 1 , 2 , 3 -RSB- for details . bits are sent from the encoder to the queue . For each frame period o , the algorithm calculates the rate ri -LRB- bits/second -RRB- at which the server of the queue will transmit Si bits into the network . The time when the server can begin sending frame i is denoted by ti . The departure time of frame i , when the server has just sent the last bit of frame i , is denoted by di . During the ith frame period , the lower bound and upper bound for the rate , denoted rLi and rUi , are derived in -LSB- 1 -RSB- as : rLi = Si D + -LRB- i ? 1 -RRB- o ? ti -LRB- 1 -RRB- and rUi = -LRB- Si -LRB- i + K -RRB- o?ti if ti < -LRB- i + K -RRB- o 1 otherwise -LRB- 2 -RRB- The rate ri is calculated at time ti where ti = maxfdi ? 1 ; -LRB- i ? 1 + K -RRB- og -LRB- 3 -RRB- So pictures i ; i + 1 ; : : : ; i ? 1 + K should have arrived and picture i ? 1 should have departed before the server can begin sending picture i . The departure time of picture i is di = ti + -LRB- Si = ri -RRB- -LRB- 4 -RRB- and the delay of picture i is delayi = di ? -LRB- i ? 1 -RRB- o -LRB- 5 -RRB- So rLi and rHi are dependent on ri ? 1 through di ? 1 and ti . The performance of the algorithm is improved by using a lookahead parameter to estimate the sizes of h future frames , where h = ; 1 ; 2 : : : H. Assuming that all future frames are sent at the current rate ri , the approximate delay for each frame is calculated . New lower and upper bounds , denoted by rLi -LRB- h -RRB- and rUi -LRB- h -RRB- , are computed by replacing i with i + h and Si with Phm = 0 Si + m in equations 1 and 2 : rLi -LRB- h -RRB- = Phm = 0 Si + m D + -LRB- i + h ? 1 -RRB- o ? ti -LRB- 6 -RRB- and rUi -LRB- h -RRB- = 8 < : Phm = 0 Si + m -LRB- i + h + K -RRB- o?ti if ti < -LRB- i + h + K -RRB- o 1 otherwise . -LRB- 7 -RRB- Since the assigned rate should be between the bounds , rLi -LRB- h -RRB- < = ri < = rUi -LRB- h -RRB- : -LRB- 8 -RRB- In both the original and the improved algorithms , if the rate assigned during the previous frame period is between the upper and lower bounds calculated for the current frame period , the rate remains the same for the current frame . However , if the previous rate is above the upper bound or below the lower bound for the current frame , then the original algorithm sets the rate to : ri = 8 > < > : rLi if ri ? 1 < rLi ; ri ? 1 if rLi < = ri ? 1 < = rUi ; rUi if ri ? 1 > rUi . -LRB- 9 -RRB- The new algorithm uses parameters and to assign the rate to be between the bounds : ri = 8 > < > : rLi + -LRB- rUi ? rLi -RRB- if ri ? 1 < rLi ; ri ? 1 if rLi < = ri ? 1 < = rUi ; rUi ? -LRB- rUi ? rLi -RRB- if ri ? 1 > rUi . -LRB- 10 -RRB- 3.1 Algorithm Specification The new smooth algorithm is specified in Figure 3 . Two auxiliary functions are used in the algorithm . The function notify -LRB- j , r -RRB- notifies a transmitter that frame j is to be sent at rate r . The function size -LRB- j , t -RRB- , shown in Figure 2 , returns either the actual or estimated size of frame j at time t . Our main contribution is the introduction of parameters and which modify the rate assignment . In order to keep the rate between the upper and lower bounds , both and should be between and 1 . When = = 0.0 , the algorithm performance is identical to the original . Additionally , the function size -LRB- i + h ; time -RRB- has been stated explictly in Figure 2 . The function size uses MPEG parameters M and N to estimate the current frame sizes if previous frame sizes are not available . The default frame sizes for the I , B , and P frames we used in our experiments are the averages for each frame type for our set of MPEG data4 . It is pointed out in -LSB- 1 -RSB- that is is not necessary that these values be accurate . Our function size -LRB- i + h ; time -RRB- assumes that the transmission sequence is the same as the display sequence . If the transmission sequence and the display sequence are different , as is discussed in the MPEG section , line 8 of function size would need to be modified to reflect the order of P-frames . The following parameters are used in the new smooth algorithm : F : total number of frames in sequence ; K : required number of complete frames buffered in the queue before the server can begin sending the next frame -LRB- 0 < = K < = F -RRB- ; specifically , the server can begin sending frame i only if frames i through i + K - 1 have arrived -LRB- each has been completely encoded -RRB- . D : maximum delay specified , in seconds , for every frame in the video sequence ; the frame delay is defined as the time difference between the arrival of the first bit of an encoded frame to the queue and the departure of its last bit from the queue ; the delay bound D is satisfiable iff D > = -LRB- K + 1 -RRB- o ; H : lookahead interval , in number of frames , which is used by the algorithm to reduce the number of rate changes required . 4The values we used are i default = 75000 bits , p default = 25000 bits , b default = 13000 bits . We set o = 1 = 30 seconds . The MPEG picture sequence parameters used in the size function are : int pic size -LSB- F -RSB- ; / * array of frame sizes Si . * / float tau ; / * the frame period o . * / int N ; / * the distance between I frames which is the number of frames in each GOP as specified by the MPEG sequence . * / int M ; / * the distance between I or P frames , as specified by MPEG . * / int I default ; / * the estimated I frame size used with first GOP . * / int P default ; / * the estimated P frame size used with first GOP . * / int B default ; / * the estimated B frame size used with first GOP . * / 1 . function size -LRB- int j ; float t -RRB- ; 2 . if -LRB- t > = j ? tau -RRB- then 3 . size : = pic size -LSB- j -RSB- ; / * use actual frame size * / 4 . elseif j > N then 5 . size : = pic size -LSB- j ? N -RSB- ; / * estimate frame size from prev . GOP * / 6 . elseif j = 1 then 7 . size : = I default ; / * if I frame of first GOP then use default I frame size * / 8 . elseif j = 1 + M or j = 1 +2 * M or j = 1 +3 * M or j = 1 +4 * M or j = 1 +5 * M ... then 9 . size : = P default ; / * if P frame of first GOP use default P frame size * / 10 . else 11 . size : = B default ; / * if B frame of first GOP use default B frame size * / 12 . endif 13 . return -LRB- size -RRB- Figure 2 : Specification of the size -LRB- j ; t -RRB- function which estimates the frame size at time t. 4 MPEG Test Sequences Throughout these experiments , frame size traces from a short sequence of 100 frames , a medium length sequence of 5000 frames , and a long sequence of 40000 frames will be used . The content of the MPEG sequence is a tennis tournament recorded from television5 . The short sequence is chosen because it is thought to be difficult to smooth due to large variations in frame size . The short sequence is used for illustrative purposes and is shown in the left plot of Figure 4 . The medium and long length sequences are used to obtain statistically significant results for the simulations . For clarity , only the I-frames of the medium length sequence is shown in the right plot of Figure 4 , which demonstrates the changes in frame sizes due to variations in scene complexity 6 in the video . The parameters used for the MPEG-1-encoder7 are shown in Table 4 . 5The sequence is available via anonymous ftp from -LSB- 5 -RSB- . 6Scene complexity relates to the amount of energy in the high order DCT coefficients . See -LSB- 2 , 3 -RSB- for MPEG coding details . 7The frame size traces were extracted from MPEG-1 sequences which have been encoded with the Berkeley MPEG-encoder -LRB- version 1.3 -RRB- which was adapted for motion-JPEG input . The frame sizes are in BITS . The videos were captured in motion-JPEG format from a VCR -LRB- VHS -RRB- with a SUN SPARCstation 20 and SunVideo . Picture Rate 30 frames per second . encoder input 384 x 288 pel color format YUV -LRB- 4:1:1 , resolution of 8 bits -RRB- quantization values I = 10 , P = 14 , B = 18 pattern IBBPBBPBBPBB GOP size 12 motion vector search ` Logarithmic ' / ` Simple ' reference frame ` Original ' slices 1 vector/range half pel / 10 Table 1 : MPEG video coding parameters 5 Bounds and Rates of Algorithms The left plot in Figure 5 shows the upper bound , lower bound , and rates assigned using the original algorithm for the 100 frame MPEG sequence using the parameters K = 1 , H = N = 12 , and D = :2 , as suggested in -LSB- 1 -RSB- . The figure shows that the assigned rates track the upper and lower bounds causing frequent small rate changes . The right plot in Figure 5 shows the upper bound , lower bound , and rate assigned using the improved algorithm with = 0.8 and = 0.7 . Since the rates assigned no longer track the bounds themselves , the number of rate changes is reduced from 45 using the original algorithm to 25 using the improved algorithm . Since the bounds change based on the rates assigned , the bounds for the original and improved algorithm are different since each algorithm assigns different rates -LRB- see Equations 3-5 -RRB- . Other performance measures as well as the method for selecting and are discussed in section 7.1 . 5.1 How Bounds are affected by Delay We now consider the effect on the bounds when the delay constraint D is relaxed . For both the original and the improved algorithm , when the maximum allowable delay is increased , the difference between the upper and lower bounds is increased , resulting in fewer rate changes . The left plot of Figure 5 shows the results of using the original algorithm with a delay constraint D = 0.2 seconds . Figure 6 shows the results of increasing the maximum delay to D = 0.3 seconds while still using the original algorithm . The resulting number of rate changes is reduced from 45 to 19 when the delay constraint is increased from 0.2 seconds to 0.3 seconds . Relaxing the delay constraint results in similar improvement when the improved algorithm is used . The performance of the improved algorithm as a function of , , and D will be examined in a later section . 6 Algorithm Performance 6.1 Performance Measures There are several ways to measure the performance of the smoothing algorithm as defined in -LSB- 1 -RSB- : number of rate changes Network scheduling is easier if fewer rate changes are required . This is the performance measure with which the improved algorithm shows the most improvement . maximum rate If the network bandwidth can not be allocated dynamically , it is necessary to reserve the maximum rate to avoid congestion . An algorithm which produces a lower maximum rate performs better . standard deviation The S.D. is a measure of the variation among rates . An algorithm which produces a lower S.D. performs better . area difference The area difference between the rates produced by the algorithm , ri , and the rates produced by an \ ideal '' smoother . The \ ideal '' smoother is a smoother which assigns a single rate equivalent to the average rate for all frames in a Group of Pictures . It is the sum of the frame sizes in each Group of Pictures -LRB- bits -RRB- divided by the Group of Pictures period : Si + Si +1 + : : : + Si + N No -LRB- 11 -RRB- The area difference is found by adding the absolute value of the difference between the frame rate assigned by the algorithm ri and the \ ideal '' frame rate8 . This \ ideal '' smoother does not guarantee a delay bound is satisfied . It also assumes that all frame sizes in each GOP are known prior to assigning a rate for all frames in the GOP . Using this performance measure , an algorithm producing a small area difference would perform well . Figure 7 shows the rates produced by the ideal smoother , the original algorithm , and the improved algorithm using the data from the short MPEG sequence . When comparing the area difference between the \ ideal '' smoother and both the original and the improved algorithm , both algorithms perform equally well . Figure 7 also shows that the number of rate changes required with the improved algorithm is less than the number required by the original algorithm . The improved algorithm \ jumps '' between rates in sizable increments whereas the original algorithm produces many smaller rate changes . For example , between frames 60 and 70 the original algorithm makes nine small rate changes , whereas the improved algorithm makes only three . 8Our \ ideal '' smoother is not delayed by one GOP as it is in -LSB- 1 -RSB- 6.2 Performance as a function of and To demonstrate the performance improvement achiveable by using various values for and , we use the long length -LRB- 40,000 frames -RRB- MPEG sequence and set K = 1 , H = N = 12 , D = :2 . We change and from to 1 in increments of 0.1 . We consider each measure of performance : 1 . Number of rate changes . Table 2 and the left plot of Figure 8 show the number of rate changes required for various values of and . The original algorithm requires 17389 changes while the improved algorithm requires only 9098 changes when = 0.3 and = 0.9 . Table 2 : Number of rate changes for long sequence = :0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.0 17389 13656 13110 13068 13552 13981 14400 14864 15509 16310 17408 0.1 15822 12026 11409 11309 11518 11780 12091 12498 12920 13589 14127 0.2 15163 11329 10677 10511 10629 10905 11149 11371 11779 12175 12711 0.3 14839 10881 10173 9997 9977 10225 10478 10660 10870 11299 11720 0.4 14628 10573 9857 9662 9617 9854 10057 10211 10437 10726 11227 0.5 14375 10331 9577 9399 9475 9459 9735 9940 10179 10454 10771 0.6 14490 10231 9431 9235 9205 9406 9615 9749 9991 10254 10608 0.7 14695 10199 9402 9176 9206 9380 9569 9694 10054 10248 10763 0.8 14797 10154 9365 9150 9149 9370 9542 9777 10176 10518 10893 0.9 14900 10119 9312 9098 9246 9386 9759 10024 10399 10996 11672 1.0 15429 10480 9677 9551 9683 9929 10372 10954 11753 12706 14719 2 . Maximum rate . The maximum rate required for various values of and are shown in the right plot of Figure 8 . Using this particular set of data , the maximum rate appears to be more sensitive to changes in rather than . The original algorithm has a maximum rate of 2.6160 Mbits/sec . When = :1 and = :1 , a maximum rate of only 2.6044 Mbits/sec is required . Generally , the maximum rate increases as increases , although the simulation results show that for some values of , the maximum rate is much smaller than for other values of . 3 . Standard Deviation . The left plot of Figure 9 shows the standard deviation of the rates . The standard deviation of the rates produced by the original algorithm is 2:7236 ? 105 bits/second . By choosing = :0 and = :9 , the standard deviation of the rates could be reduced slightly to 2:6797 ? 105 bits/second . 4 . Area Difference . The right plot of Figure 9 shows the area difference when compared to the \ ideal '' smoother . The difference produced by the original algorithm is 2:3134 ? 109 bits . When = 0.1 and = 0.7 , the difference can be reduced to 2:0800 ? 109 bits . 6.3 Reliability . Since the behavior of a particular MPEG sequence may affect the performance of the improved algorithm as well as the optimal and values , the experiments from section 6.2 were repeated using several 40,000 frame MPEG sequences . The number of rate changes required by the original algorithm are compared to the number of rate changes resulting from using the optimal and values . Table 3 shows that for all types of video , the optimal value is usually 0.3 or 0.4 and the optimal value is usually 0.8 or 0.9 . The next section shows that non-optimal values for and can still produce very good results . Table 3 : Number of rate changes for various 40000 frame MPEG sequences . Sequence Orig . Impr . Sequence Orig . Impr . Movies : \ James Bond : Goldfinger '' 13518 5204 0.3 0.6 News/Talk Show : \ Jurassic Park '' 26964 21491 0.4 0.9 TV news 30280 26553 0.3 0.9 \ Silence of the Lambs '' 34127 31363 0.3 0.9 Talk Show 17213 9937 0.3 0.8 \ Star Wars '' 31969 28590 0.4 0.9 Others : Sports : MTV video clips 18623 10044 0.3 0.9 ATP tennis 17389 9098 0.3 0.9 MTV video clips 22279 15419 0.3 0.9 formula 1 car race 13955 4460 0.4 0.7 Mr. Bean comedy 27306 22628 0.3 0.8 super bowl 17290 9097 0.5 0.8 \ The Simpsons '' 26955 21828 0.3 0.9 soccer world cup final 14994 6191 0.4 0.8 \ Asterix '' Cartoon 19559 12136 0.4 0.9 7 Performance as a Function of D , H , and K Choosing the optimal values for and is difficult because they are influenced by parameters D , H , and K , the performance measure and , to some degree , on the particular video sequence as was shown by the results of Table 3 . The original algorithm performs reasonably well for all performance measures except for the number of rate changes , so we will select and which minimize the number of rate changes . In order to show how the optimal values of and are affected by parameters D , H , and K , we fix two of the parameters and change the third while computing the number of rate changes for values and between and 1 in 0.1 increments . Although no attempt is made to pre-determine the optimal and values for all combinations of parameters , the trends of the optimal and values are shown as a function of each parameter . Simulation results using the 5000 frame sequence identify the optimal values for and . Non-optimal values of and which result in the number of rate changes close to the optimal value have also been identified In order to reduce the three-dimensional data representing , , and the number of rate changes to two dimensions , a \ good '' performing , combination is arbitrarily defined as one which results in a number of rate changes no more than 5 % above the number of rate changes required for the optimal , combination . For example , using the data from Table 2 , the number of rate changes is 9098 , so \ good '' , combinations would produce less than 9098 * 1.05 = 9553 rate changes . This could be thought of as cutting a horizontal plane at 9553 rate changes in Figure 8 which results in a two-dimensiontal region of \ good '' , combinations . In order to further reduce data representing the two-dimensional region of \ good '' , combinations , a simple search method is used to find the range of acceptable , combinations as follows . First , is set to the optimal value , Optimal , and the maximum and minimum value for is determined such that the , Optimal combination is within range . Second , is set to the optimal value , Optimal , and the maximum and minimum value for is determined such that the Optimal , combination is within range . This results in Max ; Min ; Max ; Min , which are plotted against the parameter being changed in Figures 10-12 . 7.1 Performance as a function of Delay D As was shown in the left plot of Figure 5 and Figure 6 , the bounds are highly sensitive to the delay D . When D is small -LRB- D < :1 -RRB- , the difference between the upper and lower bounds is small ; however , when D is large -LRB- D > :4 -RRB- the upper bound may be much larger than the lower bound . The left plot of Figure 10 shows the number of rate changes for Optimal and Optimal as well as the number of rate changes for the original algorithm . The improved algorithm is shown to reduce the number of rate changes when D > = :12 seconds . When D > = :25 seconds , the improved algorithm requires approximately 1/3 the number of rate changes as the original . Parameters and are sensitive to the bounds and are therefore sensitive to D as well . To determine values for and which minimize the number of rate changes for each delay , D is changed while K = 1 and H = N = 12 . The middle plot in Figure 10 shows the values for Max ; Optimal , and Min for each D while the right plot shows the values for Max ; Optimal , and Min . When D < = :12 seconds , the number of rate changes for Optimal ; Optimal is large , so any choice for and performs close to the optimal . As D increases above 0.16 seconds , the number of rate changes for Optimal ; Optimal decreases , which causes the range for Max ; Min ; Max , and Min to decrease . 7.2 Performance as a function of H The left plot in Figure 11 shows the number of rate changes for Optimal and Optimal as well as the number of rate changes for the original algorithm as a function of the lookahead , H. Parameter H was changed while D = 0.2 seconds and K = 1 . The number of rate changes is minimized for both the original and improved algorithms when H = 5 . A greater number of rate changes are required as the lookahead increases past 5 frames . The center plot in Figure 11 shows the values for Max ; Optimal , and Min for each H while the right plot shows the values for Max ; Optimal , and Min . When H > = 10 frames , the large difference between Max and Min , and between Max and Min , suggest that there are many values for and which result in close to the optimal performance . 7.3 Performance as a function of K The left plot of Figure 12 shows the number of rate changes for Optimal and Optimal as well as the number of rate changes for the original algorithm as a function of the number of buffered frames , K. Parameter K was changed while H = N = 12 and D = :1333 + -LRB- K + 1 -RRB- = 30 . The original algorithm has the minimum number of rate changes when K = 7 and the improved algorithm has the minimum number of rate changes when K = 8 . The center plot in Figure 11 shows the values for Max ; Optimal , and Min for each H while the right plot shows the values for , Max ; Optimal , and Min . Optimal remains between 0.3 and 0.5 for all K while the large difference between Max and Min suggests that any > = :5 would produce good results for most K. 8 Conclusions and Future Research An improved lossless smoothing algorithm has been introduced . The method of rate assignment is improved by using two new parameters , and . Several long , statistically significant MPEG sequences are used to simulate both the original and the improved algorithm . The improved algorithm is shown to greatly reduce the number of rate changes required , which eases network management by requiring bandwidth to be dynamically allocated a fewer number of times . Other performance measures have been examined including the rate difference from an \ ideal '' smoother , the maximum rate , and the standard deviation of the rate , for various values of and . It is found that for a given performance measure , particular instances of and can be found to significantly improve the performance of the original algorithm . As an aid to selecting values for and which reduce the number of rate changes , the trends for and have been examined as a function of algorithm parameters D , H , and K . While it is difficult to predict the optimal and which minimize the number of rate changes , many , combinations produce results very close to the results produced using the optimal , combination . An outstanding research problem is finding a better procedure for selecting and based on a given set of parameters and video sequence . Another approach to this problem is making and adaptive , such that and change based on known video information , such as the number of frames since the last rate change . References -LSB- 1 -RSB- Simon S. Lam , Simon Chow , and David K.Y. Yau , \ An Algorithm for Lossless Smoothing of MPEG Video , '' Proceedings of ACM SIGCOMM ' 94 , Vol . 24 No. 4 October 1994 , pp. 281-293 . -LSB- 2 -RSB- D. Le Gall . MPEG : A video compression standard for multimedia applications CACM , 34 -LRB- 4 -RRB- :46 -58 , April 1991 . -LSB- 3 -RSB- Coding of moving frames and associated audio , November 1991 . SC29/WG11 committee -LRB- MPEG -RRB- draft submitted to ISO-IEC/JTC1 SC29 . -LSB- 4 -RSB- T. Ott , T. Lakshman , and A. Tabatabai . A scheme for smoothing delay-sensitive traffic offered to ATM networks . Proceedings of INFOCOM ' 92 , pages 776-785 . -LSB- 5 -RSB- Oliver Rose . Statistical properties of MPEG video traffic and their impact on traffic modeling in ATM systems . University of Wuerzburg . Institute of Computer Science Research Report Series . Report No. 101 . February 1995 . The VBR MPEG video traces used in this research are available via anonymous ftp from ftp-info3 . informatik.uni-wuerzburg . de in the / pub/MPEG directory . -LSB- 6 -RSB- P. Pancha and M. El Zarki . Bandwidth requirements of variable bit rate MPEG sources in ATM networks . Proceedings of INFOCOM '93 pages 902-909 , March 1993 . -LSB- 7 -RSB- H. Kanakia , P. Mishra , and A. Reibman . An adaptive congestion control scheme for realtime packet video transport . Proceedings of ACM SIGCOMM '93 pages 20-31 , Sept 1993 . 1 . procedure new smooth -LRB- int H , K ; float D , alpha , beta -RRB- ; 2 . int i , h , sum ; 3 . float depart , time , rate , delay , lower , upper , old lower , old upper ; 4 . begin 5 . i : = 1 ; depart : = 0.0 ; 6 . while -LRB- i < = F -RRB- do 7 . time : = max -LRB- depart , -LRB- i - 1 + K -RRB- * tau -RRB- ; / * time to begin sending frame i * / 8 . h : = 0 ; sum : = 0 ; lower : = 0.0 ; upper : = 1 ; 9 . while -LRB- lower < = upper -RRB- and -LRB- h < = H -RRB- do 10 . sum : = sum + size -LRB- i + h ; time -RRB- ; 11 . old lower : = lower ; 12 . old upper : = upper ; 13 . lower : = sum / -LRB- D + -LRB- i + h - 1 -RRB- * tau - time -RRB- ; 14 . if -LRB- time > = -LRB- K + i + h -RRB- * tau -RRB- then 15 . upper : = 1 ; 16 . else 17 . upper : = sum / -LRB- -LRB- K + i + h -RRB- * tau - time -RRB- ; 18 . endif ; 19 . lower : = max -LRB- lower , old lower -RRB- ; 20 . upper : = min -LRB- upper , old upper -RRB- ; 21 . h : = h +1 ; 22 . endwhile 23 . if -LRB- lower > upper -RRB- then 24 . if -LRB- lower > old lower -RRB- then 25 . rate : = upper ; / * upper = old upper * / 26 . else 27 . rate : = lower ; / * lower = old lower , upper < old upper * / 28 . endif ; 29 . elseif -LRB- i = 1 -RRB- then / * h = H * / 30 . rate : = -LRB- lower + upper -RRB- / 2 ; / * rate for first frame * / 31 . elseif -LRB- rate > upper -RRB- then / * rate assignment modified below * / 32 . rate : = upper ? alpha ? -LRB- upper ? lower -RRB- ; 33 . elseif -LRB- rate < lower -RRB- then 34 . rate : = lower + beta ? -LRB- upper ? lower -RRB- ; 35 . endif 36 . notify -LRB- i , rate -RRB- ; / * notify transmitter of rate for frame i * / 37 . depart : = time + pic size -LSB- i -RSB- / rate ; / * departure time of frame i * / 38 . delay : = depart - -LRB- i-1 -RRB- * tau ; / * delay of frame i * / 39 . i : = i +1 ; 40 . endwhile ; 41 . end ; / * new smooth * / Figure 3 : Specification of the new smooth algorithm . I pictures P pictures B pictures 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 x 10 4 picture number size -LRB- bits -RRB- Short MPEG sequence . Coding Pattern : IBBPBBPBBPBB M = 3 N = 12 I pictures 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 2 4 6 8 10 12 14 16 18 x 10 4 picture number size -LRB- bits -RRB- Medium MPEG sequence . -LRB- I pictures only -RRB- Figure 4 : 100 and 5000 frame MPEG sequence . upper bound assigned rate lower bound 10 20 30 40 50 60 70 80 90 100 2 4 6 8 10 12 14 x 10 5 picture number bits/second K = 1 H = N = 12 D = 0.2 Alpha = 0.0 Beta = 0.0 Rt. . Ch . = 45 upper bound assigned rate lower bound 10 20 30 40 50 60 70 80 90 100 2 4 6 8 10 12 14 x 10 5 picture number bits/second K = 1 H = N = 12 D = 0.2 Alpha = .8 Beta = .7 Rt. . Ch . = 25 Figure 5 : Original and improved algorithm upper bounds , lower bounds , and rates . upper bound assigned rate lower bound 10 20 30 40 50 60 70 80 90 100 2 4 6 8 10 12 14 x 10 5 picture number bits/second K = 1 H = N = 12 D = 0.3 Alpha = 0.0 Beta = 0.0 Rt. . Ch . = 19 Figure 6 : Original algorithm bounds when D = :3 seconds `` ideal smoother '' rate -LRB- Average rate for GOP -RRB- orig . alg . rate -LRB- alpha = 0.0 beta = 0.0 Num . Rt. . Ch . = 45 -RRB- impr . alg . rate -LRB- alpha = 0.8 beta = 0.7 Num . Rt. . Ch . = 25 -RRB- 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 x 10 5 picture number -LRB- picture rate = 30 pictures/second -RRB- rate -LRB- bits/second -RRB- K = 1 H = N = 12 D = 0.2 Figure 7 : Rates of ideal smoother , original algorithm , and improved algorithm 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 0.8 1 1.2 1.4 1.6 1.8 x 10 4 alpha beta no . of rate changes Rate Changes 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 2.4 2.6 2.8 3 3.2 3.4 x 10 6 alpha beta maximum rate -LRB- bits/sec -RRB- Maximum Rate Figure 8 : Number of rate changes and maximum rate 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 2.65 2.7 2.75 2.8 2.85 x 10 5 alpha beta standard deviation of rate Standard Deviation 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 2 2.2 2.4 2.6 2.8 3 x 10 9 alpha beta area difference -LRB- bits -RRB- Area Difference Figure 9 : Standard Deviation and Area Difference beta max beta optimal beta min 0.1 0.2 0.3 0.4 0.5 0.2 0.4 0.6 0.8 1 Delay D -LRB- seconds -RRB- beta values alpha max alpha optimal alpha min 0.1 0.2 0.3 0.4 0.5 0.2 0.4 0.6 0.8 1 Delay D -LRB- seconds -RRB- alpha original algorithm optimal alpha and beta 0.1 0.2 0.3 0.4 0.5 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Delay D -LRB- seconds -RRB- number of rate changes Figure 10 : Number of Rate Changes using optimal and as a function of delay D beta max beta optimal beta min 5 10 15 20 0.2 0.4 0.6 0.8 1 Lookahead H -LRB- frames -RRB- beta alpha max alpha optimal alpha min 5 10 15 20 0.2 0.4 0.6 0.8 1 Lookahead H -LRB- frames -RRB- alpha original algorithm optimal alpha and beta 5 10 15 20 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Lookahead H -LRB- frames -RRB- number of rate changes Figure 11 : Number of Rate Changes using optimal and as a function of delay H beta max beta optimal beta min 2 4 6 8 10 12 0.2 0.4 0.6 0.8 1 Number of buffered pictures K -LRB- frames -RRB- beta alpha max alpha optimal alpha min 2 4 6 8 10 12 0.2 0.4 0.6 0.8 1 Number of buffered pictures K -LRB- frames -RRB- alpha original algorithm optimal alpha and beta 2 4 6 8 10 12 200 400 600 800 1000 1200 1400 1600 1800 2000 Number of buffered pictures K -LRB- frames -RRB- number of rate changes Figure 12 : Number of Rate Changes using optimal and as a function of delay K

O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O