Patent Publication Number: US-5253059-A

Title: Method and circuit for adjusting the size of a video frame

Description:
FIELD OF THE INVENTION 
     The present invention relates to a method and circuit for coding video signals. More specifically, the present invention relates to a technique for adjusting the size of a frame of video to be coded to control the amount of bits generated by a video coding circuit. 
     BACKGROUND OF THE INVENTION 
     Video services can be provided to subscribers by compressing a video signal using a coding circuit at a transmitting station. The compressed video signal is then transmitted via a telecommunication network such as a packet network to a remotely located subscriber station. At the subscriber station, the video signal is decompressed through use of a decoding circuit to reconstruct the original signal. It is expected that High Definition Television (HDTV) signals will be transmitted to subscribers in this manner. 
     Typically, a video coding circuit includes a processor for performing an orthogonal transform such as a discrete cosine transfer (DCT) and a quantizer for quantizing the transform coefficients resulting from the orthogonal transform. The coding circuit can code a frame of video using an intra-frame or an inter-frame compression technique. A frame of video to be coded is divided into blocks of pixels which may be 8×8 or 16×16 blocks. If the frame is coded using an intra-frame compression technique, the orthogonal transform is applied directly to each block of pixels and the resulting transform coefficients are quantized. If the frame of video is coded using an inter-frame technique, on a block-by-block basis, a decision is made to directly transform and quantize the actual block or to transform and quantize a predictive error which is the difference between the actual block and a prediction. Illustratively, the prediction is a motion-compensated prediction based on a previous frame. Some video coding circuits utilize only the intra-frame mode while other video coding circuits code some frames with an intra-frame mode and some frames with an inter-frame mode. In both the intra-frame and inter-frame mode, the quantized transform coefficients are converted by a coder into variable length (VLC) or fixed length (FLC) code words for transmission to a remote location. In some cases, the coder comprises a run-length coder followed by a variable length coder. 
     In a variety of applications, it is desirable to control the number of code bits generated by the coding circuit for particular frames. For example, if there is congestion in the network, it may be desirable to reduce the number of code bits generated by the coding circuit to reduce the average bandwidth required to transmit the video signal through the network. 
     In another example, it may be desirable to control the rate at which the coding circuit generates code bits so as to insure that code bits are transmitted from the coding circuit into the network at a smooth rate. It should be noted that for a particular frame, an intra-frame coding mode results in more code bits than an inter-frame coding mode. Furthermore, busy frames and frame portions result in the generation of more code bits than smooth frames and frame portions. Thus, in general, a video coding circuit generates code bits at a variable bit rate. 
     Some networks can handle the local fluctuations in bit rate produced by the coding circuit and the code bits are transmitted at a variable bit rate directly into the network. For other networks, it is necessary to smooth out the fluctuations in bit rate generated by the coding circuit before the code bits are transmitted into the network. The rate smoothing can be accomplished by use of a rate buffer which interfaces between the coding circuit and the network. Code bits are written into the rate buffer at a variable rate by the coding circuit and transmitted from the rate buffer into the network at a desired constant or approximately constant rate. To carry out this scheme, it is necessary to maintain the fraction of the rate buffer which is occupied within predetermined upper and lower limits. Thus, in a typical scheme, the contents of the rate buffer are fed back to the coding circuit to control the amount of code bits generated by the coding circuit. 
     The prior art discloses a number of techniques for controlling the rate at which code bits are generated by the coding circuit. 
     To adjust the rate at which code bits are produced by a coding circuit, the coarseness of the quantizer may be adjusted (see e.g. W. Chen and W. K. Pratt, &#34;Scene Adaptive Coder&#34;, IEEE Trans. Commun., Vol. COM-32, pp. 225-232, March 1984 and C. T. Chen et al, U.S. patent application Ser. No. 381,860 filed Jul. 19, 1989 and assigned to the assignee hereof). A coarse quantizer (i.e. a quantizer with a large step size) results in fewer code bits than a fine quantizer (i.e. a quantizer with a small step size). A problem with this approach for controlling the rate at which code bits are generated by a video coding circuit is that unacceptable artifacts may be produced when the quantizer is made very coarse to fit the signal within a specified bandwidth. 
     If the allowed budget of bits is too tight, another option is to decimate the signal spatially and/or temporally before coding. Typically, the source signal is decimated horizontally and vertically by a factor of two to obtain a quarter-size frame. However, use of a quarter-size frame can result in a coded image which is of poor quality. The drastic reduction in samples which occurs when there is a conversion to a quarter-size frame makes it difficult to achieve a desired amount of code bits while also maintaining a desired level of quality in the coded video signal. The coding algorithm cannot make up for the reduced number of samples in a quarter-size frame through quantization with a fine quantizer having a very small step size. There is a range of rates for which using a fine quantizer does not make up for the loss of quality resulting from use of a quarter-size frame in forming a coded image. 
     To overcome this problem, the present invention uses a coding circuit and coding method which achieve a more gradual reduction in frame size than the reduction to quarter-size frames utilized in the prior art. Preferably, the present invention utilizes spatial reduction of M/N in each of the horizontal and vertical dimensions for a total frame size reduction of M 2  /N 2 , where M and N are integers and where O&lt;M/N&lt;1. 
     The normal procedure for a conversion by a factor M/N consists of first using interpolation (sampling rate increase) by a factor of M followed by decimation (sampling rate decrease) by a factor of N (see e.g., R. E. Crochiere et al., &#34;Multirate Digital Signal Processing&#34;, Prentice Hall, Englewood Cliffs, N.J., 1983) In practice, these interpolation and decimation operations are carried out using periodically shift-varying filters that are derived from interpolation and decimation filters using a polyphase representation. However, such filters are quite complex and are difficult to implement, especially when the flexibility to select among different M/N ratios is desired. One reason for the difficulty in achieving flexibility is that the polyphase periodically shift-varying filters combine both low pass anti-aliasing filtering and interpolation and decimation. 
     In view of the foregoing, it is an object of the present invention to provide a method and circuit for coding a video image wherein the number of code bits produced can be varied while still maintaining a desired level of coded image quality. More specifically, it is an object of the present invention to provide a video coding circuit and method in which a preprocessing filter stage flexibly achieves a selected frame size reduction in both the horizontal and vertical directions to control the number of code bits generated by a video coding circuit. It is a further object of the present invention to provide a video coding circuit and method which controls the number of code bits by controlling the frame size in combination with controlling the quantizer step size. 
     SUMMARY OF THE INVENTION 
     In accordance with an illustrative embodiment of the present invention, a video coding circuit includes a preprocessing stage and a coding stage. 
     The invention may utilize any type of coding stage. In an illustrative embodiment, the coding stage includes a processor for performing a discrete cosine transform and a quantizer for quantizing the discrete cosine transform coefficients. The coding stage may utilize an intra-frame or an inter-frame compression technique. 
     The preprocessing stage achieves a desired reduction in frame size to control the number of bits generated by the coding stage. 
     The input to the preprocessing stage is a sequence of full-size video frames. The output of the preprocessing stage is a sequence of video frames of reduced size. This sequence of video frames of reduced size is input into the coding stage which outputs a coded video signal for transmission to a remotely located decoder. In the preprocessing stage, the frames of video are processed by both low pass anti-aliasing filters and shift varying sample reduction filters. 
     More specifically, in the preprocessing stage, the video frames are filtered in the horizontal direction by a low pass anti-aliasing filter with a normalized cutoff frequency which is equal to or less than M H  π/N H . The video frames are also filtered in the vertical direction by a low pass anti-aliasing filter with a normalized cutoff frequency which is equal to or less than M v  π/N v . The desired reduction factor in the horizontal direction is M H  /N H  and the desired reduction factor is in the vertical direction is M v  /N v , wherein M H ,N H , M v , and N v  are non-negative integers. In a preferred embodiment, the desired reduction factor is the same in the horizontal and vertical directions and is represented by M/N. 
     Illustratively, the low pass filters for the horizontal and vertical directions are implemented by finite impulse response (FIR) linear transverse filters. A lookup table is associated with each of the filters to provide the filter coefficients for a selected value of M/N or for selected values of M N  /N N  and M v  /N v . 
     It should be noted that the vertical and horizontal low pass filters do not change the number of pixel values (i.e. samples) in a frame. These filters only eliminate frequency components which are above the cutoff in both dimensions to suppress the aliasing that normally occurs due to reducing the number of samples in a frame. 
     To reduce the number of samples in a frame to achieve a reduction in frame size, a shift varying filter is then applied horizontally and vertically. A first particular shift varying filter out of a plurality of available shift varying filters is selected to reduce the horizontal sample rate by a selected value of M H  /N H . A second particular shift varying filter is selected to reduce the vertical sampling rate by a selected value M v  /N v . 
     The resulting reduced size frames are then coded by the coding stage and transmitted to a remotely located subscriber station. At the subscriber station, the arriving coded signal is decoded to reconstruct the reduced-size frames. The reduced-size frames are interpolated at the subscriber station to restore the original sampling rate if this is required by a display at the subscriber station. 
     In a preferred embodiment of the invention, the preprocessing stage includes a plurality of low pass filters and a plurality of shift varying filters for the horizontal and vertical directions corresponding to different values of M/N or M H  /N H  and M v  /N v . A feedback signal from the network or from a rate buffer which interfaces the coding stage to the network is applied to the preprocessing stage to select the desired frame size reduction factor to control the number of code bits generated by the coding stage. 
     The inventive preprocessing stage may be used in combination with a technique for controlling the step size of the quantizer used in the coding stage to further control the number of bits generated by the coding stage. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 schematically illustrates a video coding circuit in accordance with an illustrative embodiment of the present invention. 
     FIGS. 2A, 2B, 2C, and 2D illustrate the transmission characteristics of low pass filters utilized in the coding circuit of FIG. 1. 
     FIG. 3 schematically illustrates a low pass filter for use in the coding circuit of FIG. 1. 
     FIGS. 4A, 4B, 4C and 4D schematically illustrate the operation of a shift varying filter for use in the coding circuit of FIG. 1. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     FIG. 1 schematically illustrates a video coding circuit 10, a transmission network 20 and a remotely located subscriber station 30. 
     The coding circuit 10 includes a preprocessing stage 40 and a coding stage 50. The purpose of the preprocessing stage 40 is to receive frames of video to be coded at the input 41 and to reduce the size of the frames by a particular reduction factor. The reduced-size frames are outputted at the output 42 of the preprocessing stage 40 and are transmitted to the coding stage 50 at the input 51. The coding stage 50 converts each frame into a series of code bits at the output 52. The number of code bits generated by the coding stage 50 for each frame depends on the reduction factor used by the preprocessing stage 40 to reduce the frame size. Thus by controlling the reduction factor utilized by the preprocessing stage 40, it is possible to control the number of code bits generated for a frame by the coding stage 50. 
     The coding stage 50 is now considered. The present invention is applicable for any video coding scheme. For purposes of illustration, one particular coding scheme is briefly described herein. 
     Illustratively, the coding stage 50 comprises a DCT processor 53, a quantizer 54, a predictive error processor 55 and a coder 56. Typically, a coding stage such as the coding stage 50 can code in an intra-frame and in an interframe (e.g., a predictive) mode. To code a frame in an intra-frame mode, the frame is divided into blocks of pixels such as 8×8 or 16×16 blocks. The DCT processor 53 applies a DCT transform to each block of pixels and the resulting transform coefficients are quantized by the quantizer 54. The resulting coefficients are then coded by the coder 56. The coder 56 is, for example, a variable length coder or a run length coder in combination with a variable length coder. In addition, the output of the quantizer 54 is fed to the predictive error processor 56. In some coding units, the predictive error processor includes an inverse quantizer and an inverse DCT processor for reconstructing each block of pixels. The reconstructed blocks of pixels are stored for use in coding earlier or later frames in an inter-frame mode. In the case of a frame coded using an inter-frame mode such as a predictive mode, each block of pixels is transmitted to the predictive error processor 55. The predictive error processor 55 outputs a predictive error. Illustratively, the predictive error is the difference between the input block of pixels and a motion compensated prediction from a previous frame which is stored in the predictive error processor 55. Based on a decision criterion involving an activity index, the block of pixels is directly coded by the DCT processor 53 and quantizer 54 or the predictive error is coded by the DCT processor 53 and quantizer 54. In either case, the resulting quantized transform coefficients are processed by the coder 56 to generate code bits. The coder 56 may be a variable length or a fixed length coder. Illustratively, in a group of frames, the first frame is typically intra-frame coded and the remainder of the frames are inter-frame coded. 
     Depending on the type of coding stage utilized, code bits are generated at a constant or a variable rate. For the coding stage 50 illustrated in FIG 1 and discussed above, the bit rate at the output 52 is variable, depending for example on whether intra-frame or predictive coding is used for a given frame, as predictive coding results in fewer code bits than intra-frame coding. In addition, the number of bits generated varies with image content, with busy portions of frames resulting in more code bits than still portions of frames. In some cases, it is desirable to smooth out these local fluctuations in bit rate before the code bits are transmitted into the transmission network 20. To accomplish this, the optional rate buffer 60 is utilized. In other cases, the code bits generated by the coding stage 50 are transmitted directly into the network 20 at a variable rate. 
     The case wherein the coding stage 50 is directly connected to the transmission network 20 is considered first. In some situations, such as when there is network congestion, it may be desirable to reduce the average rate at which code bits are produced by the coding stage 50. In this case, the network 20 transmits via the feedback path 70 a signal to the preprocessing stage 40. The feedback signal is utilized by the preprocessing stage 40 to determine a frame reduction factor which in turn determines the size of the frames at the preprocessing stage output 42. For example, if it is desired to reduce the number of code bits produced by the coding stage 50 because of congestion in the network 20, the feedback path 70 transmits a signal which causes the frame size to be reduced. The size of reduction may be established once per group of frames. 
     In an alternative embodiment of the invention, it is desired to smooth out any variations in output bit rate of the coding stage 50. In this case, the rate buffer 60 is utilized. The rate buffer 60 receives code bits at a variable rate from the coding stage 50 and transmits code bits into the network 20 at a constant rate. In order for the rate buffer to work properly, the fraction of the rate buffer which is occupied has to be maintained between certain upper and lower limits. To maintain the occupancy of the buffer within these limits, a signal indicative of the occupancy fraction of the buffer is fed back via the feedback path 70&#39; to the preprocessing stage 40 to control the frame size, and therefore, the number of bits generated by the coding stage 50. If the occupancy of the rate buffer 60 is too low, the frame size is increased to increase the number of code bits produced by the coding stage 50. If the occupancy of the buffer 60 is too high, the frame size is decreased to decrease the number of code bits produced by the coding stage 50. 
     Whether or not the rate buffer 60 is utilized, the code bits generated by the coding stage 50 are transmitted to the remotely located subscriber station 30. The subscriber station 30 comprises a decoder 31, which reconstructs the reduced-size frames by reversing the operations of the coding stage 50, and an interpolator 32 which increases the size of the reduced-size frames back to full-size (i.e., reversing the frame size reducing operation of the preprocessing stage 40) if this is required by the display 33. 
     The preprocessing stage 40 is now considered in detail. The stage 40 comprises a horizontal low pass filter 43, a vertical low pass filter 44, a horizontal shift varying filter 45 and a vertical shift varying filter 46. 
     Full-size video frames are received at the input 41 of the stage 40. The frames are processed in the horizontal and vertical directions by the low pass filters 43 and 44 and then the number of samples in the horizontal and vertical directions is reduced by the shift varying filters 45 and 46 to generate reduced-sized frames at the output 42. 
     The cutoff frequency of the horizontal low pass filter 43 is M H  π/N H  and the cutoff frequency of the vertical low pass filter is M v  π/N v . The quantities M H  π/N H  and M v  π/N v  represent frequencies in radians which are normalized to the Nyquist frequency. (Thus, in an unfiltered signal the range of frequencies in radians is from 0 to π). As indicated above, M H  /N H  is the desired reduction factor in the horizontal direction and M v  /N v  is the desired reduction factor in the vertical direction. The horizontal and vertical reduction factors may be selected in response to the feedback signals transmitted via the path 70 from the network 20 or via the path 70&#39; from the rate buffer 60, if such a rate buffer is utilized. 
     In an illustrative embodiment of the invention, N H  and N v  are set to 8 and M H  and M v  may be selected from 4, 5, 6 and 7. 
     The low pass filters 45 and 46 may be implemented using finite impulse response filters with monotone passband frequency responses. A lookup table is provided so that different filter coefficients can be selected corresponding to different values of M H  /N H  and M v  /N v . FIGS. 2A, 2B, 2C and 2D illustrate the transmission characteristics for N(i.e., NM or N v )=8 and M (i.e., M H  or M v )=4, 5, 6, and 7. The quality of the reconstructed signal at the subscriber station 30 (see FIG. 1) depends on the responses of the filters utilized. In FIGS. 2A, 2B, 2C, 2D, the filters are of duration eleven (see FIG. 3) and the frequency responses are such that they result in some but not excessive aliasing. 
     The operation of an FIR filter 43 or 44 is illustrated in FIG. 3. Each of the filters 43 and 44 comprises a shift register 110 for storing video samples. The shift register 110 illustratively comprises eleven individual storage elements 120. At each clock cycle, all the video samples are shifted one location to the right, the rightmost sample is eliminated, and a new sample is stored in the leftmost storage element. For each clock cycle, an output sample appears at the output 130. Each output sample is a weighted sum of the input samples stored in the storage elements 120. To obtain the weighted sum, each of the samples stored in the storage elements 120 is multiplied by a weighting coefficient using a multiplier 140. The outputs of the multipliers 140 are added by the adder 145 to obtain an output sample at the output 130. Depending on whether M=4,5,6, or 7, different weighting coefficients are utilized. The coefficients for the different values of M are stored in the lookup table 150 and are applied to the multipliers 140 in response to an input signal on line 160, which input signal may be the feedback signal on path 70 or 70&#39; of FIG. 1. 
     After processing by the low pass filters 43 and 44 the frames are processed by the horizontal and vertical shift varying filters 45 and 46. The horizontal shift varying filter 45 reduces the number of samples in the horizontal direction by M H  /N H  and the vertical shift varying filter 46 reduces the number of samples in the vertical direction by M v  /N v . 
     The operation of a shift varying filter is illustrated in FIG. 4A, 4B, 4C and 4D. In particular, FIGS. 4A, 4B, 4C and 4D illustrate the operation of the shift varying filter 45 for the case when M/N=5/8. FIG. 4A shows the temporal locations of nine input samples for the shift varying filter 45. The input samples are labeled i 0 , i 7 , . . . ,i 8 . The temporal spacing between the input samples is T. For a 30 frame/sec video signal wherein each frame is 720×480 pixels, the sample spacing is about 9.6×10 -8  seconds. FIG. 4B shows the locations of the output samples 0 0 ,0 1 , . . . ,0 5 . Relative to the input samples, the temporal spacing of the output samples is 1.6 T. 
     The output sample O 0  is obtained by retaining the input sample i 0 . The output sample O 1 , is a weighted sum of the two nearest preceding and two nearest succeeding input samples i 0 ,i 1 ,i 2  and i 3 . Similarly, the output sample O 2 , is a weighted sum of the two nearest preceding and two nearest succeeding input samples i 2 ,i 3 ,i 4  and i 5 . The output sample O 3  is a weighted sum of i 4 ,i 5 ,i 6  and i 7  and the output sample 0 4  is a weighted sum of i 5 ,i 6 ,i 7  and i 8 . The output sample O 5  is obtained by retaining the input sample i 5 . 
     The operation of the shift varying filter 45 is illustrated in greater detail in FIG. 4C. The filter 45 illustratively comprises a shift register 200 for storing the input video samples i 0 ,i 1  . . . The shift register 200 comprises four individual storage elements 205. Four storage elements are needed because each output sample is a weighted sum of four input samples. To generate an output sample, a weighted sum of input samples is obtained using the multipliers 210. The weighting coefficients are supplied by a lookup table 230 which provides specific weighting coefficients depending on a selected value of M/N. The value of M/N may be selected by a signal applied to the input 240 of the lookup table 230. The signal applied to the input 240 of the lookup table 230 may be generated via one of the feedback paths 70 or 70&#39; (see FIG. 1). The outputs of the multipliers 210 are added by the adder 250. To obtain the appropriate output sample spacing at the output 270, the variable delay 260 is utilized. Depending on the particular value of M/N, a variable delay device 260 provides a sequence of delays to implement the proper output sample spacing. In the case where M/N is 5/8, fractional delays of 0, 0.6 T, 0.2 T, 0.8 T, and 0.4 T are used. This may be understood as follows: 
     FIG. 4D shows the contents of the shift register 200 at particular values of the discrete time variable nT. At time n=0, the output O 0  is formed by transmitting the input sample i 1  directly to the output 270. This is accomplished by setting all the weighting coefficients to zero except the coefficient of i 1  which is set to 1 and by setting the variable delay 260 to zero. The output sample O 1  is generated by obtaining a weighted sum of the content of the register at time n=1 and adding a fractional delay of 0.6 so that O 0  and O 1  are spaced apart by 1.6 T. As indicated above, the weighted sum is obtained using multipliers 210 and the delay is provided by the delay 260 of FIG. 4C. The output sample O 2  is obtained from a weighted sum of the register contents at time n=3 with a fractional delay of 0.2. The output sample O 3  is obtained from a weighted sum of the register contents at time n=4 with a fractional delay 0.8 and the output sample O 4  is obtained from a weighted sum of the register contents at time n=6 with a fractional delay of 0.4. The sequence of fractional delays 0, 0.6, 0.1, 0.8, 0.4 then repeats itself starting at time n=8 at which the output sample O 5  is set to be equal to i 8  with a zero fractional delay. 
     Thus, by selecting the appropriate weighting coefficients and fractional delays depending on a desired reduction factor M/N, the number of samples is reduced. This is accomplished in the horizontal direction using the shift varying filter 45 and in the vertical direction using the shift varying filter 46. The resulting reduced-size frames are then coded by the coding stage 50 in the manner described above. 
     In short, the number of bits used to code a frame of video is controlled by controlling the frame size. In a preprocessing stage of a video coding circuit, each frame of video is processed by a low pass anti-aliasing filter in the horizontal and vertical directions. The cutoff frequencies in the horizonal and vertical directions are determined by a selected frame reduction factor in the corresponding direction. Then to reduce the sampling rate in each direction, each frame of video is processed horizontally and vertically by a shift varying filter. When the sampling rate in both the horizontal and vertical direction is reduced by M/N, the number of code bits decreases linearly with M/N. 
     In addition, the strategy of the present invention for controlling video frame size may be used in combination with a strategy for controlling the quantizer step size to control the number of bits utilized to code a frame of video. In this case, a signal may be sent from the rate buffer 60 via path 70&#39; to the quantizer 54 to control the quantizer step size based on buffer content, for example. Alternatively, a signal may be sent from the network 20 via the path 70 to the quantizer 54 to control the quantizer step size. 
     Finally, the above described embodiments of the invention are intended to be illustrative only. Numerous alternative embodiments may be devised by those skilled in the art without departing from the spirit and scope of the following claims.