Abstract:
There is provided a method for automatically set a spatial resolution for an image to be encoded by properties of an image during an encoding process. The method includes the steps of dividing the moving image into blocks and compression encoding the moving image for each of the blocks, decoding an encoded moving image, obtaining a block distortion ratio from a decoded image, and making a resolution decision to select a first resolution lower than a current spatial resolution if the block distortion ratio is greater than a first threshold value or a second resolution higher than the current spatial resolution if the block distortion ratio is smaller than a second threshold value.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention generally relates to high efficiency compression encoding and decoding methods for video data of moving images, and particularly to a methods and an apparatus for automatic switching of spatial resolution of video signals to be encoded in accordance with properties of images during encoding of the moving images. 
     2. Description of the Related Art 
     Image compression is performed to transmit a maximum quality of images within a limited bandwidth in moving image transmissions. The image compression for a given spatial resolution, such compression methods as MPEG-4 and H.263 are employed, for example. In such compression methods for the moving images, there are requirements to switch the spatial resolution in encoding in accordance with complexity of the images in a same scene and in a same bit stream. 
     FIG. 1 shows a conventional switching apparatus for switching the spatial resolution. In FIG. 1, a switching apparatus  100  includes an encoding unit  101 , a single frame averaged quantization size calculation unit  102 , an originated data bit counter unit  103 , a resolution decision unit  104  and a memory unit  105 . A moving image signal  110  with a certain resolution is input to the encoding unit  101 . At first, a switching of the resolution takes place in accordance with a resolution  111  that is determined by the resolution decision unit  104 . Then, the encoding unit  101  encodes the moving images into a bit stream  112  by a given compression method, and also the encoding unit  101  outputs a quantization size  113  for each of blocks, which is input to the single frame averaged quantization size calculation unit  102 . The resolution decision unit  104  determines a resolution  111  by threshold values QP 1 , QP 2 , FR 1  and FR 2  as will be described later, based on an originated data bit quantity  114  (the number of information bits) from the originated bit counter unit  103 , a quantization size  115  output by the single frame averaged quantization size calculation unit  102 , and a previous resolution  116  output by the memory  105 . 
     An example of conventional technology for the resolution decision unit  104  may be referred to ITU-T Document Q15-C-15 “Video Codec Test Model, Near-Term, Version 9” issued in December 1997. FIG. 2 shows a process flow of the resolution decision method disclosed in this document. This method is based on a principle that a product of an average quantization size (QPpre in FIG. 2) expresses a complexity, i.e., a degree of difficulty in encoding, and an originated data size (B in FIG.  2 ). 
     FIG. 2 represents operations for one frame. In a step  201 , necessary parameters are provided. Th 1  and Th 2  represent threshold values. QPpre represents a single frame averaged quantization size of a frame encoded most recently. B represents an originated data volume of the frame that was encoded most recently. QP 1 , QP 2 , FR 1  and FR 2  are parameters to determine the threshold values. TB represents a targeted bit rate. Here, the threshold value Th 1  is a threshold value with respect to image complexity in a high spatial resolution. If a product of the single frame averaged quantization size QPpre for the most recent encoded frame, as described in above, and the originated data volume B of the most recent encoded frame is larger than the threshold value Th 1 , then it is determined that the image is exceedingly complex, and the resolution for the image to be encoded is chosen to be low. Conversely, the threshold value Th 2  is a threshold value with respect to the image complexity in a low spatial resolution. If the product of the single frame averaged quantization size QPpre for the most recent encoded frame, as described in above, and the originated data volume B is smaller than the threshold value Th 2 , then it is determined that the image is not complex, and the resolution for the image to be encoded is chosen to be high. FR 1  is a frame rate corresponding to the high resolution, and FR 2  is a frame rate corresponding to the low resolution. FR 1  and FR 2  may be equal. 
     In a step  202 , the threshold values Th 1  and Th 2  are determined. The threshold value Th 1  is calculated by multiplying the parameter QP 1  to the target bit rate per frame TB/FR 1  for the high resolution. Similarly, the threshold value Th 2  is calculated by multiplying the parameter QP 2  to the target bit rate per frame TB/FR 2  for the low resolution. 
     In a step  203 , a present spatial resolution, whether high or low, is checked. If the present spatial resolution is high, then the process proceeds to a step  204 , otherwise it proceeds to a step  205 . 
     In a step  204 , a decision is made whether or not the resolution for the next frame should be lowered, given that the present spatial resolution is high. Actually, the threshold value Th 1  is compared with the product of the single frame averaged quantization size QPpre for the most recently encoded frame and the originated data volume B for the most recently encoded frame, and if the product of QPpre and B is larger than Th 1 , then the spatial resolution for the next frame is lowered in a step  206 . 
     In a step  205 , a decision is made whether or not the resolution for the next frame should be heightened, given that the present spatial resolution is low. Actually, the threshold value Th 2  is compared with the product of the single frame averaged quantization size QPpre for the most recently encoded frame and the originated data volume B for the most recently encoded frame, and if the product of QPpre and B is lower than Th 2 , then the spatial resolution for the next frame is heightened in a step  207 . 
     In conventional technologies as described above, there is a problem. If a video content is of relatively still with modest movement, that is, if there is no discontinuity, such as scene changes, or abrupt movements in the contents, the product of the single frame averaged quantization size QPpre of the most recent encoded frame and the originated data volume B for the most recently encoded frame may be used as a standard to express the complexity of images. However, where the contents include images with such discontinuity as scene changes and abrupt movements, the product cannot be used as the standard for the image complexity. However, the product has been used as the standard to express the image complexity in conventional technologies. 
     For this reason, as described above, parameters have had to be manually adjusted in encoding when a video program contains images with abrupt motions, while the motions may not be highly visible to an audience, which has made a realtime encoding impossible and required a certain expertise to set the parameters. 
     SUMMARY OF THE INVENTION 
     It is a general object of the present invention to provide a method and an apparatus that substantially obviates one or more of the problems caused by the limitations and disadvantages of the related art. 
     Another object of the present invention to provide a method and an apparatus that automatically switches the spatial resolution of an image to be encoded by properties of the image, even during the encoding process. 
     The above objects of the present invention are achieved by an automatic setting method of a spatial resolution for a moving image, comprising the steps of: dividing the moving image into blocks and compression encoding the moving image for each of the blocks; decoding an encoded moving image thus obtained; obtaining a block distortion ratio from a decoded image; and making a resolution decision to select a first resolution lower than a current spatial resolution if the block distortion ratio is greater than a first threshold value or a second resolution higher than the current spatial resolution if the block distortion ratio is smaller than a second threshold value. In this manner, block distortions generated by a larger quantization size that has been introduced to compress the data volume to the required bit rate are suppressed. 
     The above-mentioned objects of the present invention are also achieved by an apparatus for automatically setting a spatial resolution for a moving image, comprising: a first unit dividing the moving image into blocks and compression encoding the moving image for each of the blocks; a second unit decoding an encoded moving image thus obtained; a third unit obtaining a block distortion ratio from a decoded image; and a fourth unit making a resolution decision to select a first resolution lower than a current spatial resolution if the block distortion ratio is greater than a first threshold value or a second resolution higher than the current spatial resolution if the block distortion ratio is smaller than a second threshold value. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     Other objects, features and advantages of the present invention will become more apparent from the following detailed description when read in conjunction with the accompanying drawings, in which: 
     FIG. 1 shows a conventional spatial resolution switching apparatus; 
     FIG. 2 is a flow chart showing a decision process for a resolution for a spatial resolution switching apparatus under conventional technologies; 
     FIG. 3 is a block diagram of a spatial resolution switching apparatus of the present invention; 
     FIG. 4 shows adjacent image blocks and pixels that are used to calculate a block distortion ratio; 
     FIG. 5 shows a calculation process for the block distortion ratio; and 
     FIG. 6 is a flow chart for the embodiment of the present invention. 
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     In the following, embodiments of the present invention will be described with reference to the accompanying drawings. 
     FIG. 3 shows an embodiment of the present invention. More particularly, FIG. 3 is a block diagram of a spatial resolution switching apparatus  300 . The spatial resolution switching apparatus  300  includes an encoding unit  301 , a block distortion measuring unit  302 , memory units  303  and  305  and a resolution decision unit  304 . 
     The encoding unit  301  receives input moving images  310  that have a certain resolution. At first, the resolution is determined by a resolution  311  that is decided by the resolution decision unit  304 . Then, the encoding unit  301  compresses the moving images as a bit stream  312  by a chosen compression method. If a differential encoding is employed as the encoding method, a local decode output image  313  is output from the encoding unit  301  to the block distortion measuring unit  302 , concurrently. Here, the local decode output image is one that is decoded in the encoding unit and that is used as a standard when a difference from the input image is calculated. The resolution of the local decode output image is set at same as that of the input image. If a differential encoding is not employed as the encoding method, the encoding unit  301  outputs an image  313  that is decoded from the compressed image, concurrently. The resolution decision unit  304  decides the resolution  311  according to operation flow as will be described with reference to FIGS. 4 through 6, based on a past block distortion ratio  315  supplied from the memory  303 , which is based on the block distortion ratio  314  from the block distortion measuring unit  302  and a resolution  316  provided by the memory  305 . 
     A calculation method for the block distortion ratio is described with reference to FIGS. 4 through 6. The block distortion ratio is defined as a ratio of an accumulated sum of pixel value differences at block boundaries both vertically and horizontally to an accumulated sum of pixel value differences within a block. 
     FIG. 4 shows adjacent image blocks and pixels used in the calculation of the block distortion ratio. A pixel that is uv pixels apart from a first pixel on the left of a block border is called Pvai. A pixel that is vv pixels apart from a first pixel on the right of the block border is called Pvbi. A pixel that is xv pixels apart toward the left from the pixel Pvai is called Pvci. A pixel that is yv pixels apart toward the right from the pixel Pvbi is called Pvdi. Further, a pixel that is uh pixels apart from a first pixel above the block border is called Phai. A pixel that is vh pixels apart from a first pixel below the block border is called Phbi. A pixel that is xh apart upward from the pixel Phai is called Phci. A pixel that is yh pixels apart downward from the pixel Phbi is called Phdi. Distances uv, vv, xv, yv, uh, vh, xh and yh are selected in accordance with a current resolution. Where the present resolution is low, adjacent cells may represent results from a low pass filter processing, implying a high correlation among adjacent pixels, thereby an unsatisfactory value may be obtained. In case the resolution is the highest, uv=vv=0 and xv=yv=1 are acceptable. Similarly, uh=vh=0 and xh=yh=1 are acceptable in that case. 
     FIG. 5 describes a calculation method of the block distortion ratio. First, a calculation method for the vertical direction is described. In a step  501 , pixels are chosen as described above for the calculation. Here, a distortion ratio at the block boundary is called Dv 1  and a distortion ratio in the block is called Dv 2 . 
     In a step  502 , the block boundary distortion ratio Dv 1  and the in-block distortion ratio Dv 2  are calculated. The block boundary distortion ratio Dv 1  is a sum of absolute values of differences between Pvai and Pvbi for all pixels (i=1, N) which occupy both blocks of the boundary in FIG.  4 . To calculate the in-block distortion ratio, a sum of absolute values of differences between Pvai and Pvci and a sum of absolute values of differences between Pvbi and Pdi are totaled for all pixels (i=1, N) in the block in FIG.  4 . Instead of the sums of absolute values, sums of the values raised to a higher power, e.g., square of difference values, may be calculated. 
     The block distortion ratio in the horizontal direction is described. In a step  503 , pixels are chosen for calculation as described above. Here, a block boundary distortion ratio is called Dh 1  and an in-block distortion ratio is called Dh2. 
     In a step  504 , the block boundary distortion ratio Dh 1  and the in-block distortion ratio Dh 2  are calculated. The block boundary distortion ratio Dh 1  is a sum of absolute values of differences between Phai and Phbi for all pixels (i=1, N) that occupy both blocks in FIG.  4 . Further, the in-block distortion ratio Dh 2  is a sum of absolute values of differences between Phai and Phci and a sum of absolute values of differences between Phbi and Phdi for all pixels in the block of FIG. 4 (i=1, N) are totaled. Instead of the sums of absolute values, sums of the values raised to a higher power, e.g., square of difference values, may be calculated. 
     In a step  505 , a block distortion ratio D is calculated by the following formula. D=(Dv 1 +Dh 1 ) ×2/(Dv 2 +Dh 2 ). The block distortion ratio D is output in a step  506 , to finish the process. 
     An operation detail of the resolution decision unit  304  based on the block distortion ratio as above calculated is described with reference to FIG. 6 that is a flow chart of the embodiment example of the present invention. In the embodiment in FIG. 6, there are two resolutions, high and low. There may be three or more resolutions that can be decided in a similar manner to the embodiment. 
     In a step  601 , a threshold value for a high spatial resolution is called Th 1  and another for a low spatial resolution is called Th 2 . The block distortion ratio D as calculated above is used. Further, frame numbers A and B are introduced, which are the numbers of the frames that will be examined before renewal of the threshold values. 
     In a step  602 , a present resolution is checked, whether high or low. If the resolution is high, then the process proceeds to a step  603 , otherwise to a step  604 . 
     In the step  603 , if a number of frames that have been processed since the resolution was determined high is smaller than B, then the process proceeds to a step  605  where the threshold value Th 1  is substituted by D, and then the spatial resolution is kept at high in a step  607 . The process then proceeds to a step  615  to end itself. 
     On the other hand, if the number of the frames that have been processed since the resolution was determined high is larger than B, then the process proceeds to a step  606  to determine whether or not the resolution is to be changed. In the step  606 , if the lock distortion ratio D exceeds the threshold value Th 1 , then the process proceeds to a step  608  where a test is performed to determine whether the resolution is a lowest resolution. In the step  608 , if the resolution is determined not to be the lowest resolution, then the process proceeds to a step  609  where the resolution is changed to a next lower resolution for the next frame to use the renewed resolution in encoding. On the other hand, if the block distortion ratio D is not greater than the threshold value Th 1 , then the process proceeds to the step  607  to maintain the resolution as described above. 
     In a step  604 , if a number of frames that have been processed since the resolution was determined low is less than A, then the process proceeds to a step  610  where the threshold value Th 2  is substituted with D and the spatial resolution is not changed in a step  611 . Then, the process proceeds to the step  615  to end the process. 
     On the other hand, if the number of frames that have been processed since the resolution was determined low is greater than A, then the process proceeds to a step  612  where a decision is made as to whether or not the resolution shall be changed. In the step  612 , in case that the block distortion ratio D is smaller than the threshold value Th 2 , the process proceeds to a step  613  for a checking whether the resolution is a highest resolution. In the step  613 , if the resolution is determined not to be the highest resolution, then the process proceeds to a step  614  where the resolution is heightened to a next higher resolution for encoding a next frame to use the renewed resolution. On the other hand, if the block distortion ratio D is not smaller than the threshold value Th 2  in the step  613 , then the process proceeds to the step  611  where no resolution change takes place as described above. 
     Through the process described above, the spatial resolution for a frame to be encoded next is determined. In the above embodiment, D was substituted to the threshold values Th 1  and Th 2 . However, one or both may be fixed. 
     As described above, the present invention provides a method and an apparatus that automatically align the spatial resolution for moving pictures while suppressing such undesired matters as the block distortion by lowering the spatial resolution and maintaining the quantization size constant when a large amount of data is necessary for encoding pictures with abrupt motions. 
     Further, the present invention is not limited to these embodiments, but various variations and modifications may be made without departing from the scope of the present invention. 
     The present application is based on Japanese priority application No. 2000-354895 filed on Nov. 21, 2000 with the Japanese Patent Office, the entire contents of which are hereby incorporated by reference.