Abstract:
The present invention provides an adaptive intra-macroblock refresh method, by which an intra-macroblock refresh technology is applied in a manner of considering an error recovering method of a decoder. The present invention includes the steps of measuring a correlation between a motion vector of a currently encoded macroblock and a motion vector of a neighbor macroblock, recording a measurement result of the correlation in a refresh map by block unit, and performing encoding using the refresh map.

Description:
[0001]     This application claims the benefit of the Korean Application No. P2004-0019387 filed on Mar. 22, 2004, which is hereby incorporated by reference.  
       BACKGROUND OF THE INVENTION  
       [0002]     1. Field of the Invention  
         [0003]     The present invention relates to an adaptive intra-macroblock refresh method, by which video information lost or damaged by an error on a system for compressed video transmission is effectively and quickly recovered or concealed.  
         [0004]     2. Discussion of the Related Art  
         [0005]     Generally, there are an intra-refresh method and error concealing method to recover video information damaged by an error under a channel environment exposed to errors effectively and quickly.  
         [0006]     In the intra-refresh method, an encoder encodes video information periodically or non-periodically into intra-frame or intra-macroblock to transfer to minimize influence and propagation caused by damaged data in case that the video information is damaged by channel error in a receiving end.  
         [0007]     In the error concealing method, a decoder conceals video information damaged or lost by transmission error using normally received data.  
         [0008]     The intra-refresh methods can be representatively categorized into a periodic intra-refresh method that periodically encodes video information into intra-frame or intra-macroblock to transfer and an adaptive intra-refresh technology recommended by MPEG4.  
         [0009]     The intra-macroblock refresh is a method of inserting a predetermined number of intra-macroblocks decided by an encoder forcibly to enable a decoder to quickly recover the video information damage or loss occurring due to channel error.  
         [0010]     In most cases, an intra-refresh unit is a frame unit. Yet, periodical insertion of intra-frame, which brings about serious video delay phenomenon and serious degradation of image quality under the same bit rate environment, is inefficient in a low bit rate real-time transmission system.  
         [0011]     Hence, a video compression standard for low bit rate real-time video transmission such as H.263 and MPEG4 uses a refresh method of macroblock unit. As a representative method, the adaptive intra-macroblock refresh technology is recommended in the MPEG4 Video Compression Standard Appendix H.  
         [0012]     The adaptive intra-macroblock refresh technology recommended by MPEG4 is explained as follows.  
         [0013]     First of all, the number of intra-macroblocks in VOP (video object plane) is decided according to a bit rate or frame rate prior to being fixed or encoded.  
         [0014]     Once the number of the intra-macroblocks is decided in VOP, the number of macroblocks to apply adaptive intra-refresh by the number of intra-macroblocks of VOP and the number of macroblocks having applied periodic intra-refresh thereto are decided.  
         [0015]     A macroblock of which refresh map is set to ‘1’ applies adaptive intra-macroblock refresh amounting to the number of macroblocks to which adaptive intra-refresh will be applied. A macroblock of which refresh map is set to ‘0’ applies periodic intra-macroblock refresh amounting to the number of macroblocks to which adaptive intra-refresh will be applied.  
         [0016]     In the adaptive intra-macroblock refresh, a motion of each of the macroblocks is preferentially measured. In case of a macroblock having many motions, macroblocks amounting to the decided number are encoded into intra-macroblocks only.  
         [0017]     In doing so, a motion measurement result is recorded in a refresh map by macroblock unit. An encoder decides whether to encode the currently encoded macroblocks into intra-macroblocks with reference to the recorded refresh map.  
         [0018]     In this case, the motion is measured by comparison of SAD (sum absolute difference) of a current macroblock and predefined SAD threshold. If the SAD of the current macroblock is greater than the SAD threshold, the current macroblock is decided as a macroblock having a motion and is then encoded into intra-macroblock. The SAD threshold uses mean SAD by macroblock unit of frame.  
         [0019]     An adaptive intra-macroblock refresh method according to a related art is explained with reference to the drawings as follows.  
         [0020]      FIGS. 1A  to  1 M are diagrams for explaining an adaptive intra-macroblock refresh method according to a related art.  
         [0021]     All macroblocks of a first VOP  100   a,  as shown in  FIG. 1A , are encoded into intra-macroblocks. Since a previous VOP fails to exist, a refresh map  110   a,  as shown in  FIG. 1B , is recorded as ‘0’. Reference numbers  101   a  to  101   d  mean specific video information such as a human face.  
         [0022]     A second VOP  100   b,  as shown in  FIG. 1C , is encoded into P-VOP. In dong so, an encoder encodes each macroblock using the refresh map  110   a  generated in  FIG. 1B .  
         [0023]     Since all values of the refresh map  110   a  in  FIG. 1B  are ‘0’, the encoder does not perform intra-macroblock refresh on the second VOP  100   b  but measures each motion of the macroblocks using an SAD of a current macroblock and an SAD threshold.  
         [0024]     Each motion of the macroblocks is measured according to the variation of the video information  101   b  shown in  FIG. 1C .  
         [0025]     If there exists an area  105   a  having the SAD of the current macroblock greater than the SAD threshold, a value of an area  115   a  having the SAD of the current macroblock greater than the SAD threshold like a refresh map  110   b  in  FIG. 1E  is updated to ‘1’ from ‘0’, in the refresh map  110   a  in  FIG. 1B .  
         [0026]     Subsequently, a third VOP  100   c,  as shown in  FIG. 1F , is encoded into intra-macroblocks  130   a.  In doing so, the encoder encodes the macroblock of the area  115   a  having the SAD greater than the SAD threshold as many as the number of predefined intra-macroblocks into the intra-macroblocks  130   a  using the refresh map generated in  FIG. 1E . In the present embodiment, it is exemplary shown that a case that the number of the intra-blocks is 2.  
         [0027]     Namely, the encoder checks whether the current macroblock is a target of intra-macroblock refresh. If it is the target of the intra-macroblock refresh, the encoder encodes it into the intra-macroblocks  130   a.    
         [0028]     In other words, the encoder encodes the intra-macroblock refresh target into the intra-macroblocks  130   a  in  FIG. 1F  and then updates the refresh map  110   b  in  FIG. 1E  like  FIG. 1G . Namely, the encoder changes a value of the refresh map of the macroblock encoded into the intra-macroblocks into ‘0’ and then performs the corresponding update like  FIG. 1G .  
         [0029]     And, the encoder measures the motion of each macroblock using the SAD of the current macroblock and the SAD threshold.  
         [0030]     If there exists an area  105   b,  as shown in  FIG. 11H , having the SAD of the current macroblock greater than the threshold as a result of the macroblock measurement result, a value of an area  115   b  having the SAD of the current macroblock greater than the threshold is updated to ‘1’, as shown in  FIG. 1I , in a refresh map  110   c  in  FIG. 1G .  
         [0031]     Subsequently, a fourth VOP  100   d  is encoded into intra-macroblocks  130 , as shown in  FIG. 1J , like the encoding method of the third VOP  100   c.  In doing so, the encoder encodes the macroblock of the area  115   b  having the SAD greater than the SAD threshold as many as the number of predefined intra-macroblocks into the intra-macroblocks  130   b  using the refresh map generated in  FIG. 1E .  
         [0032]     Namely, the encoder checks whether the current macroblock is a target of intra-macroblock refresh. If it is the target of the intra-macroblock refresh, the encoder encodes it into the intra-macroblocks  130   b.    
         [0033]     In other words, the encoder encodes the intra-macroblock refresh target into the intra macroblocks  130   b  in  FIG. 1J  and then updates the refresh map  110   c  in  FIG. 1I  like  FIG. 1J . Namely, the encoder changes a value of the refresh map of the macroblock encoded into the intra-macroblocks into ‘0’ and then performs the corresponding update like  FIG. 1K .  
         [0034]     And, the encoder measures the motion of each macroblock using the SAD of the current macroblock and the SAD threshold.  
         [0035]     If there exists an area  105   c,  as shown in  FIG. 1L , having the SAD of the current macroblock greater than the threshold as a result of the macroblock measurement result, a value of an area  115   c  having the SAD of the current macroblock greater than the threshold is updated to ‘1’, as shown in  FIG. 1M , in a refresh map  110   d  in  FIG. 1K .  
         [0036]     In the related art adaptive intra-macroblock refresh method, if a mean frame SAD is greater than the threshold, the intra-refresh map is updated to ‘1’ to perform the intra-macroblock refresh.  
         [0037]     However, since a case that the refresh map becomes ‘1’ relatively increases in case of an object having a great motion like the face area in a mobile terminal system, the intra-macroblock refresh area is unable to catch up with the substantially needed refresh area. Hence, the effect of the adaptive intra-macroblock refresh effect is reduced.  
         [0038]     Moreover, the effect of the adaptive intra-macroblock refresh increases or decreases according to the error concealing method in the decoder. Yet, since the error concealing method is not taken into consideration, image quality degradation and error propagation problems take place.  
       SUMMARY OF THE INVENTION  
       [0039]     Accordingly, the present invention is directed to an adaptive intra-macroblock refresh method that substantially obviates one or more problems due to limitations and disadvantages of the related art.  
         [0040]     An object of the present invention is to provide an adaptive intra-macroblock refresh method, by which an intra-macroblock refresh technology is applied in a manner of considering an error recovering method of a decoder.  
         [0041]     Another object of the present invention is to provide an adaptive intra-macroblock refresh method, by which image quality degradation and error propagation are maximally prevented for recovery in a manner of preferentially performing intra-macroblock refresh on a macroblock having a greatest possibility of image degradation in error concealment.  
         [0042]     A further object of the present invention is to provide an adaptive intra-macroblock refresh method, by which a high-quality video service can be provided under a channel environment exposed to an error like a mobile communication environment and in which video information lost and damaged by an error is effectively and quickly recovered or concealed on a mobile terminal system for compressed video transmission.  
         [0043]     Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.  
         [0044]     To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, an adaptive intra-macroblock refresh method according to the present invention includes the steps of measuring a correlation by comparing motion vectors between macroblocks adjacent to each other and encoding the macroblocks into intra-macroblocks or inter-macroblocks according to the correlation.  
         [0045]     In another aspect of the present invention, an adaptive intra-macroblock refresh method includes the steps of measuring a correlation between a motion vector of a currently encoded macroblock and a motion vector of a neighbor macroblock, recording a measurement result of the correlation in a refresh map by block unit, and performing encoding using the refresh map.  
         [0046]     In a further aspect of the present invention, an adaptive intra-macroblock refresh method includes the steps of comparing a threshold to an absolute value difference between a motion vector of a currently encoded macroblock and a motion vector of a neighbor macroblock, if the absolute value difference is greater than the threshold, deciding the currently encoded macroblock as a macroblock having a small correlation, updating a value of the small-correlation macroblock to ‘1’ from ‘0’ in a refresh map, and encoding macroblocks amounting to a predefined number among the macroblocks having the macroblock value of ‘1’ in the refresh map into intra-macroblocks.  
         [0047]     It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0048]     The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principle of the invention. In the drawings:  
         [0049]      FIGS. 1A  to  1 M are diagrams for explaining an adaptive intra-macroblock refresh method according to a related art;  
         [0050]      FIG. 2  is a flowchart of an adaptive intra-macroblock refresh method according to the present invention; and  
         [0051]      FIGS. 3A  to  1 M are diagrams for explaining an adaptive intra-macroblock refresh method according to the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0052]     Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.  
         [0053]      FIG. 2  is a flowchart of an adaptive intra-macroblock refresh method according to the present invention.  
         [0054]     Referring to  FIG. 2 , an encoder measures correlation between a motion vector of a currently encoded macroblock and a motion vector of a neighbor macroblock (S 200 ).  
         [0055]     Namely, the encoder measures the correlation between the motion vector of the currently encoded macroblock and the motion vector of the neighbor macroblock and then encodes the macroblock having small correlation into an intra-macroblock.  
         [0056]     The correlation is measured via comparison of an absolute difference value between the motion vector of the currently encoded macroblock and the motion vector of the neighbor macroblock and a threshold of the absolute difference value between the motion vectors.  
         [0057]     The threshold is experimental data and can vary according to a video size and bit rate. In case of encoding a QCIF size video at 64 Kbps, the threshold is set to ‘5’ to provide best performance.  
         [0058]     If the neighbor macroblock is an intra-macroblock when the encoder compares the correlation between the motion vectors, the encoder preferably measures the correlation using the motion vector of the neighbor macroblock recovered to an inter-macroblock.  
         [0059]     In the description of the present invention, an embodiment of measuring correlation using a motion vector of a macroblock in the vicinity of an upper side.  
         [0060]     In case of a macroblock situated at an upper image boundary, there is no upper macroblock. Hence, a usable neighbor macroblock is limited. Preferably, the correlation is measured using a motion vector of a macroblock at the same location of a former frame.  
         [0061]     In this case, the correlation is measured using the motion vector of the macroblock in the vicinity of the upper side. This is because a case that the error concealing method in the decoder uses the motion vector in the vicinity of the upper side is assumed.  
         [0062]     Namely, as another object of the present invention is to solve the problem that the adaptive intra-macroblock refresh is increased or decreased according to the error concealing method in the decoder, the intra-macroblock refresh is preferentially performed on the macroblock having the great possibility of image quality degradation in error concealment.  
         [0063]     For instance, the error concealing method in the decoder assumed by the present invention refers to the motion vector of the macroblock in the vicinity of the upper side in the macroblock having the error occurred therein.  
         [0064]     The error is concealed in a manner of bringing a portion corresponding to the error-occurring macroblock in a previous frame using the macroblock in the vicinity of the upper side and the motion vector of the macroblock in the vicinity of the upper side and then inserting the brought portion in the error-occurring macroblock of a present frame.  
         [0065]     The above-explained error concealing method raises the possibility of the image quality degradation in error concealment in case that correlation between a motion vector of an arbitrary macroblock and a motion vector of a macroblock in the vicinity of an upper side of the arbitrary macroblock is small.  
         [0066]     Hence, by measuring the correlation between a motion vector of a currently encoded macroblock and a motion vector of a macroblock in the vicinity of an upper side and by encoding the macroblock having small correlation into intra-macroblock preferentially, intra-macroblock refresh associated with the error concealing method in the decoder is enabled.  
         [0067]     Thus, the present invention can perform the intra-macroblock refresh more effectively in case that the error concealing method in the decoder uses the motion vectors. In doing so, the macroblock of which correlation is measured preferably has the same location of the macroblock of which motion vector is used in the error concealing method.  
         [0068]     Once the step  200  is performed, the encoder records the correlation measurement result in a refresh map by macroblock unit (S 202 ).  
         [0069]     Namely, the encoder measures the correlation between the motion vector absolute value difference and the threshold. If the motion vector absolute value difference is greater than the threshold, the encoder records the value of the refresh map as ‘1’. If the motion vector absolute value difference is smaller than the threshold, the encoder records the value of the refresh map as ‘0’.  
         [0070]     After completion of the step S 202 , the encoder performs encoding using the recorded refresh map (S 204 ). Namely, the encoder encodes the macroblock having the refresh map value of 1 into intra-macroblock or encodes the macroblock having the refresh map value of 0 into inter-macroblock.  
         [0071]     An encoding method using the refresh map is explained with reference to  FIGS. 3A  to  3 M as follows.  
         [0072]      FIGS. 3A  to  1 M are diagrams for explaining an adaptive intra-macroblock refresh method according to the present invention.  
         [0073]     All macroblocks of a first VOP  300   a,  as shown in  FIG. 3A , are encoded into intra-macroblocks. Since a previous VOP fails to exist, a refresh map  310   a,  as shown in  FIG. 3B , is recorded as ‘0’. Reference numbers  301   a  to  301   d  mean specific video information such as a human face.  
         [0074]     A second VOP  300   b,  as shown in  FIG. 3C , is encoded into P-VOP. In dong so, an encoder encodes each macroblock using the refresh map  310   a  generated in  FIG. 3B .  
         [0075]     Since all values of the refresh map  310   a  in  FIG. 3B  are ‘0’, the encoder does not perform intra-macroblock refresh on the second VOP  100   b  but measures correlation using an absolute value difference between a current macroblock motion vector and a neighbor macroblock motion vector and a threshold.  
         [0076]     Preferably, the correlation is measured using the threshold and the absolute value difference between the motion vector of a current macroblock and a motion vector of a macroblock in the vicinity of an upper side of the current macroblock.  
         [0077]     If there exists an area  305   a,  as shown in  FIG. 3D , in which the absolute value difference between the motion vector of the current macroblock and the motion vector of the macroblock in the vicinity of the upper side of the current macroblock is greater than the predefined threshold as a result of the macroblock correlation measurement, the refresh map  310   a  in  FIG. 3B  is updated like  FIG. 3E .  
         [0078]     Arrows in  FIG. 3D  indicate motion vectors. The macroblock failing to be provided with the arrow means that there exists no motion.  
         [0079]     Namely, the value of the refresh map of the area  305   a,  in which the absolute value difference between the motion vector of the current macroblock and the motion vector of the neighbor (upper) macroblock is greater than the predefined threshold, is updated to ‘1’ from ‘0’.  
         [0080]     Subsequently, a third VOP  300   c  is encoded like  FIG. 3F . In doing so, the encoder encodes each macroblock using the refresh map  310   b  generated in  FIG. 3E .  
         [0081]     In the present embodiment, it is exemplary shown that a case that the number of the intra-blocks is 2.  
         [0082]     The encoder checks whether the current macroblock is a target of intra-macroblock refresh. If it is the target of the intra-macroblock refresh, the encoder encodes it into intra-macroblocks  330   a.    
         [0083]     In other words, the encoder encodes the intra-macroblock refresh target area into the intra-macroblocks  330   a  in  FIG. 3F  and then updates the refresh map  310   b  in  FIG. 3E  like  FIG. 3G . Namely, the encoder changes a value of the refresh map  310   c  of the macroblock  313   a  encoded into the intra-macroblocks into ‘0’ and then performs the corresponding update like  FIG. 3G .  
         [0084]     Subsequently, the encoder measures the correlation using the predefined threshold and the absolute value difference between a motion vector of a current macroblock and a motion vector of a neighbor macroblock.  
         [0085]     If there exists an area  305   b,  as shown in  FIG. 3H , in which the absolute value difference between the motion vector of the current macroblock and the motion vector of the neighbor macroblock is greater than the predefined threshold as a result of the macroblock correlation measurement, the refresh map  310   c  in  FIG. 3G  is updated like  FIG. 3I . Namely, the value of the refresh map of the area  305   b,  in which the absolute value difference between the motion vector of the current macroblock and the motion vector of the neighbor macroblock is greater than the predefined threshold, is updated to ‘1’ from ‘0’.  
         [0086]     Subsequently, a fourth VOP  300   d  is encoded like  FIG. 3J . In doing so, the encoder encodes each macroblock using the refresh map  310   c  generated in  FIG. 3I .  
         [0087]     The encoder checks whether the current macroblock is a target of intra-macroblock refresh. If it is the target of the intra-macroblock refresh, the encoder encodes it into intra-macroblocks  303   b.    
         [0088]     In other words, the encoder encodes the intra-macroblock refresh target area into the intra-macroblocks  303   b  in  FIG. 3J  and then updates the refresh map  310   c  in  FIG. 3G  like  FIG. 3K . Namely, the encoder changes a value of the refresh map  310   d  of the macroblock  313   b  encoded into the intra-macroblocks into ‘0’ and then performs the corresponding update like  FIG. 3K .  
         [0089]     Subsequently, the encoder measures the correlation using the predefined threshold and the absolute value difference between a motion vector of a current macroblock and a motion vector of a neighbor macroblock.  
         [0090]     If there exists an area  305   c,  as shown in  FIG. 3L , in which the absolute value difference between the motion vector of the current macroblock and the motion vector of the neighbor macroblock is greater than the predefined threshold as a result of the macroblock correlation measurement, the refresh map  310   d  in  FIG. 3K  is updated like  FIG. 3M . Namely, the value of the refresh map of the area  305   c,  in which the absolute value difference between the motion vector of the current macroblock and the motion vector of the neighbor macroblock is greater than the predefined threshold, is updated to ‘1’ from ‘0’.  
         [0091]     If the number of intra-macroblocks is decided in VOP, the number of macroblocks, to which adaptive intra-refresh will be applied, and the number of macroblocks, to which periodic intra-refresh is applied, can be decided by the number of the intra-macroblocks in VOP.  
         [0092]     Adaptive intra-macroblock refresh is applicable to the macroblock of which refresh map is decided as ‘1’ as many as the number of the macroblocks to which the adaptive intra-refresh will be applied. And, periodic intra-macroblock refresh is applicable to the macroblock of which refresh map is decided as ‘0’ as many as the number of the macroblocks to which the periodic intra-refresh will be applied.  
         [0093]     The present invention repeats the above-explained steps to perform the intra-macroblock refresh.  
         [0094]     Accordingly, the present invention is capable of restoring and concealing the video information lost or damaged by errors under a transmission environment exposed to the errors such as a mobile communication environment effectively and quickly. Therefore, the present invention can provide the adaptive intra-macroblock refresh method capable of providing the high-quality video service in the products such as a next generation mobile communication terminal enabling video communications.  
         [0095]     It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.