Source: https://patents.google.com/patent/EP1829383B1/en
Timestamp: 2019-10-22 14:31:48
Document Index: 396384223

Matched Legal Cases: ['art 50', 'art 50', 'arts 70', 'arts 50', 'arts 50', 'arts 50', 'art 70', 'art 70', 'art 90', 'art 100', 'arts 90', 'art 90', 'art 100', 'art 110', 'art 110', 'art 110', 'art 130', 'art 130']

EP1829383B1 - Temporal estimation of a motion vector for video communications - Google Patents
Temporal estimation of a motion vector for video communications Download PDF
EP1829383B1
EP1829383B1 EP05855322A EP05855322A EP1829383B1 EP 1829383 B1 EP1829383 B1 EP 1829383B1 EP 05855322 A EP05855322 A EP 05855322A EP 05855322 A EP05855322 A EP 05855322A EP 1829383 B1 EP1829383 B1 EP 1829383B1
EP05855322A
EP1829383A1 (en
Nien-Chung Feng
2004-12-22 Priority to US11/022,362 priority Critical patent/US20060133495A1/en
2005-12-22 Priority to PCT/US2005/046739 priority patent/WO2006069297A1/en
2007-09-05 Publication of EP1829383A1 publication Critical patent/EP1829383A1/en
2011-01-05 Publication of EP1829383B1 publication Critical patent/EP1829383B1/en
Embodiments of the present invention pertain to the decoding (decompressing) of video data.
Conventional temporal error concealment techniques are based in the pixel domain. Consider a frame (the current frame) in which a motion vector associated with an area (e.g., a macroblock of interest) in the frame is missing. A set of motion vectors is formed by selecting motion vectors associated with macroblocks that surround the macroblock of interest in the current frame and motion vectors associated with macroblocks that surround the co-located macroblock in the reference frame (the co-located macroblock is the macroblock that is at the same position in the reference frame as the macroblock of interest is in the current frame). With a pixel-domain approach, a measure of distortion is calculated for each of the motion vectors in the set. To evaluate the distortion, pixel values are taken from the reconstructed frame buffer. In a motion select technique, the motion vector that minimizes the distortion measure is chosen as the replacement for the absent motion vector. In a motion search technique, a search for a motion vector that minimizes the distortion measure is performed within, for example, a 3x3 window of macroblocks.
Further attention is drawn to the document EP-A-1 395 061 . The document describes a method for an approximation of motion vector for image block, by deriving first set of vectors from motion vectors of neighboring blocks in same frame and corresponding block and its neighboring blocks in one or more preceding and/or subsequent frames.
Attention is also drawn to the paper by Kim D. W. et al, entitled "Block motion estimation based on spatio-temporal correlation" TENCON '96. PROCEEDINGS., 1996 IEEE TENON. DIGITAL SIGNAL PROCESSING APPLICATIONS PERTH, WA, AUSTRALIA 26-29 NOV. 1996, NEW YORK, NY, USA, IEEE, US, vol. 2, 26 November 1996 (1996-11-26), pages 955-960, XP010236811 ISBN: 0-7803-3679-8.
In accordance with the present invention, a method as set forth in claim 1, and a system, as set forth in claim 7, is provided. Embodiments of the invention are claimed in the dependent claims.
Figure 1 is a block diagram of one example of a system for decoding video data.
Figure 2 illustrates an example of two frames of image data organized as macroblocks.
Figure 3 illustrates an example of two image frames showing the motion of an object from one frame to the next.
Figure 4 is a data flow diagram showing the flow of data from a data encoding process to a data decoding process.
Figure 5 is a flowchart of a motion vector domain-based temporal error concealment method.
Figure 6 illustrates the flow of information according to the method of Figure 5.
Figure 7 is a flowchart of a method for selecting candidate motion vectors used in a motion vector domain-based temporal error concealment process.
Figure 8 illustrates the flow of information according to the method of Figure 7.
Figure 9 is a flowchart of a method for detecting frame-to-frame motion change and used for selecting candidate motion vectors used in a motion vector domain-based temporal error concealment process.
Figure 10 is a flowchart of another method for detecting frame-to-frame motion change and used for selecting candidate motion vectors used in a motion vector domain-based temporal error concealment process.
Figure 11 is a flowchart of a method for locating a motion boundary within a frame and used for selecting candidate motion vectors used in a motion vector domain-based temporal error concealment process.
Figure 12 illustrates the flow of information according to the method of Figure 11.
Figure 13 is a flowchart of a method that uses the trajectory of a moving object to select a candidate motion vector used in a motion vector domain-based temporal error concealment process.
Figure 14 illustrates the method of Figure 13.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present embodiments, discussions utilizing terms such as "selecting" or "determining" or "comparing" or "counting" or "deciding" or the like, refer to the actions and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
Figure 1 is a block diagram of an example of a system 10 upon which embodiments may be implemented. The system 10 shows the components of an execution platform for implementing certain functionality of the embodiments. As depicted in Figure 1, the system 10 includes a microprocessor 12 (e.g., an Advanced Reduced Instruction Set Computer Machine, or ARM, processor) coupled to a digital signal processor (DSP) 15 via a host interface 11. The host interface 11 translates data and commands passing between the microprocessor 12 and the DSP 15 into their respective formats. In the present embodiment, both the microprocessor 12 and the DSP 15 are coupled to a memory 17 via a memory controller 16. In the system 10 embodiment, the memory 17 is a shared memory, whereby the memory 17 stores instructions and data for both the microprocessor 12 and the DSP 15. Access to the shared memory 17 is through the memory controller 16. The shared memory 16 also includes a video frame buffer for storing pixel data that drives a coupled display 18.
Embodiments of the present invention can be used with Moving Pictures Experts Group (MPEG) compression (encoding) schemes such as MPEG-1, MPEG-2, MPEG-4, and International Telecommunication Union (ITU) encoding schemes such as H.261, H.263 and H.264; however, the present invention is not so limited. In general, embodiments can be used with encoding schemes that make use of temporal redundancy or motion compensation - in essence, encoding schemes that use motion vectors to increase the amount of compression (the compression ratio).
Figure 2 illustrates an example of two frames 21 and 22 of image or video data. In the example of Figure 2, the frame 21 (also referred to herein as the first frame or the reference frame) precedes the frame 22 (also referred to herein as the second frame or current frame) in display order. Each of the frames 21 and 22 is organized as a plurality of macroblocks, exemplified by the macroblock 23. In one embodiment, a macroblock has dimensions of 16 pixels by 16 pixels; however, the present invention is not so limited - macroblocks can have dimensions other than 16x16 pixels. Although Figure 2 shows a certain number of macroblocks, the present invention is not so limited.
In the example of Figure 2, a motion vector is associated with each macroblock. A motion vector has a dimension that describes its length (magnitude) and a dimension that describes its direction (angle). A motion vector may have a magnitude of zero. For purposes of illustration, a motion vector that is properly received at a decoder is represented herein as an arrow (e.g., arrow 24) within a macroblock. Macroblocks (e.g., macroblock 25) for which an associated motion vector was not properly received are indicated in Figure 2 by shading. The frame 22 also includes a macroblock 28 of interest, indicated by an "X," where a motion vector was not properly received. A motion vector may not be properly received if the data describing the motion vector is late, corrupted or missing.
As indicated in Figure 2, there may be instances in which motion vectors for several macroblocks (that is, a slice of macroblocks consisting of one or more consecutive macroblocks) are not properly received. As will be seen, according to the embodiments, a motion vector can be estimated for each macroblock in the slice for which a motion vector was not properly received. However, in general, a motion vector can be estimated for any macroblock where there is a desire to do so.
In one embodiment, to estimate a motion vector for a macroblock 28 in the current frame 22, a macroblock 29 in the reference frame 21 is identified. The macroblock 29 is in the same position within the frame 21 as the macroblock 28 of interest is in the frame 22. Accordingly, the macroblock 28 and the macroblock 29 are said to be co-located. Further, a first plurality (window 26) of macroblocks in the current frame 22 that neighbor the macroblock 28 is identified, and a second plurality (window 27) of macroblocks in the reference frame 21 that neighbor the macroblock 29 in the reference frame 21 is also identified. In one embodiment, the window 27 is in the same position within the frame 21 as the window 26 is in the frame 22. Accordingly, the window 26 and the window 27 are also said to be co-located. In general, the term "co-located" is used to describe a region (e.g., a macroblock or a window of macroblocks) of one frame and a corresponding region in another frame that are in the same positions within their respective frames. A pair of co-located macroblocks 108 and 109 is also indicated; that is, macroblock 108 is at a position within window 27 that is the same as the position of macroblock 109 within window 26.
In one embodiment, the array of macroblocks in the window 26 surrounds the macroblock 28 of interest. In one such embodiment, the window 26 and the window 27 each include a 3x3 array of macroblocks. Windows of different dimensions, including windows that are not square-shaped, can be selected. Also, a window does not necessarily have to surround the macroblock of interest, in particular for those instances in which the macroblock of interest is at the edge of a frame.
Figure 3 illustrates two consecutive image frames (a first frame 32 and a second frame 34) according to one embodiment. In the example of Figure 3, the second frame 34 follows the first frame 32 in display order. In an MPEG compression scheme, the first frame 32 can correspond to, for example, an I-frame or a P-frame, and the second frame 34 can correspond to, for example, a P-frame. In general, the first frame 32 and the second frame 34 are "inter-coded" (e.g., inter-coded frames are encoded dependent on other frames).
In the example of Figure 3, an object 33 is located at a certain position within the first frame 32, and the same object 33 is located at a different position within the second frame 34. The MPEG compression scheme works by encoding the differences between frames. A motion vector 35 is used as the simplest way of communicating the change in the image between the frames 32 and 34; that is, the image of the object 33 does not have to be sent again just because it moved. In a similar manner, a motion vector can be associated with a macroblock in a frame (e.g., the macroblock 23 of Figure 2).
Figure 4 is a data flow diagram 40 showing the flow of data from an encoder to a decoder according to one embodiment. In an encoder, an encoding process 42 compresses (encodes) data 41 using an encoding scheme such as MPEG-1, MPEG-2, MPEG-4, H.261, H.263 or H.264. The compressed data 43 is sent from the encoder to a decoder (e.g., the system 10 of Figure 1) via the channel 44, which may be a wired or wireless channel. The received data 45 may include both properly received data and corrupted data. Also, some data may also be lost during transmission or may arrive late at the decoder. The decoding process 46 decompresses (reconstructs) the received data 45 to generate the reconstructed data 47.
Figure 5 is a flowchart 50 of one embodiment of a motion vector domain-based temporal error concealment method. Although specific steps are disclosed in the flowchart 50 of Figure 5 (as well as in the flowcharts 70, 90, 100, 110 and 130 of Figures 7, 9, 10, 11 and 13, respectively), such steps are exemplary. That is, other embodiments may be formulated by performing various other steps or variations of the steps recited in the flowcharts 50, 70, 90, 100, 110 and 130. It is appreciated that the steps in the flowcharts 50, 70, 90, 100, 110 and 130 may be performed in an order different than presented, and that the steps in the flowcharts 50, 70, 90, 100, 110 and 130 are not necessarily performed in the sequence illustrated.
Figure 5 is described with reference also to Figure 6. Figure 6 shows a 3x3 window 63 of macroblocks selected from a reference frame 61, and a 3x3 window 64 of macroblocks selected from a current frame 62. It is understood that the reference frame 61 and the current frame 62 each include macroblocks in addition to the macroblocks included in the windows 63 and 64, respectively.
The window 63 and the window 64 are co-located. In the present embodiment, the macroblock (MB) 68 of interest - that is, the macroblock for which a motion vector is to be estimated - lies at the center of the window 64, but as mentioned above, that does not have to be the case.
It is understood that the windows 63 and 64 can be other than 3x3 windows. For instance, 5x5 windows may be used. Also, if the macroblock of interest is along one edge of the current frame 62, then a window that is not square in shape (e.g., a 3x2 or a 2x3 window) may be used.
In one embodiment, the reference frame 61 precedes the current frame 62 in display order. In another embodiment, the reference frame 61 may be a frame that comes after the current frame 62 in display order; that is, the reference frame 61 may be a "future frame." In yet another embodiment, both the frame preceding the current frame 62 and the future frame following the current frame 62 may be considered for the error concealment methods described herein.
In one embodiment, in a block 51 of Figure 5, the windows 63 and 64 are identified. Correctly received motion vectors associated with the macroblocks in the window 63, and correctly received motion vectors associated with the macroblocks in the window 64, are accessed.
In a block 52, in one embodiment, a determination is made as to whether motion vectors in the reference frame 61 (specifically, in the window 63) are eligible to be included in the set 65 of candidate motion vectors. Embodiments of methods used to make this determination are described in conjunction with Figures 7, 8, 9 and 10, below.
In a block 53 of Figure 5, in one embodiment, if motion vectors in the reference frame can be included in the set 65 of candidate motion vectors, then motion vectors from the window 63 and from the window 64 are intelligently selected and included in the set 65. Embodiments of methods used to select motion vectors from the windows 63 and 64 are described in conjunction with Figures 11, 12, 13 and 14, below.
In a block 54 of Figure 5, in one embodiment, if motion vectors in the reference frame are not eligible to be included in the set 65 of candidate motion vectors, then only motion vectors from window 64 are selected and included in the set 65. Note that it is possible that there may be instances in which the window 64 contains no properly received motion vectors. The method of Figures 7 and 8 can be used to address those instances.
In a block 55 of Figure 5, in one embodiment, a statistical measure of the set 65 of candidate motion vectors is determined. The statistical measure defines a motion vector 67 for the macroblock 68 of interest. The motion vector 67 can then be applied to the macroblock 68 of interest.
For an array of N m-dimensional vectors, V = (
for i =1,2,...,N, the median vector
is the vector that satisfies the following constraint:
∑ i = 1 N ‖ ν → VM - ν → i ‖ p ≤ ∑ i = 1 N ‖ ν → j - ν → i ‖ p ; j = 1 , 2 , … , N ;
where p denotes the p-norm metrics between the vectors. For simplicity, in one embodiment, p = 1 is used. For a two-dimensional vector
= (v(x), v(y)), the 1-norm distance between
‖ ν 0 → - ν 1 → ‖ p = 1 = ν 0 x - ν 1 x + ν 0 y - ν 1 y .
Figure 7 is a flowchart 70 of one example of a method for selecting candidate motion vectors used in a motion vector domain-based temporal error concealment process. Flowchart 70 describes a method for implementing blocks 52, 53 and 54 of Figure 5. Figure 7 is described with reference also to Figure 8.
In a block 71 of Figure 7, the window 83 (in a reference frame 81) and the window 84 (in the current frame 82) are identified. It is understood that the reference frame 81 and the current frame 82 each include macroblocks in addition to the macroblocks included in the windows 83 and 84, respectively.
In a block 73, if there is a properly received motion vector for a macroblock in the window 84, that motion vector is included in the set 85 of candidate motion vectors, and the motion vector for the co-located macroblock in the window 83 is not included in the set 85. For example, there is a properly received motion vector for the macroblock 87 (in the window 83 in the reference frame 81) and a properly received motion vector for the macroblock 89 (in the window 84 in the current frame 82). According to one example, the motion vector associated with the macroblock 89 (current frame 82) is included in the set 85, and the motion vector associated with the macroblock 87 (reference frame 81) is not included in the set 85.
As described above, a statistical measure of the set 85 of candidate motion vectors is determined (refer to the discussion of Figures 5 and 6).
Figure 9 is a flowchart 90 of one example of a method for detecting frame-to-frame motion change. Figure 10 is a flowchart 100 of another embodiment of a method for detecting frame-to-frame motion change. Either or both of the methods of the flowcharts 90 and 100 can be used to determine whether motion vectors from a reference frame should be included in the set of candidate motion vectors, in order to address the points mentioned in the preceding paragraph.
With reference first to Figure 9, the flowchart 90 describes a method for implementing the block 52 of Figure 5. In a block 91, a first range of values for motion vectors associated with a reference frame is determined. In a block 92, a second range of values for motion vectors associated with the current frame is determined. In a block 93, the first and second ranges of values are compared, and the motion vectors associated with the reference frame are included in the set of candidate motion vectors according to the results of the comparison.
Figure 9 is described further with reference also to Figure 2. In the block 91 motion vector statistics are calculated for the properly received motion vectors associated with the reference frame 21.
In the block 92, motion vector statistics are calculated for the properly received motion vectors associated with the current frame 22.
In one example, all of the motion vectors associated with the reference frame 21 and the current frame 22 are included in the calculations of motion vector statistics. In another example, only subsets of the motion vectors are used instead of all of the motion vectors. In the latter example, the subsets may include only the motion vectors associated with macroblocks for which motion vectors for both frames were properly received. That is, for example, a motion vector for a macroblock in the reference frame 21 is only included in a first subset if the motion vector for the co-located macroblock in the current frame 22 was also properly received. Similarly, a motion vector for a macroblock in the current frame 22 is only included in a second subset if the motion vector for the co-located macroblock in the reference frame 21 was also properly received.
In one example, for each frame, the statistics calculated include the mean and standard deviation of the motion vector dimensions (magnitude/length and direction/angle). Let / be the set of indices of the motion vectors
that are included in the calculations of motion vector statistics, and let M be the size of the set /. Then the means and standard deviations (std) for the magnitudes (mag) and angles (ang) are calculated as follows for the reference frame 21 and the current frame 22:
mean mag_frm = 1 M ∑ i ∈ l mag ν → frm i ;
mean ang_frm = 1 M ∑ i ∈ l ang ν → frm i ;
std mag_frm = 1 M ∑ i ∈ l ( mag ⁢ ν → frm i - mean mag_frm 2 ;
std ang_frm = 1 M ∑ i ∈ l ( ang ⁢ ν → frm i - mean mag_frm 2 ;
where the subscript "frm" refers to either the current frame or the reference frame. Once the means and standard deviations are calculated, the ranges (meanmag_frm - stdmag_frm, meanmag_frm + stdmag-frm ) and (meanang_frm - stdang_frm, meanang_frm + stdang_frm ) are formed for each of the current and reference frames.
In the block 93, the ranges of the motion vector magnitudes for the reference frame 21 and for the current frame 22 are compared, and the ranges of the motion vector angles for the reference frame 21 and for the current frame 22 are also compared. If the range of motion vector magnitudes for the reference frame 21 overlaps the range of motion vector magnitudes for the current frame 22, and if the range of motion vector angles for the reference frame 21 overlaps the range of motion vector angles for the current frame 22, then the reference frame 21 and the current frame 22 are judged to have similar motion. Accordingly, motion vectors from the reference frame 21 are eligible for inclusion in the set of candidate motion vectors (e.g., the set 65 of Figure 6).
With reference now to Figure 10, the flowchart 100 describes another embodiment of a method for implementing the block 52 of Figure 5. In a block 101, the dimensions of pairs of motion vectors are compared to determine whether motion vectors in each of the pairs are similar to each other. Each of the pairs of motion vectors includes a first motion vector associated with a first macroblock at a position in a reference frame, and a second motion vector associated with a second macroblock at the position in the current frame.
Figure 10 is described further with reference also to Figure 2. In the block 101, in one embodiment, the dimensions of each pair of co-located macroblocks are compared. The macroblocks 108 and 109 of Figure 2 are an example of a pair of co-located macroblocks.
In one embodiment, to facilitate the comparison, each received motion vector in the reference frame 21 and each received motion vector in the current frame 22 is given a magnitude label and a direction label. In one such embodiment, the magnitude label has a value of either zero (0) or one (1), depending on its relative magnitude. For example, a motion vector having a magnitude of less than or equal to two (2) pixels is assigned a magnitude label of 0, and a motion vector having a magnitude of more than 2 pixels is assigned a magnitude label of 1. In one embodiment, the direction label has a value of 0, 1, 2 or three (3). For example, relative to a vertical line in a frame, a motion vector having an angle greater than or equal to -45 degrees but less than 45 degrees could be assigned a direction label of 0, a motion vector having an angle greater than or equal to 45 degrees but less than 135 degrees could be assigned a direction label of 1, and so on. Other schemes for labeling the magnitude and direction of motion vectors can be used.
In the block 102 of Figure 10, in one embodiment, the number of pairs of co-located macroblocks that contain similar motion vectors is counted. In other words, the number of pairs of similar motion vectors is counted.
In the block 103, in one embodiment, motion vectors from the reference frame 21 are eligible for inclusion in the set of candidate motion vectors (e.g., the set 65 of Figure 6) if the count made in the block 102 exceeds a threshold. In one embodiment, the threshold is equal to one-half of the number of macroblocks in either of the two frames 21 or 22.
In the neighborhood of a macroblock of interest, there may be a motion boundary - objects on one side of a motion boundary may move differently from objects on the other side of the motion boundary. Figure 11 is a flowchart 110 of one embodiment of a method for locating a motion boundary. The flowchart 110 describes one embodiment of a method of implementing the block 53 of Figure 5. Note that, in one embodiment, the block 53 (and hence the method of the flowchart 110) is implemented depending on the outcome of the block 52 of Figure 5.
In a block 111 of Figure 11, a motion boundary is identified in a reference frame. In a block 112, the set of candidate motion vectors includes only those motion vectors that are associated with macroblocks in the reference frame that lie on the same side of the motion boundary as a macroblock in the reference frame that is co-located with a macroblock of interest in the current frame.
Figure 11 is described further with reference also to Figure 12. Figure 12 shows a window 125 in a reference frame 121, and a window 126 in a current frame 122. It is understood that the reference frame 121 and the current frame 122 each include macroblocks in addition to the macroblocks included in the windows 125 and 126, respectively.
In one embodiment, in the block 111, a motion boundary 129 is identified in the reference frame 121. In one embodiment, the motion boundary 129 is identified in the following manner. Each of the motion vectors associated with the macroblocks in the window 125 in the reference frame 121 is assigned a magnitude label and a direction label. The discussion above in conjunction with Figure 10 describes one method for labeling motion vectors.
In one embodiment, in the block 112, only those motion vectors associated with the window 125 that are in the same class as the motion vector associated with the macroblock 124 are included in the set 126 of candidate motion vectors. In other words, in the present embodiment, only the motion vectors in the window 125 in the reference frame 121 that are on the same side of the motion boundary 129 as the macroblock 124 (the macroblock co-located with the macroblock 123 of interest) are included in the set 128 of candidate motion vectors. That is, in the example of Figure 12, only the motion vectors classified as class 0 are included in the set 128. As described above, a statistical measure of the set 128 of candidate motion vectors is then determined (refer to the discussion of Figures 5 and 6).
Figure 13 is a flowchart 130 of one embodiment of a method that uses the trajectory of a moving object to select a candidate motion vector. The flowchart 130 describes one embodiment of a method of implementing the block 53 of Figure 5.
In a block 131 of Figure 13, an object in a first macroblock in a reference frame is identified. In a block 132, a motion vector associated with the object is included in the set of candidate motion vectors if the object sufficiently overlaps a co-located second macroblock in the current frame (that is, the first macroblock and the second macroblock are in the same position within their respective frames).
Figure 13 is described further with reference also to Figure 14. Figure 14 shows a window 147 of a reference frame 141 and a window 148 of a current frame 142. The macroblock 143 is co-located with the macroblock 146. It is understood that the reference frame 141 and the current frame 142 each include macroblocks in addition to the macroblocks included in the windows 147 and 148, respectively.
In the block 132, in one embodiment, a determination is made as to whether the macroblock 145 that contains the object 144 overlaps the macroblock 146 by a sufficient amount. If so, the motion vector associated with the object 144 can be included in the set of candidate motion vectors (e.g., the set 65 of Figure 6). If not, the motion vector associated with the object 144 is not included in the set.
Note that method described in conjunction with Figures 13 and 14 can be similarly applied to any of the macroblocks within the windows 147 and 148. That is, although described for the center macroblock of the windows 147 and 148, the present invention is not so limited.
The embodiments of Figures 9-14 have been described separately in order to more clearly describe certain aspects of the embodiments; however, it is appreciated that the embodiments may be implemented by combining different aspects of these embodiments. In one embodiment, one of the methods described in conjunction with Figures 9 and 10 is combined with one of the methods described in conjunction with Figures 11-14.
The concepts described herein can be used for applications other than error concealment.
A method of estimating a lost or corrupted motion vector of a macroblock of a frame, the method comprising:
determining that a motion vector for a macroblock of interest of a current frame was not properly received at a video decoding device;
estimating the motion vector for the macroblock of interest of the current frame, wherein estimating the motion vector for the macroblock of interest of the current frame includes:
identifying a set of candidate motion vectors from potential candidate motion vectors associated with macroblocks of the current frame and macroblocks of a reference frame, wherein identifying the set of candidate motion vectors comprises the following steps
a) identifying (51) as potential candidate motion vectors, motion vectors associated with macroblocks that neighbor the macroblock of interest of the current frame and motion vectors associated with macroblocks that neighbor a co-located macroblock in the reference frame;
b) assigning magnitude labels and direction labels to the potential candidate motion vectors indicative of the relative magnitude and angle of each of the potential candidate motion vectors;
c) comparing the magnitude a direction labels of motion vectors of co-located pairs of macroblocks in the reference frame and in the current frame, respectively, to determine (52) whether the motion vectors of the reference-frame are eligible for inclusion in the set of candidate motion vectors
d) selecting (53) one set of candidate motion vectors from the potential candidate motion vectors , if motion vectors from the reference frame are eligible for inclusion, otherwise selecting (54) the candidate motion vectors from the current frame; and
determining a statistical measure of the set of candidate motion vectors; and
wherein said estimating the motion vector for the macroblock of interest of the current frame is based on the statistical measure.
decoding the macroblock of interest of the current frame based on the estimated motion vector.
The method of Claim 1 wherein the determining a statistical measure comprises determining the median of the set of candidate motion vectors.
The method of claim 1, wherein said comparing comprises
counting the pairs that are similar; and
determining whether the motion vectors of the reference frame are eligible for inclusion in the set of candidate motion vectors based on whether the count exceeds a specified threshold.
The method of claim 1, wherein selecting the set of candidate motion vectors from the potential candidate motion vectors comprises:
identify a motion boundary in the reference frame:
comparing the motion vector magnitude labels and motion vector direction labels associated with each of the potential candidate motion vectors of the reference frame with magnitude labels and motion vector direction labels associated with the motion vector associated with the current macroblocky, respectively;
determining whether the candidate motion vectors lie on the same side of the motion boundary as the current macroblock based on the comparing; and
determining whether the motion vectors of the reference frame are eligible for inclusion in the set of candidate motion vectors based on whether the motion vectors lie on the same side of the motion boundary as the current macroblock.
The method of claim 1, wherein selecting the set of candidate motion vectors from the potential candidate motion vectors further comprises:
identify an object that is contained in one of the macroblocks of the reference frame, wherein the one of the macroblocks is co-located with the macroblock of the current frame; and
determining whether the motion vectors of the reference frame are eligible for inclusion in the set of candidate motion vectors based on whether the object overlaps the macroblock by a specified amount.
A system that is specifically adapted to execute any of the methods recited in claims 1-6.
A computer-usable medium having computer-readable program code embodied therein for causing a decoding device to perform any of the methods recited in claims 1-6.
a memory unit coupled to the microprocessor, the memory unit containing instructions that when executed by the microprocessor implement any of the methods of claims 1-6.
EP05855322A 2004-12-22 2005-12-22 Temporal estimation of a motion vector for video communications Active EP1829383B1 (en)
US11/022,362 US20060133495A1 (en) 2004-12-22 2004-12-22 Temporal error concealment for video communications
PCT/US2005/046739 WO2006069297A1 (en) 2004-12-22 2005-12-22 Temporal estimation of a motion vector for video communications
EP1829383A1 EP1829383A1 (en) 2007-09-05
EP1829383B1 true EP1829383B1 (en) 2011-01-05
ID=36177977
EP05855322A Active EP1829383B1 (en) 2004-12-22 2005-12-22 Temporal estimation of a motion vector for video communications
US (2) US20060133495A1 (en)
EP (1) EP1829383B1 (en)
JP (2) JP5021494B2 (en)
KR (1) KR100964407B1 (en)
CN (1) CN101116345B (en)
AT (1) AT494735T (en)
DE (1) DE602005025808D1 (en)
TW (1) TW200637375A (en)
WO (1) WO2006069297A1 (en)
JP5038134B2 (en) 2004-07-01 2012-10-03 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ Method and apparatus for using frame rate up-conversion technique in scalable video coding
JP2008507915A (en) 2004-07-20 2008-03-13 クゥアルコム・インコーポレイテッドＱｕａｌｃｏｍｍ Ｉｎｃｏｒｐｏｒａｔｅｄ Method and apparatus for encoder assisted frame rate up conversion for video compression
KR20100118978A (en) * 2008-01-11 2010-11-08 조란(프랑스) Sparse geometry for super resolution video processing
CN102131095B (en) * 2010-01-18 2013-03-20 联发科技股份有限公司 Motion prediction method and video encoding method
CN103765896B (en) * 2011-06-27 2017-05-03 三星电子株式会社 A method and apparatus for encoding motion information and a method and apparatus for decoding motion information
JP5649523B2 (en) * 2011-06-27 2015-01-07 日本電信電話株式会社 Video encoding method, apparatus, video decoding method, apparatus, and program thereof
EP1301044B1 (en) 2001-10-05 2006-05-24 Mitsubishi Electric Information Technology Centre Europe B.V. Method and apparatus for compensating for motion vector errors in image data
AU2003279015A1 (en) * 2002-09-27 2004-04-19 Videosoft, Inc. Real-time video coding/decoding
2004-12-22 US US11/022,362 patent/US20060133495A1/en not_active Abandoned
2005-12-22 CN CN2005800480510A patent/CN101116345B/en active IP Right Grant
2005-12-22 JP JP2007548507A patent/JP5021494B2/en active Active
2005-12-22 TW TW094145929A patent/TW200637375A/en unknown
2005-12-22 AT AT05855322T patent/AT494735T/en not_active IP Right Cessation
2005-12-22 DE DE602005025808T patent/DE602005025808D1/en active Active
2005-12-22 EP EP05855322A patent/EP1829383B1/en active Active
2005-12-22 WO PCT/US2005/046739 patent/WO2006069297A1/en active Application Filing
2005-12-22 KR KR1020077015762A patent/KR100964407B1/en active IP Right Grant
2010-01-27 US US12/694,522 patent/US8817879B2/en active Active
2011-07-19 JP JP2011158191A patent/JP5420600B2/en active Active
DE602005025808D1 (en) 2011-02-17
WO2006069297A1 (en) 2006-06-29
TW200637375A (en) 2006-10-16
KR100964407B1 (en) 2010-06-15
KR20070090242A (en) 2007-09-05
CN101116345B (en) 2010-12-22
AT494735T (en) 2011-01-15
JP5021494B2 (en) 2012-09-05
JP2011254508A (en) 2011-12-15
EP1829383A1 (en) 2007-09-05
US20060133495A1 (en) 2006-06-22
US20100118970A1 (en) 2010-05-13
JP2008526119A (en) 2008-07-17
CN101116345A (en) 2008-01-30
US8817879B2 (en) 2014-08-26
JP5420600B2 (en) 2014-02-19
KR100244291B1 (en) 2000-02-01 Method for motion vector coding of moving picture
KR101155767B1 (en) 2012-06-12 Selecting encoding types and predictive modes for encoding video data
JP4777889B2 (en) 2011-09-21 Mode decision for intermediate prediction in video coding
RU2518635C2 (en) 2014-06-10 Method and apparatus for encoding and decoding coding unit of picture boundary
US8693540B2 (en) 2014-04-08 Method and apparatus of temporal error concealment for P-frame
Inventor name: DANE, GOKCE
Inventor name: YE, YAN
Inventor name: NI, KARL
Inventor name: TSAI, MING-CHANG
Inventor name: FENG, NIEN-CHUNG
Inventor name: LEE, YEN-CHI
Ref document number: 602005025808
Ref document number: E010772