Source: http://www.google.ca/patents/US8374245
Timestamp: 2017-12-16 11:15:26
Document Index: 140608947

Matched Legal Cases: ['Application No. 60', 'Application No. 2003116281', 'Application No. 03', 'Application No. 03', 'Application No. 2003204477', 'Application No. 2003204477', 'Application No. 2003204477', 'Application No. 2003', 'Application No. 10', 'Application No. 10', 'Application No. 2003', 'Application No. 2003', 'Application No. 03141275', 'Application No. 03141275', 'Application No. 2003116281', 'Application No. 2', 'Application No. 2', 'Application No. 2003', 'Application No. 03141275']

Patent US8374245 - Spatiotemporal prediction for bidirectionally predictive(B) pictures and ... - Google Patents
Several improvements for use with Bidirectionally Predictive (B) pictures within a video sequence are provided. In certain improvements Direct Mode encoding and/or Motion Vector Prediction are enhanced using spatial prediction techniques. In other improvements Motion Vector prediction includes temporal...http://www.google.ca/patents/US8374245?utm_source=gb-gplus-sharePatent US8374245 - Spatiotemporal prediction for bidirectionally predictive(B) pictures and motion vector prediction for multi-picture reference motion compensation
Publication number US8374245 B2
Application number US 11/525,059
Also published as CA2430460A1, CA2430460C, CN1471320A, CN100455018C, EP1369820A2, EP1369820A3, US8873630, US9185427, US9571854, US20040001546, US20070014358, US20130148737, US20150016527, US20160134890, US20170118488
Publication number 11525059, 525059, US 8374245 B2, US 8374245B2, US-B2-8374245, US8374245 B2, US8374245B2
Inventors Alexandros Tourapis, Shipeng Li, Feng Wu
Patent Citations (252), Non-Patent Citations (105), Referenced by (17), Classifications (25), Legal Events (2)
Spatiotemporal prediction for bidirectionally predictive(B) pictures and motion vector prediction for multi-picture reference motion compensation
US 8374245 B2
Several improvements for use with Bidirectionally Predictive (B) pictures within a video sequence are provided. In certain improvements Direct Mode encoding and/or Motion Vector Prediction are enhanced using spatial prediction techniques. In other improvements Motion Vector prediction includes temporal distance and subblock information, for example, for more accurate prediction. Such improvements and other presented herein significantly improve the performance of any applicable video coding system/logic.
1. A method for use in encoding video data in a video encoder, the method comprising:
making a spatial/temporal motion vector prediction decision for at least one direct mode macroblock in a B-picture, wherein the spatial/temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock;
selectively encoding the at least one direct mode macroblock, wherein the encoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
selecting a reference picture for the given direct mode macroblock from among reference pictures used for surrounding portions of the B-picture, wherein the selecting the reference picture for the given direct mode macroblock comprises selecting a minimum reference picture index for the given direct mode macroblock from among the reference picture indices used for the surrounding portions of the B-picture;
performing spatial motion vector prediction for the given direct mode macroblock; and
performing motion compensation for the given direct mode macroblock; and
signaling spatial/temporal motion vector prediction decision information for the at least one direct mode macroblock in a header that includes header information for plural macroblocks in the B-picture, wherein the signaling of the spatial/temporal motion vector prediction decision information in the header communicates to a video decoder the spatial/temporal motion vector prediction decision for the at least one direct mode macroblock.
2. The method of claim 1 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
3. The method of claim 1 wherein the at least one direct mode macroblock comprises plural direct mode macroblocks.
4. The method of claim 3 wherein the plural direct mode macroblocks are 16×16 macroblocks.
5. The method of claim 4 wherein each of the 16×16 macroblocks includes four 8×8 sub-blocks.
6. The method of claim 1 wherein the surrounding portions are surrounding macroblocks.
7. The method of claim 1 wherein the spatial motion vector prediction comprises median motion vector prediction.
analyzing the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
analyzing motion flow within the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
analyzing whether collocated blocks of a subsequent picture have zero motion, the subsequent picture following the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
analyzing temporal distance between the B-picture and pictures around the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the analysis.
identifying a scene change around the B-picture, wherein the spatial/temporal motion vector prediction decision is based at least in part on the identification of the scene change.
13. A method for use in decoding video data in a video decoder, the method comprising:
receiving signaled spatial/temporal motion vector prediction decision information for at least one direct mode macroblock in a header that includes header information for plural macroblocks in a B-picture;
from the signaled spatial/temporal motion vector prediction decision information in the header, determining a spatial/temporal motion vector prediction decision for the at least one direct mode macroblock, wherein the spatial or temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock; and
decoding the at least one direct mode macroblock, wherein the decoding includes for a given direct mode macroblock of the at least one direct mode macroblock:
performing motion compensation for the given direct mode macroblock.
14. The method of claim 13 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
15. The method of claim 13 wherein the at least one direct mode macroblock comprises plural direct mode macroblocks.
16. The method of claim 15 wherein the plural direct mode macroblocks are 16×16 macroblocks.
17. The method of claim 16 wherein each of the 16×16 macroblocks includes four 8×8 sub-blocks.
18. The method of claim 13 wherein the surrounding portions are surrounding macroblocks.
displaying visual results of the decoding of the video data; and
reproducing audio data associated with the video data.
20. A video decoder implemented with a computing device, wherein the video decoder is adapted to perform a method comprising:
21. The video decoder of claim 20 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
22. The video decoder of claim 20 wherein the at least one direct mode macroblock comprises plural direct mode macroblocks.
23. The video decoder of claim 20 wherein the surrounding portions are surrounding macroblocks.
24. The video decoder of claim 20 wherein the computing device further includes an audio reproduction module for reproducing audio data.
25. The video decoder of claim 20 wherein the computing device is a hand-held device that includes a display, a network interface, one or more processors and memory.
26. The video decoder of claim 20 wherein the computing device is a portable communication device that includes a display, a network interface, one or more processors and memory.
27. The video decoder of claim 20 wherein the computing device is a set-top box that includes a network interface, one or more processors and memory.
28. One or more memory devices having stored thereon computer-executable instructions for causing a computing device programmed thereby to perform a method of decoding video data, the method comprising:
29. The one or more memory devices of claim 28 wherein the surrounding portions are surrounding macroblocks.
30. The one or more memory devices of claim 28 wherein the surrounding portions are surrounding macroblocks.
31. A portable communication device that includes a display, a network interface, one or more processors, memory, an audio reproduction module for reproducing audio, and a video decoder, wherein the video decoder is adapted to perform a method comprising:
receiving signaled spatial/temporal motion vector prediction decision information for at least one direct mode macroblock in a header that includes header information for plural macroblocks in a slice of a B-picture;
32. A video encoder implemented with a computing device, wherein the video encoder is adapted to perform a method comprising:
analyzing one or more pictures of a video sequence;
based at least in part on results of the analyzing, making a spatial/temporal motion vector prediction decision for at least one direct mode macroblock in a B-picture among the one or more pictures of the video sequence, wherein the spatial/temporal motion vector prediction decision indicates use of spatial motion vector prediction for the at least one direct mode macroblock;
33. The video encoder of claim 32 wherein the plural macroblocks in the B-picture are in a slice of the B-picture.
34. The video encoder of claim 32 wherein the surrounding portions are surrounding macroblocks.
35. The video encoder of claim 32 wherein the analyzing comprises analyzing the B-picture.
36. The video encoder of claim 32 wherein the analyzing comprises analyzing motion flow within the B-picture.
37. The video encoder of claim 32 wherein the analyzing comprises analyzing whether collocated blocks of a subsequent picture have zero motion, the subsequent picture following the B-picture.
38. The video encoder of claim 32 wherein the analyzing comprises analyzing temporal distance between the B-picture and pictures around the B-picture.
39. The video encoder of claim 32 wherein the analyzing comprises identifying a scene change around the B-picture.
This application is a continuation of U.S. patent application Ser. No. 10/444,511, filed May 23, 2003, the disclosure of which is hereby incorporated by reference, which claims the benefit of, and hereby incorporates by reference the entire disclosure of, U.S. Provisional Application No. 60/385,965, filed Jun. 3, 2002, and titled “Spatialtemporal Prediction for Bidirectionally Predictive (B) Frames and Motion Vector Prediction for Multi-Frame Reference Motion Compensation.”
The motivation for increased coding efficiency in video coding has led to the adoption in the Joint Video Team (JVT) (a standards body) of more refined and complicated models and modes describing motion information for a given macroblock. These models and modes tend to make better advantage of the temporal redundancies that may exist within a video sequence. See, for example, IUU-T, Video Coding Expert Group (VCEG), “JVT Coding—(ITU-T H.26L & ISO/IEC JTC1 Standard)—Working Draft Number 2 (WD-2)”, ITU-T JVT-B118, March 2002; and/or Heiko Schwarz and Thomas Wiegand, “Tree-structured macroblock partition”, Doc. VCEG-N17, Dec. 2001.
There is continuing need for further improved methods and apparatuses that can support the latest models and modes and also possibly introduce new models and modes to take advantage of improved coding techniques.
The above state needs and other are addressed, for example, by a method for use in encoding video data within a sequence of video frames. The method includes identifying at least a portion of at least one video frame to be a Bidirectionally Predictive (B) picture, and selectively encoding the B picture using at least spatial prediction to encode at least one motion parameter associated with the B picture. In certain exemplary implementations the B picture may include a block, a macroblock, a subblock, a slice, or other like portion of the video frame. For example, when a macroblock portion is used, the method produces a Direct Macroblock.
In certain further exemplary implementations, the method further includes employing linear or non-linear motion vector prediction for the B picture based on at least one reference picture that is at least another portion of the video frame. By way of example, in certain implementations, the method employs median motion vector prediction to produce at least one motion vector.
In still other exemplary implementations, in addition to spatial prediction, the method may also process at least one other portion of at least one other video frame to further selectively encode the B picture using temporal prediction to encode at least one temporal-based motion parameter associated with the B picture. In some instances the temporal prediction includes bidirectional temporal prediction, for example based on at least a portion of a Predictive (P) frame.
In certain other implementations, the method also selectively determines applicable scaling for a temporal-based motion parameter based at least in part on a temporal distance between the predictor video frame and the frame that includes the B picture. In certain implementations temporal distance information is encoded, for example, within a header or other like data arrangement associated with the encoded B picture.
FIG. 3 is an illustrative diagram depicting spatial predication associated with portions of a picture, in accordance with certain exemplary implementations of the present invention.
FIG. 4 is an illustrative diagram depicting Direct Prediction in B picture coding, in accordance with certain exemplary implementations of the present invention.
FIG. 5 is an illustrative diagram depicting what happens when a scene change happens or even when the collocated block is intra-coded, in accordance with certain exemplary implementations of the present invention.
FIG. 6 is an illustrative diagram depicting handling of collocated intra within existing codecs wherein motion is assumed to be zero, in accordance with certain exemplary implementations of the present invention.
FIG. 7 is an illustrative diagram depicting how Direct Mode is handled when the reference picture of the collocated block in the subsequent P picture is other than zero, in accordance with certain exemplary implementations of the present invention.
FIG. 8 is an illustrative diagram depicting an exemplary scheme wherein MVFW and MVBW are derived from spatial prediction, in accordance with certain exemplary implementations of the present invention.
FIG. 9 is an illustrative diagram depicting how spatial prediction solves the problem of scene changes and the like, in accordance with certain exemplary implementations of the present invention.
FIG. 10 is an illustrative diagram depicting joint spatio-temporal prediction for Direct Mode in B picture coding, in accordance with certain exemplary implementations of the present invention.
FIG. 11 is an illustrative diagram depicting Motion Vector Prediction of a current block considering reference picture information of predictor macroblocks, in accordance with certain exemplary implementations of the present invention.
FIG. 12 is an illustrative diagram depicting how to use more candidates for Direct Mode prediction especially if bidirectional prediction is used within the B picture, in accordance with certain exemplary implementations of the present invention.
FIG. 13 is an illustrative diagram depicting how B pictures may be restricted in using future and past reference pictures, in accordance with certain exemplary implementations of the present invention.
FIG. 14 is an illustrative diagram depicting projection of collocated Motion Vectors to a current reference for temporal direct prediction, in accordance with certain exemplary implementations of the present invention.
FIGS. 15 a-c are illustrative diagrams depicting Motion Vector Predictors for one MV in different configurations, in accordance with certain exemplary implementations of the present invention.
FIGS. 16 a-c are illustrative diagrams depicting Motion Vector Predictors for one MV with 8×8 partitions in different configurations, in accordance with certain exemplary implementations of the present invention.
FIGS. 17 a-c are illustrative diagrams depicting Motion Vector Predictors for one MV with additional predictors for 8×8 partitioning, in accordance with certain exemplary implementations of the present invention.
Several improvements for use with Bidirectionally Predictive (B) pictures within a video sequence are described below and illustrated in the accompanying drawings. In certain improvements Direct Mode encoding and/or Motion Vector Prediction are enhanced using spatial prediction techniques. In other improvements Motion Vector prediction includes temporal distance and subblock information, for example, for more accurate prediction. Such improvements and other presented herein significantly improve the performance of any applicable video coding system/logic.
Bus 136 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnects (PCI) bus also known as Mezzanine bus.
Encoding Bidirectionally Predictive (B) Pictures And Motion Vector Prediction
This section describes several exemplary improvements that can be implemented to encode Bidirectionally Predictive (B) pictures and Motion Vector prediction within a video coding system or the like. The exemplary methods and apparatuses can be applied to predict motion vectors and enhancements in the design of a B picture Direct Mode. Such methods and apparatuses are particularly suitable for multiple picture reference codecs, such as, for example, JVT, and can achieve considerable coding gains especially for panning sequences or scene changes.
Bidirectionally Predictive (B) pictures are an important part of most video coding standards and systems since they tend to increase the coding efficiency of such systems, for example, when compared to only using Predictive (P) pictures. This improvement in coding efficiency is mainly achieved by the consideration of bidirectional motion compensation, which can effectively improve motion compensated prediction and thus allow the encoding of significantly reduced residue information. Furthermore, the introduction of the Direct Prediction mode for a Macroblock/block within such pictures can further increase efficiency considerably (e.g., more than 10-20%) since no motion information is encoded. Such may be accomplished, for example, by allowing the prediction of both forward and backward motion information to be derived directly from the motion vectors used in the corresponding macroblock of a subsequent reference picture.
By way of example, FIG. 4 illustrates Direct Prediction in B picture at time t+1 coding based on P frames at times t and t+2, and the applicable motion vectors (MVs). Here, an assumption is made that an object in the picture is moving with constant speed. This makes it possible to predict a current position inside a B picture without having to transmit any motion vectors. The motion vectors ({right arrow over (MV)}fw,{right arrow over (MV)}tw) of the Direct Mode versus the motion vector {right arrow over (MW)} of the collocated MB in the first subsequent P reference picture are basically calculated by:
MV → fw = TR B · MV → TR D and MV → bw = ( TR B - TR D ) · MV → TR D ,
where TRB is the temporal distance between the current B picture and the reference picture pointed by the forward MV of the collocated MB, and TRD is the temporal distance between the future reference picture and the reference picture pointed by the forward MV of the collocated MB.
Unfortunately there are several cases where the existing Direct Mode does not provide an adequate solution, thus not efficiently exploiting the properties of this mode. In particular, existing designs of this mode usually force the motion parameters of the Direct Macroblock, in the case of the collocated Macroblock in the subsequent P picture being Intra coded, to be zero. For example, see FIG. 6, which illustrates handling of collocated intra within existing codecs wherein motion is assumed to be zero. This essentially means that, for this case, the B picture Macroblock will be coded as the average of the two collocated Macroblocks in the first subsequent and past P references. This immediately raises the following concern; if a Macroblock is Intra-coded, then how does one know how much relationship it has with the collocated Macroblock of its reference picture. In some situations, there may be little if any actual relationship. Hence, it is possible that the coding efficiency of the Direct Mode may be reduced. An extreme case can be seen in the case of a scene change as illustrated in FIG. 5. FIG. 5 illustrates what happens when a scene change occurs in the video sequence and/or what happens when the collocated block is intra. Here, in this example, obviously no relationship exists between the two reference pictures given the scene change. In such a case bidirectional prediction would provide little if any benefit. As such, the Direct Mode could be completely wasted. Unfortunately, conventional implementations of the Direct Mode restrict it to always perform a bidirectional prediction of a Macroblock.
FIG. 7 is an illustrative diagram depicting how Direct Mode is handled when the reference picture of the collocated block in the subsequent P picture is other than zero, in accordance with certain implementations of the present invention.
An additional issue with the Direct Mode Macroblocks exists when multi-picture reference motion compensation is used. Until recently, for example, the JVT standard provided the timing distance information (TRB and TRD), thus allowing for the proper scaling of the parameters. Recently, this was changed in the new revision of the codec (see, e.g., Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, “Joint Committee Draft (CD) of Joint Video Specification (ITU-T Rec. H.264|ISO/IEC 14496-10 AVC)”, ITU-T JVT-C167, May, 2002, which is incorporated herein by reference). In the new revision, the motion vector parameters of the subsequent P picture are to be scaled equally for the Direct Mode prediction, without taking in account the reference picture information. This could lead to significant performance degradation of the Direct Mode, since the constant motion assumption is no longer followed.
Nevertheless, even if the temporal distance parameters were available, it is not always certain that the usage of the Direct Mode as defined previously is the most appropriate solution. In particular for the B pictures which are closer to a first forward reference picture, the correlation might be much stronger with that picture, than the subsequent reference picture. An extreme example which could contain such cases could be a sequence where scene A changes to scene B, and then moves back to scene A (e.g., as may happen in a news bulletin, etc.). All the above could deter the performance of B picture encoding considerably since Direct Mode will not be effectively exploited within the encoding process.
With these and other concerns in mind, unlike the previous definitions of the Direct Mode where only temporal prediction was used, in accordance with certain aspects of the present invention, a new Direct Macroblock type is introduced wherein both temporal prediction and/or spatial prediction is considered. The type(s) of prediction used can depend on the type of reference picture information of the first subsequent P reference picture, for example.
In accordance with certain other aspects of the present invention, one may also further considerably improve motion vector prediction for both P and B pictures when multiple picture references are used, by taking in consideration temporal distances, if such are available.
These enhancements are implemented in certain exemplary methods and apparatuses as described below. The methods and apparatuses can achieve significant bitrate reductions while achieving similar or better quality.
Direct Mode Enhancements:
In most conventional video coding systems, Direct Mode is designed as a bidirectional prediction scheme where motion parameters are always predicted in a temporal way from the motion parameters in the subsequent P images. In this section, an enhanced Direct Mode technique is provided in which spatial information may also/alternatively be considered for such predictions.
One or more of the following exemplary techniques may be implemented as needed, for example, depending on the complexity and/or specifications of the system.
One technique is to implement spatial prediction of the motion vector parameters of the Direct Mode without considering temporal prediction. Spatial prediction can be accomplished, for example, using existing Motion Vector prediction techniques used for motion vector encoding (such as, e.g., median prediction). If multiple picture references are used, then the reference picture of the adjacent blocks may also be considered (even though there is no such restriction and the same reference, e.g. 0, could always be used).
Motion parameters and reference pictures could be predicted as follows and with reference to FIG. 3, which illustrates spatial predication associated with portions A-E (e.g., macroblocks, slices, etc.) assumed to be available and part of a picture. Here, E is predicted in general from A, B, C as Median (A,B,C). If C is actually outside of the picture then D is used instead. If B, C, and D are outside of picture, then only A is used, where as if A does not exist, such is replaced with (0,0). Those skilled in the art will recognize that spatial prediction may be done at a subblock level as well.
In general spatial prediction can be seen as a linear or nonlinear function of all available motion information calculated within a picture or a group of macroblocks/blocks within the same picture.
There are various methods available that may be arranged to predict the reference picture for Direct Mode. For example, one method may be to select a minimum reference picture among the predictions. In another method, a median reference picture may be selected. In certain methods, a selection may be made between a minimum reference picture and median reference picture, e.g., if the minimum is zero. In still other implementations, a higher priority could also be given to either vertical or horizontal predictors (A and B) due to their possibly stronger correlation with E.
If one of the predictions does not exist (e.g., all surrounding macroblocks are predicted with the same direction FW or BW only or are intra), then the existing one is only used (single direction prediction) or such could be predicted from the one available. For example if forward prediction is available then:
MV → bw = ( TR B - TR D ) · MV → fw TR B
Temporal prediction is used for Macroblocks if the subsequent P reference is non intra as in existing codecs. Attention is now drawn to FIG. 8, in which MVFW and MVBW are derived from spatial prediction (Median MV of surrounding Macroblocks). If either one is not available (i.e., no predictors) then one-direction is used. If a subsequent P reference is intra, then spatial prediction can be used instead as described above. Assuming that no restrictions exist, if one of the predictions is not available then Direct Mode becomes a single direction prediction mode.
This could considerably benefit video coding when the scene changes, for example, as illustrated in FIG. 9, and/or even when fading exists within a video sequence. As illustrated in FIG. 9, spatial prediction may be used to solve the problem of a scene change.
If temporal distance information is not available within a codec, temporal prediction will not be as efficient in the direct mode for blocks when the collocated P reference block has a non-zero reference picture. In such a case, spatial prediction may also be used as above. As an alternative, one may estimate scaling parameters if one of the surrounding macroblocks also uses the same reference picture as the collocated P reference block. Furthermore, special handling may be provided for the case of zero motion (or close to zero motion) with a non-zero reference. Here, regardless of temporal distance forward and backward motion vectors could always be taken as zero. The best solution, however, may be to always examine the reference picture information of surrounding macroblocks and based thereon decide on how the direct mode should be handled in such a case.
More particularly, for example, given a non-zero reference, the following sub cases may be considered:
Case A: Temporal prediction is used if the motion vectors of the collocated P block are zero.
Case B: If all surrounding macroblocks use different reference pictures than the collocated P reference, then spatial prediction appears to be a better choice and temporal prediction is not used.
Case C: If motion flow inside the B picture appears to be quite different than the one in the P reference picture, then spatial prediction is used instead.
Case D: Spatial or temporal prediction of Direct Mode macroblocks could be signaled inside the image header. A pre-analysis of the image could be performed to decide which should be used.
Case E: Correction of the temporally predicted parameters based on spatial information (or vice versa). Thus, for example, if both appear to have the same or approximately the same phase information then the spatial information could be a very good candidate for the direct mode prediction. A correction could also be done on the phase, thus correcting the sub pixel accuracy of the prediction.
FIG. 10 illustrates a joint spatio-temporal prediction for Direct Mode in B picture coding. Here, in this example, Direct Mode can be a 1- to 4-direction mode depending on information available. Instead of using Bi-directional prediction for Direct Mode macroblocks, a multi-hypothesis extension of such mode can be done and multiple predictions used instead.
Combined with the discussion above, Direct Mode macroblocks can be predicted using from one up to four possible motion vectors depending on the information available. Such can be decided, for example, based on the mode of the collocated P reference image macroblock and on the surrounding macroblocks in the current B picture. In such a case, if the spatial prediction is too different than the temporal one, one of them could be selected as the only prediction in favor of the other. Since spatial prediction as described previously, might favor a different reference picture than the temporal one, the same macroblock might be predicted from more than 2 reference pictures.
The JVT standard does not restrict the first future reference to be a P picture. Hence, in such a standard, a picture can be a B as illustrated in FIG. 12, or even a Multi-Hypothesis (MH) picture. This implies that more motion vectors are assigned per macroblock. This means that one may also use this property to increase the efficiency of the Direct Mode by more effectively exploiting the additional motion information.
In FIG. 12, the first subsequent reference picture is a B picture (pictures B8 and B9). This enables one to use more candidates for Direct Mode prediction especially if bidirectional prediction is used within the B picture.
In particular one may perform the following:
a.) If the collocated reference block in the first future reference is using bidirectional prediction, the corresponding motion vectors (forward or backward) are used for calculating the motion vectors of the current block. Since the backward motion vector of the reference corresponds to a future reference picture, special care should be taken in the estimate of the current motion parameters. Attention is drawn, for example to FIG. 12 in which the first subsequent reference picture is a B picture (pictures B8 and B9). This enables one to use more candidates for Direct Mode prediction especially if bidirectional prediction is used within the B picture. Thus, as illustrated, the backward motion vector of B8 {right arrow over (MV)}B8bw can be calculated as 2×{right arrow over (MV)}B7bw due to the temporal distance between B8, B7 and P6. Similarly for B9 the backward motion vector can be taken as {right arrow over (MV)}B7bw, if though these refer to the B7. One may also restrict these to refer to the first subsequent P picture, in which case these motion vectors can be scaled accordingly. A similar conclusion can be deduced about the forward motion vectors. Multiple picture reference or intra macroblocks can be handled similar to the previous discussion.
b.) If bidirectional prediction for the collocated block is used, then, in this example, one may estimate four possible predictions for one macroblock for the direct mode case by projecting and inverting the backward and forward motion vectors of the reference.
c.) Selective projection and inversion may be used depending on temporal distance. According to this solution, one selects the motion vectors from the reference picture which are more reliable for the prediction. For example, considering the illustration in FIG. 12, one will note that B8 is much closer to P2 than P6. This implies that the backward motion vector of B7 may not be a very reliable prediction. In this case, direct mode motion vectors can therefore be calculated only from the forward prediction of B7. For B9, however, both motion vectors seem to be adequate enough for the prediction and therefore may be used. Such decisions/information may also be decided/supported within the header of the image. Other conditions and rules may also be implemented. For example, additional spatial confidence of a prediction and/or a motion vector phase may be considered. Note, in particular, that if the forward and backward motion vectors have no relationship, then the backward motion vector might be too unreliable to use.
Single Picture Reference for B Pictures:
A special case exists with the usage of only one picture reference for B pictures (although, typically a forward and a backward reference are necessary) regardless of how many reference pictures are used in P pictures. From observations of encoding sequences in the current JVT codec, for example, it was noted that, if one compares the single-picture reference versus the multi-picture reference case using B pictures, even though encoding performance of P pictures for the multi-picture case is almost always superior to that of the single-picture, the some is not always true for B pictures.
One reason for this observation is the overhead of the reference picture used for each macroblock. Considering that B pictures rely more on motion information than P pictures, the reference picture information overhead reduces the number of bits that are transmitted for the residue information at a given bitrate, which thereby reduces efficiency. A rather easy and efficient solution could be the selection of only one picture reference for either backward or forward motion compensation, thus not needing to transmit any reference picture information.
This is considered with reference to FIGS. 13 and 14. As illustrated in FIG. 13, B pictures can be restricted in using only one future and past reference pictures. Thus, for direct mode motion vector calculation, projection of the motion vectors is necessary. A projection of the collocated MVs to the current reference for temporal direct prediction is illustrated in FIG. 14 (note that it is possible that TDD,0>TDD,1). Thus, in this example, Direct Mode motion parameters are calculated by projecting motion vectors that refer to other reference pictures to the two reference pictures, or by using spatial prediction as in FIG. 13. Note that such options not only allow for possible reduced encoding complexity of B pictures, but also tend to reduce memory requirements since fewer B pictures (e.g., maximum two) are needed to be stored if B pictures are allowed to reference B pictures.
In certain cases a reference picture of the first future reference picture may no longer be available in the reference buffer. This could immediately generate a problem for the estimate of Direct Mode macroblocks and special handling of such cases is required. Obviously there is no such problem if a single picture reference is used. However, if multiple picture references are desired, then possible solutions include projecting the motion vector(s) to either the first forward reference picture, and/or to the reference picture that was closest to the non available picture. Either solution could be viable, whereas again spatial prediction could be an alternative solution.
Refinements of the Motion Vector Prediction for Single- and Multi-Picture Reference Motion Compensation
Motion vector prediction for multi-picture reference motion compensation can significantly affect the performance of both B and P picture coding. Existing standards, such as, for example, JVT, do not always consider the reference pictures of the macroblocks used in the prediction. The only consideration such standards do make is when only one of the prediction macroblocks uses the same reference. In such a case, only that predictor is used for the motion prediction. There is no consideration of the reference picture if only one or all predictors are using a different reference.
In such a case, for example, and in accordance with certain further aspects of the present invention, one can scale the predictors according to their temporal distance versus the current reference. Attention is drawn to FIG. 11, which illustrates Motion Vector prediction of a current block (C) considering the reference picture information of predictor macroblocks (Pr) and performance of proper adjustments (e.g., scaling of the predictors)
If predictors A, B, and C use reference pictures with temporal distance TRA, TRB, and TRC respectively, and the current reference picture has a temporal distance equal to TR, then the median predictor is calculated as follows:
MV → pred - TR × Median ( MV → A TR A , MV → B TR B , MV → C TR C )
If integer computation is to be used, it may be easier to place the multiplication inside the median, thus increasing accuracy. The division could also be replaced with shifting, but that reduces the performance, whereas it might be necessary to handle signed shifting as well (−1>>N=−1). It is thus very important in such cases to have the temporal distance information available for performing the appropriate scaling. Such could also be available within the header, if not predictable otherwise.
Motion Vector prediction as discussed previously is basically median biased, meaning that the median value among a set of predictors is selected for the prediction. If one only uses one type of macroblock (e.g., 16×16) with one Motion Vector (MV), then these predictors can be defined, for example, as illustrated in FIG. 15. Here, MV predictors are shown for one MV. In FIG. 15 a, the MB is not in the first row or the last column. In FIG. 15 b, the MB is in the last column. In FIG. 15 c, the MB is in the first row.
The JVT standard improves on this further by also considering the case that only one of the three predictors exists (i.e. Macroblocks are intra or are using a different reference picture in the case of multi-picture prediction). In such a case, only the existing or same reference predictor is used for the prediction and all others are not examined.
Intra coding does not always imply that a new object has appeared or that scene changes. It might instead, for example, be the case that motion estimation and compensation is inadequate to represent the current object (e.g., search range, motion estimation algorithm used, quantization of residue, etc) and that better results could be achieved through Intra Coding instead. The available motion predictors could still be adequate enough to provide a good motion vector predictor solution.
What is intriguing is the consideration of subblocks within a Macroblock, with each one being assigned different motion information. MPEG4 and H.263 standards, for example, can have up to four such subblocks (e.g., with size 8×8), where as the JVT standard allows up to sixteen subblocks while also being able to handle variable block sizes (e.g., 4×4, 4×8, 8×4, 8×8, 8×16, 16×8, and 16×16). In addition JVT also allows for 8×8 Intra subblocks, thus complicating things even 23 further.
Considering the common cases of JVT and MPEG-4/H.263 (8×8 and 16×16), the predictor set for a 16×16 macroblock is illustrated in FIGS. 16 a-c having a similar arrangement to FIGS. 15 a-c, respectively. Here, Motion Vector predictors are shown for one MV with 8×8 partitions. Even though the described predictors could give reasonable results in some cases, it appears that they may not adequately cover all possible predictions.
Attention is drawn next to FIGS. 17 a-c, which are also in a similar arrangement to FIGS. 15 a-c, respectively. Here, in FIGS. 17 a-c there are two additional predictors that could also be considered in the prediction phase (C1 and A2). If 4×4 blocks are also considered, this increases the possible predictors by four.
Instead of employing a median of the three predictors A, B, and C (or A1, B, and C2) one may now have some additional, and apparently more reliable, options. Thus, for example, one can observe that predictors A1, and C2 are essentially too close with one another and it may be the case that they may not be too representative in the prediction phase. Instead, selecting predictors A1, C1, and B seems to be a more reliable solution due to their separation. An alternative could also be the selection of A2 instead of A1 but that may again be too close to predictor B. Simulations suggest that the first case is usually a better choice. For the last column A2 could be used instead of A1. For the first row either one of A1 and A2 or even their average value could be used. Gain up to 1% was noted within JVT with this implementation.
The previous case adds some tests for the last column. By examining FIG. 7 b, for example, it is obvious that such tends to provide the best partitioning available. Thus, an optional solution could be the selection of A2, C1, and B (from the upper-left position). This may not always be recommended however, since such an implementation may adversely affect the performance of right predictors.
An alternative solution would be the usage of averages of predictors within a Macroblock. The median may then be performed as follows:
{right arrow over (MV)} pred=Median(Ave({right arrow over (MV)} C 1 , {right arrow over (MV)} C 2 ), Ave({right arrow over (MV)} A 1 , {right arrow over (MV)} A 2 ), {right arrow over (MV)} B)
For median row/column calculation, the median can be calculated as:
{right arrow over (MV)} pred=Median(Median({right arrow over (MV)} C 1 ,{right arrow over (MV)} C 2 , {right arrow over (MV)} D), . . . Median({right arrow over (MV)} D , {right arrow over (MV)} A 1 , {right arrow over (MV)} C 1 ), Median({right arrow over (MV)}B , {right arrow over (MV)} A 1 , {right arrow over (MV)} A 2 ))
Another possible solution is a Median5 solution. This is probably the most complicated solution due to computation (quick-sort or bubble-sort could for example be used), but could potentially yield the best results. If 4×4 blocks are considered, for example, then Median9 could also be used:
{right arrow over (MV)} pred=Median({right arrow over (MV)} C 1 , {right arrow over (MV)} C 2 , {right arrow over (MV)} D , {right arrow over (MV)} B , {right arrow over (MV)} A 1 , {right arrow over (MV)} A 2 )
Considering that JVT allows the existence of Intra subblocks within an Inter Macroblock (e.g., tree macroblock structure), such could also be taken in consideration within the Motion Prediction. If a subblock (e.g., from Macroblocks above or left only) to be used for the MV prediction is Intra, then the adjacent subblock may be used instead. Thus, if A1 is intra but A2 is not, then A1 can be replaced by A2 in the prediction. A further possibility is to replace one missing Intra Macroblock with the MV predictor from the upper-left position. In FIG. 17 a, for example, if C1 is missing then D may be used instead.
In the above sections, several improvements on B picture Direct Mode and on Motion Vector Prediction were presented. It was illustrated that spatial prediction can also be used for Direct Mode macroblocks; where as Motion Vector prediction should consider temporal distance and subblock information for more accurate prediction. Such considerations should significantly improve the performance of any applicable video coding system.
US4661849 3 Jun 1985 28 Apr 1987 Pictel Corporation Method and apparatus for providing motion estimation signals for communicating image sequences
US4661853 1 Nov 1985 28 Apr 1987 Rca Corporation Interfield image motion detector for video signals
US4695882 30 Jan 1985 22 Sep 1987 Kokusai Denshin Denwa Co., Ltd. Movement estimation system for video signals using a recursive gradient method
US4796087 1 Jun 1987 3 Jan 1989 Jacques Guichard Process for coding by transformation for the transmission of picture signals
US4849812 24 Feb 1988 18 Jul 1989 U.S. Philips Corporation Television system in which digitized picture signals subjected to a transform coding are transmitted from an encoding station to a decoding station
US4862267 31 May 1988 29 Aug 1989 Sony Corp. Motion compensated interpolation of digital television images
US5089887 22 Sep 1989 18 Feb 1992 Thomson Consumer Electronics Method and device for the estimation of motion in a sequence of moving images
US5089889 27 Apr 1990 18 Feb 1992 Victor Company Of Japan, Ltd. Apparatus for inter-frame predictive encoding of video signal
US5117287 1 Mar 1991 26 May 1992 Kokusai Denshin Denwa Co., Ltd. Hybrid coding system for moving image
US5132792 12 Oct 1990 21 Jul 1992 Sony Corporation Video signal transmitting system
US5157490 13 Mar 1991 20 Oct 1992 Kabushiki Kaisha Toshiba Television signal scanning line converting apparatus
US5193004 3 Dec 1990 9 Mar 1993 The Trustees Of Columbia University In The City Of New York Systems and methods for coding even fields of interlaced video sequences
US5235618 6 Nov 1990 10 Aug 1993 Fujitsu Limited Video signal coding apparatus, coding method used in the video signal coding apparatus and video signal coding transmission system having the video signal coding apparatus
US5260782 31 Aug 1992 9 Nov 1993 Matsushita Electric Industrial Co., Ltd. Adaptive DCT/DPCM video signal coding method
US5412435 25 Jun 1993 2 May 1995 Kokusai Denshin Denwa Kabushiki Kaisha Interlaced video signal motion compensation prediction system
US5424779 24 Nov 1993 13 Jun 1995 Kabushiki Kaisha Toshiba Video coding apparatus
US5442400 29 Apr 1993 15 Aug 1995 Rca Thomson Licensing Corporation Error concealment apparatus for MPEG-like video data
US5448297 9 Sep 1993 5 Sep 1995 Intel Corporation Method and system for encoding images using skip blocks
US5453799 5 Nov 1993 26 Sep 1995 Comsat Corporation Unified motion estimation architecture
US5467086 18 Jun 1993 14 Nov 1995 Samsung Electronics Co., Ltd. Apparatus and method of coding/decoding video data
US5467136 17 Feb 1994 14 Nov 1995 Kabushiki Kaisha Toshiba Video decoder for determining a motion vector from a scaled vector and a difference vector
US5477272 22 Jul 1993 19 Dec 1995 Gte Laboratories Incorporated Variable-block size multi-resolution motion estimation scheme for pyramid coding
US5565922 24 Jan 1996 15 Oct 1996 General Instrument Corporation Of Delaware Motion compensation for interlaced digital video signals
US5594504 6 Jul 1994 14 Jan 1997 Lucent Technologies Inc. Predictive video coding using a motion vector updating routine
US5598215 23 May 1994 28 Jan 1997 Nippon Telegraph And Telephone Corporation Moving image encoder and decoder using contour extraction
US5617144 25 May 1995 1 Apr 1997 Daewoo Electronics Co., Ltd. Image processing system using pixel-by-pixel motion estimation and frame decimation
US5619281 30 Dec 1994 8 Apr 1997 Daewoo Electronics Co., Ltd Method and apparatus for detecting motion vectors in a frame decimating video encoder
US5621481 8 Apr 1994 15 Apr 1997 Sony Corporation Motion vector detecting apparatus for determining interframe, predictive error as a function of interfield predictive errors
US5648819 29 Mar 1995 15 Jul 1997 U.S. Philips Corporation Motion estimation using half-pixel refinement of frame and field vectors
US5677735 19 Jan 1995 14 Oct 1997 Kabushiki Kaisha Toshiba Motion picture coding apparatus
US5687097 13 Jul 1995 11 Nov 1997 Zapex Technologies, Inc. Method and apparatus for efficiently determining a frame motion vector in a video encoder
US5691771 26 Dec 1995 25 Nov 1997 Sony Corporation Processing of redundant fields in a moving picture to achieve synchronized system operation
US5699476 9 May 1996 16 Dec 1997 U.S. Philips Corporation Method and apparatus for transmitting and/or storing a series of hierarchically encoded digital image data blocks
US5701164 19 Dec 1996 23 Dec 1997 Sony Corporation Macroblock coding including difference between motion vectors
US5717441 29 Apr 1996 10 Feb 1998 Matsushita Electric Ind. Picture data memory with high access efficiency in detecting motion vectors, a motion vector detection circuit provided with the picture data memory, and an address conversion circuit
US5731850 7 Jun 1995 24 Mar 1998 Maturi; Gregory V. Hybrid hierarchial/full-search MPEG encoder motion estimation
US5734755 11 Mar 1994 31 Mar 1998 The Trustees Of Columbia University In The City Of New York JPEG/MPEG decoder-compatible optimized thresholding for image and video signal compression
US5786860 22 Feb 1995 28 Jul 1998 Korean Advanced Institute Of Science And Technology High speed block matching for bi-directional motion vector estimation
US5796438 23 Jun 1995 18 Aug 1998 Sony Corporation Methods and apparatus for interpolating picture information
US5798788 1 Feb 1996 25 Aug 1998 David Sarnoff Research Center, Inc. Method and apparatus for evaluating field display functionality of a video decoder
US5822541 9 Oct 1996 13 Oct 1998 Hitachi, Ltd. Compressed video data amount reducing device, compressed video data amount reducing system and compressed video data amount reducing method
US5847776 24 Jun 1996 8 Dec 1998 Vdonet Corporation Ltd. Method for entropy constrained motion estimation and coding of motion vectors with increased search range
US5874995 2 Sep 1997 23 Feb 1999 Matsuhita Electric Corporation Of America MPEG video decoder having a high bandwidth memory for use in decoding interlaced and progressive signals
US5901248 6 Aug 1996 4 May 1999 8X8, Inc. Programmable architecture and methods for motion estimation
US5926573 29 Mar 1996 20 Jul 1999 Matsushita Electric Corporation Of America MPEG bit-stream format converter for changing resolution
US5929940 24 Oct 1996 27 Jul 1999 U.S. Philips Corporation Method and device for estimating motion between images, system for encoding segmented images
US5946042 2 Jul 1997 31 Aug 1999 Sony Corporation Macroblock coding including difference between motion vectors
US5949489 31 Jul 1998 7 Sep 1999 Mitsubishi Denki Kabushiki Kaisha Image signal coding system
US5963258 31 Jul 1998 5 Oct 1999 Mitsubishi Denki Kabushiki Kaisha Image signal coding system
US5963673 18 Dec 1996 5 Oct 1999 Sanyo Electric Co., Ltd. Method and apparatus for adaptively selecting a coding mode for video encoding
US5970173 4 Jun 1996 19 Oct 1999 Microsoft Corporation Image compression and affine transformation for image motion compensation
US5970175 26 Oct 1998 19 Oct 1999 Mitsubishi Denki Kabushiki Kaisha Image signal coding system
US5973743 2 Dec 1997 26 Oct 1999 Daewoo Electronics Co., Ltd. Mode coding method and apparatus for use in an interlaced shape coder
US5973755 4 Apr 1997 26 Oct 1999 Microsoft Corporation Video encoder and decoder using bilinear motion compensation and lapped orthogonal transforms
US5982438 29 Oct 1997 9 Nov 1999 Microsoft Corporation Overlapped motion compensation for object coding
US6002439 27 May 1999 14 Dec 1999 Mitsubishi Denki Kabushiki Kaisha Image signal coding system
US6005980 21 Jul 1997 21 Dec 1999 General Instrument Corporation Motion estimation and compensation of video object planes for interlaced digital video
US6011596 11 Jun 1997 4 Jan 2000 British Broadcasting Video image motion compensation using an algorithm involving at least two fields
US6055012 27 Dec 1996 25 Apr 2000 Lucent Technologies Inc. Digital multi-view video compression with complexity and compatibility constraints
US6067322 4 Jun 1997 23 May 2000 Microsoft Corporation Half pixel motion estimation in motion video signal encoding
US6081209 12 Nov 1998 27 Jun 2000 Hewlett-Packard Company Search system for use in compression
US6094225 2 Dec 1997 25 Jul 2000 Daewoo Electronics, Co., Ltd. Method and apparatus for encoding mode signals for use in a binary shape coder
US6130963 22 Nov 1996 10 Oct 2000 C-Cube Semiconductor Ii, Inc. Memory efficient decoding of video frame chroma
US6154495 30 Apr 1998 28 Nov 2000 Kabushiki Kaisha Toshiba Video coding and video decoding apparatus for changing a resolution conversion according to a reduction ratio setting information signal
US6167090 23 Dec 1997 26 Dec 2000 Nippon Steel Corporation Motion vector detecting apparatus
US6175592 12 Mar 1997 16 Jan 2001 Matsushita Electric Industrial Co., Ltd. Frequency domain filtering for down conversion of a DCT encoded picture
US6188794 20 May 1999 13 Feb 2001 Mitsubishi Denki Kabushiki Kaisha Image signal coding system
US6205176 28 Jul 1998 20 Mar 2001 Victor Company Of Japan, Ltd. Motion-compensated coder with motion vector accuracy controlled, a decoder, a method of motion-compensated coding, and a method of decoding
US6205177 24 Mar 2000 20 Mar 2001 Netergu Networks Video coder/decoder
US6263024 11 Dec 1997 17 Jul 2001 Matsushita Electric Industrial Co., Ltd. Picture encoder and picture decoder
US6269121 13 Aug 1998 31 Jul 2001 Daewoo Electronics Co., Ltd. Adaptive motion estimation method and apparatus
US6271885 23 Jun 1999 7 Aug 2001 Victor Company Of Japan, Ltd. Apparatus and method of motion-compensated predictive coding
US6282243 18 Nov 1997 28 Aug 2001 Fujitsu Limited Apparatus and method for interframe predictive video coding and decoding with capabilities to avoid rounding error accumulation
US6295376 8 Jun 1998 25 Sep 2001 Hitachi, Ltd. Image sequence coding method and decoding method
US6307887 30 May 2000 23 Oct 2001 Microsoft Corporation Video encoder and decoder using bilinear motion compensation and lapped orthogonal transforms
US6307973 4 Dec 2000 23 Oct 2001 Mitsubishi Denki Kabushiki Kaisha Image signal coding system
US6320593 20 Apr 1999 20 Nov 2001 Agilent Technologies, Inc. Method of fast bi-cubic interpolation of image information
US6324216 30 May 1995 27 Nov 2001 Sony Corporation Video coding selectable between intra-frame prediction/field-based orthogonal transformation and inter-frame prediction/frame-based orthogonal transformation
US6377628 15 Dec 1997 23 Apr 2002 Thomson Licensing S.A. System for maintaining datastream continuity in the presence of disrupted source data
US6381279 30 Jun 2000 30 Apr 2002 Hewlett-Packard Company Method for providing motion-compensated multi-field enhancement of still images from video
US6404813 27 Mar 1997 11 Jun 2002 At&T Corp. Bidirectionally predicted pictures or video object planes for efficient and flexible video coding
US6427027 7 Apr 2000 30 Jul 2002 Sony Corporation Picture encoding and/or decoding apparatus and method for providing scalability of a video object whose position changes with time and a recording medium having the same recorded thereon
US6459812 7 Apr 2000 1 Oct 2002 Sony Corporation Picture encoding and/or decoding apparatus and method for providing scalability of a video object whose position changes with time and a recording medium having the same recorded thereon
US6483874 27 Jan 1999 19 Nov 2002 General Instrument Corporation Efficient motion estimation for an arbitrarily-shaped object
US6496601 8 Mar 2000 17 Dec 2002 Viewpoint Corp. System and method for asynchronous, adaptive moving picture compression, and decompression
US6519287 13 Jul 1998 11 Feb 2003 Motorola, Inc. Method and apparatus for encoding and decoding video signals by using storage and retrieval of motion vectors
US6529632 29 Feb 2000 4 Mar 2003 Hitachi, Ltd. Image coding method
US6539056 21 Jul 1999 25 Mar 2003 Sony Corporation Picture decoding method and apparatus
US6636565 12 Jan 2000 21 Oct 2003 Lg Electronics Inc. Method for concealing error
US6647061 9 Jun 2000 11 Nov 2003 General Instrument Corporation Video size conversion and transcoding from MPEG-2 to MPEG-4
US6650781 8 Jun 2001 18 Nov 2003 Hitachi, Ltd. Image decoder
US6683987 8 Mar 2000 27 Jan 2004 Victor Company Of Japan, Ltd. Method and apparatus for altering the picture updating frequency of a compressed video data stream
US6735345 19 Mar 2003 11 May 2004 Microsoft Corporation Efficient macroblock header coding for video compression
US6785331 27 Jan 1998 31 Aug 2004 Nippon Telegraph And Telephone Corporation Predictive encoding and decoding methods of video data
US6798364 5 Feb 2002 28 Sep 2004 Intel Corporation Method and apparatus for variable length coding
US6798837 18 Feb 2000 28 Sep 2004 Matsushita Electric Industrial Co., Ltd. Video coding method and video coding apparatus
US6807231 4 Sep 1998 19 Oct 2004 8×8, Inc. Multi-hypothesis motion-compensated video image predictor
US6816552 11 Jul 2001 9 Nov 2004 Dolby Laboratories Licensing Corporation Interpolation of video compression frames
US6873657 27 Dec 2001 29 Mar 2005 Koninklijke Philips Electronics, N.V. Method of and system for improving temporal consistency in sharpness enhancement for a video signal
US6876703 27 Apr 2001 5 Apr 2005 Ub Video Inc. Method and apparatus for video coding
US6920175 9 Aug 2001 19 Jul 2005 Nokia Corporation Video coding architecture and methods for using same
US6975680 12 Jul 2001 13 Dec 2005 Dolby Laboratories, Inc. Macroblock mode decision biasing for video compression systems
US6980596 20 Nov 2002 27 Dec 2005 General Instrument Corporation Macroblock level adaptive frame/field coding for digital video content
US6999513 17 Sep 2002 14 Feb 2006 Korea Electronics Technology Institute Apparatus for encoding a multi-view moving picture
US7003035 27 Jun 2002 21 Feb 2006 Microsoft Corporation Video coding methods and apparatuses
US7154952 15 Jul 2003 26 Dec 2006 Microsoft Corporation Timestamp-independent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures
US7233621 6 Jan 2003 19 Jun 2007 Lg Electronics, Inc. Method of determining a motion vector for deriving motion vectors of bi-predictive block
US7346111 10 Dec 2003 18 Mar 2008 Lsi Logic Corporation Co-located motion vector storage
US7362807 8 Jan 2003 22 Apr 2008 Matsushita Electric Industrial Co., Ltd. Motion vector coding method and motion vector decoding method
US7388916 7 Jul 2001 17 Jun 2008 Electronics And Telecommunications Research Institute Water ring scanning apparatus and method, and apparatus and method for encoding/decoding video sequences using the same
US7733960 * 26 Oct 2007 8 Jun 2010 Panasonic Corporation Motion vector calculation method
US20010019586 13 Feb 2001 6 Sep 2001 Kang Hyun Soo Motion estimation method and device
US20010040926 15 May 2001 15 Nov 2001 Miska Hannuksela Video coding
US20020105596 13 Dec 2000 8 Aug 2002 Steve Selby Method and apparatus for detecting motion between odd and even video fields
US20020114388 13 Apr 2001 22 Aug 2002 Mamoru Ueda Decoder and decoding method, recorded medium, and program
US20020122488 11 Oct 2001 5 Sep 2002 Kuniaki Takahashi Motion vector conversion method and conversion apparatus
US20020154693 2 Mar 2001 24 Oct 2002 Demos Gary A. High precision encoding and decoding of video images
US20020186890 3 May 2001 12 Dec 2002 Ming-Chieh Lee Dynamic filtering for lossy compression
US20030053537 5 Mar 2002 20 Mar 2003 Chang-Su Kim Systems and methods for reducing frame rates in a video data stream
US20030099292 20 Nov 2002 29 May 2003 Limin Wang Macroblock level adaptive frame/field coding for digital video content
US20030099294 20 Nov 2002 29 May 2003 Limin Wang Picture level adaptive frame/field coding for digital video content
US20030112864 17 Sep 2001 19 Jun 2003 Marta Karczewicz Method for sub-pixel value interpolation
US20030113026 16 Dec 2002 19 Jun 2003 Microsoft Corporation Skip macroblock coding
US20030142748 27 Jun 2002 31 Jul 2003 Alexandros Tourapis Video coding methods and apparatuses
US20030142751 22 Jan 2003 31 Jul 2003 Nokia Corporation Coding scene transitions in video coding
US20030156646 17 Dec 2002 21 Aug 2003 Microsoft Corporation Multi-resolution motion estimation and compensation
US20030202590 30 Apr 2002 30 Oct 2003 Qunshan Gu Video encoding using direct mode predicted frames
US20030206589 6 Jan 2003 6 Nov 2003 Lg Electronics Inc. Method for coding moving picture
US20040001546 23 May 2003 1 Jan 2004 Alexandros Tourapis Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation
US20040008899 13 Jun 2003 15 Jan 2004 Alexandros Tourapis Optimization techniques for data compression
US20040047418 15 Jul 2003 11 Mar 2004 Alexandros Tourapis Timestamp-independent motion vector prediction for predictive (P) and bidirectionally predictive (B) pictures
US20040141651 24 Oct 2003 22 Jul 2004 Junichi Hara Modifying wavelet division level before transmitting data stream
US20040146109 16 Apr 2003 29 Jul 2004 Satoshi Kondo Method for calculation motion vector
US20040234143 9 Jun 2003 25 Nov 2004 Makoto Hagai Image encoding method and picture decoding method
US20050013497 18 Jul 2003 20 Jan 2005 Microsoft Corporation Intraframe and interframe interlace coding and decoding
US20050013498 18 Jul 2003 20 Jan 2005 Microsoft Corporation Coding of motion vector information
US20050036759 30 Jun 2004 17 Feb 2005 Microsoft Corporation Efficient motion vector coding for video compression
US20050053137 27 May 2004 10 Mar 2005 Microsoft Corporation Predicting motion vectors for fields of forward-predicted interlaced video frames
US20050053147 15 Sep 2004 10 Mar 2005 Microsoft Corporation Motion vector prediction in bi-directionally predicted interlaced field-coded pictures
US20050053149 15 Sep 2004 10 Mar 2005 Microsoft Corporation Direct mode motion vectors for Bi-directionally predicted interlaced pictures
US20050100093 15 Nov 2004 12 May 2005 Microsoft Corporation Signaling field type information
US20050129120 28 Jan 2005 16 Jun 2005 Jeon Byeong M. Method of determining a motion vector for deriving motion vectors of a bi-predictive image block
US20050135484 7 Jun 2004 23 Jun 2005 Daeyang Foundation (Sejong University) Method of encoding mode determination, method of motion estimation and encoding apparatus
US20050147167 24 Dec 2003 7 Jul 2005 Adriana Dumitras Method and system for video encoding using a variable number of B frames
US20050185713 24 Feb 2004 25 Aug 2005 Lsi Logic Corporation Method and apparatus for determining a second picture for temporal direct-mode block prediction
US20050207490 11 Mar 2005 22 Sep 2005 Wang Jason N Stored picture index for AVC coding
US20050249291 2 May 2005 10 Nov 2005 Stephen Gordon Method and system for generating a transform size syntax element for video decoding
US20050254584 19 Jul 2005 17 Nov 2005 Chang-Su Kim Systems and methods for enhanced error concealment in a video decoder
US20060013307 29 Jun 2005 19 Jan 2006 Yannick Olivier Method or device for coding a sequence of source pictures
US20060072662 9 Dec 2005 6 Apr 2006 Microsoft Corporation Improved Video Coding
US20060280253 21 Aug 2006 14 Dec 2006 Microsoft Corporation Timestamp-Independent Motion Vector Prediction for Predictive (P) and Bidirectionally Predictive (B) Pictures
US20070064801 9 Nov 2006 22 Mar 2007 General Instrument Corporation Picture Level Adaptive Frame/Field Coding for Digital Video Content
US20070177674 12 Jan 2007 2 Aug 2007 Lg Electronics Inc. Processing multiview video
US20080043845 19 Jul 2007 21 Feb 2008 Fujitsu Limited Motion prediction processor with read buffers providing reference motion vectors for direct mode coding
US20080069462 29 Oct 2007 20 Mar 2008 Kiyofumi Abe Moving picture coding method and moving picture decoding method
US20080075171 31 Oct 2007 27 Mar 2008 Yoshinori Suzuki Moving picture encoding method and decoding method
US20090238269 5 Jul 2007 24 Sep 2009 Purvin Bibhas Pandit Method and Apparatus for Decoupling Frame Number and/or Picture Order Count (POC) for Multi-View Video Encoding and Decoding
USRE35910 12 May 1994 29 Sep 1998 Matsushita Electric Industrial Co., Ltd. Moving image signal encoding apparatus and decoding apparatus
USRE36507 21 Oct 1997 18 Jan 2000 Matsushita Electric Corporation Of America Apparatus and method for processing groups of fields in a video data compression system to encode a single frame as an I-field and a P-field
USRE36822 2 Oct 1998 15 Aug 2000 Victor Company Of Japan, Ltd. Moving image signal coding apparatus and coded signal decoding apparatus
USRE37222 19 Jul 1994 12 Jun 2001 Sony Corporation Video signal transmitting system
USRE38563 19 Nov 2001 10 Aug 2004 Gen Instrument Corp Prediction and coding of bi-directionally predicted video object planes for interlaced digital video
EP0279053B1 11 Dec 1987 10 Apr 1991 ANT Nachrichtentechnik GmbH Method for the transmission and reproduction of sequences of television pictures
EP0397402B1 3 May 1990 10 Jul 1996 Matsushita Electric Industrial Co., Ltd. Moving image signal encoding- and decoding apparatus
EP0526163B1 28 Jul 1992 4 Mar 1998 Matsushita Electric Industrial Co., Ltd. Image coding method and image coding apparatus
EP0535746B1 29 Sep 1992 29 Jan 1997 Philips Electronics Uk Limited Motion vector estimation, motion picture encoding and storage
EP0588653B1 16 Sep 1993 22 Dec 1999 Fujitsu Limited Image data coding and restoring method and apparatus for restoring the same
EP0614318B1 3 Mar 1994 1 Dec 1999 Kabushiki Kaisha Toshiba Video encoder and decoder
EP0625853B1 20 May 1994 3 Mar 1999 Nippon Telegraph And Telephone Corporation Moving image encoder and decoder
EP0771114A2 24 Oct 1996 2 May 1997 Hyundai Electronics Industries Co., Ltd. Method of generating object-adaptive code block patterns
EP0782343A3 27 Dec 1996 20 Dec 2000 Matsushita Electric Industrial Co., Ltd. Video coding method
EP0786907A2 24 Jan 1997 30 Jul 1997 Texas Instruments Incorporated Video encoder
EP0830029A3 25 Jun 1997 7 May 2003 Robert Bosch Gmbh Method for video signal data reduction
EP0863673A3 27 Feb 1998 11 Dec 2002 General Instrument Corporation Intra-macroblock DC and AC coefficient prediction for interlaced digital video
EP0863674B1 27 Feb 1998 26 Mar 2014 Motorola Mobility LLC Prediction and coding of bi-directionally predicted video object planes for interlaced digital video
EP0863675A3 27 Feb 1998 9 Apr 2003 General Instrument Corporation Motion estimation and compensation of video object planes for interlaced digital video
EP0874526B1 23 Apr 1998 25 Aug 2004 Victor Company Of Japan, Limited Motion compensation encoding apparatus and motion compensation encoding method for high-efficiency encoding of video information through selective use of previously derived motion vectors in place of motion vectors derived from motion estimation
EP0884912B1 8 Jun 1998 27 Aug 2003 Hitachi, Ltd. Image sequence decoding method
EP0901289A3 3 Sep 1998 2 Jul 2003 Matsushita Electric Industrial Co., Ltd. Apparatus for layered video coding
EP0944245B1 20 Mar 1998 25 Jul 2001 SGS-THOMSON MICROELECTRONICS S.r.l. Hierarchical recursive motion estimator for video images encoder
EP1006732A3 8 Nov 1999 22 Oct 2003 Mitsubishi Denki Kabushiki Kaisha Motion compensated interpolation for digital video signal processing
EP1427216A1 9 Jun 2003 9 Jun 2004 Matsushita Electric Industrial Co., Ltd. Image encoding method and image decoding method
3 Anonymous, "DivX Multi Standard Video Encoder," 2 pp.
5 Chujoh et al., "Verification result on the combination of spatial and temporal," JVT-E095, 5 pp. (Oct. 2002).
6 Decision on Grant dated Apr. 2, 2007, from Russian Patent Application No. 2003116281, 5 pp.
7 Ericsson, "Fixed and Adaptive Predictors for Hybrid Predictive/Transform Coding," IEEE Transactions on Comm., vol. COM-33, No. 12, pp. 1291-1302 (1985).
8 European Official Communication dated Oct. 28, 2005, from European Patent Application No. 03 011 935.8, 3 pp.
9 European Search Report dated Dec. 4, 2003, from European Patent Application No. 03 011 935.8, 5 pp.
10 Examiner's First Report dated Nov. 29, 2007, from Australian Patent Application No. 2003204477, 2 pp.
11 Examiner's Report No. 2 dated Jun. 24, 2008, from Australian Patent Application No. 2003204477, 2 pp.
12 Flierl et al., "Multihypothesis Motion Estimation for Video Coding," Proc. DCC, 10 pp. (Mar. 2001).
13 Fogg, "Survey of Software and Hardware VLC Architectures," SPIE, vol. 2186, pp. 29-37 (1994).
14 Girod, "Efficiency Analysis of Multihypothesis Motion-Compensated Prediction for Video Coding," IEEE Transactions on Image Processing, vol. 9, No. 2, pp. 173-183 (Feb. 2000).
15 Girod, "Motion-Compensation: Visual Aspects, Accuracy, and Fundamental Limits," Motion Analysis and Image Sequence Processing, Kluwer Academic Publishers, pp. 125-152 (1993).
16 Grigoriu, "Spatio-temporal compression of the motion field in video coding," 2001 IEEE Fourth Workshop on Multimedia Signal Processing, pp. 129-134 (Oct. 2001).
17 Gu et al., "Introducing Direct Mode P-picture (DP) to reduce coding complexity," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, Document No. JVT-C044, 10 pp. (Mar. 2002).
18 Horn et al., "Estimation of Motion Vector Fields for Multiscale Motion Compensation," Proc. Picture Coding Symp. (PCS 97), pp. 141-144 (Sep. 1997).
19 Hsu et al., "A Low Bit-Rate Video Codec Based on Two-Dimensional Mesh Motion Compensation with Adaptive Interpolation," IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, No. 1, pp. 111-117 (Jan. 2001).
20 Huang et al., "Hardware architecture design for variable block size motion estimation in MPEG-4 AVC/JVT/ITU-T H.264," Proc. of the 2003 Int'l Symposium on Circuits & Sys. (ISCAS '03), vol. 2, pp. 796-799 (May 2003).
21 ISO/IEC, "Information Technology-Coding of Audio-Visual Objects: Visual, ISO/IEC 14496-2, Committee Draft," 330 pp. (1998).
22 ISO/IEC, "Information Technology—Coding of Audio-Visual Objects: Visual, ISO/IEC 14496-2, Committee Draft," 330 pp. (1998).
23 ISO/IEC, "ISO/IEC 11172-2: Information Technology-Coding of Moving Pictures and Associated Audio for Storage Mediaat up to About 1.5 Mbit/s," 122 pp. (1993).
24 ISO/IEC, "ISO/IEC 11172-2: Information Technology—Coding of Moving Pictures and Associated Audio for Storage Mediaat up to About 1.5 Mbit/s," 122 pp. (1993).
25 ISO/IEC, "MPEG-4 Video Verification Model Version 10.0," ISO/IEC JTC1/SC29/WG11, MPEG98/N1992, 305 pp. (1998).
26 ISO/IEC, "MPEG-4 Video Verification Model Version 18.0," ISO/IEC JTC1/SC29/WG11 N3908, Pisa, pp. 1-10, 299-311 (Jan. 2001).
27 ITU-Q15-F-24, "MVC Video Codec-Proposal for H.26L," Study Group 16, Video Coding Experts Group (Question 15), 28 pp. (document marked as generated in 1998).
28 ITU—Q15-F-24, "MVC Video Codec—Proposal for H.26L," Study Group 16, Video Coding Experts Group (Question 15), 28 pp. (document marked as generated in 1998).
29 ITU-T, "ITU-T Recommendation H.261: Video Codec for Audiovisual Services at p x 64 kbits," 28 pp. (1993).
30 ITU-T, "ITU-T Recommendation H.262: Information Technology-Generic Coding of Moving Pictures and Associated Audio Information: Video," 218 pp. (1995).
31 ITU-T, "ITU-T Recommendation H.262: Information Technology—Generic Coding of Moving Pictures and Associated Audio Information: Video," 218 pp. (1995).
32 ITU-T, "ITU-T Recommendation H.263: Video Coding for Low Bit Rate Communication," 167 pp. (1998).
33 Jeon et al., "B picture coding for sequence with repeating scene changes," JVT-C120, 9 pp. (document marked May 1, 2002).
34 Jeon, "Clean up for temporal direct mode," JVT-E097, 13 pp. (Oct. 2002).
35 Jeon, "Direct mode in B pictures," JVT-D056, 10 pp. (Jul. 2002).
36 Jeon, "Motion vector prediction and prediction signal in B pictures," JVT-D057, 5 pp. (Jul. 2002).
37 Ji et al., "New Bi-Prediction Techniques for B Pictures Coding," IEEE Int'l Conf. on Multimedia and Expo, pp. 101-104 (Jun. 2004).
38 Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, "Joint Model No. 1, Revision 1 (JM-1r1)," JVT-A003r1, Pattaya, Thailand, 80 pp. (Dec. 2001).
39 Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, "Study of Final Committee Draft of Joint Video Specification," JVT-F100, Awaji Island, 242 pp. (Dec. 2002).
40 Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, Working Draft No. 2, Revision 0 (WD-2), JVT-B118r1, 105 pp. (Jan. 2002).
41 Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, Working Draft No. 2, Revision 2 (WD-2), JVT-B118r2, 106 pp. (Jan. 2002).
42 Joint Video Team of ISO/IEC MPEG and ITU-T VCEG, "Joint Final Committee Draft (JFCD) of Joint Video Specification (ITU-T Recommendation H.264, ISO/IEC 14496-10 AVC," JVT-D157 (Aug. 2002).
43 Joint Video Team of ISO/IEC MPEG and ITU-T VCEG, "Text of Committee Draft of Joint Video Specification (ITU-T Rec. H.264, ISO/IEC 14496-10 AVC)," Document JVT-C167, 143 pp. (May 2002).
44 Kadono et al., "Memory reduction for temporal technique of direct mode," JVT-E076, 12 pp. (Oct. 2002).
45 Ko et al., "Fast Intra-Mode Decision Using Inter-Frame Correlation for H.264/AVC," Proc. IEEE ISCE 2008, 4 pages (Apr. 2008).
46 Kondo et al., "New Prediction Method to Improve B-picture Coding Efficiency," VCEG-O26, 9 pp. (document marked Nov. 26, 2001).
47 Kondo et al., "Proposal of Minor Changes to Multi-frame Buffering Syntax for Improving Coding Efficiency of B-pictures," JVT-B057, 10 pp. (document marked Jan. 23, 2002).
48 Konrad et al., "On Motion Modeling and Estimation for Very Low Bit Rate Video Coding," Visual Comm. & Image Processing (VCIP '95), 12 pp. (May 1995).
49 Kossentini et al., "Predictive RD Optimized Motion Estimation for Very Low Bit-rate Video Coding," IEEE J. on Selected Areas in Communications, vol. 15, No. 9 pp. 1752-1763 (Dec. 1997).
50 Lainema et al., "Skip Mode Motion Compensation," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-C027, 8 pp. (May 2002).
51 Microsoft Corporation, "Microsoft Debuts New Windows Media Players 9 Series, Redefining Digital Media on the PC," 4 pp. (Sep. 4, 2002) [Downloaded from the World Wide Web on May 14, 2004].
53 Notice of Acceptance dated Feb. 26, 2009, from Australian Patent Application No. 2003204477, 3 pp.
54 Notice of Allowance dated Apr. 1, 2011, from Japanese Patent Application No. 2003-157240, 6 pp.
55 Notice of Allowance dated Apr. 12, 2010, from Korean Patent Application No. 10-2003-0035240, 2 pp.
56 Notice of Preliminary Rejection dated Sep. 24, 2009, from Korean Patent Application No. 10-2003-0035240, 4 pp.
57 Notice of Rejection dated Dec. 8, 2009, from Japanese Patent Application No. 2003-157240, 5 pp.
58 Notice of Rejection dated Sep. 17, 2010, from Japanese Patent Application No. 2003-157240, 3 pp.
59 Notice on Grant of Patent Right for Invention dated Sep. 19, 2008, from Chinese Patent Application No. 03141275.0, 4 pp.
60 Notice on Office Action dated Sep. 1, 2006, from Chinese Patent Application No. 03141275.0, 9 pp.
61 Official Action dated Dec. 11, 2006, from Russian Patent Application No. 2003116281, 10 pp.
62 Official Action dated Mar. 22, 2012, from Canadian Patent Application No. 2,430,460, 3 pp.
63 Official Action dated Sep. 21, 2011, from Canadian Patent Application No. 2,430,460, 3 pp.
64 Official Notice of Final Rejection dated Apr. 9, 2010, from Japanese Patent Application No. 2003-157240, 4 pp.
65 Panusopone et al., "Direct Prediction for Predictive (P) Picture in Field Coding mode," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, Document JVT-D046, 8 pp. (Jul. 2002).
66 Pourazad et al., "An H.264-based Video Encoding Scheme for 3D TV," EURASIP European Signal Processing Conference-EUSIPCO, Florence, Italy, 5 pages (Sep. 2006).
67 Pourazad et al., "An H.264-based Video Encoding Scheme for 3D TV," EURASIP European Signal Processing Conference—EUSIPCO, Florence, Italy, 5 pages (Sep. 2006).
68 Printouts of FTP directories from http://ftp3.itu.ch, 8 pp. (downloaded from the World Wide Web on Sep. 20, 2005).
69 Reader, "History of MPEG Video Compression-Ver. 4.0," 99 pp. (document marked Dec. 16, 2003).
70 Reader, "History of MPEG Video Compression—Ver. 4.0," 99 pp. (document marked Dec. 16, 2003).
71 Schwarz et al., "Core Experiment Results on Improved Macroblock Prediction Modes," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-B054, 10 pp. (Jan.-Feb. 2002).
72 Schwarz et al., "Tree-structured macroblock partition," ITU-T SG16/Q.6 VCEG-O17, 6 pp. (Dec. 2001).
74 Suzuki et al., "Study of direct mode," JVT-E071 rl, 7 pp. (Oct. 2002).
75 Suzuki, "Handling of reference pictures and MVs for direct mode," JVT-D050, 11 pp. (Jul. 2002).
76 The Second Office Action dated Nov. 16, 2007, from Chinese Patent Application No. 03141275.0, 8 pp.
77 Tourapis et al., "B picture and ABP finalization," JVT-E018, 2 pp. (Oct. 2002).
78 Tourapis et al., "Direct Mode Coding for Bipredictive Slices in the H.264 Standard," IEEE Trans. on Circuits and Systems for Video Technology, vol. 15, No. 1, pp. 119-126 (Jan. 2005).
79 Tourapis et al., "Direct Prediction for Predictive (P) and Bidirectionally Predictive (B) frames in Video Coding ," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-C128, 11 pp. (May 2002).
80 Tourapis et al., "Motion Vector Prediction in Bidirectionally Predictive (B) frames with regards to Direct Mode," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-C127, 7 pp. (May 2002).
81 Tourapis et al., "Performance Comparison of Temporal and Spatial Direct mode," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-E026, 7 pp. (Oct. 2002).
82 Tourapis et al., "Temporal Interpolation of Video Sequences Using Zonal Based Algorithms," IEEE, pp. 895-898 (Oct. 2001).
83 Tourapis et al., "Timestamp Independent Motion Vector Prediction for P and B frames with Division Elimination," Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-D040, 18 pp. (Jul. 2002).
84 U.S. Appl. No. 10/462,085.
85 U.S. Appl. No. 10/622,378.
86 U.S. Appl. No. 10/942,524.
87 U.S. Appl. No. 11/275,103.
88 U.S. Appl. No. 11/465,938.
89 U.S. Appl. No. 11/525,059.
90 U.S. Appl. No. 11/824,550.
91 U.S. Appl. No. 12/364,325.
92 U.S. Appl. No. 12/474,821.
93 U.S. Appl. No. 13/459,809.
94 U.S. Appl. No. 60/341,674, filed Dec. 17, 2001, Lee et al.
95 U.S. Appl. No. 60/488,710, filed Jul. 18, 2003, Srinivasan et al.
96 U.S. Appl. No. 60/501,081, filed Sep. 7, 2003, Srinivasan et al.
97 Wang et al., "Adaptive frame/field coding for JVT Video Coding," ITU-T SG16 Q.6 JVT-B071, 24 pp. (Jan. 2002).
98 Wang et al., "Interlace Coding Tools for H.26L Video Coding," ITU-T SG16/Q.6 VCEG-O37, pp. 1-20 (Dec. 2001).
99 Wiegand et al., "Long-term Memory Motion Compensated Prediction," IEEE Transactions on Circuits & Systems for Video Technology, vol. 9, No. 1, pp. 70-84 (Feb. 1999).
100 Wiegand et al., "Motion-compensating Long-term Memory Prediction," Proc. Int'l Conf. on Image Processing, 4 pp. (Oct. 1997).
101 Wiegand, "H.26L Test Model Long-Term No. 9 (TML-9) draft 0," ITU-Telecommunications Standardization Sector, Study Group 16, VCEG-N83, 74 pp. (Dec. 2001).
102 Wien, "Variable Block-Size Transforms for Hybrid Video Coding," Dissertation, 182 pp. (Feb. 2004).
103 Winger et al., "HD temporal direct-mode verification & text," JVT-E037, 8 pp. (Oct. 2002).
104 Wu et al., "Joint estimation of forward and backward motion vectors for interpolative prediction of video," IEEE Transactions on Image Processing, vol. 3, No. 5, pp. 684-687 (Sep. 1994).
105 Yu et al., "Two-Dimensional Motion Vector Coding for Low Bitrate Videophone Applications," Proc. Int'l Conf on Image Processing, Los Alamitos, US, pp. 414-417, IEEE Comp. Soc. Press (1995).
US9392293 * 21 May 2014 12 Jul 2016 Alcatel Lucent Accelerated image processing
US9438929 * 18 Mar 2014 6 Sep 2016 Samsung Electronics Co., Ltd. Method and apparatus for encoding and decoding an image by using an adaptive search range decision for motion estimation
US9445120 3 Feb 2015 13 Sep 2016 Sun Patent Trust Moving picture coding method, moving picture coding apparatus, moving picture decoding method, moving picture decoding apparatus and moving picture coding and decoding apparatus
US9456214 2 Aug 2012 27 Sep 2016 Sun Patent Trust Moving picture coding method, moving picture coding apparatus, moving picture decoding method, moving picture decoding apparatus, and moving picture coding and decoding apparatus
US9479777 4 Mar 2013 25 Oct 2016 Sun Patent Trust Moving picture coding method, moving picture decoding method, moving picture coding apparatus, moving picture decoding apparatus, and moving picture coding and decoding apparatus
US9485518 24 May 2012 1 Nov 2016 Sun Patent Trust Decoding method and apparatus with candidate motion vectors
US9591328 16 Jan 2013 7 Mar 2017 Sun Patent Trust Methods and apparatuses for encoding and decoding video using temporal motion vector prediction
US9609320 * 24 Jun 2013 28 Mar 2017 Sun Patent Trust Image decoding method and image decoding apparatus
US9609356 3 Feb 2015 28 Mar 2017 Sun Patent Trust Moving picture coding method and apparatus with candidate motion vectors
US9615107 24 May 2012 4 Apr 2017 Sun Patent Trust Image coding method, image coding apparatus, image decoding method, image decoding apparatus, and image coding and decoding apparatus
US9648323 30 May 2013 9 May 2017 Sun Patent Trust Image coding method and image coding apparatus
US9723322 15 Sep 2016 1 Aug 2017 Sun Patent Trust Decoding method and apparatus with candidate motion vectors
US9819961 28 Apr 2016 14 Nov 2017 Sun Patent Trust Decoding method and apparatuses with candidate motion vectors
US20130188713 * 28 Feb 2013 25 Jul 2013 Sungkyunkwan University Foundation For Corporate Collaboration Bi-prediction coding method and apparatus, bi-prediction decoding method and apparatus, and recording medium
US20140016702 * 24 Jun 2013 16 Jan 2014 Panasonic Corporation Image decoding method and image decoding apparatus
US20140270555 * 18 Mar 2014 18 Sep 2014 Samsung Electronics Co., Ltd. Method and apparatus for encoding and decoding an image by using an adaptive search range decision for motion estimation
US20170163987 * 16 Feb 2017 8 Jun 2017 Sun Patent Trust Image coding method, image coding apparatus, image decoding method, image decoding apparatus, and image coding and decoding apparatus
U.S. Classification 375/240.15, 375/240.16, 375/240.24
International Classification H04N7/12, H04N11/02, H03M7/36, G06T9/00, H04N11/04, H04N19/593
Cooperative Classification H04N19/58, H04N19/577, H04N19/56, H04N19/107, H04N19/87, H04N19/52, H04N19/176, H04N19/102, H04N19/105, H04N19/142, H04N19/137, H04N19/593, H04N19/573, H04N19/51, H04N19/61, H04N19/513