Source: https://patents.google.com/patent/US10404994B2/en
Timestamp: 2019-12-12 01:18:08
Document Index: 244155106

Matched Legal Cases: ['art 1000', 'Application No. 201080068069', 'Application No. 201080068069', 'Application No. 1100658', 'Application No. 2013', 'Application No. 11830181', 'Application No. 10860988', 'Application No. 11830181', 'Application No. 2011', 'Application No. 100123109', 'Application No. 10860988', 'Application No. 10', 'Application No. 2010', 'Application No. 2011', 'Application No. 104130109']

US10404994B2 - Methods and systems for motion vector derivation at a video decoder - Google Patents
Methods and systems for motion vector derivation at a video decoder Download PDF
US10404994B2
US10404994B2 US15/960,120 US201815960120A US10404994B2 US 10404994 B2 US10404994 B2 US 10404994B2 US 201815960120 A US201815960120 A US 201815960120A US 10404994 B2 US10404994 B2 US 10404994B2
US15/960,120
US20180242012A1 (en
2015-06-11 Priority to US14/737,437 priority patent/US9955179B2/en
2018-04-23 Application filed by Intel Corp filed Critical Intel Corp
2018-04-23 Priority to US15/960,120 priority patent/US10404994B2/en
2018-08-23 Publication of US20180242012A1 publication Critical patent/US20180242012A1/en
2019-09-03 Publication of US10404994B2 publication Critical patent/US10404994B2/en
238000009795 derivation Methods 0 description title 30
Method and apparatus for deriving a motion vector at a video decoder. A block-based motion vector may be produced at the video decoder by utilizing motion estimation among available pixels relative to blocks in one or more reference frames. The available pixels could be, for example, spatially neighboring blocks in the sequential scan coding order of a current frame, blocks in a previously decoded frame, or blocks in a downsampled frame in a lower pyramid when layered coding has been used.
This application is a continuation of U.S. patent application Ser. No. 14/737,437, filed on 11 Jun. 2015, entitled “METHODS AND SYSTEMS FOR MOTION VECTOR DERIVATION AT A VIDEO DECODER”, which is a continuation of U.S. patent application Ser. No. 12/567,540, filed on 25 Sep. 2009, now U.S. Pat. No. 9,654,792, entitled “METHODS AND SYSTEMS FOR MOTION VECTOR DERIVATION AT A VIDEO DECODER”, which is a Non-Provisional Application of U.S. Provisional Patent Application Ser. No. 61/222,984, filed on 3 Jul. 2009, entitled “METHODS AND SYSTEMS FOR MOTION VECTOR DERIVATION AT A VIDEO DECODER”, all of which are incorporated by reference in their entireties for all purposes.
Motion estimation (ME) in video coding may be used to improve video compression performance by removing or reducing temporal redundancy among video frames. For encoding an input block, traditional motion estimation may be performed at an encoder within a specified search window in reference frames. This may allow determination of a motion vector that meets a predefined requirement, such as the minimization of a metric such as the sum of absolute differences (SAD) between the input block and the reference block. The motion vector (MV) information can then be transmitted to a decoder for motion compensation. The video decoder may then utilize the received motion vector information to displace the pixels from the reference frames to form reconstructed output pixels. This displacement may be used to represent the motion compensation.
Note that in the description below, the terms “frame” and “picture” are used interchangeably, as would be understood by persons of ordinary skill in the art.
FIG. 1 illustrates MV derivation using already decoded blocks from a current frame, according to an embodiment.
FIG. 2 is a flow chart illustrating the process of MV derivation using already decoded blocks from a current frame, according to an embodiment.
FIG. 4 is a flow chart illustrating MV derivation using already decoded blocks from a current frame, where the motion search is based on blocks in two different reference frames, according to an embodiment.
FIG. 5 illustrates MV derivation using previously decoded blocks from a previously decoded frame, according to an embodiment.
FIG. 6 is a flow chart illustrating MV derivation using previously decoded blocks from a previously decoded frame, according to an embodiment.
FIG. 7 illustrates MV derivation using previously decoded blocks from already decoded previous and succeeding frames, according to an embodiment.
FIG. 8 is a flow chart illustrating MV derivation using previously decoded blocks from already decoded previous and succeeding frames, according to an embodiment.
FIG. 9 illustrates MV derivation using a previously decoded block from a lower level in a layered coding context, according to an embodiment.
FIG. 10 is a flow chart illustrating MV derivation using a previously decoded block from a lower level in a layered coding context, according to an embodiment.
FIG. 11 illustrates a computing context of an exemplary software embodiment.
FIG. 12 is a block diagram showing a self MV derivation module in the context of a H.264 encoder, according to an embodiment.
FIG. 13 is a block diagram showing a self MV derivation module in the context of a H.264 decoder, according to an embodiment.
FIG. 14 is a block diagram illustrating a system, according to an embodiment.
The following applies to video compression. The system and method described below may allow derivation of a motion vector (MV) at a video decoder. This can reduce the amount of information that needs to be sent from a video encoder to the video decoder. A block-based motion vector may be produced at the video decoder by performing motion estimation on available previously decoded pixels with respect to blocks in one or more reference frames. The available pixels could be, for example, spatially neighboring blocks in the sequential scan coding order of the current frame, blocks in a previously decoded frame, or blocks in a downsampled frame in a lower pyramid when layered coding has been used. In an alternative embodiment, the available pixels can be a combination of the above-mentioned blocks.
Already Decoded Spatially Neighboring Blocks in the Current Frame
In an embodiment, pixels that can be used to determine an MV may come from spatially neighboring blocks in the current frame, where these blocks have been decoded prior to the decoding of the target block in the current frame. FIG. 1 shows an example 100 utilizing one or more blocks 140 that are above and to the left of the target block 130 in a current frame 110. To determine a motion vector for the target block 130 that needs to be decoded in the current frame 110, motion search may be performed for one or more of the blocks 140 above and to the left of the target block 130, relative to the blocks 150 of reference frame 120, where blocks 150 correspond to blocks 140. Such an approach may be useful in decoding of predictive frames, also called P-frames, which hold only the changes relative to a previous frame.
In an embodiment, the raster scan coding order may be used to identify the spatial neighbor blocks that are above, to the left, above and to the left, and above and to the right of the target block.
Generally, this approach may be applied to available pixels of spatially neighboring blocks in the current frame, as long as the neighboring blocks were decoded prior to the target block in sequential scan coding order. Moreover, this approach may apply motion search with respect to reference frames in the reference frame list for a current frame.
The processing for this embodiment is illustrated as process 200 in FIG. 2. At 210, one or more blocks of pixels may be identified, where these blocks neighbor the target block of the current frame. Such neighboring blocks may or may not be immediately adjacent to the target block. At 220, motion search may be performed for the identified blocks. The motion search may be based on corresponding blocks of a reference frame, and at 230 yields motion vectors that connects the corresponding blocks of the reference frame to the identified blocks. Note that in an embodiment, 220 and 230 may precede 210, such that the motion vectors for the identified blocks may be known and the identified blocks decoded, prior to the identified blocks being used in this process. At 240, the motion vectors of the identified blocks are used to derive the motion vector for the target block, which may then be used for motion compensation for the target block. This derivation may be performed using any suitable process known to persons of ordinary skill in the art. Such a process may be, for example and without limitation, weighted averaging or median filtering. The process 200 concludes at 250.
FIG. 3 shows an embodiment 300 that may utilize one or more neighboring blocks 340 (shown here as blocks above and to the left of the target block 330) in a current frame 310. This may allow generation of a motion vector based on one or more corresponding blocks 350 and 355 in a previous reference frame 320 and a subsequent reference frame 360, respectively, where the terms “previous” and “subsequent” refer to temporal order. The motion vector can then be applied to target block 330. Here, the motion search may operate over an additional reference frame, in contrast to the embodiments of FIGS. 1 and 2. In an embodiment, a raster scan coding order may be used to determine spatial neighbor blocks above, to the left, above and to the left, and above and to the right of the target block. This approach may be used for bi-directional (B) frames, which use both the preceding and following frames for decoding.
The process of the embodiment of FIG. 3 is shown as process 400 of FIG. 4. At 410, one or more blocks of pixels may be identified in the current frame, where the identified blocks neighbor the target block of the current frame. At 420, motion search for the identified blocks may be performed, based on corresponding blocks in a temporally subsequent reference frame and on corresponding blocks in a previous reference frame. At 430, the motion search may result in motion vectors for the identified blocks. As in the case of FIG. 2, 420 and 430 may precede 410, such that the motion vectors of the neighboring blocks may be determined prior to identification of those blocks. At 440, the motion vectors may be used to derive the motion vector for the target block, which may then be used for motion compensation for the target block. This derivation may be performed using any suitable process known to persons of ordinary skill in the art. Such a process may be, for example and without limitation, weighted averaging or median filtering. The process concludes at 450.
Already Decoded Blocks in Previously Decoded Frames
In this embodiment, pixels that can be used to determine an MV may come from a corresponding block in a previously reconstructed frame. FIG. 5 shows an example 500 of utilizing a block 540 from a previous frame 515, where the block 540 may be in a position corresponding to a target block 530 in a current frame 510. Here the MV can be derived from the relationship between the corresponding block 540 of the previously decoded frame 515 relative to one or more blocks 550 in a reference frame 520.
The processing for such an embodiment is shown in FIG. 6 as process 600. At 610, a block of pixels may be identified in a previous frame, where the identified block corresponds to a target block of a current frame. At 620, a motion vector may be determined for the identified block relative to a corresponding block in a reference frame. In an alternative embodiment, 620 may precede 610, such that the motion vector for the block of the previous frame may be derived prior to identifying the block for use with respect to the target block of the current frame. At 630, the motion vector may be used for the target block. The process may conclude at 640.
Another embodiment may use neighboring blocks next to the corresponding block of the previous frame to do the motion search in a reference frame. Examples of such neighboring blocks could be the blocks above, below, to the left, or to the right of the corresponding block in the previously reconstructed frame.
In an alternative embodiment, the available pixels can come from the corresponding blocks of previous and succeeding reconstructed frames in temporal order. This approach is illustrated in FIG. 7 as embodiment 700. To encode a target block 730 in a current frame 710, already decoded pixels may be used, where these pixels may be found in a corresponding block 740 of a previous frame 715, and in a corresponding block 765 of a succeeding frame 755. A first motion vector may be derived for corresponding block 740, by doing a motion search through one or more blocks 750 of reference frame 720. Block(s) 750 may neighbor a block in reference frame 720 that corresponds to block 740 of previous frame 715. A second motion vector may be derived for corresponding block 765 of succeeding frame 755, by doing a motion search through one or more blocks 770 of reference frame 760. Block(s) 770 may neighbor a block in reference frame 760 that corresponds to block 765 of succeeding frame 755. Based on the first and second motion vectors, forward and/or backward motion vectors for target block 730 may be determined. These latter motion vectors may then be used for motion compensation for the target block
This process is described as process 800 of FIG. 8. At 810, a block may be identified in a previous frame, where this identified block may correspond to the target block of the current frame. At 820, a first motion vector may be determined for this identified block of the previous frame, where the first motion vector may be defined relative to a corresponding block of a first reference frame. In 830, a block may be identified in a succeeding frame, where this block may correspond to the target block of the current frame. A second motion vector may be determined at 840 for this identified block of the succeeding frame, where the second motion vector may be defined relative to the corresponding block of a second reference frame. At 850, one or two motion vectors may be determined for the target block using the respective first and second motion vectors above. Process 800 may conclude at 860.
In another embodiment, neighboring blocks next to the corresponding block in the previous and succeeding reconstructed frames may be used to do the motion search based on their respective reference frames. An example of the neighboring blocks may be the blocks above, below, to the left, or to the right of the collocated blocks in the reconstructed frames, for example. Moreover, this approach can use motion search using reference frames in the reference frame lists, in both forward and backward temporal order.
Generally, the approach of FIGS. 3 and 7 may be used in the codec processing of bi-directional (B) frames.
Already Decoded Blocks in a Downsampled Frame in a Lower Pyramid of Layered Coding
In an embodiment, pixels that can be used to determine an MV for a target block may come from corresponding blocks in a lower layer whose video is downsampled from an original input in a scalable video coding scenario. FIG. 9 shows an example 900 utilizing a lower layer block 940 corresponding to the target block 930 of the current picture 910. The block 940 may occur in a picture 915 that corresponds to current picture 910. The corresponding block 940 can be used to perform the motion search, given one or more blocks 950 and 970 in respective reference pictures 920 and 960 in the lower layer. The reference pictures in the lower layer can be the forward or backward (previous or succeeding) pictures in temporal order. Since the motion vector may be derived in the downsampled layer, the motion vector may be upscaled before it is applied to the target block 930 in the target layer.
This approach may also be applied to already-decoded blocks that are spatial neighbors to the block 940 in the lower layer corresponding to the target frame 930 in the current picture 910.
The processing of FIG. 9 is shown as a flowchart 1000 in FIG. 10. At 1010, given a target block in a current frame, a corresponding block may be identified in a corresponding frame in a lower layer. At 1020, a motion vector may be determined for the corresponding block in the lower layer, relative to one or more reference frames in the lower layer. At 1030, the determined motion vector may be used for motion estimation for the target block in the current frame. The process may conclude at 1040.
In an alternative embodiment, 1020 may precede 1010, so that the motion vector is determined at the lower layer, prior to identifying the block in the lower layer for ME purposes for the target layer.
A rate distortion optimization (RDO) model may be used to determine which coding mode is selected, given the options of motion estimation at video encoder side and motion estimation at video decoder side. The RDO model for motion estimation at the video encoder may generate a cost metric, and may include the costs of both coding distortion and MV bits, and the cost function for the motion estimation at the decoder may include only the coding distortion. In an embodiment, the video encoder may compare the costs for these two motion estimation options and determine which one to pick. In an embodiment, the video encoder may identify the chosen coding mode with a flag bit during communications between the encoder and the decoder. The video decoder may then act according to the state of the flag bit. If the flag bit indicates that motion estimation at the decoder side is utilized, the video decoder may derive the motion vector autonomously.
Such a mode selection process is illustrated in FIG. 11, as process 1100. At 1120, traditional encoder side motion estimation (ME) may first be performed to get an MV for this coding mode. At 1130, the corresponding RDO cost metric may be calculated. Let this cost be J0. At 1140, ME is performed at the decoder as described in any of the above embodiments, to get an MV for this coding mode. At 1150, the corresponding RDO cost metric may be calculated to be J1. At 1160, if J1<J0, then at 1170, the decoder side ME based result may be chosen. Otherwise, the result from the traditional ME based coding mode may be chosen at 1180. The process may conclude at 1190. In an alternative embodiment, more than two modes may be similarly evaluated, where the mode having the lowest RDO cost metric may be chosen. A flag can be used to signal the chosen mode in the communications between the encoder and decoder.
Logic to perform the processing described above may be incorporated in a self MV derivation module that is used in a larger codec architecture. FIG. 12 illustrates an exemplary H.264 video encoder architecture 1200 that may include a self MV derivation module 1240, where H.264 is a video codec standard. Current video information may be provided from a current video block 1210 in a form of a plurality of frames. The current video may be passed to a differencing unit 1211. The differencing unit 1211 may be part of the Differential Pulse Code Modulation (DPCM) (also called the core video encoding) loop, which may include a motion compensation stage 1222 and a motion estimation stage 1218. The loop may also include an intra prediction stage 1220, and intra interpolation stage 1224. In some cases, an in-loop deblocking filter 1226 may also be used in the loop.
The current video may be provided to the differencing unit 1211 and to the motion estimation stage 1218. The motion compensation stage 1222 or the intra interpolation stage 1224 may produce an output through a switch 1223 that may then be subtracted from the current video 1210 to produce a residual. The residual may then be transformed and quantized at transform/quantization stage 1212 and subjected to entropy encoding in block 1214. A channel output results at block 1216.
The output of motion compensation stage 1222 or inter-interpolation stage 1224 may be provided to a summer 1233 that may also receive an input from inverse quantization unit 1230 and inverse transform unit 1232. These latter two units may undo the transformation and quantization of the transform/quantization stage 1212. The inverse transform unit 1232 may provide dequantized and detransformed information back to the loop.
A self MV derivation module 1240 may implement the processing described herein for derivation of a motion vector from previously decoded pixels. Self MV derivation module 1240 may receive the output of in-loop deblocking filter 1226, and may provide an output to motion compensation stage 1222.
FIG. 13 illustrates an H.264 video decoder 1300 with a self MV derivation module 1310. Here, a decoder 1300 for the encoder 1200 of FIG. 12 may include a channel input 1338 coupled to an entropy decoding unit 1340. The output from the decoding unit 1340 may be provided to an inverse quantization unit 1342 and an inverse transform unit 1344, and to self MV derivation module 1310. The self MV derivation module 1310 may be coupled to a motion compensation unit 1348. The output of the entropy decoding unit 1340 may also be provided to intra interpolation unit 1354, which may feed a selector switch 1323. The information from the inverse transform unit 1344, and either the motion compensation unit 1348 or the intra interpolation unit 1354 as selected by the switch 1323, may then be summed and provided to an in-loop de-blocking unit 1346 and fed back to intra interpolation unit 1354. The output of the in-loop deblocking unit 1346 may then be fed to the self MV derivation module 1310.
The encoder and decoder described above, and the processing performed by them as described above, may be implemented in hardware, firmware, or software, or some combination thereof. In addition, any one or more features disclosed herein may be implemented in hardware, software, firmware, and combinations thereof, including discrete and integrated circuit logic, application specific integrated circuit (ASIC) logic, and microcontrollers, and may be implemented as part of a domain-specific integrated circuit package, or a combination of integrated circuit packages. The term software, as used herein, refers to a computer program product including a computer readable medium having computer program logic stored therein to cause a computer system to perform one or more features and/or combinations of features disclosed herein.
A software or firmware embodiment of the processing described above is illustrated in FIG. 14. System 1400 may include a processor 1460 and a body of memory 1410 that may include one or more computer readable media that store computer program logic 1420. Memory 1410 may be implemented as a hard disk and drive, a removable media such as a compact disk and drive, or a read-only memory (ROM) device, for example. Processor 1460 and memory 1410 may be in communication using any of several technologies known to one of ordinary skill in the art, such as a bus. Logic contained in memory 1410 may be read and executed by processor 1460. One or more I/O ports and/or I/O devices, shown as I/O 1470, may also be connected to processor 1460 and memory 1410.
Computer program logic 1420 may include decoded block identification logic 1430. This module of computer program logic, when executed on processor 1460, identifies a block of pixels that may ultimately be used to determine a motion vector for a target block. Computer program logic 1420 may also include motion vector determination logic 1440. This module of computer program logic, when executed on processor 1460, determines a motion vector on the basis of the identified block of pixels identified by decoded block identification logic 1430, relative to one or more reference frames. Computer program logic 1420 may also include motion vector application logic 1450. This module of computer program logic, when executed on processor 1460, uses the motion vector determined by logic module 1440 to perform motion estimation for the target block.
Alternatively, any of the logic modules shown in computer program logic 1420 may be implemented in hardware.
Methods and systems are disclosed herein with the aid of functional building blocks, such as those listed above, describing the functions, features, and relationships thereof. At least some of the boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries may be defined so long as the specified functions and relationships thereof are appropriately performed. In addition, the encoder and decoder described above may by incorporated in respective systems that encode a video signal and decode the resulting encoded signal respectively using the processes noted above.
1. A video decoder apparatus comprising:
determine a target block of a current frame;
identify a first block of a previous frame based on the first block being a neighbor of a second block of the previous frame collocated with the target block; and
decode the target block of the current frame based at least in part on a motion vector corresponding to the first block, wherein the motion vector is relative to a block of a reference frame distinct from the previous frame.
2. The apparatus of claim 1, wherein the processor is further to:
identify a third block of the current frame above and to the left of the target block; and
decode the target block of the current frame based at least in part on a second motion vector corresponding to the third block.
3. The apparatus of claim 2, wherein the processor is further to:
identify a fourth block of the current frame above and immediately adjacent the target block; and
decode the target block of the current frame based at least in part on a third motion vector corresponding to the fourth block.
identify a fifth block of the current frame to the left and immediately adjacent the target block; and
decode the target block of the current frame based at least in part on a fourth motion vector corresponding to the fifth block.
5. The apparatus of claim 1, wherein the previous frame is temporally previous to the current frame.
6. The apparatus of claim 1, wherein the processor is further to:
decode the first block and the second block prior to the decode of the target block of the current frame.
7. The apparatus of claim 1, wherein the current frame comprises a P-frame or a B-frame.
8. The apparatus of claim 1, wherein the processor is to decode the target block based at least in part on the motion vector in response to a flag to indicate use of at least one motion vector for the block of decoded pixels.
a display communicatively coupled to the one or more processor circuitry, the display to display an image based at least in part on the decoded target block.
11. At least one non-transitory computer-readable medium comprising instructions stored thereon, which if executed by one or more processors, cause the one or more processors to:
12. The medium of claim 11, comprising further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to:
13. The medium of claim 12, comprising further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to:
14. The medium of claim 13, comprising further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to:
15. The medium of claim 11, wherein the previous frame is temporally previous to the current frame and the medium comprises further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to decode the first block and the second block prior to the decode of the target block of the current frame.
16. A computer-implemented method of video decoding comprising:
determining a target block of a current frame;
identifying a first block of a previous frame based on the first block being a neighbor of a second block of the previous frame collocated with the target block; and
decoding the target block of the current frame based at least in part on a motion vector corresponding to the first block, wherein the motion vector is relative to a block of a reference frame distinct from the previous frame.
identifying a third block of the current frame above and to the left of the target block; and
decoding the target block of the current frame based at least in part on a second motion vector corresponding to the third block.
18. The method of claim 17, comprising further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to:
19. The method of claim 18, comprising further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to:
20. The method of claim 16, wherein the previous frame is temporally previous to the current frame and the medium comprises further instructions stored thereon, which if executed by the one or more processors, cause the one or more processors to decode the first block and the second block prior to the decode of the target block of the current frame.
US15/960,120 2009-07-03 2018-04-23 Methods and systems for motion vector derivation at a video decoder Active US10404994B2 (en)
US14/737,437 US9955179B2 (en) 2009-07-03 2015-06-11 Methods and systems for motion vector derivation at a video decoder
US15/960,120 US10404994B2 (en) 2009-07-03 2018-04-23 Methods and systems for motion vector derivation at a video decoder
US14/737,437 Continuation US9955179B2 (en) 2009-07-03 2015-06-11 Methods and systems for motion vector derivation at a video decoder
US20180242012A1 US20180242012A1 (en) 2018-08-23
US10404994B2 true US10404994B2 (en) 2019-09-03
ID=43412656
US12/567,540 Active 2030-12-06 US9654792B2 (en) 2009-07-03 2009-09-25 Methods and systems for motion vector derivation at a video decoder
US14/737,437 Active 2030-07-14 US9955179B2 (en) 2009-07-03 2015-06-11 Methods and systems for motion vector derivation at a video decoder
US15/960,120 Active US10404994B2 (en) 2009-07-03 2018-04-23 Methods and systems for motion vector derivation at a video decoder
US (3) US9654792B2 (en)
PL400344A1 (en) 2012-08-13 2014-02-17 Politechnika Poznanska Method for determining the the motion vector predictor
GB2248361A (en) 1990-09-28 1992-04-01 Sony Broadcast & Communication Motion dependent video signal processing
TW494686B (en) 1999-07-30 2002-07-11 Koninkl Philips Electronics Nv A system and method for motion compensation of image planes in color sequential displays
US20030103568A1 (en) 2001-11-30 2003-06-05 Samsung Electronics Co., Ltd. Pixel data selection device for motion compensated interpolation and method thereof
TW200708115A (en) 2005-05-16 2007-02-16 Intel Corp Parallel execution of media encoding using multi-threaded single instruction multiple data processing
CN101018334A (en) 2007-02-13 2007-08-15 武汉大学 A method for quickly implementing flexible time domain coding of the dual frame reference video stream
US20070223585A1 (en) 2006-02-28 2007-09-27 Tatsuro Fujisawa Video decoding apparatus and video decoding method
US20080101701A1 (en) 2006-10-31 2008-05-01 Roope Rainisto Communication terminal
EP1936998A2 (en) 2006-12-19 2008-06-25 Hitachi, Ltd. Decoding method and coding method
TWI298597B (en) 2004-12-30 2008-07-01 Intel Corp Method, apparatus and system for processing digital video data, and apparatus for processing video data
TWI303944B (en) 2004-10-12 2008-12-01 Via Tech Inc Audio and image data integrating system for a recording device
US20090238268A1 (en) 2008-03-20 2009-09-24 Mediatek Inc. Method for video coding
JP4352189B2 (en) 1998-08-20 2009-10-28 株式会社安川電機 Magnetic encoder and motor with magnetic encoder
US20120093217A1 (en) 2009-03-30 2012-04-19 Korea University Research And Business Foundation Method and Apparatus for Processing Video Signals
TWI357086B (en) 2008-07-03 2012-01-21 Advanced Semiconductor Eng Transformer and method for adjusting mutual induct
TWM494686U (en) 2014-09-02 2015-02-01 guo-dong Liu Multifunctional leather trimming cutter structure
2009-09-25 US US12/567,540 patent/US9654792B2/en active Active
2015-06-11 US US14/737,437 patent/US9955179B2/en active Active
2018-04-23 US US15/960,120 patent/US10404994B2/en active Active
TWI301953B (en) 2005-03-14 2008-10-11 Qisda Corp Methods and apparatuses for video encoding
US20080159398A1 (en) 2006-12-19 2008-07-03 Tomokazu Murakami Decoding Method and Coding Method
"A Comparison of Motion-Compensated Interlace-to-Progressive Conversion Methods", Signal Processing Image Communication, vol. 12, 1998, 209-229.
"New Algorithm for Nonlinear Vector-Based Upconversion with Center Weighted Medians", Journal of Electronic Imaging 6(3), Jul. 1997, 368-378.
"No English Translation Available", Second Office Action for Chinese Patent Application No. 201080068069.8 dated Oct. 20, 2015, 5 pages.
"No English Translation Available", Third Office Action for Chinese Patent Application No. 201080068069.8 dated Apr. 1, 2016, 3 pages.
"Nonlinear Vector Error Tolerant Interploation of Intermediate Video Images by Weighted Medians", Signal Processing Image Communication, vol. 14, (Search Report PCTUS00/18386), 851-868.
"Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services-Coding of moving video", International Telecommunication Union, H.262,Amendment 2, Jan. 2007, Information technology-Generic coding of moving pictures and associated audio information: Video Amendment 2: Support for colour spaces, 14 pages.
"Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video", International Telecommunication Union, H.262, Corrigendum 2, Information technology—Generic coding of moving pictures and associated audio information: Video Technical Corrigendum 2, May 2006, 14 pages.
"Series H: Audiovisual and Multimedia Systems Infrastructure of audiovisual services—Coding of moving video", International Telecommunication Union, H.262,Amendment 2, Jan. 2007, Information technology—Generic coding of moving pictures and associated audio information: Video Amendment 2: Support for colour spaces, 14 pages.
Baxes, , "Digital Image Processing: Principles and Applications", John Wiley & Sons, 1994, 88-91.
Bjontegaard, "H.26L Test Model Long Term 8 (TML-8) draft0", ITU Study Group 16, Apr. 2, 2001, 1-2, 16-19.
Bjontegaard, "Video Coding Experts Group (Question 15)", ITU-Telecommunications Standardization Section, Document Q15-F-11, Sixth Meeting: Seoul, Korea, Nov. 3-6, 1998.
Bjontegaard, "Video Coding Experts Group (Question 15)", ITU-Telecommunications Standardization Section, Document Q15-J-72, Tenth Meeting: Osaka, May 16-18, 2000.
Bjontegaard, "Video Coding Experts Group (Question 15)", ITU—Telecommunications Standardization Section, Document Q15-F-11, Sixth Meeting: Seoul, Korea, Nov. 3-6, 1998.
Bjontegaard, "Video Coding Experts Group (Question 15)", ITU—Telecommunications Standardization Section, Document Q15-J-72, Tenth Meeting: Osaka, May 16-18, 2000.
Chang, A. et al., "Fast integer motion estimation for H.264 video coding standard", 2004 IEEE International Conference on Multimedia and Expo: Jun. 27-30, 2004, vol. 1, pp. 289-292, Taipei, Taiwan, IEEE Operations Center, Piscataway, NJ.
Chang, et al., "11 Fast integer motion estimation for H.264 video coding standard", 11, 2004 IEEE International Conference on Multimedia and Expo : June 27-30, 2004, Taipei, Taiwan, IEEE Operations Center, Piscataway, NJ, vol. 1, Jun. 27, 2004.
Chiu, et al., "Report of Self Derivation of Motion Estimation Improvement in TMuC", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, URL: http://phenix,int-evry.fr/jct/doc_end_user/documents/3_Guangzhou/wg11/JCTVC-C127-m18151-v1-JCTVC-C127.zip (see JCTVC-C127.doc), Oct. 3, 2010 (13 pages).
Chiu, et al., "TE1: Fast Techniques to Improve Self Derivation of Motion Estimation", 2. JCT-VC Meeting; Jul. 21, 2010 to Jul. 28, 2010; Geneva; Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-T SG.1; URL: http//WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/; No. JCTVC-B047; Jul. 28, 2010, XP030007627; ISSN: 0000-0048.
Chiu, et al., "TE1: Report of Self Derivation of Motion Estimation Improvement in TMuC", 3. JCT-VC Meeting; 94. MPEG Meeting; Oct. 7, 2010-Oct. 15, 10; Guangzhou; (Joint Collaborative Team on Video Coding of ISO/IEC JTC1/SC29/WG11 and ITU-TSG.16); http://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE. No. JCTVC-C127, Oct. 3, 2010 XP030007834.
Combined Search and Examination Report for United Kingdom Patent Application No. 1100658.2 dated May 16, 2011, 6 pages.
Combined Search and Examination Report received for GB Patent Application No. GB1100658.2,dated May 16, 2011.
Decision of Grant for Japanese Patent Application No. 2013-540202 dated Feb. 24, 2015, 2 pages.
Decision of Rejection for Taiwan Patent Applicaiton No. 100101277 dated Jan. 22, 2014, 6 pages.
EP Office Action, dated Aug. 11, 2016, for European Patent Application No. 11830181.1.
European Office Action dated Mar. 16, 2016 for EP Patent Application No. 10860988.4.
Extended European Search Report for European Patent Application No. 11830181.1 dated Dec. 11, 2014, 9 pages.
Final Notification of Reasons for Refusal for Japanese Patent Application No. 2011-004871 dated Nov. 20, 2012, 4 pages.
Final Office Action dated Jun. 15, 2016 for U.S. Appl. No. 12/567,540.
Final Office Action for U.S. Appl. No. 12/566,823 dated Apr. 14, 2014, 13 pages.
Final Office Action for U.S. Appl. No. 12/566,823 dated Aug. 6, 2012, 18 pages.
Final Office Action for U.S. Appl. No. 12/567,540 dated Jan. 30, 2013, 12 pages.
Final Office Action for U.S. Appl. No. 12/567,540 dated Jun. 15, 2016, 14 pages.
Final Office Action for U.S. Appl. No. 12/657,168 dated Jan. 17, 2013, 16 pages.
Final Office Action for U.S. Appl. No. 13/575,233 dated Feb. 18, 2016, 14 pages.
Final Office Action for U.S. Appl. No. 13/575,233 dated May 14, 2015, 14 pages.
Final Office Action for U.S. Appl. No. 14/578,802 dated May 13, 2016, 5 pages.
Final Office Action, dated Jan. 4, 2017 for U.S. Appl. No. 13/575,233.
First Office Action and Search Report for Chinese Patent Applicaiton No. 201110056040.4 dated Sep. 2, 2013, 11 pages.
Huang, et al., "TE1: decoder-Side Motion Vector Derivation with Switchable Template Matching", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/ IEC JTC1/SC29/WG11, Document: JCTVC-B076, Jul. 21-28, 2010, 11 pages.
Kamp, Steffen et al., "Fast decoder side motion vector derivation for inter frame video coding", Picture Coding Symposium 2009, May 6, 2009, Chicago, IL.
Murakami, T. et al., "Advanced B Skip Mode with Decoder-side Motion Estimation", 37 VCEG Meeting (Video Coding Experts Group of ITU-T SG.16), Apr. 11, 2009, Yokohama, Japan.
Nisar, et al., "Fast Motion Estimation Algorithm Based on Spatio-Temporal Correlation and Direction of Motion Vectors", Electronics Letters, Dept. of Mechatronics, Gwangju Inst. of Sci & Technol. 11/23/20016, vol. 42, No. 24, 2.
Non-Final Office Action for U.S. Appl. No. 12/566,823 dated Jan. 10, 2012, 10 pages.
Non-Final Office Action for U.S. Appl. No. 12/566,823 dated Sep. 18, 2013, 12 pages.
Non-Final Office Action for U.S. Appl. No. 12/567,540 dated Apr. 6, 2012, 11 pages.
Non-Final Office Action for U.S. Appl. No. 12/567,540 dated Oct. 21, 2013, 11 pages.
Non-Final Office Action for U.S. Appl. No. 12/567,540 dated Sep. 24, 2014, 10 pages.
Non-Final Office Action for U.S. Appl. No. 12/657,168 dated Mar. 16, 2012, 15 pages.
Non-Final Office Action for U.S. Appl. No. 13/575,233 dated Dec. 18, 2014, 10 pages.
Non-Final Office Action for U.S. Appl. No. 13/575,233 dated Jul. 14, 2016, 14 pages.
Non-Final Office Action for U.S. Appl. No. 13/575,233 dated Jun. 2, 2017.
Non-Final Office Action for U.S. Appl. No. 13/575,233 dated Sep. 11, 2015, 11 pages.
Non-Final Office Action for U.S. Appl. No. 13/895,939 dated Dec. 4, 2015, 15 pages.
Non-Final Office Action for U.S. Appl. No. 13/895,939 dated Mar. 4, 2015, 10 pages.
Non-Final Office Action for U.S. Appl. No. 14/578,802 dated Dec. 23, 2015, 5 pages.
Non-Final Office Action for U.S. Appl. No. 14/737,437, dated May 5, 2017.
Notice of Allowance for Taiwan Patent Applicaiton No. 099121807 dated Aug. 12, 2015, 3 pages.
Notice of Allowance for U.S. Appl. No. 12/566,823 dated Aug. 14, 2014, 8 pages.
Notice of Allowance for U.S. Appl. No. 12/567,540, dated Jan. 11, 2017.
Notice of Allowance for U.S. Appl. No. 12/582,061 dated Jan. 30, 2013, 16 pages.
Notice of Allowance for U.S. Appl. No. 13/895,939 dated Apr. 11, 2016, 7 pages.
Notice of Allowance for U.S. Appl. No. 13/895,939 dated Sep. 12, 2014, 10 pages.
Notice of Allowance for U.S. Appl. No. 13/996,582, dated Jul. 15, 2016.
Notice of Allowance for U.S. Appl. No. 14/578,802 dated Aug. 16, 2016, 8 pages.
Notice of Allowance for U.S. Appl. No. 14/737,437 dated Dec. 27, 2017.
Notice of Last Preliminary Rejection for Korean Patent Applicaiton No. 10-2010-4254 dated Jun. 19, 2013, 11 pages.
Notice of Preliminary Rejection for Korean Patent Applicaiton No. 10-2010-4254 dated Dec. 26, 2012, 5 pages.
Notice of Preliminary Rejection for Korean Patent Applicaiton No. 10-2010-4254 dated Jun. 12, 2012, 6 pages.
Office Action and Search Report for Taiwan Patent Applicaiton No. 099121807 dated Feb. 15, 2015, 8 pages.
Office Action and Search Report for Taiwan Patent Applicaiton No. 100101277 dated Aug. 7, 2013, 19 pages.
Office Action and Search Report for Taiwan Patent Application No. 100123109 dated Oct. 22, 2014, 18 pages.
Office Action for European Patent Application No. 10860988.4 dated Mar. 16, 2016, 9 pages.
Office Action for German Patent Applicaiton No. 10 2010 025 816.4 dated Jan. 30, 2015, 11 pages.
Office Action for German Patent Application No. 10 2011 008 630.7 dated Oct. 30, 2015, 18 pages.
Office Action received for Chinese Patent Application No. 2010-10270056.0,dated May 27, 2013.
Poynton, "A Technical Introduction to Digital Video", John Wley & Sons, 1996, 8-11.
Reasons for Rejection for Japanese Patent Application No. 2011-004871 dated Aug. 14, 2012, 4 pages.
Search Report dated Jul. 26, 2016, for TW Patent Application No. 104130109.
Second Office Action for Chinese Patent Applicaiton No. 201110056040.4 dated Mar. 31, 2014, 7 pages.
Sullivan, et al., "Meeting report of the second meeting of the Joint Collaborative Team on Video Coding (JCT-VC), Geneva, CH, Jul. 21-28, 2010", URL: http://wftp3.itu.int/av-arch/jctvc-site/, Oct. 7, 2010.
Ugur (Nokia), K. et al., "11 Description of video coding technology", proposal by Tandberg, Nokia, Ericsson, 1. JCT-VC Meeting; Apr. 15, 2010-Apr. 23, 2010; Dresden; URL: http://wftp3.itu.int/av-arch/jctvc-site/, Apr. 12, 2010.
Wiseman, "An Introduction to MPEG Video Compression", members.aol.com/symbandgrl printed Apr. 14, 2000.
Y.-J. CHIU, L. XU, W. ZHANG, H. JIANG (INTEL): "TE1: Report of self derivation of motion estimation improvement in TMuC", 3. JCT-VC MEETING; 94. MPEG MEETING; 7-10-2010 - 15-10-2010; GUANGZHOU; (JOINT COLLABORATIVE TEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL: HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/, 3 October 2010 (2010-10-03), XP030007834
Y.-JEN CHIU, L. XU, W. ZHANG, H. JIANG (INTEL): "TE1: Fast techniques to improve self derivation of motion estimation", 2. JCT-VC MEETING; 21-7-2010 - 28-7-2010; GENEVA; (JOINT COLLABORATIVETEAM ON VIDEO CODING OF ISO/IEC JTC1/SC29/WG11 AND ITU-T SG.16 ); URL:HTTP://WFTP3.ITU.INT/AV-ARCH/JCTVC-SITE/, no. JCTVC-B047, JCTVC-B047, 28 July 2010 (2010-07-28), XP030007627
Yi, et al., "Fast techniques to improve seif derivation of motion estimation-presentation slides", Jul. 28, 2010 (Jul. 28, 2010), XP055255145, Retrieved from the Internet: URL:http://wftp3.itu.int/av-arch/jctvc-site/2010_07_B_Geneva/JCTVC-B047.zip.
Yi, et al., "Fast techniques to improve seif derivation of motion estimation—presentation slides", Jul. 28, 2010 (Jul. 28, 2010), XP055255145, Retrieved from the Internet: URL:http://wftp3.itu.int/av-arch/jctvc-site/2010_07_B_Geneva/JCTVC-B047.zip.
US20150350667A1 (en) 2015-12-03
US20110002390A1 (en) 2011-01-06
US9654792B2 (en) 2017-05-16
US20180242012A1 (en) 2018-08-23
US9955179B2 (en) 2018-04-24
JP2008503947A (en) 2008-02-07 Method and apparatus for forming encoded picture data and method and apparatus for decoding encoded picture data
TWI495328B (en) 2015-08-01 Methods and apparatus for adaptively choosing a search range for motion estimation