Source: https://patents.justia.com/patent/10715799
Timestamp: 2020-08-15 02:47:43
Document Index: 465133072

Matched Legal Cases: ['Application No. 60', 'Application No. 60', 'art 10', 'art 10', 'Application No. 03730648', 'Application No. 16194007']

US Patent for Methods and systems for image intra-prediction mode management Patent (Patent # 10,715,799 issued July 14, 2020) - Justia Patents Search
Justia Patents Predictor (epo)US Patent for Methods and systems for image intra-prediction mode management Patent (Patent # 10,715,799)
Jun 7, 2016 - Dolby Labs
This application is a continuation of U.S. application Ser. No. 13/672,553, filed Nov. 8, 2012, which is a continuation of U.S. application Ser. No. 12/058,679, filed Mar. 29, 2008, which is a continuation of U.S. application Ser. No. 10/404,298, filed Mar. 31, 2003, which claims the benefit of U.S. Provisional Application No. 60/319,272 filed May 28, 2002, and which claims the benefit of U.S. Provisional Application No. 60/319,390, filed Jul. 11, 2002. Each of the foregoing applications is incorporated herein by reference in its entirety.
Embodiments of the present invention relate to intra-prediction for an image. Digital video requires a large amount of data to represent each and every frame of a digital video sequence (e.g., series of frames) in an uncompressed manner. It is not feasible for most applications to transmit uncompressed digital video across computer networks because of bandwidth limitations. In addition, uncompressed digital video requires a large amount of storage space. The digital video is normally encoded in some manner to reduce the storage requirements and reduce the bandwidth requirements.
FIG. 9 is a block diagram illustrating mode estimation in some embodiments of the present invention;
A digital image may be divided into blocks for more efficient processing or for other reasons. As illustrated in FIG. 1, a target block “C” 12 may be situated adjacent to adjacent block “A,” 14| which is located immediately above target block “C” 12. Another adjacent block “B” 16 is located immediately to the left of target block “C” 12. Other blocks that share boundaries with target block “C” 12 may also be considered to be adjacent blocks to block “C” 12.
Blocks may comprise various numbers of pixels in different configurations. For example, a block may comprise a 4.times.4 array of pixels. A block may also comprise a 16.times.16 array of pixels or an 8.times.8 array. Other pixel configurations, including both square and rectangular arrays may also make up a block.
Each pixel in a target block may be predicted with reference to data regarding pixels in adjacent blocks. This adjacent pixel data or adjacent block data comprises the prediction modes used to predict those adjacent blocks or adjacent pixels. Specific adjacent pixels and pixels within a target block may be referenced using an alphanumeric index as illustrated in FIG. 2. FIG. 2 illustrates a 4.times.4 target block, such as block “C” 12 comprising 16 pixels designated by lower case alphabetic characters 22. Pixels in an adjacent block immediately above the target block are designated by capital alphabetic characters 24. Pixels in an adjacent block immediately to the left of the target block are designated by capital alphabetical characters 26. The bottom right pixel 25 in an adjacent block above and to the left of the target block 12 is designated by the capital letter “Q”.
d, j, I, p may be predicted by D
i, j, k, I, may be predicted by K
a may be predicted by (A + 2B + C + I + 2J + K + 4) >> 3 b, e may be predicted by (B + 2C + D + J + 2K + L + 4) >> 3 c, f, i may be predicted by (C + 2D + E + K + 2L + M + 4) >> 3 d, g, j, m may be predicted by (D + 2E + F + L + 2M + N + 4) >> 3 h, k, n may be predicted by (E + 2F + G + M + 2N + O + 4) >> 3 l, o may be predicted by (F + 2G + H + N + 2O + P + 4) >> 3 p may be predicted by (G + H + O + P + 2) >> 2
m may be predicted by (J + 2K + L + 2) >> 2 i, n may be predicted by (I + 2J + K + 2) >> 2 e, j, o may be predicted by (Q + 2I + J + 2) >> 2 a, f, k, p may be predicted by (A + 2Q + I + 2) >> 2 b, g, l may be predicted by (Q + 2A + B + 2) >> 2 c, h may be predicted by (A + 2B + C + 2) >> 2 d may be predicted by (B + 2C + D + 2) >> 2
a, j may be predicted by (Q + A + 1) >> 1 b, k may be predicted by (A + B + 1) >> 1 c, l may be predicted by (B + C + 1) >> 1 d may be predicted by (C + D + 1) >> 1 e, n may be predicted by (I + 2Q + A + 2) >> 2 f, o may be predicted by (Q + 2A + B + 2) >> 2 g, p may be predicted by (A + 2B + C + 2) >> 2 h may be predicted by (B + 2C + D + 2) >> 2 i may be predicted by (Q + 2I + J + 2) >> 2 m may be predicted by (I + 2J + K + 2) >> 2
a, g may be predicted by (Q + I + 1) >> 1 b, h may be predicted by (I + 2Q + A + 2) >> 2 c may be predicted by (Q + 2A + B + 2) >> 2 d may be predicted by (A + 2B + C + 2) >> 2 e, k may be predicted by (I + J + 1) >> 1 f, l may be predicted by (Q + 2I + J + 2) >> 2 i, o may be predicted by (J + K + 1) >> 1 j, p may be predicted by (I + 2J + K + 2) >> 2 m may be predicted by (K + L + 1) >> 1 n may be predicted by (J + 2K + L + 2) >> 2
a may be predicted by (2A + 2B + J + 2K + L + 4) >> 3 b, i may be predicted by (B + C + 1) >> 1 c, j may be predicted by (C + D + 1) >> 1 d, k may be predicted by (D + E + 1) >> 1 l may be predicted by (E + F + 1) >> 1 e may be predicted by (A + 2B + C + K + 2L + M + 4) >> 3 f, m may be predicted by (B + 2C + D + 2) >> 2 g, n may be predicted by (C + 2D + E + 2) >> 2 h, o may be predicted by (D + 2E + F + 2) >> 2 p may be predicted by (E + 2F + G + 2) >> 2
a may be predicted by (B + 2C + D + 2I + 2J + 4) >> 3 b may be predicted by (C + 2D + E + I + 2J + K + 4) >> 3 c, e may be predicted by (J + K + 1) >> 1 d, f may be predicted by (J + 2K + L + 2) >> 2 g, i may be predicted by (K + L + 1) >> 1 h, j may be predicted by (K + 2L + M + 2) >> 2 l, n may be predicted by (L + 2M + N + 2) >> 2 k, m may be predicted by (L + M + 1) >> 1 o may be predicted by (M + N + 1) >> 1 p may be predicted by (M + 2N + 0 + 2) >> 2
In some embodiments of the present invention, the horizontal prediction (mode 0) and the vertical prediction (mode 1) may be reversed, if desired. Also it is to be understood that the diagonal down/left prediction mode and the horizontal down prediction mode may be reversed, if desired. In addition, it is to be understood the diagonal down/right prediction (mode 5), the vertical right prediction (mode 6), the vertical left prediction (mode 7), and the horizontal up prediction (mode 8) may be reordered, if desired. Further, it is desirable that the DC prediction is between the mode 0/mode 1 set and the mode 3/mode 4 set, but may be located between mode 3/mode 4 set and mode 5/mode 6/mode 7/mode 8 set, if desired, or any other location. Moreover, the angled modes 3-8 may be renumbered as desired without significant impact on the encoding efficiency.
Unfortunately, the substitution of the previous mode numbers with the new mode numbers (e.g., a substitution of numbers into the cells of known data tables), while perhaps an improvement, still results in a generally unordered set of data
(iii) if the prediction mode of block A equals the prediction mode of block B, then intra prediction mode order for block C is {infra prediction block mode A, other modes in ascending order}.
1. A method for decoding a digital image, the method comprising:
decoding each of blocks into which an image is divided;
performing intra-prediction for predicting values of pixels located along a specified direction in a target block to be decoded;
estimating a prediction mode for the target block;
obtaining first information indicating whether the estimated prediction mode is to be selected as an intra prediction mode of the target block; and
responsive to determining that the first information indicates that the estimated prediction mode is not to be selected as the intra prediction mode of the target block, obtaining second information indicating the intra prediction mode for the target block;
the intra-prediction is based on at least a vertical prediction mode using a prediction value being a pixel value of a first block located adjacent to and above the target block, a horizontal prediction mode using a prediction value being a pixel value of a second block located adjacent to the left side of the target block, a DC prediction mode using a prediction value being an average of the pixel values of the first block located adjacent to and above the target block and the second block located adjacent to and to the left side of the target block, a Diagonal Down/Left prediction mode using the specified direction being diagonally downward to the left at approximately a 45 degree angle, a Diagonal Down/Right prediction mode using the specified direction being diagonally downward to the right at approximately a 45 degree angle as the prediction mode,
the prediction modes are numbered with increasingly larger numbers, in a predetermined order of the vertical prediction mode, the horizontal prediction mode, the DC prediction mode, the Diagonal Down/Left prediction mode and the Diagonal Down/Right prediction mode,
an estimated prediction mode is determined to have the lower mode number among the prediction mode of the first block located adjacent to and above the target block and the prediction mode of the second block located adjacent to the left side of the target block as the prediction mode for the target block, wherein a mode number associated with each of the prediction mode of the first block and the prediction mode of the second block is in accordance with the predetermined order of prediction modes.
6005622 December 21, 1999 Haskell et al.
6108448 August 22, 2000 Song et al.
6122320 September 19, 2000 Bellifemine et al.
6341144 January 22, 2002 Haskell et al.
6532306 March 11, 2003 Boon
6556625 April 29, 2003 Haskell et al.
6668095 December 23, 2003 Rackett
7236524 June 26, 2007 Sun
7289672 October 30, 2007 Sun
7450640 November 11, 2008 Kim
7706444 April 27, 2010 Conklin
7751476 July 6, 2010 Tanizawa
8228986 July 24, 2012 Sun
8233538 July 31, 2012 Sun
RE43567 August 7, 2012 Sun
8279927 October 2, 2012 Sun
8311111 November 13, 2012 Xu
8325796 December 4, 2012 Wilkins
8326075 December 4, 2012 Wilkins
8331449 December 11, 2012 Kim
8331450 December 11, 2012 Sun
8559738 October 15, 2013 Guo
8780971 July 15, 2014 Bankoski
8780996 July 15, 2014 Bankoski
8781004 July 15, 2014 Bankoski
8885706 November 11, 2014 Bankoski
9131073 September 8, 2015 Bankoski
9154799 October 6, 2015 Bankoski
9262670 February 16, 2016 Pietila
9344729 May 17, 2016 Grange
9374590 June 21, 2016 Sun
9374592 June 21, 2016 Mathew
9392272 July 12, 2016 Wang
9578324 February 21, 2017 Wang
9774868 September 26, 2017 Sun
20020044601 April 18, 2002 Haskell et al.
20030080184 May 1, 2003 Takami
20030081838 May 1, 2003 Teng
20030223495 December 4, 2003 Sun
20030223496 December 4, 2003 Sun et al.
20030223645 December 4, 2003 Sun et al.
20030231795 December 18, 2003 Karczewicz
20030233496 December 18, 2003 Sun
20060126955 June 15, 2006 Srinivasan
20080175319 July 24, 2008 Sun
20100034268 February 11, 2010 Kusakabe
20110249733 October 13, 2011 Zhao
20110249734 October 13, 2011 Segall
20160044318 February 11, 2016 Sun et al.
20160150246 May 26, 2016 Sun et al.
08/183758 July 1993 JP
08/186823 July 1996 JP
2000/295627 October 2000 JP
00/33583 June 2000 WO
Joint Committee Draft (CD), Document JVT-C167, Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, May 2002.
Greg Conklin, “New Intra Prediction Modes,” document VCEG-N54 doc, ITU-T Video Coding Experts Group (VCEG) Meeting Santa Barbara, CA, USA Sep. 24-28, 2001.
Minhua Zhou, “Scalable Intra Prediction,” document JVT-C033 doc, Joint Vide Team (JVT) of ISO/IEC MPEG & ITU-T VCEG Meeting, Fairfax, Virginia, USA May 6-10, 2002.
H.26L Test Model Long Term No. 6 (TML-6) draft0. [Online] , ITU-Telecommunications Standardization Sector Study Group 16 Video Coding Experts Group (VCEG), 2001, pp. 11-14 [retrieved on May 28, 2003].
Richardson, “H.264 I MPEG-4 Part 10 White Paper”, H.264 / MPEG-4 Part 10: Intra Prediction, www.vcodex.com, Apr. 30, 2003, 6 pages.
Bojun Meng, Oscar C. Au, “Fast Intra-Prediction Mode Selection for 4x4 Blocks in H.264”, ICASSP 2003 IEEE International conference, (Apr. 6-10, 2003), IEEE, pp. III-389-III-392.
“H.26L Test Model Long Term No. 6 (TML-6) Draft O” ITU-T Telcommunication Standarization Sector of ITU Geneva, CH, Jan. 9, 2001(Jan. 9, 2001), pp. 1-35, X00108915 *paragraph '3.4 Intra prediction mode (Intra_pred_mode)!* *tables 1,2*.
Weigland T: “Joint Model No. 1, Revision 1(JM-IRL)” ITU Study Group 16-Video Coding Experts Group, Dec. 3, 2001 (Dec. 3, 2001), pp. 1, 3-75, XP001086627 *paragraph '3.4.3 Intra Prediction Mode (Intra_pred_mode)!*.
Pan F et al: “Fast Mode Decision for Intra Prediction” Joint Video Team (JVT) of ISO/IEC MPEG & ITU-TVCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document JVT-G013, ′Online! Mar. 7, 2003 (Mar. 7, 2003), pp. 1-22, XP002325066 Retrieved from the internet: URL:http://ftp3.itu.int/av-arch/jvt-site/2003_03_Pattaya/JVT-G013.doc> 'retrieved on Mar. 23, 2005! *the whole document*.
U.S. Appl. No. 10/404,211—Office Action dated May 31, 2006.
U.S. Appl. No. 10/404,211—Office Action dated Nov. 14, 2006.
U.S. Appl. No. 10/404,211—Notice of Allowance dated Jun. 21, 2007.
U.S. Appl. No. 10/404,211—Notice of Allowance dated Aug. 2, 2007.
U.S. Appl. No. 10/404,293—Office Action dated Aug. 12, 2006.
U.S. Appl. No. 10/404,293—Notice of Allowance dated Jan. 23, 2007.
U.S. Appl. No. 10/404,293—Notice of Allowance dated Mar. 6, 2007.
U.S. Appl. No. 10/404,298—Office Action dated Sep. 20, 2006.
U.S. Appl. No. 10/404,298—Office Action dated Mar. 19, 2007.
U.S. Appl. No. 10/404,298—Office Action dated Jul. 11, 2007.
U.S. Appl. No. 10/404,298—Notice of Allowance dated Feb. 7, 2008.
European Search Report issued in European Patent Application No. EP 03 73 0648 dated May 4, 2005.
European Search Report issued in European Patent Application No. EP 05 00 5349 dated Jun. 10, 2005.
European Search Report issued in European Patent Application No. EP 06 01 9237 dated Sep. 18, 2007.
European Search Report issued in European Patent application No. EP 08 00 7745 dated Jan. 28, 2011.
European Search Report issued in European Patent Application No. EP 10 17 9562 dated Jan. 28, 2011.
European Search Report issued in European Patent Application No. EP 10 17 9572 dated Jan. 28, 2011.
Karczewicz et al., “Analysis and Simplification of Intra Prediction”, Nokia Research Center, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document: JVT-D025, Jul. 22-26, 2002, 8 pages, XP030005299.
Sun, “Intra-Prediction-Mode Ordering and Coding”, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6), Document: JVT-D027, Jul. 22-26, 2002, 8 pages, XP030005301.
Supplementary European Search Report issued in European Patent Application No. 03730648. 7 dated Apr. 22, 2005.
European Communication pursuant to Article 94(3) EPC in European Application No. 16194007.7, dated Jan. 2, 2018.
“Joint Final Committee Draft (JFCD) of Joint Video Specification (ITU-T Rec. H264|ISO/IEC 14496-10 AVC,” Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, Jul. 2002, 4th Meeting, 207 pages.
Patent number: 10715799
Patent Publication Number: 20160295212
Inventors: Shijun Sun (Vancouver, WA), Louis J. Kerofsky (Camas, WA)
Application Number: 15/175,962
Current U.S. Class: Predictor (epo) (375/E7.133)
International Classification: H04N 19/105 (20140101); H04N 19/196 (20140101); H04N 19/176 (20140101); H04N 19/60 (20140101); H04N 19/593 (20140101); H04N 19/154 (20140101); H04N 19/11 (20140101); H04N 19/44 (20140101); H04N 19/182 (20140101); H04N 19/159 (20140101); H04N 19/63 (20140101);