Source: http://www.google.com/patents/US7876818?ie=ISO-8859-1&dq=4316055
Timestamp: 2014-03-13 08:11:45
Document Index: 197611826

Matched Legal Cases: ['art 1', 'art 2', 'art 2', 'art 3', 'art 8', 'art 5', 'art 10', 'art 4', 'art 1003', 'art 1003', 'art 151', 'art 151', 'art 153', 'art 155', 'art 159', 'art 153', 'art 161', 'art 161', 'art 1012', 'art 1014', 'art 57', 'art 51', 'art 410', 'art 124', 'art, 19', 'art, 20', 'art 19', 'art 65', 'art 502', 'art 51', 'art 67', 'art 66', 'art 67', 'art 210', 'art 66', 'art 65', 'art 57', 'art 57', 'art 66', 'art 153']

Patent US7876818 - Image encoding device and image decoding device - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsMultiplexing means of an image encoding device multiplexes object display speed information or image display absolute time information, and an image decoding device performs image processing on the basis of the multiplexed object display speed information or image display absolute time information, whereby...http://www.google.com/patents/US7876818?utm_source=gb-gplus-sharePatent US7876818 - Image encoding device and image decoding deviceAdvanced Patent SearchPublication numberUS7876818 B2Publication typeGrantApplication numberUS 11/835,771Publication dateJan 25, 2011Filing dateAug 8, 2007Priority dateOct 20, 1997Also published asCN1282490A, CN1642283A, CN1652607A, CN100350803C, CN100377596C, CN100388794C, CN101267570A, CN101304523A, CN101304523B, DE69842007D1, EP1032220A1, EP1032220A4, EP1585341A2, EP1585341A3, EP1585342A2, EP1585342A3, EP1585342B1, EP1909501A2, EP1909501A3, EP1909502A2, EP1909502A3, EP1909503A2, EP1909503A3, EP2242277A1, EP2278809A2, EP2278809A3, EP2278810A2, EP2278810A3, EP2278811A2, EP2278811A3, EP2278812A2, EP2278812A3, US6983014, US7356078, US20050232364, US20050232365, US20080049833, WO1999021367A1, WO1999021370A1Publication number11835771, 835771, US 7876818 B2, US 7876818B2, US-B2-7876818, US7876818 B2, US7876818B2InventorsShinichi Kuroda, Shunichi Sekiguchi, Kohtaro Asai, Hirofumi Nishikawa, Yoshimi Isu, Yuri HasegawaOriginal AssigneeMitsubishi Denki Kabushiki KaishaExport CitationBiBTeX, EndNote, RefManPatent Citations (14), Non-Patent Citations (17), Classifications (28) External Links: USPTO, USPTO Assignment, EspacenetImage encoding device and image decoding deviceUS 7876818 B2Abstract Multiplexing means of an image encoding device multiplexes object display speed information or image display absolute time information, and an image decoding device performs image processing on the basis of the multiplexed object display speed information or image display absolute time information, whereby image decoding process can be performed smoothly and accurately.
1. An image coding device that encodes a sequence of moving images and generates an encoded bit stream, comprising:
a multiplexer configured to multiplex header information of the moving image sequence,
wherein the multiplexer is configured to multiplex display time multiplex identification information indicating whether frame rate information that uniquely identifies display time of each image contained in a moving image sequence is multiplexed into the header information of the moving image sequence, and to multiplex the frame rate information according to value of the display time multiplex identification information, wherein the frame rate information represents predetermined frame rate value or parameter value to be used to derive frame rate value, and wherein the multiplexing of time is independent of the frame rate information.
2. An image coding method for encoding a sequence of moving images and generating an encoded bit stream, comprising:
multiplexing, utilizing a multiplexer, header information of the moving image sequence;
multiplexing display time multiplex identification information indicating whether frame rate information that uniquely identifies display time of each image contained in a moving image sequence is multiplexed into the header information of the moving image sequence; and
multiplexing the frame rate information according to value of the display time multiplex identification information, wherein the frame rate information represents predetermined frame rate value or parameter value to be used to derive frame rate value, and wherein the multiplexing of time is independent of the frame rate information.
This application is a Divisional of application Ser. Nos. 11/155,483 and 11/155,611, now U.S. Pat. No. 7,356,078, both filed on Jun. 20, 2005, which are Divisionals of application Ser. No. 09/545,172 filed on Apr. 6, 2000 now U.S. Pat. No. 6,983,014, and for which priority is claimed under 35 U.S.C �120. Application Ser. No. 09/545,172 is a continuation of International Application No. PCT/JP98/00941, whose international filing date is Mar. 6, 1998. The entire contents of each of the above-identified applications are hereby incorporated by reference.
FIG. 26 is a block diagram depicting another example of the configuration of the header analysis pail of the VOP decoder part according to the eighth embodiment;
The MPEG-4 system is a system that regards a moving picture sequence as a set of moving picture objects taking arbitrary forms temporally and spatially and performs encoding and decoding for each moving picture object. In FIG. 1 there is depicted the video data structure in MPEG-4. In MPEG-4: the moving picture object containing the time axis is called a video object [Video Object (hereinafter referred to as VO)]; a component of the VO is called a video object layer {Video Object Layer (hereinafter referred to as VOL)]; a component of the VOL is called a group of video object planes (Group of Video Object Planes (hereinafter referred to as GOV)]; and image data which represents the state of the GOP at each time and forms the basic unit for encoding is called a video object plane [Video Object Plane (hereinafter referred to as VOP)] The VO corresponds, for example, to each speaker or the background in a video conference scene. The VOL forms the basic unit having inherent temporal and spatial resolutions of the speaker or background. And the VOP is image data of such a VOL at each time (corresponding to a frame). The GOV is a data structure that forms the basic unit for editing a plurality of VOLs or random access thereto; this data structure need not always be used for encoding.
In this instance, no shape data exists and only the texture data is encoded.
A description will be given below of the image encoding device of Embodiment 1. This is based on an MPEG-4 video encoder, which will hereinafter be referred to as a VOP encoder since it performs encoding for each VOP. The operation of the existing VOP encoder is disclosed, for example, in ISO/IEC JTC1/SC29/WG11/N1796, and hence it will not be described here, but instead a description will be given of a VOP encoder that contains constituents of Embodiment 1.
Next, the operation of the header multiplexing part will be described. The VO header multiplexing part 1 multiplexes VOP header information to creates a bit stream, and outputs it to the VOL header multiplexing part 2.
The VOL header multiplexing part 2 multiplexes VOL header information onto the input bit stream, and outputs the multiplexed bit stream to the GOV header multiplexing selection part 3.
A VOP start code multiplexing part 8 in the VOP header multiplexing part 5 outputs to a modulo time base multiplexing pan 9 and a VOP time increment multiplexing part 10 a bit stream obtained by multiplexing a VOP start code onto the input bit stream.
Table 3 shows examples of multiplexing of the VOP rate 1026. In this instance, when the VOP rate 1026 is 2/sec, �000� is multiplexed as the VOP rate information. When the VOP rate is 5/sec, �001� is multiplexed. When the VOP rate is 25/sec, �001� is multiplexed. When the VOP rate is 30/sec, �011� is multiplexed.
For other VOP rates (for example, when the VOP rate is 10/sec), �100� is multiplexed. Incidentally, a decision as to whether to multiplex the VOP rate information is made independently of the VOP flag value described later on. The multiplexing of the VOP rate may also be done as exemplified in Table 4. In this case, when all VOPs are related to exactly the same image in the VOL, the image is regarded as a still picture and �010� is multiplexed as the VOP rate information.
The GOV header multiplexing part 4 multiplexes the GOV header information onto the input bit stream, and outputs the multiplexed bit stream to the VOP header multiplexing part 1003 FIG. 10 illustrates the VOP header multiplexing part 1003 in detail. Reference numeral 1004 denotes a management time generating part.
A description will be given first of the configuration and operation of the image decoding device (VOP decoder) of Embodiment 3. Since the operation of the existing VOP decoder is disclosed, for example, in ISO/IEC JTC1/SC29/WG11N1796, the VOP decoder of a novel configuration according to this embodiment will be described without referring to the existing VOP decoder itself. The VOP decoder of this embodiment is one that is able to decode the encoded bit stream generated by the VOP encoder described previously with reference to Embodiment 1.
Referring to FIG. 11, the operation of the decoder will be described in detail. The encoded VOP bit stream 150 is input into the header analysis part 151, wherein the header information is analyzed following a predetermined syntax. The bit stream having the header information analyzed in the header analysis part 151 is fed into the video signal analysis part 153, wherein it is analyzed into the encoded shape data 154, the encoded texture data 157 and the motion information 158. The shape decoding part 155 decodes the encoded shape data input thereinto, and outputs the decoded shape data 156 The motion compensation part 159 generates the predictive texture data 160 from the reference data 165 read out of the memory 164 and the motion information 158 provided from the video signal analysis part 153, and provides the predictive texture data 160 to the texture decoding part 161. Based on the encoded texture data 157 and the predictive texture data 160, the texture decoding part 161 reconstructs image data by the method prescribed in MPEG-4, generating the decoded texture data 162. The decoded texture data 162 is written in the memory 164 so that it is used afterward for VOP decoding.
In this instance, the decoded VOP images a204 to c206 are all mapped into the first image frame at each second in the decoded image 211; the decoded VOP image a204 is mapped into every five image frames including the first at each second; the decoded VOP image b205 is mapped into every 10 image frames including the first at each second; and the decoded VOP image c206 is mapped into every 15 images frames including the first at each second. By this, it is possible to display a pictorial image with a plurality of objects synthesized in the image frames in accordance with their display speeds.
Based on the count value and the VOP rate information fed thereto from the counter part 1012 a and the time code 1011, the decision means 1012 b calculates the absolute time that the VOP candidate for decoding has. For example, in the case where the count value is 4, the VOP rate is 2/sec and the absolute time is 0 h10 m0 sec0 msec, the absolute of the VOP candidate for decoding has is calculated to be 0 h10 m02 sec0 msec. If the thus calculated absolute time of the VOP candidate for decoding and the externally-set display control information 1009 are equal to each other, the VOP is decided to be decoded.
The VOP time increment analysis part 1014 analyzes the VOP time increment, and outputs a bit stream to the video information header analysis part 57. The video information header analysis pan 57 analyzes the video information header, and outputs a bit stream to the start code analysis part 51.
As described above, the decoder according to Embodiment 5 has control means which controls the image reconstruction by specifying the display time of the image at each time for decoding on the basis of the display speed information when the display speed identification information decoded by the display speed information decoding means indicates a fixed speed and on the basis of display time information multiplexed for each image at each time in the case where the display speed identification information indicates a variable speed.
Assume that 01:01:00 is decoded as the time code of the first VOP of the decoded object image c406 Assuming that the time code of the first image frame of the decoded image 411 defined in the composition part 410 is 01:00:00, the decoded object image a404 is mapped into the first frame of the decoded image 411, the decoded object image b405 is mapped 10 seconds after the first frame of the decoded image 411, and the decoded object image c406 is mapped one minute after the first frame of the decoded image 411; thus, the decoded objects can be displayed in the respective flames. By this, it is possible to display a pictorial image with a plurality of video objects synthesized in the image frames in correspondence to the reference absolute times.
Embodiment 9 A ninth embodiment (Embodiment 9) of the present invention is directed to a VOP encoder that implements an improved scheme for encoding the modulo time base (corresponding to first time information) which is used in combination with the VOP time increment (corresponding to second time information) in MPEG-4
FIG. 28 illustrates the internal configuration of the header multiplexing part 124 in Embodiment 9 Reference numeral 500 denotes a VOP header multiplexing part, 19 a bit length calculating part, 20 a modulo time base, 21 a shifted modulo time base, 22 an information bit indicating a repeat count, and 501 a modulo time base multiplexing part.
The operation of the bit length calculation part 19 will be concretely described below. With the abovesaid threshold value set at 4, if the modulo time base 20 is �1111111110,� the shift-repeat count is two and the shifted modulo time base 21 is �10,� If expressed by a fixed two-bit length, the information bit 22 indicating the shift-repeat count is �10,�
The modulo time base analysis part 65 in the VOP header analysis part 502 analyzes the shifted modulo time base 69 and the information bit 70 indicating the shift-repeat count contained in the bit stream fed from the start code analysis part 51, and outputs the shifted modulo time base 69 and the information bit 70 indicating the shift-repeat count to the modulo time base calculation part 67 and the bit stream to the VOP time increment analysis part 66 The modulo time base calculation part 67 calculates the modulo time base from the shifted modulo time base 69 and the information bit 70 indicating the shift-repeat count, and outputs it to the composition part 210. More specifically, the value of the modulo time base is restored by reversing the procedure described previously with reference to Embodiment 9. In the case where a preset positive threshold value (The decoder side also required to set exactly the same value as the threshold value described in respect of the encoder of Embodiment 9) and the shifted modulo time base 69 is �10� and the information bit 70 indicating the shift-repeat count is �10,� �1111111110� with �11111111� added to the high-order bit of �10� is the restored value of the modulo time base. The thus obtained restored value of the modulo time base is used to define the display time of the VOP concerned, together with the VOP time increment information.
The VOP time increment analysis part 66 analyzes the VOP time increment contained in the bit stream fed from the modulo time base analysis part 65, and outputs the analyzed bit stream to the video inform-nation header analysis part 57. The video information header analysis part 57 analyzes the video information header contained in the bit stream fed from the VOP time increment analysis part 66, and outputs the analyzed bit stream to the video signal analysis part 153.
In the case where the modulo time base of the immediately previously encoded VOP is �11110� (decimal numeral: 30) and the modulo time base of the VOP to be encoded is �11110� (decimal numeral: 62), the difference bit string becomes �100000� (decimal numeral: 32). Then, the number of bits �1� contained in the thus calculated difference bit string �100000� is one. In the case of calculating the difference modulo time base by such a conversion table as Table 2, the difference modulo time base corresponding to one bit �1� is �10� and consequently, �10� is output as the difference modulo time base Table 2 is an example of the conversion table, and other conversion tables may also be defined.
As described above, the encoder according to Embodiment 11 is adapted to express the modulo time base as the difference modulo time base and multiplex the difference modulo time base instead of encoding the modulo time base in the form presently prescribed in MPEG-4; hence, the amount of information generated can be made smaller than in the case of using the method prescribed in MPEG-4
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5748789 *Oct 31, 1996May 5, 1998Microsoft CorporationTransparent block skipping in object-based video coding systemsUS5790196Feb 14, 1997Aug 4, 1998Mitsubishi Electric Information Technology Center America, Inc.Adaptive video coding methodUS5969764Jul 18, 1997Oct 19, 1999Mitsubishi Electric Information Technology Center America, Inc.Adaptive video coding methodUS5990957Nov 4, 1997Nov 23, 1999Samsung Electronics Co., Ltd.Video signal bit amount control using adaptive quantizationUS6011868 *Apr 4, 1997Jan 4, 2000Hewlett-Packard CompanyBitstream quality analyzerUS6028539Feb 4, 1998Feb 22, 2000Matsushita Electric Industrial Co., Ltd.Buffer control method, and decoding apparatus performing buffer controlUS6031575Mar 19, 1997Feb 29, 2000Sony CorporationMethod and apparatus for encoding an image signal, method and apparatus for decoding an image signal, and recording mediumUS6057884Jun 5, 1997May 2, 2000General Instrument CorporationTemporal and spatial scaleable coding for video object planesUS6075576Jul 3, 1997Jun 13, 2000Matsushita Electric Industrial Co., Ltd.Method for display time stamping and synchronization of multiple video object planesUS6233356Jul 7, 1998May 15, 2001At&T Corp.Generalized scalability for video coder based on video objectsUS6266370Aug 29, 1997Jul 24, 2001Nippon Telegraph And Telephone CorporationBrightness-variation compensation method and coding/decoding apparatus for moving picturesJPH07107514A Title not availableJPH08294127A Title not availableJPH09116897A Title not available* Cited by examinerNon-Patent CitationsReference1"MPEG97/N1796: Bitstream Syntax," ISO/IEC JTC1/SC29/WG11, Jul. 1997, pp. 114-168, XP002243400.2"Video Bitstream Syntax and Semantics, ISO Standard 13818.2, Jul. 1, 1995, pp. 23-62, XP002243439".3"Video Bitstream Syntax and Semantics," ISO Standard 13818.2, Jul. 1, 1995, pp. 23-62, XP002243439.4Aign et al., MPEG-4 Video Transmission via DAB: Error Detection and Error Concealment and "MPEG-4 Video Verification Model Version 7.0," ISO-IEC/JTC1/SC29/WG11 N1642, Apr. 1997, Section 2.1, MPEG-4 Video Coding.5D. K. Fibush, "Timing and Synchronization Using MPEG-2 Transport Streams," SMPTE Journal, SMPTE Inc., Scarsdale, N.Y., U.S. vol. 105, No. 7, Jul. 1, 1996, pp. 395-400, XP000597142.6Ebrahimi, T., C. Horne, J. Ostermann and E.S. Jang, "Working Draft 4.0 of ISO/IEC 14496-2," International Organisation for Standardisation, Video & SNHC Groups, Jul. 27, 1997, pp. i-x and 1-140, Stockholm.7Ebrahimi, T., C. Horne, J. Ostermann, A. Puri and Y. Nakaya, "MPEG-4 Version 2 Visual Working Draft Rev 2.0," International Organisation for Standardisation, Video & SNHC Groups, Feb. 6, 1998, pp. i-xi and 1-42, San Jose.8European Search Report for EP 05013384, dated Apr. 12, 2006.9Fibush, David K., Timing and Synchronization Using MPEG-2 Transport Streams, Jul. 1996, SMPTE Journal, pp. 395-400.10Fribourg, Coding of Moviing Pictures and Audio, Oct. 1997, International Organisation For Standardisation Organisation Internationale De Normalisation, ISO/IEC JTC1/SC29/WG11, MPEG 97/2688 (XP-002374597).11Fribourg, Coding of Moving Pictures and Audio, Oct. 1997, International Organisation For Standardisation Organisation Internationale De Normalisation, ISO/IEC JTC1/SC29/WG11, MPEG97/M2747 (XP-002374596).12MPEG-2, 1995-2006 Tsinghua Tongfang Optical Disc. Co., Ltd., Apr. 30, 1996.13MPEG-4 Requirements, International Organisation for Standardisation Organisation Internationale De Normalisation ISO/IEC JTC1/SC29/WG11, Coding of Moving Pictures and Audio, Version 3, Apr. 1997, pp. 1-25, XP-001091541.14Puri, MPEG-4: An Object-based Multimedia Coding Standard Supporting Mobile Applications, Aug. 1997, ACM Mobile Networks and Applications Journal, Special Issue on Mobile Multimedia Communications, see Fig. 11, Section 4.1, Section 6.4.15Sikora, The MPEG-4 Video Standard Verification Model, Feb. 1997, IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, No. 1, pp. 19-31.16Supplementary Partial European Search Report, dated Jun. 5, 2003 or EP 98 90 5800.17Tan, T.K., and S.M. Shen, "VOP Time Increment Resolution," International Organisation for Standardisation, Proposal, Jul. 1997, pp. 1-3.Classifications U.S. Classification375/240.02, 375/240.01International ClassificationH04N7/12, H04N7/52, H04N7/24, H04N7/50, H04N7/26, H04N7/32, H04N7/62Cooperative ClassificationH04N19/00545, H04N19/00884, H04N19/00781, H04N19/00387, H04N21/44012, H04N21/234318, H04N7/52, H04N21/23412, H04N21/4307European ClassificationH04N7/50, H04N7/26J2A, H04N21/2343J, H04N7/52, H04N7/26A10S, H04N21/43S2, H04N21/234C, H04N21/44R, H04N7/26J, H04N7/26YRotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google