Source: http://www.google.com/patents/US7973806?dq=3691140
Timestamp: 2017-04-25 15:10:34
Document Index: 68363823

Matched Legal Cases: ['Application No. 2005', 'Application No. 2005', 'Application No. 2005', 'Application No. 2005', 'Application No. 2005', 'Application No. 2005', 'Application No. 2005']

Patent US7973806 - Reproducing apparatus capable of reproducing picture data - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA reproducing apparatus includes a graphics processing unit that outputs graphics data of an RGB color space, a video decoder that outputs video data of a YUV color space, a conversion unit that converts a color space of the graphics data from the RGB color space to the YUV color space, a blending process...http://www.google.com/patents/US7973806?utm_source=gb-gplus-sharePatent US7973806 - Reproducing apparatus capable of reproducing picture dataAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS7973806 B2Publication typeGrantApplication numberUS 12/758,743Publication dateJul 5, 2011Filing dateApr 12, 2010Priority dateJan 4, 2005Fee statusPaidAlso published asUS7728851, US20060176312, US20100194993Publication number12758743, 758743, US 7973806 B2, US 7973806B2, US-B2-7973806, US7973806 B2, US7973806B2InventorsShinji KunoOriginal AssigneeKabushiki Kaisha ToshibaExport CitationBiBTeX, EndNote, RefManPatent Citations (52), Non-Patent Citations (52), Referenced by (3), Classifications (25), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetReproducing apparatus capable of reproducing picture data
US 7973806 B2Abstract
This application is a continuation of U.S. patent application 11/326,108, entitled “REPRODUCING APPARATUS CAPABLE OF REPRODUCING PICTURE DATA,” and filed on Dec. 30, 2005, which is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2005-000250, filed Jan. 4, 2005, the entire contents of both of which are incorporated herein by reference.
The HD-standard main video has a resolution of, e.g., 1920×1080 pixels or 1280×720 pixels. Each of the sub-video data, sub-picture data and navigation data has a resolution of, e.g., 720×480 pixels.
The GPU 120 is a graphics controller that generates graphics data (also referred to as graphics picture data), which forms a graphics screen picture, from data that is written by the CPU 11 in the video memory (VRAM) 131 that is assigned to the partial memory area of the main memory 13. The GPU 120 generates graphics data using a graphics arithmetic function such as bit block transfer. For example, in a case where picture data (sub-video, sub-picture, navigation) are written in three planes in the VRAM 131 by the CPU 11, the GPU 120 executes a blending process, with use of bit block transfer, which blends the picture data corresponding to the three planes on a pixel-by-pixel basis, thereby generating graphics data for forming a graphics screen picture with the same resolution (e.g., 1920×1080 pixels) as the main video. The blending process is executed using alpha data that are associated with the picture data of sub-video, sub-picture and navigation, respectively. The alpha data is a coefficient representative of the degree of transparency (or non-transparency) of each pixel of the associated picture data. The alpha data corresponding to the sub-video, sub-picture and navigation are multiplexed on the stream along with the picture data of the sub-video, sub-picture and navigation. Specifically, each of the sub-video, sub-picture and navigation included in the stream contains picture data and alpha data.
In one embodiment, the video decoder 25 is a decoder that supports the H.264/AVC standard. The video decoder 25 decodes HD-standard main video data and generates a digital YUV video signal that forms a video screen picture with a resolution of, e.g., 1920×1080 pixels. The digital YUV video signal is sent to the blending process unit 30.
The main video data (MAIN VIDEO) is sent to the video decoder 25 via the PCI bus 21. The main video data (MAIN VIDEO) is decoded by the video decoder 25. The decoded main video data has a resolution of 1920×1080 pixels according to the HD standard, and is sent to the blending process unit 30 as a digital YUV video signal. The main audio data (MAIN AUDIO) is sent to the audio decoder 24 via the PCI bus 21. The main audio data (MAIN AUDIO) is decoded by the audio decoder 24. The decoded main audio data (MAIN AUDIO) is sent to the audio mixer 31 as an I2S-format digital audio signal.
The GPU 120 generates graphics data for forming a graphics screen picture of 1920×1080 pixels, on the basis of the decoded results of the sub-picture decoder 104, sub-video decoder 105 and advanced navigation decoder 106, that is, picture data corresponding to the sub-picture data, sub-video data and advanced navigation data, which are written in the VRAM 131 by the CPU 11. In this case, the three picture data corresponding to the sub-picture data, sub-video data and advanced navigation data are blended by an alpha blending process that is executed by a mixer (MIX) unit 121 of the GPU 120.
In this alpha blending process, alpha data corresponding to the three picture data written in the VRAM 131 are used. Specifically, each of the three picture data written in the VRAM 131 contains RGB data and alpha data. The mixer (MIX) unit 121 executes the blending process on the basis of the alpha data of the three picture data and position information of each of the three picture data, which is told from the CPU 11. Thereby, the mixer (MIX) unit 121 generates a graphics screen picture, which includes, for instance, the three picture data that are at least partly blended. As regards an area where the picture data are blended, new alpha data corresponding to the area is calculated by the mixer (MIX) unit 121. The colors of the pixels in that area in the graphics screen picture of 1920×1080 pixels, which includes no effective picture data, are black. The alpha value corresponding to the pixels in the area, which includes no effective picture data, is a value (alpha=0) that indicates that these pixels are transparent.
In this way, the GPU 120 generates the graphics data (RGB) that form the graphics screen picture of 1920×1080 pixels, and the alpha data corresponding to the graphics data, on the basis of the decoded results of the sub-picture decoder 104, sub-video decoder 105 and advanced navigation decoder 106. As regards a scene in which only one of the pictures of the sub-picture data, sub-video data and advanced navigation data, or the GPU 120 generates graphics data that corresponds to a graphics screen picture, in which the picture (e.g., 720×480) is disposed on the surface of 1920×1080 pixels, and alpha data corresponding to the graphics data.
Assume now that main video data (Video) with a resolution of 1920×1080 pixels was input to the blending process unit 30 as picture data C, and graphics data with a resolution of 1920×1080 pixels was input to the blending process unit 30 as picture data G. In this case, on the basis of alpha data (A) with a resolution of 1920×1080 pixels, the blending process unit 30 executes an arithmetic operation for overlaying the picture data G on the picture data C in units of a pixel. This arithmetic operation is executed by the following equation (1):
V=α×G+(1−α)C (1)
Assume now that graphics data with a resolution of 1920×1080 pixels is generated from the sub-picture data and sub-video data that are written in the VRAM 131. Each of the sub-picture data and sub-video data has a resolution of, e.g., 720×480 pixels. In this case, each of the sub-picture data and sub-video data is accompanied with alpha data with a resolution of, e.g., 720×480 pixels.
G=Go×αo+Gu(1−αo) (2)
α=αo+αu×(1−αo) (3)
In this way, the MIX unit 121 of the GPU 120 blends the sub-picture data and sub-video data by using that one of the alpha data corresponding to the sub-picture data and the alpha data corresponding to the sub-video data, which is to be used as the oversurface. Thereby, the MIX unit 121 generates graphics data for forming a screen picture of 1920×1080 pixels. Further, the MIX unit 121 of the GPU 120 calculates the alpha value of each pixel of the graphics data for forming a screen picture of 1920×1080 pixels, on the basis of the alpha data corresponding to the sub-picture data and the alpha data corresponding to the sub-video data.
Specifically, the MIX unit 121 of the GPU 120 executes the blending process for blending a surface of 1920×1080 pixels (the color of pixels=black, the alpha value of pixels=0), a surface of sub-video data of 720×480 pixels, and a surface of sub-picture data of 720×480 pixels. Thereby, the MIX unit 121 calculates graphics data for forming a screen picture of 1920×1080 pixels, and alpha data of 1920×1080 pixels. The surface of 1920×1080 pixels is used as a lowest surface, the surface of the sub-video data is used as a second lowest surface, and the surface of the sub-picture data is used as an uppermost surface.
In the screen picture of 1920×1080 pixels, the color of each pixel in the area, where neither sub-picture data nor sub-video data is present, is black. The color of each pixel in the area, where only sub-picture data is present, is the same as the normal color of each associated pixel of the sub-picture data. Similarly, the color of each pixel in the area, where only sub-video data is present, is the same as the normal color of each associated pixel of the sub-video data.
In the screen picture of 1920×1080 pixels, the alpha value corresponding to each pixel in the area, where neither sub-picture data nor sub-video data is present, is zero. The alpha value of each pixel in the area, where only sub-picture data is present, is the same as the normal alpha value of each associated pixel of the sub-picture data. Similarly, the alpha value of each pixel in the area, where only sub-video data is present, is the same as the normal alpha value of each associated pixel of the sub-video data.
FIG. 6 shows a state in which sub-video data of 720×480 pixels is overlaid on main video data of 1920×1080 pixels.
In FIG. 6, graphics data is generated by a blending process that blends a surface of 1920×1080 pixels (the color of pixels=black, the alpha value of pixels=0) and a surface of sub-video data of 720×480 pixels on a pixel-by-pixel basis.
In the graphics data of 1920×1080 pixels, the alpha value of each pixel in the area, where the sub-video data of 720×480 pixels is absent, is zero. Accordingly, the area where the sub-video data of 720×480 pixels is absent is transparent. In this area, the main video data is displayed with the degree of non-transparency of 100%.
Each pixel of the sub-video data of 720×480 pixels is displayed on the main video data with a degree of transparency that is designated by the alpha data corresponding to the sub-video data. For example, a pixel of sub-video data with an alpha value=1 is displayed with 100% non-transparency, and a pixel of main video data corresponding to this pixel position is not displayed.
As is shown in FIG. 7, main video data, which is reduced to a resolution of 720×480 pixels, can be displayed on a partial area of sub-video data that is enlarged to a resolution of 1920×1080 pixels.
Specifically, in accordance with an instruction from the CPU 11, the GPU 120 executes such a scaling process as to gradually increase the resolution (picture size) of sub-video data up to 1920×1080 pixels. This scaling process is executed using pixel interpolation. As the resolution of the sub-video data becomes higher, the size of the area where the sub-video data of 720×480 pixels is not present (i.e. area with alpha value=0) gradually decreases within the graphics data of 1920×1080 pixels. Thereby, the size of the sub-video data, which is overlaid on the main video data and displayed, gradually increases, while the size of the area with the alpha value=0 gradually decreases. If the resolution (picture size) of the sub-video data reaches 1920×1080 pixels, the GPU 120 executes a blending process that overlays, on a pixel-by-pixel basis, a surface of, e.g., 720×480 pixels (the color of pixels=black, the alpha value of pixels=0) on the sub-video data of 1920×1080 pixels. Thus, the area of 720×480 pixels with the alpha value=0 is disposed on the sub-video data of 1920×1080 pixels.
On the other hand, in accordance with an instruction from the CPU 11, the video decoder 25 executes the scaling process that reduces the resolution of main video data to 720×480 pixels.
The main video data that is reduced to 720×480 pixels is displayed on an area of 720×480 pixels with the alpha value=0, which is disposed on the sub-video data of 1920×1080 pixels. Specifically, the alpha data that is output from the GPU 120 can also be used as a mask for limiting the area where the main video data is to be displayed.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5027212Dec 6, 1989Jun 25, 1991Videologic LimitedComputer based video/graphics display systemUS5809245Jan 23, 1996Sep 15, 1998Kabushiki Kaisha ToshibaMultimedia computer systemUS5890190Jun 7, 1995Mar 30, 1999Intel CorporationFrame buffer for storing graphics and video dataUS5912710Dec 18, 1997Jun 15, 1999Kabushiki Kaisha ToshibaSystem and method for controlling a display of graphics data pixels on a video monitor having a different display aspect ratio than the pixel aspect ratioUS6128015Jun 30, 1998Oct 3, 2000Kabushiki Kaisha ToshibaMultimedia computer systemUS6230209Dec 28, 1999May 8, 2001Kabushiki Kaisha ToshibaMultimedia computer systemUS6311204Oct 11, 1996Oct 30, 2001C-Cube Semiconductor Ii Inc.Processing system with register-based process sharingUS6356277Jan 22, 1998Mar 12, 2002Seiko Epson CorporationYUV-RGB digital conversion circuit and picture display device and electronic equipment using the sameUS6377309Jan 10, 2000Apr 23, 2002Canon Kabushiki KaishaImage processing apparatus and method for reproducing at least an image from a digital data sequenceUS6463208Dec 22, 1997Oct 8, 2002Lg Electronics Inc.Optical disc reproducing apparatus and methodUS6486922Feb 8, 1999Nov 26, 2002Matsushita Electric Industrial Co., Ltd.Synchronization method of image decoderUS6552750Mar 21, 2000Apr 22, 2003Tvia, Inc.Apparatus for improving the presentation of graphics data on a television displayUS6606127Jun 9, 2000Aug 12, 2003Enseo, Inc.System and method for synchronizing, storing and accurately reproducing video signalsUS6741617Dec 13, 1999May 25, 2004Koninklijke Philips Electronics N.V.Arrangement for decoding digital video signalsUS6828987Aug 7, 2001Dec 7, 2004Ati Technologies, Inc.Method and apparatus for processing video and graphics dataUS6912350Dec 8, 1999Jun 28, 2005Intel CorporationDVD subpicture rendering without loss of color resolutionUS6947485Nov 20, 2001Sep 20, 2005Stmicroelectronics, Inc.System, method and apparatus for an instruction driven digital video processorUS7035531Jan 18, 2001Apr 25, 2006Cyberlink Corp.Device and method for supplying commentary informationUS7068324Jan 8, 2001Jun 27, 2006Micronas GmbhSystem for displaying graphics in a digital television receiverUS7136078Aug 31, 2004Nov 14, 2006Via Technologies Inc.Connection device capable of mixing an RGB graphics signal and a YUV video signal and related methodUS7365757Dec 17, 1998Apr 29, 2008Ati International SrlMethod and apparatus for independent video and graphics scaling in a video graphics systemUS7539391Jun 27, 2002May 26, 2009Nxp B.V.Method and apparatus for trick-mode support of audio/video/data streams with conditional accessUS20030194213Apr 15, 2002Oct 16, 2003Schultz Mark AlanDisplay of closed captioned information during video trick modesUS20040001402Apr 14, 2003Jan 1, 2004Pioneer CorporationInformation recording apparatus and information recording methodUS20040207755Apr 16, 2004Oct 21, 2004Tzu-Ping LinApparatus and method for signal prcoessing of format conversion and combination of video signalsUS20040233215Jun 24, 2004Nov 25, 2004Dawson Thomas PatrickGraphics and video integration with alpha and video blendingUS20050122341Jan 20, 2005Jun 9, 2005Broadcom CorporationVideo and graphics system with parallel processing of graphics windowsUS20050140695Dec 24, 2003Jun 30, 2005Dunton Randy R.Method and apparatus to communicate graphics overlay informationUS20050185928Oct 1, 2004Aug 25, 2005Samsung Electronics Co., Ltd.Storage medium storing interactive graphics stream activated in response to user's command, and reproducing apparatus for reproducing from the sameUS20050281342Jun 22, 2004Dec 22, 2005Broadcom CorporationSlow motion and high speed for digital videoJP2000305546A Title not availableJP2000324419A Title not availableJP2001053956A Title not availableJP2001320673A Title not availableJP2002199277A Title not availableJP2002297198A Title not availableJP2002325171A Title not availableJP2003162276A Title not availableJP2003179850A Title not availableJP2003224862A Title not availableJP2004328226A Title not availableJP2004328718A Title not availableJPH0946722A Title not availableJPH1153580A Title not availableJPH06335022A Title not availableJPH08205092A Title not availableJPH10145780A Title not availableJPH10320922A Title not availableJPH11168701A Title not availableJPH11203444A Title not availableWO2004049710A1Nov 14, 2003Jun 10, 2004Sony CorporationReproduction device, reproduction method, reproduction program, and recording mediumWO2004090860A1Apr 1, 2004Oct 21, 2004Matsushita Electric Industrial Co., Ltd.Video combining circuitNon-Patent CitationsReference1Advisory Action issued Sep. 24, 2008 in U.S. Appl. No. 11/326,585 in 3 pages.2Amendment after Final filed Sep. 30, 2008 in U.S. Appl. No. 11/326,105 in 9 pages.3Amendment after Final filed Sep. 9, 2008 in U.S. Appl. No. 11/326,585 in 10 pages.4Amendment Appeal Brief filed Jan. 14, 2010 in U.S. Appl. No. 11/326,105 in 23 pages.5Amendment filed Dec. 2, 2008 in U.S. Appl. No. 11/326,108 in 10 pages.6Amendment filed Jan. 16, 2009 in U.S. Appl. No. 11/326,105 in 9 pages.7Amendment filed Jan. 18, 2008 in U.S. Appl. No. 11/326,108 in 8 pages.8Amendment filed Mar. 30, 2009 in U.S. Appl. No. 11/326,585 in 10 pages.9Amendment filed Mar. 31, 2008 in U.S. Appl. No. 11/326,105 in 9 pages.10Amendment filed Mar. 31, 2008 in U.S. Appl. No. 11/326,585 in 10 pages.11Amendment filed Sep. 25, 2009 in U.S. Appl. No. 11/326,106 in 8 pages.12An English Translation of Final Notice of Rejection mailed by the Japan Patent office for Japanese Patent Application No. 2005-000246 on Sep. 8, 2009.13An English Translation of Final Notice of Rejection mailed by the Japan Patent office for Japanese Patent Application No. 2005-000247 on Sep. 8, 2009.14An English Translation of Notice of Reasons for Rejection mailed by the Japan Patent Office for Japanese Patent Application No. 2005-000248 on Aug. 18, 2009.15An English Translation of Notification of Reasons for Rejection mailed by Japan Patent Office for Japanese Patent Application No. 2005-000246 on May 26, 2009.16An English Translation of Notification of Reasons for Rejection mailed by Japan Patent Office for Japanese Patent Application No. 2005-000247 on May 12, 2009.17An English Translation of Notification of Reasons for Rejection mailed by Japan Patent Office for Japanese Patent Application No. 2005-000248 on May 12, 2009.18Appeal Brief filed Dec. 23, 2009 in U.S. Appl. No. 11/326,105 in 23 pages.19Appeal Brief filed Dec. 30, 2009 in U.S. Appl. No. 11/326,585 in 24 pages.20English Translation of Communication from the Japanese Patent Office dated Jan. 26, 2010 for Japanese Patent Application 2005-000247.21Examiner's Answer to Appeal Brief Filed on Dec. 23, 2009, mailed Apr. 15, 2010, issued in U.S. Appl. No. 11/326,105 in 21 pages.22Final Office Action dated Apr. 17, 2008 issued in U.S. Appl. No. 11/326,108 in 15 pages.23Final Office Action dated Apr. 7, 2009 issued in U.S. Appl. No. 11/326,105 in 25 pages.24Final Office Action dated Jan. 25, 2010 issued in U.S. Appl. No. 11/326,106 in 13 pages.25Final Office Action dated Jul. 31, 2008 issued in U.S. Appl. No. 11/326,105 in 17 pages.26Final Office Action dated Jul. 6, 2009 issued in U.S. Appl. No. 11/326,585 in 25 pages.27Final Office Action dated Jul. 9, 2008 issued in U.S. Appl. No. 11/326,585 in 17 pages.28Final Office Action dated Mar. 11, 2009 issued in U.S. Appl. No. 11/326,108 in 14 pages.29Final Office Action dated Nov. 12, 2010, issued in U.S. Appl. No. 11/326,106 in 12 pages.30Non-Final Amendment filed Aug. 27, 2010 in U.S. Appl. No. 11/326,106 in 8 pages.31Notice of Allowance dated Aug. 24, 2009 issued in U.S. Appl. No. 11/326,108 in 6 pages.32Notice of Allowance dated Jan. 11, 2010 issued in U.S. Appl. No. 11/326,108 in 9 pages.33Notice of Appeal and Pre Appeal Request for Conference filed Sep. 3, 2009 in U.S. Appl. No. 11/326,105 on in 7 pages.34Notice of Appeal and Pre-Appeal Brief Request for Review filed Sep. 30, 2009 in U.S. Appl. No. 11/326,585 in 7 pages.35Office Action dated Dec. 3, 2007 issued in U.S. Appl. No. 11/326,105 in 14 pages.36Office Action dated Dec. 31, 2007 issued in U.S. Appl. No. 11/326,105 in 14 pages.37Office Action dated Dec. 31, 2007 issued in U.S. Appl. No. 11/326,585 in 11 pages.38Office Action dated Dec. 31, 2008 issued in U.S. Appl. No. 11/326,585 in 19 pages.39Office Action dated Jun. 26, 2009 issued in U.S. Appl. No. 11/326,106 in 12 pages.40Office Action dated Mar. 29, 2010 issued in U.S. Appl. No. 11/326,585 in 21 pages.41Office Action dated Oct. 17, 2008 issued in U.S. Appl. No. 11/326,105 in 20 pages.42Office Action dated Oct. 18, 2007 issued in U.S. Appl. No. 11/326,108 in 13 pages.43Office Action dated Sep. 8, 2008 issued in U.S. Appl. No. 11/326,108 in 12 pages.44Office Action dates May 27, 2010 issued in U.S. Appl. No. 11/326,106 in 16 pages.45Office Action issued in corresponding Japanese Application No.: 2005-000250, mailed on Jan. 25, 2011.46Preliminary Amendment filed Apr. 12, 2010 in U.S. Appl. No. 12/758,743, in 3 pages.47RCE and Amendment filed Jun. 10, 2009 in U.S. Appl. No. 11/326,108 in 13 pages.48RCE and RCE Transmittal filed on Jul. 14, 2008 in U.S. Appl. No. 11/326,108 in 10 pages.49RCE filed Nov. 23, 2009 in U.S. Appl. No. 11/326,108 in 3 pages.50RCE filed Oct. 9, 2008 in U.S. Appl. No. 11/326,585 in 3 pages.51Reply Bried in Response to Examiner's Answer filed Jun. 15, 2010, in U.S. Appl. No. 11/326,105 15 pages.52Response to Advisory Action filed Oct. 9, 2008 in U.S. Appl. No. 11/326,585 in 9 pages.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8385726Jan 25, 2007Feb 26, 2013Kabushiki Kaisha ToshibaPlayback apparatus and playback method using the playback apparatusUS20070223877 *Jan 25, 2007Sep 27, 2007Shinji KunoPlayback apparatus and playback method using the playback apparatusWO2013043420A1 *Sep 12, 2012Mar 28, 2013Microsoft CorporationLow-complexity remote presentation session encoder* Cited by examinerClassifications U.S. Classification345/629, 345/612, 345/592, 345/638International ClassificationG09G5/00, G09G5/02Cooperative ClassificationG09G2320/0242, G09G5/14, G09G2340/10, H04N9/76, H04N5/85, H04N5/45, H04N21/4325, H04N21/4348, H04N21/4355, H04N21/4402, H04N21/4307, H04N21/42653, H04N21/4316, H04N21/42646, H04N21/8146European ClassificationH04N5/85, G09G5/14, H04N9/76, H04N5/45Legal EventsDateCodeEventDescriptionApr 12, 2010ASAssignmentOwner name: KABUSHIKI KAISHA TOSHIBA, JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KUNO, SHINJI;REEL/FRAME:024220/0493Effective date: 20060323Dec 17, 2014FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services