Source: http://www.google.com/patents/US7844122?dq=6272333
Timestamp: 2015-07-29 12:17:33
Document Index: 289443723

Matched Legal Cases: ['in fine', 'art 2', 'art 2', 'art 2', 'art 2', 'art 1', 'art 1']

Patent US7844122 - Chroma temporal rate reduction and high-quality pause system and method - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA system and method are provided for compressing data. In use, luminescence data of a frame is updated at a first predetermined rate, while chrominance data of the frame is updated at a second predetermined rate that is less than the first predetermined rate. Moreover, pause information may be inserted...http://www.google.com/patents/US7844122?utm_source=gb-gplus-sharePatent US7844122 - Chroma temporal rate reduction and high-quality pause system and methodAdvanced Patent SearchPublication numberUS7844122 B2Publication typeGrantApplication numberUS 10/447,514Publication dateNov 30, 2010Filing dateMay 28, 2003Priority dateJun 21, 2002Fee statusPaidAlso published asUS20030235340, US20120106621Publication number10447514, 447514, US 7844122 B2, US 7844122B2, US-B2-7844122, US7844122 B2, US7844122B2InventorsSteven E. Saunders, Krasimir D. Kolarov, William C. LynchOriginal AssigneeDroplet Technology, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (19), Non-Patent Citations (11), Classifications (7), Legal Events (6) External Links: USPTO, USPTO Assignment, EspacenetChroma temporal rate reduction and high-quality pause system and method
US 7844122 B2Abstract
A system and method are provided for compressing data. In use, luminescence data of a frame is updated at a first predetermined rate, while chrominance data of the frame is updated at a second predetermined rate that is less than the first predetermined rate. Moreover, pause information may be inserted with the compressed data, where the pause information may be used when the video data is paused during the playback thereof to increase the quality of a still frame.
1. A method for compressing video data comprising:
updating luminescence data of a frame at a first predetermined rate; and
updating chrominance data of the frame at a second predetermined rate that is less than the first predetermined rate;
wherein one or more frequency bands of the chrominance data is omitted utilizing a wavelet filter.
2. The method as recited in claim 1, and further comprising interpolating omitted portions of the chrominance data upon decompression of the video data.
3. A computer program product embodied on a computer readable medium for compressing video data, comprising:
computer code for updating luminescence data of a frame at a first predetermined rate; and
computer code for updating chrominance data of the frame at a second predetermined rate that is less than the first predetermined rate;
4. The computer program product as recited in claim 3, and further comprising interpolating omitted portions of the chrominance data upon decompression of the video data. Description
The present application is a continuation-in-part of a patent application filed Apr. 17, 2003 under Ser. No. 10/418,863 and naming the same inventors, and claims priority from a first provisional application filed Jun. 21, 2002 under Ser. No. 60/390,345, and a second provisional application filed Jun. 21, 2002 under Ser. No. 60/390,492, which are each incorporated herein by reference in their entirety.
Lossy digital video compression systems operate on digitized video sequences to produce much smaller digital representations. The reconstructed visible result looks much like the original video but may not generally be a perfect match. For these systems, it is important that the information lost in the process correspond to aspects of the video that are not easily seen or not readily noticed by viewers.
A typical digital video compression system operates in a sequence of stages, comprising a transform stage, a quantization stage, and an entropy-coding stage. Some compression systems such as MPEG and other DCT-based codec algorithms add other stages, such as a motion compensation search, etc. 2D and 3D Wavelets are current alternatives to the DCT-based codec algorithms. Wavelets have been highly regarded due to their pleasing image quality and flexible compression ratios, prompting the JPEG committee to adopt a wavelet algorithm for its JPEG2000 still image standard.
When using a wavelet transform as the transform stage in a video compressor, such algorithm operates as a sequence of filter pairs that split the data into high-pass and low-pass components or bands. Standard wavelet transforms operate on the spatial extent of a single image, in 2-dimensional fashion. The two dimensions are handled by combining filters that work horizontally with filters that work vertically. Typically, these alternate in sequence, H-V-H-V, though strict alternation is not necessary. It is known in the art to apply wavelet filters in the temporal direction as well: operating with samples from successive images in time. In addition, wavelet transforms can be applied separately to brightness or luminance (luma) and color-difference or chrominance (chroma) components of the video signal.
One may use a DCT or other non-wavelet spatial transform for spatial 2-D together with a wavelet-type transform in the temporal direction. This mixed 3-D transform serves the same purpose as a 3-D wavelet transform. It is also possible to use a short DCT in the temporal direction for a 3-D DCT transform.
The temporal part of a 3-D wavelet transform typically differs from the spatial part in being much shorter. Typical sizes for the spatial transform are 720 pixels horizontally and 480 pixels vertically; typical sizes for the spatial transform are two, four, eight, or fifteen frames. These temporal lengths are smaller because handling many frames results in long delays in processing, which are undesirable, and requires storing frames while they are processed, which is expensive.
When one looks at a picture or a video sequence to judge its quality, or when one visually compares two pictures or two video sequences, some defects or differences are harder to detect than others. This is a consequence of the human visual system having greater sensitivity for some aspects of what one sees than for others. For instance, one may see very fine details only when they are at high contrast, but can see medium-scale details which are very subtle in contrast. These differences are important for compression. Compression processes are designed to make the differences and errors as unnoticeable as possible. Thus, a compression process may produce good fidelity in the middle sizes of brightness contrast, while allowing more error in fine details.
There is thus a continuing need to exploit various psychophysics opportunities to improve compression algorithms, without significantly sacrificing perceived quality.
The foregoing compression systems are often used in Personal Video Recorders, Digital Video Recorders, Cable Set-Top Boxes, and the like. A common feature of these applications and others is that users have the possibility of pausing the video, keeping a single frame displayed for an extended time as a still image.
It is known in the art to process a video sequence, or other sequence of images, to derive a single image of higher resolution than the input images. This processing is very expensive in computing, however, as it must identify or match moving objects in the scene, camera motion, lighting shifts, and other changes and compensate for each change individually and in combination. Contemporary applications, however, do not presently support such computational extravagance for a simple pause function.
There is thus a continuing need to exploit various psychophysics opportunities to present a paused image that is of substantially higher visual quality than would be produced by simply repeating a frame of decompressed video.
A system and method are provided for compressing data. In use, luminescence data of a frame is updated at a first predetermined rate, while chrominance data of the frame is updated at a second predetermined rate that is less than the first predetermined rate.
Thus, the amount of compression is increased. To accomplish this, in one embodiment, one or more frequency bands of the chrominance data may be omitted. Moreover, the one or more frequency bands may be omitted utilizing a filter. Such filter may include a wavelet filter. Thus, upon the decompression of the video data, the omitted portions of the chrominance data may be interpolated.
Another system and method are provided for compressing data. Such system and method involves compressing video data, and inserting pause information with the compressed data. Thus, the pause information is used when the video data is paused during the playback thereof.
In one embodiment, the pause information may be used to improve a quality of the played back video data during a pause operation. Moreover, the pause information may include a high-resolution frame. Still yet, the pause information may include data capable of being used to construct a high-resolution frame.
FIG. 2 illustrates a method for compressing data with chrominance (chroma) temporal rate reduction, in accordance with one embodiment.
FIG. 2A illustrates a method for compressing data with a high-quality pause capability during playback, in accordance with one embodiment.
FIG. 2 illustrates a method 200 for compressing data with chrominance temporal rate reduction, in accordance with one embodiment. In one embodiment, the present method 200 may be carried out in the context of the transform module 102 of FIG. 1 and the manner in which it carries out a reversible transform. It should be noted, however, that the method 200 may be implemented in any desired context.
In operation 202, luminescence (luma) data of a frame is updated at a first predetermined rate. In operation 204, chrominance (chroma) data of the frame is updated at a second predetermined rate that is less than the first predetermined rate.
In a digital video compression system, it is thus possible to vary the effective rate of transmitting temporal detail for different components of the scene. For example, one may arrange the data stream so that some components of the transformed signal are sent more frequently than others. In one example of this, one may compute a three-dimensional (spatial+temporal) wavelet transform of a video sequence, and transmit the resulting luma coefficients at the full frame rate.
Moreover, one may omit one or more higher-frequency bands from the chroma signal, thus in effect lowering the temporal response rate—the temporal detail fidelity—of the video for chroma information. During reconstruction of the compressed video for viewing, one may fill in or interpolate the omitted information with an approximation, rather than showing a “zero level” where there was no information transmitted. This may be done in the same way as for omitted spatial detail. Most simply it can be done by holding the most-recently received level until new information is received. More generally, it can be done by computing an inverse wavelet filter, using zero or some other default value for the omitted information, to produce a smoothed variation with the correct overall level but reduced temporal detail.
One particular example of this sort of chroma rate compression is as follows: for the chroma components, one may compute an average across two frames (four fields) of the spatially transformed chroma values. This may be accomplished by applying a double Haar wavelet filter pair and discarding all but the lowest frequency component. One may transmit only this average value. On reconstruction, one can hold the received value across two frames (four fields) of chroma. It has been found that viewers do not notice this, even when they are critically examining the compression method for flaws.
The following stage of the video compression process, quantization, discards information by grouping similar values together and transmitting only a representative value. This discards detail about exactly how bright an area is, or exactly what color it is. When one finds that a transformed component is near zero, zero is chosen for the representative value (denoting no change at a particular scale). One can omit sending zero and have the receiver assume zero as its default value. This helps compression by lowering the amount of data sent. Again, human visual sensitivity to levels is known to differ between luma and chroma.
Thus, one can take advantage of this fact by applying different levels of quantization to luma and chroma components, discarding more information from the chroma. When this different quantization is done following a temporal or 3-D transform, the effect is to reduce the temporal detail in the chroma band along with the spatial detail. In a typical case, for ordinary video material, the temporal transform results in low frequency components that are much larger than higher frequency components. Then, applying quantization to this transformed result groups the small values with zero, in effect getting them omitted from the compressed representation and lowering the temporal resolution of the chroma components.
FIG. 2A illustrates a method 250 for compressing data with a high-quality pause capability during playback, in accordance with one embodiment. In one embodiment, the present method 250 may be carried out in the context of the framework of FIG. 1. It should be noted, however, that the method 250 may be implemented in any desired context.
In operation 252, video data is compressed. In one embodiment, the data compression may be carried out in the context of the coder portion 101 of the framework of FIG. 1. Of course, such compression may be implemented in any desired context.
In operation 254, pause information is inserted with the compressed data. In one embodiment, the pause information may be used to improve a quality of the played back video data. Moreover, the pause information may include a high-resolution frame. Still yet, the pause information may include data capable of being used to construct a high-resolution frame.
Thus, in operation 256, the pause information may be used when the video data is paused during the playback thereof. In the present method, the compressed video is equipped with a set of extra information especially for use when the video is paused. This extra information may include a higher-quality frame, or differential information that when combined with a regular compressed frame results in a higher-quality frame.
In order to keep the compression bit rate at a useful level, this extra information need not be included for every frame, but rather only for some frames. Typically, the extra information may be included for one frame of every 15 or so in the image, allowing a high-quality pause operation to occur at a time granularity of � second. This may be done in accord with observations of video pausing behavior. One can, however, include the extra information more often than this, at a cost in bit rate. One can also include it less often to get better compression performance, at a cost in user convenience. The tradeoff may be made over a range from two frames to 60 or more frames.
In one embodiment, the extra information may include a whole frame of the video, compressed using a different parameter set (for example, quantizing away less information) or using a different compression method altogether (for example, using JPEG-2000 within an MPEG stream). These extra frames may be computed when the original video is compressed, and may be carried along with the regular compressed video frames in the transmitted or stored compressed video.
In another embodiment, the extra information may include extra information for the use of the regular decompression process rather than a complete extra frame. For example, in a wavelet video compressor, the extra information might consist of a filter band of data that is discarded in the normal compression but retained for extra visual sharpness when paused. For another example, the extra information might include extra low-order bits of information from the transformed coefficients, and additional coefficients, resulting from using a smaller quantization setting for the chosen pausable frames.
In another embodiment, the extra information may include data for the use of a decompression process that differs from the regular decompression process, and is not a complete frame. This information, after being decompressed, may be combined with one or more frames of video decompressed by the regular process to produce a more detailed still frame.
More information regarding an exemplary wavelet-based transformation will now be set forth, which may be employed in combination with the various features of FIGS. 1-2A. It should be highly noted, however, that such wavelet-based transformation is set forth for illustrative purposes only and should not be construed as limiting in any manner. For example, it is conceived that the various features of FIGS. 1-2A may be implemented in the context of a DCT-based algorithm or the like.
FIG. 3 illustrates a method 300 for compressing/decompressing data, in accordance with one embodiment. In one embodiment, the present method 300 may be carried out in the context of the transform module 102 of FIG. 1 and the manner in which it carries out a reversible transform. It should be noted, however, that the method 300 may be implemented in any desired context.
The method 500 provides a technique for generating edge filters for a wavelet filter pair. Initally, in operation 502, a wavelet scheme is analyzed to determine local derivatives that a wavelet filter approximates. Next, in operation 504, a polynomial order is chosen to use for extrapolation based on characteristics of the wavelet filter and a numbers of available samples. Next, extrapolation formulas are derived for each wavelet filter using the chosen polynomial order. See operation 506. Still yet, in operation 508, specific edge wavelet cases are derived utlizing the extrapolation formulas with the available samples in each case.
Moreover, additional optional information regarding exemplary extrapolation formulas and related information will now be set forth in greater detail.
To approximate Y2N−1 from the left, one may fit a quadratic polynomial from the left. Approximating the negative of half the 2nd derivative at 2N−1 using the available values yields Equation #1.1.R.
Equation #1.1.R may be used in place of Equation #1.1 when point one is right-most. The apparent multiply by 3 can be accomplished with a shift and add. The division by 3 is trickier. For this case where the right-most index is 2N−1, there is no problem calculating Y2N−2 by means of Equation #1.2. In the case where the index of the right-most point is even (say 2N), there is no problem with Equation #1.1, but Equation #1.2 involves missing values. Here the object is to subtact an estimate of Y from the even X using just the previously calculated odd indexed Ys, Y1 and Y3 in the case in point. This required estimate at index 2N can be obtained by linear extrapolation, as noted above. The appropriate formula is given by Equation #1.2.R.
In such formulation, certain coefficients are computed with an offset or bias of �, in order to avoid the additions mentioned above. It is to be noted that, although there appear to be many additions of � in this formulation, these additions need not actually occur in the computation. In Equations #3.1 and 3.1R, it can be seen that the effects of the additions of � cancel out, so they need not be applied to the input data. Instead, the terms in parentheses (Y0+�) and the like may be understood as names for the quantities actually calculated and stored as coefficients, passed to the following level of the wavelet transform pyramid.
As can be seen here, the values taken as input to the inverse computation are the same terms produced by the forward computation in Equations #3.1˜3.2L and the corrections by � need never be calculated explicitly.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4561012 *Dec 27, 1983Dec 24, 1985Rca CorporationPre-emphasis and de-emphasis filters for a composite NTSC format video signalUS4979041 *Jan 28, 1988Dec 18, 1990Massachusetts Institute Of TechnologyHigh definition television systemUS5003377 *Jan 12, 1989Mar 26, 1991Massachusetts Institute Of TechnologyExtended definition television systemsUS5893145Dec 2, 1996Apr 6, 1999Compaq Computer Corp.System and method for routing operands within partitions of a source register to partitions within a destination registerUS6125201 *Jun 25, 1997Sep 26, 2000Andrew Michael ZadorMethod, apparatus and system for compressing dataUS6141673May 25, 1999Oct 31, 2000Advanced Micro Devices, Inc.Microprocessor modified to perform inverse discrete cosine transform operations on a one-dimensional matrix of numbers within a minimal number of instructionsUS6144773Feb 27, 1996Nov 7, 2000Interval Research CorporationWavelet-based data compressionUS6148110Feb 3, 1998Nov 14, 2000Matsushita Electric Industrial Co., Ltd.Image data processing apparatus and methodUS6195465Jul 3, 1995Feb 27, 2001Ricoh Company, Ltd.Method and apparatus for compression using reversible wavelet transforms and an embedded codestreamUS6229929May 14, 1998May 8, 2001Interval Research CorporationBorder filtering of video signal blocksUS6272180Apr 19, 1999Aug 7, 2001Sharp Laboratories Of America, Inc.Compression and decompression of reference frames in a video decoderUS6332043Nov 27, 1998Dec 18, 2001Sony CorporationData encoding method and apparatus, data decoding method and apparatus and recording mediumUS6360021Nov 30, 1999Mar 19, 2002The Regents Of The University Of CaliforniaApparatus and methods of image and signal processingUS6381280May 29, 1998Apr 30, 2002Interval Research CorporationSingle chip motion wavelet zero tree codec for image and video compressionUS6396948May 14, 1998May 28, 2002Interval Research CorporationColor rotation integrated with compression of video signalUS6407747May 7, 1999Jun 18, 2002Picsurf, Inc.Computer screen image magnification system and methodUS6516030May 14, 1998Feb 4, 2003Interval Research CorporationCompression of combined black/white and color video signalUS6650773 *Sep 29, 2000Nov 18, 2003Hewlett-Packard Development Company, L.P.Method including lossless compression of luminance channel and lossy compression of chrominance channelsUS6990246 *Aug 17, 2000Jan 24, 2006Vics LimitedImage coding* Cited by examinerNon-Patent CitationsReference1D. LeGall and A. Tabatabai, "Sub-band coding of digital images using symmetric short kernel filters and arithmetic coding techniques", IEEE International Conference on Acoustics, Speech and Signal Processing, New York, NY, pp. 761-765, 1988.2ISO/IEC 10918-1 Digital compression and coding of continuous-tone still images: Requirements and guidelines, 1992.3ISO/IEC 11172-2:1993 Coding of moving pictures and audio at up to 1,5 Mbit/s-Part 2: Video.4ISO/IEC 11172-2:1993 Coding of moving pictures and audio at up to 1,5 Mbit/s—Part 2: Video.5ISO/IEC 13818-2:1995 Coding of moving pictures and associated audio: Part 2. Video.6ISO/IEC 14496-2 MPEG-4 Part 2: Visual, Apr. 1999.7ISO/IEC 15444-1 JPEG 2000 image coding system-Part 1: Core coding system, Mar. 2000.8ISO/IEC 15444-1 JPEG 2000 image coding system—Part 1: Core coding system, Mar. 2000.9ISO/IEC JTC 1/SC 29/WG 1 N1646 Coding of Still Pictures, Mar. 16, 2000.10Kolarov et al., "Video Compression for the Portable and Wireless Markets," Dec. 2002, Droplet Technology, Inc., White Paper.11R. Crochiere and L. Rabiner, Multirate Digital Signal Processing, Prentice Hall 1983.Classifications U.S. Classification382/239International ClassificationG06K9/36, G06K9/46Cooperative ClassificationH04N19/167, H04N19/187, H04N19/30, H04N19/635Legal EventsDateCodeEventDescriptionMay 28, 2003ASAssignmentOwner name: DROPLET TECHNOLOGY, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SAUNDERS, STEVEN E.;KOLAROV, KRASIMIR D.;LYNCH, WILLIAM C.;REEL/FRAME:014124/0610;SIGNING DATES FROM 20030527 TO 20030528Owner name: DROPLET TECHNOLOGY, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SAUNDERS, STEVEN E.;KOLAROV, KRASIMIR D.;LYNCH, WILLIAM C.;SIGNING DATES FROM 20030527 TO 20030528;REEL/FRAME:014124/0610Apr 18, 2013ASAssignmentOwner name: INNOVATIVE COMMUNICATIONS TECHNOLOGY, INC., VIRGINFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DROPLET TECHNOLOGY, INC.;REEL/FRAME:030244/0608Effective date: 20130410May 17, 2013ASAssignmentOwner name: STRAIGHT PATH IP GROUP, INC., VIRGINIAFree format text: CHANGE OF NAME;ASSIGNOR:INNOVATIVE COMMUNICATIONS TECHNOLOGIES, INC.;REEL/FRAME:030442/0198Effective date: 20130418Feb 7, 2014ASAssignmentOwner name: SORYN TECHNOLOGIES LLC, NEW JERSEYFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:STRAIGHT PATH IP GROUP, INC.;REEL/FRAME:032169/0557Effective date: 20140130Feb 25, 2014FPAYFee paymentYear of fee payment: 4Apr 28, 2015ASAssignmentOwner name: STRAIGHT PATH IP GROUP, INC., VIRGINIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SORYN TECHNOLOGIES LLC;REEL/FRAME:035511/0492Effective date: 20150419RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services