Source: https://www.lens.org/lens/patent/US_8824829_B2/fulltext
Timestamp: 2018-08-15 11:26:37
Document Index: 305254748

Matched Legal Cases: ['application No. 13', 'application No. 12', 'application No. 12', '§119', 'Application No. 60', '§120']

US 8824829 B2 - Enhancing Dynamic Ranges Of Images - The Lens - Free & Open Patent and Scholarly Search
US 8824829 B2
lens.org/012-153-589-729-606
*US08824829B2*
US008824829B2
(12) United States Patent (10) Patent No.: US 8,824,829 B2
et al. (45) Date of Patent: *Sep. 2, 2014
(73) Assignee: Dolby Laboratories Licensing Coporation, San Francisco, CA (US), Type: US Company
(21) Appl. No.: 14/076,038
(22) Filed: Nov. 8, 2013
US 2014/0064634 A1 Mar. 6, 2014
Continuation of application No. 13/488,228, filed on Jun. 4, 2012, now Pat. No. 8,582,913 , which is a continuation of application No. 12/183,033, filed on Jul. 30, 2008, now Pat. No. 8,233,738 , which is a continuation-in-part of application No. 12/182,121, filed on Jul. 29, 2008, now Pat. No. 8,135,230 .
(51) Int. Cl. G06K 009/40 (20060101); G06K 009/32 (20060101); G09G 005/10 (20060101)
(52) U.S. Cl. 382/274; 382/299; 345/690
(58) Field of Search 382/254, 263, 274, 298, 299; 358/3.27; 345/102, 690; 375/240.19, 240.21
7,265,784 B1 9/2007 Frank
7,280,705 B1 10/2007 Frank
8,228,560 B2 * 7/2012 Hooper 358/3.27
22 Claims, 13 Drawing Sheets, and 16 Figures
[0001] This application is a continuation of U.S. patent application Ser. No. 13/488,228 filed on 4 Jun. 2012, which is a continuation of U.S. patent application Ser. No. 12/183,033 filed on 30 Jul. 2008 and issued as U.S. Pat. No. 8,233,738, which is a continuation-in-part of U.S. patent application Ser. No. 12/182,121 filed on 29 Jul. 2008 and issued as U.S. Pat. No. 8,135,230, both of which claim the benefit under 35 U.S.C. §119 of U.S. Patent Application No. 60/962,708 filed on 30 Jul. 2007, all of which are entitled ENHANCING DYNAMIC RANGES OF IMAGES and are hereby incorporated herein by reference. This application claims the benefit under 35 U.S.C. §120 of U.S. patent application Ser. No. 12/183,033 filed on 30 Jul. 2008 and entitled ENHANCING DYNAMIC RANGES OF IMAGES.
h(x)=A(x)∫ξεN(x)ƒ(ξ)c(ξ−x)s(ƒ(ξ)−ƒ(x))dξ (2)
where h(x) is the output of the filter for the pixel at location x; A(x) is a normalization factor, ƒ(x) is the pixel value at location x; c(ξ−x) is a weighting function that drops off with the distance between the pixel at location ξ and the pixel at location x(c) may be called a ‘closeness’ function); and s(ƒ(ξ)−ƒ(x)) is a weighting function that drops off with the difference between ƒ(x) and ƒ(ξ) (s may be called a ‘similarity’ function). The equation (2) integral may be evaluated over a neighbourhood N(x) of the location x.
where δ is a suitable measure of the distance in intensity space between the pixel values at locations ξ and x and σr is a parameter defining the variance (i.e. how quickly s falls off with increasing difference between ƒ(ξ) and ƒ(x)).
[0038] In some embodiments, a modified function is used for the similarity function (s), such that the variance σr of the similarity function (s) increases with the value of ƒ(x). In such embodiments, it may be desirable to stretch the variance σr in proportion to the stretch introduced by the non-linear intensity mapping for the local pixel value in block 30 such that, after the block 30 stretching, the photometric variance σr is equal to a fixed number, preferably two, of quantization levels.
[0039] The effect of making σr vary with ƒ(x) as described above is similar to performing a bilateral filter with fixed variance prior to the block 30 stretching. However, performing the block 40 bilateral filter after the block 30 stretching can be advantageous because after the block 30 stretching, the block 40 bilateral filter may be performed in fixed point arithmetic. Since performing bilateral filtering can be computationally expensive, where computational resources are limited, it is desirable to operate the bilateral filter on relatively small neighbourhoods N(x) of each pixel. For example, in some embodiments, the block 40 bilateral filter may be performed on neighbourhoods that include only pixels within four or so pixel spaces of the current pixel.
[0054] Where the methods described herein are being applied to generate an image to be displayed on a television, then it may be desirable that the standard deviation of the blur filter be at least about 0.025 of the horizontal resolution of the display and more advantageously at least about 0.033 (where ‘about’ means ±15%) of the horizontal resolution of the display. For example, for a display having a horizontal resolution of 1920 pixels, the standard deviation of the blur filter is advantageously at least about 50 pixels, and more advantageously at least about 65 pixels. As noted above, good results on a display of this horizontal resolution have been achieved with a standard deviation of 150 pixels.
[0064] In the illustrated embodiment, smooth component 53A is then obtained from down-sampled mask 73 via loop 74. Loop 74 comprises N iterations, with each iteration involving: application of a blur filter in block 74A (which may comprise applying a Gaussian blur having a small kernel—e.g. a Gaussian blur applied to a 3×3 pixel neighbourhood of each pixel); and then up-sampling the result in block 74B (which may involve nearest-neighbour interpolation). This technique may be described as an image pyramid technique. The use of image pyramids is described in Burt P. and Adelson E., 1983, The Laplacian pyramid as a compact image code, IEEE Trans. on Communication 31, 4, 532-540. The result of method 70 is smooth component 53A.
[0091] In block 124 (FIG. 6A), saturation extension image 123 is multiplied (e.g. pixel-wise multiplication) with linearized image data 101B to yield a HDR image 125. In some embodiments, block 124 may involve a mapping (e.g. to values 1-a) prior to carrying out the multiplication. In other embodiments, block 124 may comprise some other mapping which takes as input saturation extension image 123 and linearized image data 101B and outputs HDR image 125.
1. A controller for use in displaying images, the controller configured to:
obtain image data;
downsample the image data to yield downsampled image data;
process the downsampled image data to both:
yield driving values for elements of a light source layer in a display; and
yield an enhancement function for use in modifying the image data to yield higher-dynamic-range image data, the higher-dynamic-range image data having at least one enhancement region comprising enhancement-region pixels;
wherein compared to the image data, the higher-dynamic-range image data comprises increased luminance values of the enhancement-region pixels in the enhancement region.
2. A controller according to claim 1 wherein, in processing the downsampled image data to yield the enhancement function, the controller is configured to further downsample the downsampled image data.
3. A controller according to claim 2 wherein, in processing the downsampled image data to yield the enhancement function, the controller is configured to process the downsampled image data to yield a binarized mask.
4. A controller according to claim 3 wherein the controller is configured to compare pixel values in the downsampled image data to a threshold to yield binary pixel values for the binarized mask.
5. A controller according to claim 3 wherein the controller is configured to upsample the binarized mask to yield a grey scale image.
6. A controller according to claim 5 wherein upsampling the binarized mask to yield the grey scale image comprises applying a blur.
7. A controller according to claim 3 comprising a linearizer arranged to linearize the image data before the controller performs the downsampling of the image data.
8. A controller according to claim 2 wherein, in processing the downsampled image data to yield the enhancement function, the controller is configured to process the downsampled image data to yield a mask and to subsequently upsample the mask wherein upsampling the mask comprises applying a DILATION operation to the mask.
9. A controller according to claim 8 wherein upsampling the mask comprises applying a gradient image as an edge stop.
10. A controller according to claim 1 wherein the controller is incorporated into a video processor chip or a display driver chip.
11. A method for displaying images, the method comprising:
obtaining image data defining an image;
downsampling the image data to yield downsampled image data;
processing the downsampled image data to both:
yield an enhancement function for use in modifying the image data to yield higher-dynamic-range image data, the higher-dynamic-range image data having at least one enhancement region comprising enhancement-region pixels wherein the enhancement-region pixels have increased luminance values as compared to corresponding pixels in the image data.
12. A method according to claim 11 wherein processing the downsampled image data to yield the enhancement function comprises further downsampling the downsampled image data.
13. A method according to claim 12 wherein processing the downsampled image data to yield the enhancement function comprises processing the downsampled image data to yield a binarized mask.
14. A method according to claim 13 comprising comparing pixel values in the downsampled image data to a threshold to yield binary pixel values for the binarized mask.
15. A method according to claim 13 comprising upsampling the binarized mask to yield a grey scale image.
16. A method according to claim 15 wherein upsampling the binarized mask to yield the grey scale image comprises applying a blur.
17. A method according to claim 13 comprising linearizing the image data before downsampling the image data.
18. A method according to claim 12 wherein processing the downsampled image data to yield the enhancement function comprises processing the downsampled image data to yield a mask and subsequently upsampling the mask wherein upsampling the mask comprises applying a DILATION operation to the mask.
19. A method according to claim 18 wherein upsampling the mask comprises applying a gradient image as an edge stop.
20. A method according to claim 11 wherein processing the downsampled image data to yield driving values for the elements of the light source layer comprises clamping the downsampled image data.
21. A method according to claim 11 wherein processing the downsampled image data to yield driving values for the elements of the light source layer comprises applying a blur filter to the downsampled image data.
22. A method according to claim 11 wherein processing the downsampled image data to yield driving values for the elements of the light source layer comprises performing an exchange operation to increase the intensity of light delivered by elements of the light source layers to areas of a second modulator that correspond to enhancement regions.