Source: http://www.google.com/patents/US7778478?dq=7350717
Timestamp: 2014-07-13 22:22:12
Document Index: 329648258

Matched Legal Cases: ['art 14', 'art 15', 'art 15', 'art 15', 'art 16', 'art 15', 'art 15']

Patent US7778478 - Image processing device, image processing program, image processing method ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign in<nobr>Advanced Patent Search</nobr>PatentsAn image processing device changes degree of noise reduction for image data in accordance with tone correction to be performed according to the image data and includes a change rate acquisition part and a noise reduction part. The change rate acquisition part obtains, at a plurality of portions in the...http://www.google.com/patents/US7778478?utm_source=gb-gplus-sharePatent US7778478 - Image processing device, image processing program, image processing method, and electronic camera for controlling degree of noise reduction in image dataAdvanced Patent SearchPublication numberUS7778478 B2Publication typeGrantApplication numberUS 11/443,047Publication dateAug 17, 2010Filing dateMay 31, 2006Priority dateDec 3, 2003Fee statusPaidAlso published asEP1708490A1, EP1708490A4, EP1708490B1, US20060215925, WO2005055588A1Publication number11443047, 443047, US 7778478 B2, US 7778478B2, US-B2-7778478, US7778478 B2, US7778478B2InventorsHideyasu KunibaOriginal AssigneeNikon CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (18), Non-Patent Citations (1), Referenced by (9), Classifications (12), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetImage processing device, image processing program, image processing method, and electronic camera for controlling degree of noise reduction in image dataUS 7778478 B2Abstract An image processing device changes degree of noise reduction for image data in accordance with tone correction to be performed according to the image data and includes a change rate acquisition part and a noise reduction part. The change rate acquisition part obtains, at a plurality of portions in the image data, a change rate of a signal level of the image data before and after tone correction. The noise reduction part controls a degree of noise reduction for each portion in the image data according to the change rate.
SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing technique for properly reducing conspicuous noise by tone correction.
DESCRIPTION OF THE PREFERRED EMBODIMENTS [Explanation of the Configuration of the Present Embodiment]
Va ⁡ ( x , y ) = ∑ j , k = - r r ⁢ exp ⁢ [ - { V ⁡ ( x + j , y + k ) - V ⁡ ( x , y ) } 2 / σ 2 ] � V ⁡ ( x + j , y + k ) ∑ j , k = - r r ⁢ exp ⁢ [ - { V ⁡ ( x + j , y + k ) - V ⁡ ( x , y ) } 2 / σ 2 ] ( 1 ) Here, (x, y) in the expression (1) is a pixel position to be processed; r is a parameter corresponding to a size of a local range to be smoothed; and cr is a parameter corresponding to a threshold value when distinguishing between large change in amplitude at a picture pattern portion and very small change in amplitude caused by noise. Preferably, r=20, σ=30 approximately, for example.
Vb ⁡ ( x , y ) = { 1 - Vl Vu � Va ⁡ ( x , y ) V ⁢ ⁢ max + Vl if ⁢ ⁢ Va ⁡ ( x , y ) V ⁢ ⁢ max < Vu 1 if ⁢ ⁢ Va ⁡ ( x , y ) V ⁢ ⁢ max ≥ Vu ( 2 ) Here, Vmax in the expression (2) is the maximum range of the pixel value Va (x, y); Vu is a threshold value of the upper limit; and V1 is a lower limit value of the normalized image Vb. Preferably, as a specific value, Vu=0.5 and V1=0.25 approximately, for example.
Vd ⁡ ( x , y ) = ∑ i ⁢ Wi � Vi ⁡ ( x , y ) ( 4 ) Here, a weighting coefficient Wi in the expression (4) may be set all together to the same value. Further, it may also be possible to adjust the magnitude of the weighting coefficient Wi in accordance with the coincidence degree between the individual logarithmic images Vi and the visual characteristics.
( ∂ 2 ∂ x 2 + ∂ 2 ∂ y 2 ) ⁢ V ⁡ ( x , y ) = P ⁡ ( x , y ) ( 5 ) Note that, specifically, it is only necessary to perform local multiplication and addition operation by the following Laplacian filter A on the brightness image V.
A = ( 0 1 0 1 - 4 1 0 1 0 ) ( 6 ) [Step S32] Here, based on the differential image P, a processed image Un that stores the local brightness contrast is generated. First, the tone correction coefficient operation part 14 performs the repetitive operation of the following recursive formula after initializing all initial values Uo (x, y) of the processed image Un to a constant C. Note that, it is preferable to set the constant C to the average value of the V signal.
U n ⁡ ( x , y ) = 1 4 ⁢ ( U n - 1 ⁡ ( x + 1 , y ) + U n ⁡ ( x - 1 , y ) + U n - 1 ⁡ ( x , y + 1 ) + U n ⁡ ( x , y - 1 ) ) - 1 4 ⁢ P ⁡ ( x , y ) ( 7 ) In the expression (7), due to the term of the differential value P (x, y) on the right-hand side, the local brightness contrast of the processed image Un is kept substantially the same as that of the brightness image V before differentiation. Further, by repetition of the recursive formula, the local brightness contrast gradually spreads to the peripheral area.
U ⁡ ( x , y ) = R ( max - min ) ⁢ ( U n ⁡ ( x , y ) - min ) ( 8 ) (Here, max is the maximum level of the processed image Un, min is the minimum level of the processed image Un, and R is the width of the signal range of the brightness image V).
g ⁡ ( x , y ) = { sR ⁡ ( x , y ) m s + R ⁡ ( x , y ) m } 1 / m ( 12 ) Here, S in the expression (12) is a parameter for the upper limit value and m is a parameter for the degree how the limit is set. Preferably, S=20 and m=1 approximately, for example.
g ⁡ ( x , y ) = [ { sR ⁡ ( x , y ) m s + R ⁡ ( x , y ) m } 1 / m if ⁢ ⁢ R ⁡ ( x , y ) > ( s s - 1 ) 1 / m 1 if ⁢ ⁢ R ⁡ ( x , y ) ≤ ( s s - 1 ) 1 / m ( 13 ) The tone correction operation part 15 performs tone correction on image data by multiplying the change rate g (x, y) found for each pixel by each pixel value of the image data.
Vo(x,y)=V(x,y)�g(x,y) Ro(x,y)=R(x,y)�g(x,y) (14) Go(x,y)=R(x,y)�g(x,y) Ro(x,y)=R(x,y)�g(x,y) (15)
Lo(x,y)=L(x,y)�g(x,y) ao(x,y)=a(x,y)�g(x,y)0.75 bo(x,y)=b(x,y)�g(x,y)0.75 (16)
Lo(x,y)=L(x,y)g(x,y) ao(x,y)=Ta{a(x,y)g(x,y)} bo(x,y)=Tb{b(x,y)g(x,y)} (17)
{ 0.5 ⁢ a + 0.15 ⁢ b + L = 100 if ⁢ ⁢ b ≥ - 0.6 0.95 ⁢ a ⁢ ⁢ and ⁢ ⁢ b ≥ - 0.65 0.08 ⁢ a - 0.1 ⁢ a - 0.8 ⁢ b + L = 100 if ⁢ ⁢ b < - 0.6 0.95 ⁢ a ⁢ ⁢ and ⁢ ⁢ b < 0.05 0.87 ⁢ a - 0.15 ⁢ a + 0.07 ⁢ b + L = 100 if ⁢ ⁢ b ≥ 0.05 0.87 ⁢ a ⁢ ⁢ and ⁢ ⁢ b < - 0.65 0.08 ⁢ a ( 18 ) Next, an original (L, a, b) and a (Lo, ao, bo) after multiplied by the change rate are connected by a straight line and a crossing (Lx, ax, bx) of the straight line and any one of the above-mentioned three planes k1a+k2b+L=100 is calculated using the following expression.
Lx = L + ( Lo - L ) ⁢ t ⁢ ⁢ ax = a + ( ao - a ) ⁢ t ⁢ ⁢ bx = b + ( bo - b ) ⁢ t ⁢ ⁢ where , t = 100 - ( k 1 ⁢ a + k 2 ⁢ b + L ) k 1 ⁡ ( ao - a ) + k 2 ⁡ ( bo - b ) + ( Lo - L ) ( 19 ) Next, the tone correction operation part 15 calculates
Rl = ( Lo - L ) 2 + ( ao - a ) 2 + ( bo - b ) 2 ( Lx - L ) 2 + ( ax - a ) 2 + ( bx - b ) 2 ( 20 ) R1 found here corresponds to the distance between (L, a, b) and (Lo, ao, bo) when it is assumed that the distance between (L, a, b) and the crossing (Lx, ax, bx) is a numerical value �1�. When the distance R1 exceeds 1, the sRGB color region is exited. Therefore, the tone correction operation part 15 applies soft limit to R1 using the following expression.
Lo′=L+(Lx−L)t′ ao′=a+(ax−a)t′ bo′=b+(bx−b)t′ (22)
[Step S51] The change rate acquisition part 16 obtains the change rate of the signal level before and after tone correction from the tone correction operation part 15. For the tone correction of the brightness image V or the RGB image described above, it is only necessary to obtain the change rate g (x, y) as is. Further, for the tone correction of the Lab image described above, it is only necessary to obtain the signal level before and after that from the tone correction operation part 15, divide it for each pixel, and calculate the change rate. Here, about the pixel whose signal level before tone correction is zero, it is preferable to set the change rate to a predetermined value (for example, �1�).
Zi ⁡ ( x , y ) = ∑ j , k = - ri ⁢ ri ⁢ exp ⁡ [ - { Pi ⁡ ( x + j , y + k ) - Pi ⁡ ( x , y ) } 2 / σ ⁢ ⁢ i 2 ] � Pi ⁡ ( x + j , y + k ) ∑ j , k = - ri ri ⁢ exp ⁡ [ - { Pi ⁡ ( x + j , y + k ) - Pi ⁡ ( x , y ) } 2 / σ ⁢ ⁢ i 2 ] ( 24 ) Here, Pi in the expression (24) corresponds to each component of the color image data. Incidentally, here, explanation is given on the assumption that P0 corresponds to the brightness component, such as L, and P1,2 corresponds to the color components, such as a and b.
r 0(x,y)=└0.1�g 0(x,y)2┘σ0(x,y)=0.3�g 0(x,y)2 r 1,2(x,y)=└0.75�g 1,2(x,y)2┘σ1,2(x,y)=2.25�g 1,2(x,y)2 (25)
r 0 ⁡ ( x , y ) = { ⌊ 0.1 � g 0 ⁡ ( x , y ) 2 ⌋ if ⁢ ⁢ ⌊ 0.1 � g 0 ⁡ ( x , y ) 2 ⌋ < r 0 , max r 0 , max if ⁢ ⁢ ⁢ ⌊ 0.1 � g 0 ⁡ ( x , y ) 2 ⌋ ≥ r 0 , max ⁢ ⁢ r 1 , 2 ⁡ ( x , y ) = { ⌊ 0.75 � g 1 , 2 ⁡ ( x , y ) 2 ⌋ if ⁢ ⁢ ⌊ 0.75 � g 1 , 2 ⁡ ( x , y ) 2 ⌋ < r 1 , 2 , max r 1 , 2 , max if ⁢ ⁢ ⌊ 0.75 � g 1 , 2 ⁡ ( x , y ) 2 ⌋ ⁢ r 1 , 2 , max ( 26 ) In addition, the amount of noise in the image data changes considerably depending on an image pickup sensitivity A. Therefore, as shown in the following expression (27), it is preferable to increase or decrease the noise reduction parameters ri and σi depending on the image pickup sensitivity A.
r 0(x,y)=└0.1√{square root over (A/200)}�g 0(x,y)2┘σ0(x,y)=0.3√{square root over (A/200)}�g 0(x,y)2 r 1,2(x, y)=└0.75√{square root over (A/200)}�g 12 (x, y)2┘σ1,2(x,y)=2.25√{square root over (A/200)}�g 1,2(x,y)2 (27)
r 0(x,y)=└0.1{0.1S 0 �g 0(x,y)}2┘σ0(x,y)=0.3{0.1S 0 �g 0(x,y)}2 r 1,2(x,y)=└09.75{0.1S 1,2 �g 1,2(x,y)}2┘σ1,2(x,y)=2.25{0.1S 1,2 �g 1,2(x, y)}2 (28)
Zi ⁡ ( x , y ) = Pi ⁡ ( x , y ) + ∑ j , k = - ri ri ⁢ exp [ - { Pi ⁡ ( x + j , y + k ) - Pi ⁡ ( x , y ) } 2 / σ ⁢ ⁢ i ⁢ 2 ] ⁡ [ Pi ⁡ ( x + j , y + k ) - Pi ⁡ ( x , y ) ] ∑ j , k = - ri ri ⁢ exp ⁡ ( 0 ) ( 29 ) In addition, in the embodiment described above, the strength of the noise filter is changed by changing the parameter of the smoothing filter according to the change rate g (x, y). However, the present invention is not limited thereto. For example, it may also be possible to perform weighted average of the image subjected to the noise filter processing with a uniform strength and the image subjected to moderate noise filter processing or no processing. By changing the weighted ratio according to the change rate g (x, y), the strength adjustment of the noise filter processing is substantially realized.
Iout ⁡ ( x , y ) = [ 1 - g ⁡ ( x , y ) - 1 z - 1 ] � I ⁡ ( x , y ) + [ g ⁡ ( x , y ) - 1 z - 1 ] � I ′ ⁡ ( x , y ) ( 30 ) Further, in the embodiment described above, the parameter S relating to the upper limit value is used in the expression (12), (13), or (21). However, the present invention is not limited thereto. For example, it may also be possible to change the parameter S in accordance with the parameter m for the degree how the limit is set. For example, it is preferable to replace the parameter S with the expression Sm.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5420633Jun 19, 1992May 30, 1995Canon Kabushiki KaishaApparatus for suppressing pseudo-contours in video signalsUS5715334 *Mar 8, 1995Feb 3, 1998The University Of ConnecticutDigital pixel-accurate intensity processing method for image information enhancementUS5724456 *Mar 31, 1995Mar 3, 1998Polaroid CorporationBrightness adjustment of images using digital scene analysisUS5991456May 29, 1996Nov 23, 1999Science And Technology CorporationMethod of improving a digital imageUS6215911 *Dec 12, 1997Apr 10, 2001Siemens AktiengesellschaftMethod for correcting the intensity of an imageUS6252931Oct 22, 1999Jun 26, 2001U.S. Philips CorporationProcessing method for an original imageUS6535648 *Dec 8, 1998Mar 18, 2003Intel CorporationMathematical model for gray scale and contrast enhancement of a digital imageUS6813389Dec 15, 1999Nov 2, 2004Eastman Kodak CompanyDigital image processing method and system including noise reduction and tone scale adjustmentsUS6856702Jun 28, 2001Feb 15, 2005Andrew C. GallagherSystem and method for selecting an image processing path using estimated appearance of noiseUS7386185 *Feb 12, 2003Jun 10, 2008Matsushita Electric Industrial Co., Ltd.Image processing device and image processing methodUS20010030707Mar 2, 2001Oct 18, 2001Shinichi FujiiDigital cameraUS20040184667 *Mar 19, 2003Sep 23, 2004Ramesh RaskarEnhancing low quality images of naturally illuminated scenesEP1345170A1Feb 22, 2002Sep 17, 2003AGFA-GEVAERT naamloze vennootschapNoise reduction method.JP2001251549A Title not availableJP2002044484A Title not availableJP2002125139A Title not availableJP2003101887A Title not availableJPH04373361A Title not available* Cited by examinerNon-Patent CitationsReference1Stahl M. et al., "Noise-Resistant weak-structure enhancement for digital radiography." SPIE Conference on Image Processing, San Diego, CA, vol. 3661. Feb. 1999.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS7924288Jun 21, 2005Apr 12, 2011Nikon CorporationImage processing device emphasizing on texture, image processing program, electronic camera, and image processing methodUS8023733 *Jun 4, 2007Sep 20, 2011Panasonic CorporationImage processing device, image processing method, image processing program, and integrated circuitUS8040387 *Jul 11, 2008Oct 18, 2011Nikon CorporationImage processing apparatus, image processing program, image processing method, and electronic camera for correcting texture of imageUS8149299 *Mar 26, 2008Apr 3, 2012Kabushiki Kaisha ToshibaApparatus and method for generating a picture having a high visibilityUS8155474Jul 9, 2007Apr 10, 2012Samsung Electronics Co., Ltd.Method, medium, and system compensating shadow areasUS8320701 *Mar 14, 2008Nov 27, 2012Kabushiki Kaisha ToshibaPicture processing apparatus and methodUS8340419Aug 16, 2011Dec 25, 2012Panasonic CorporationImage processing device, image processing method, image processing program, and integrated circuitUS8606008Nov 27, 2012Dec 10, 2013Panasonic CorporationImage processing device, image processing method, image processing program, and integrated circuitUS20110317045 *Sep 13, 2011Dec 29, 2011Zoran CorporationAdvanced noise reduction in digital cameras* Cited by examinerClassifications U.S. Classification382/254, 382/276International ClassificationG06K9/40, G06T7/00, H04N5/21, G06T5/00Cooperative ClassificationH04N2101/00, G06T7/0004, G06T5/007, G06T5/002European ClassificationG06T7/00B1, G06T5/00DLegal EventsDateCodeEventDescriptionJan 22, 2014FPAYFee paymentYear of fee payment: 4May 31, 2006ASAssignmentOwner name: NIKON CORPORATION, JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:KUNIBA, HIDEYASU;REEL/FRAME:017935/0545Effective date: 20060511RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google