Abstract:
A method for adjusting the color and tonal characteristics of a digital image, the method comprises receiving a digital image having two or more digital image channels; (b) separating each digital image channel into two signals; (c) modifying one of the signals with a multidimensional transform for adjusting the color and tonal characteristics of the digital image; and (d) combining the output of (c) with the remaining signal for producing an enhanced digital image.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     The present application is related to U.S. application Ser. No. 09/091,848, filed Sep. 30 1998, by, Andrew Gallagher, and entitled, “A METHOD FOR PRESERVING IMAGE DETAIL WHEN ADJUSTING THE CONTRAST OF A DIGITAL IMAGE”. 
    
    
     FIELD OF THE INVENTION 
     The invention relates generally to the field of image processing, and more particularly, to adjusting a method for adjusting the tone scale of a digital image. 
     BACKGROUND OF THE INVENTION 
     Existing methods for modifying the tone scale and color characteristics of a digital color image include applying a tone scale function to each of the color channels independently, applying the tone scale function to the neutral channel and preserving the original pixel color difference signal, or applying the tone scale function to the low frequency image. In addition, a 3 dimensional look-up-table (3 D LUT) may be applied to the digital color image in order to modify the color or tone scale of the digital color image. All of these methods of tone scale application alter the sharpness of the detail of the modified image, and the latter can produce displeasing artifacts. 
     Traditional methods of applying a tone scale function to a digital color image modify the apparent sharpness of the image because the tone scale modifies the amplitudes of high frequency detail information. This phenomenon occurs when the tone scale function is applied to each of the color channels independently, or when the tone scale function is applied to the neutral channel and original pixel color difference signals are preserved. 
     In an effort to apply a tone scale function to a digital image without distorting the detail information, in U.S. Pat. No. 5,012,333 Lee et al. proposed separating the image into a high frequency and a low frequency image by using FIR filters. The tone scale function is then applied to only the low frequency image, and the high frequency image is added back to the tone scaled low frequency image. 
     Also, in U.S. Pat. No. 5,454,044 Nakajima suggests modifying the image contrast by the formula 
     
       
           Sproc=Sorg+f ( Sus ). 
       
     
     The low frequency image (Sus) is passed through function f( ) which is a monotonically decreasing function. This signal is added to the original (Sorg) to create the processed image Sproc. 
     Both of these methods preserve the high frequencies of the image, but this may lead to the creation of unsharp mask type artifacts (overshoot and undershoot) in the neighborhood of large edges (characteristic of occlusion boundaries or dark shadows). 
     There exist in the prior art methods to modify the color and tone of a digital color image with a 3 D LUT. For example, the modification of a digital image by a 3 D LUT is described in U.S. Pat. No. 4,500,919. However, this method also modifies the appearance of detail in the processed digital image. None of the methods described in the prior art address the need for detail preservation when modifying the color and tonal characteristics of a digital color image with the use of a multidimensional look-up-table. 
     Thus there exists a need for a method to allow the application of a multidimensional look-up-table to a digital image to adjust the macro contrast of the image and modify the color characteristics of the image, while preserving detail information, without introducing artifacts in the neighborhood of large intensity transitions. 
     SUMMARY OF THE INVENTION 
     The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, the invention resides in a method for adjusting the color and tonal characteristics of a digital image, the method comprising: (a) receiving a digital image having two or more digital image channels; (b) separating each digital image channel into two signals; (c) modifying one of the signals with a multidimensional transform for adjusting the color and tonal characteristics of the digital image; and (d) combining the output of (c) with the remaining signal for producing an enhanced digital image. 
     These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiments and appended claims, and by reference to the accompanying drawings. 
     ADVANTAGEOUS EFFECT OF THE INVENTION 
     The present invention has the following advantages of permitting the application of a multidimensional look-up-table to a digital image to adjust the macro contrast of the image and modify the color characteristics of the image, while preserving detail information, without introducing artifacts in the neighborhood of large intensity transitions. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a perspective view of a computer system for implementing the present invention; 
     FIG. 2 is a block diagram illustrating an overview of the present invention; 
     FIG. 3 is an exploded view of the pedestal splitter of FIG. 1; 
     FIG. 4 is an exploded view of the avoidance signal generator of FIG. 2; and 
     FIG. 5 is an example of an artifact avoidance LUT. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     In the following description, the present invention will be described in the preferred embodiment as a software program. Those skilled in the art will readily recognize that the equivalent of such software may also be constructed in hardware. 
     Still further, as used herein, a computer readable storage medium may comprise, for example; magnetic storage media such as a magnetic disk (such as a floppy disk) or magnetic tape; optical storage media such as an optical disc, optical tape, or machine readable bar code; solid state electronic storage devices such as random access memory (RAM), or read only memory (ROM); or any other physical device or medium employed to store a computer program. 
     It is also instructive to note that the present invention will utilize a digital image of size 1024 pixels by 1536 pixels, although those skilled in the art will recognize that digital images of different dimensions may be used with equal success. 
     Referring to FIG. 1, there is illustrated a computer system  1  for implementing the present invention. Although the computer system  1  is shown for the purpose of illustrating a preferred embodiment, the present invention is not limited to the computer system  1  shown, but may be used on any electronic processing system. The computer system  1  includes a microprocessor-based unit  2  for receiving and processing software programs and for performing other processing functions. A display  3  is electrically connected to the microprocessor-based unit  2  for displaying user-related information associated with the software. A keyboard  4   a  is also connected to the microprocessor based unit  2  for permitting a user to input information to the software. As an alternative to using the keyboard  4   a  for input, a mouse  4   b  may be used for moving a selector  5  on the display  3  and for selecting an item on which the selector  5  overlays, as is well known in the art. 
     A compact disk-read only memory (CD-ROM)  6  is connected to the microprocessor based unit  2  for receiving software programs and for providing a means of inputting the software programs and other information to the microprocessor based unit  2  via a compact disk  7   a , which typically includes a software program. In addition, a floppy disk  7   b  may also include a software program, and is inserted into the microprocessor-based unit  2  for inputting the software program. Still further, the microprocessor-based unit  2  may be programmed, as is well know in the art, for storing the software program internally. A printer  8  is connected to the microprocessor-based unit  2  for printing a hardcopy of the output of the computer system  1 . 
     Images may also be displayed on the display  3  via a personal computer card (PC card)  7   c  or, as it was formerly known, a personal computer memory card international association card (PCMCIA card) which contains digitized images electronically embodied in the card  7   c . The PC card  7   c  is ultimately inserted into the microprocessor based unit  2  for permitting visual display of the image on the display  3 . 
     Referring to FIG. 2, there is illustrated an overview of the present invention. In this regard, the input digital image consists of several input channels. These channels are typically the red, green, and blue image channels, and they are typically each referred to as color channels. Each channel of the input digital image is respectively passed as an input to one of a plurality of pedestal splitters  30 . It is instructive to note that each color channel is passed to a different pedestal splitter  30 . For example, the red color channel is passed to the pedestal splitter  30   a . The digital image is also passed to a pre-avoidance channel generator  20 , which generates a pre-avoidance channel that is also passed to each pedestal splitter  30 . In the preferred embodiment in which there are three color channels, the pre-avoidance channel for each of the pedestal splitters  30  is equal to a linear combination of all the digital image channels representing the image. In this regard, the pre-avoidance channel is calculated as follows: 
     
       
           pac ( x,y )=0.3 red( x,y )+0.6green( x,y )+0.1 blue ( x,y )  Eq. 1 
       
     
     where pac(x,y) represents the intensity of the pre-avoidance channel at the x th  y th  pixel location, red(x,y) represents the intensity of the red channel at the x th  y th  pixel location, green(x,y) represents the intensity of the green channel at the x th  y th  pixel location, and blue(x,y) represents the intensity of the blue channel at the x th  y th  pixel location. 
     Those skilled in the art will recognize that different pre-avoidance channel signals may be used. As an alternative embodiment, the pre-avoidance channel may be equal to any one of the three color image channels of the digital image. As a further alternative embodiment, the pre-avoidance channel need not be equal for each of the pedestal splitters  30   a ,  30   b , and  30   c.    
     Each color channel of the digital image channel is divided into two portions by a pedestal splitter  30 —a pedestal signal and a texture signal, as will be described in detail below. The pedestal signal is a smooth representation of a digital image channel except for large transition edges occurring in the neighborhood of large transition edges in the digital image channel. 
     The present invention operates by applying a multidimensional transformation to a collection of pedestal signals. One example of a multidimensional transformation is a 3D LUT. Another example of a multidimensional transformation is a set of multivariate equations. The application of a 3D LUT is similar to evaluating three equations to determine modified red, green, and blue signals. For example, the output red value may be expressed as: 
     
       
           r′=ƒ   r ( r,g,b ) 
       
     
     where r′ represents the modified red signal, ƒr(r,g,b) represents a function of the red, green, and blue channel values used to determine the modified red signal, and r, g, and b represent the red, green, and blue channels respectively. A similar operation may be performed to obtain the modified green and blue signals. The 3D LUT or the set of equations, one per color channel, that are a function of more than one color signal shall be referred to as a multi-dimensional transform. 
     A three dimensional look up table (3D LUT) is applied to the collection of pedestal signals output from the pedestal splitters  30  by the 3D LUT applicator  36  in order to modify the color characteristics and tone scale of the digital image for image enhancement or preparation for an output device. A 3D LUT outputs three values for three input values. Typically a 3D LUT is a function of three variables where the three input variables are the red, green, and blue intensities and the three output values are transformed red, green and blue pixel values. By applying this 3D LUT to every pixel of the digital color image, the tone scale and color characteristics of the digital color image are transformed. The purpose of applying a 3D LUT to an image is to allow non-linear modifications to the color and tonal characteristics of the digital image. 
     Thus, the output of the 3D LUT applicator  36  is a collection of modified pedestal signals. A 3D LUT applicator  36  is implemented by any technique as is well known in the art. U.S. Pat. No. 4,500,919 describes a generalized method of constructing and applying a 3D-LUT to a digital color image. The 3D-LUT consists of a table of three output values (red, green, and blue) corresponding to three input values (red, green, and blue). The range of input values is typically chosen to span the full extent of numerical input pixel values. The output values are chosen to achieve the desired color and tone scale modifications. U.S. Pat. No. 5,583,666 describes a method for populating a 3D LUT to achieve a tone scale modification. Since a table representing all possible combinations of red, green, and blue pixel input values can consume a large amount of computer memory, most 3D LUT applications sparsely sample the range of input pixel values. For these applications, the input pixel values will not always exactly correspond to one of the 3D LUT table entries. Thus an interpolation scheme is typically employed which provides approximate output values by interpolating several of the 3D LUT table entries. A variety of 3D LUT interpolation methods exist, e.g. nearest neighbor, tri-linear, tetrahedral are all different interpolation methods. U.S. Pat. No. 4,275,413 describes the tetrahedral interpolation method. The present invention may be used with any of the methods of interpolation. 
     The texture signal of each digital image channel may be amplified by the texture modifier  33  if desired, or altered in some other manner as those skilled in the art may desire. The texture modifier  33  operates independently on the red texture signal, the green texture signal, and the blue texture signal. This texture modifier  33  may be a multiplication of the texture signal by a scalar constant. In the preferred embodiment, the texture modifier  33  amplifies the red texture signal, the green texture signal, and the blue texture signal by a scalar constant determined by the slope calculator  37 . The purpose of the slope calculator  37  is to estimate the midtone gain of the 3D LUT. This may be accomplished by evaluating the following equation:        m   =       D        [         f   3          (     p   1     )       ,       f   3          (     p   2     )         ]         D        [       p   1     ,     p   2       ]                                
     where D[x,y] indicates the Euclidian distance measured between point x and point y in a 3 dimensional space, ƒ 3 (x) indicated the output of the 3D LUT for the input point x. The calculation of the Euclidian distance is well known by those skilled in the art and will not be further explained. Points p 1  and p 2  represent the coordinates of two points in the midtone range appropriate for the image data. In the preferred embodiment,p 1 =(1200, 1200, 1200) and p 2 =(1800, 1800, 1800). The texture modifier  33  outputs three signals: a modified red texture signal, a modified green texture signal, and a modified blue texture signal. In the preferred embodiment, the modified red texture signal may be calculated with the expression: 
     
       
         
           r′ 
           txt 
           =m*r 
           txt 
         
       
     
     where r′ txt  represents the modified red texture signal, m represents the scalar constant determined by the slope calculator  37 , and r txt  represents the red texture signal input to the texture modifier  33 . The modified green and blue texture signals are calculated in a similar manner, dependent only upon the green and blue texture signals input to the texture modifier  33 , respectively. 
     The modified texture signals and the modified pedestal signals are then summed together by an adder  39 , forming a transformed digital image. The addition is preformed on each of the red, green, and blue channels independently. The red channel of the transformed digital image is formed by calculation of the following formula: 
     
       
         
           R′=r′ 
           txt 
           +r′ 
           ped 
         
       
     
     Where R′ represents the red channel of the transformed digital image, r′ txt  represents the modified red texture signal output from the texture modifier  33 , and r′ ped  represents the modified red pedestal signal output from the 3D LUT applicator  36 . In a similar manner the green and blue channels of the transformed digital image are computed by the following formulas: G′=g′ txt +g′ ped  and B′=b′ txt +b′ ped  respectively. 
     Referring to FIG. 3, the digital image channel input into the pedestal splitter  30  is split into a high pass signal and a low pass signal by a frequency splitter  70 . The spatial filtering operation preformed by the frequency splitter  70  is preferably with a Finite Impulse Response (FIR) Gaussian filter having a width of 2.5 pixels. The preferred value of the standard deviation of the Gaussian filter may vary with image size. The value of 2.5 pixels for the sigma value was derived by optimizing the present invention with 1024 by 1536 pixel size images. This Gaussian filter is a two-dimensional, lowpass filter whose filter coefficients may be derived by the following formula which is well known in the art: 
     
       
           G ( i,j )=1/(sigma sqrt(2π)) exp[−( i   2   +j   2 )/(2 sigma 2 )] 
       
     
     where 
     G(i,j)=the Gaussian filter coefficient at the (i,j) th  pixel 
     sigma=the standard deviation of the Gaussian filter (2.5) 
     π=the constant approximately 3.1415 . . . 
     Spatial filtering an image channel with a digital filter ( referred to in the art as two dimensional convolution) is well known in the art. A spatial filtering operation may be represented by the following expression:          y        (     m   ,   n     )       =       ∑     i   ,   j                         a     i   ,   j            x        (       m   -   i     ,     n   -   j       )                                  
     where y(m,n) represents the value of the signal output by the spatial filtering operation at location (m,n), x(m,n) represents the value of the signal input to the spatial filtering operation at location (m,n), and a i,j  represents the coefficients of the spatial filter. In the preferred embodiment, a i,j  is set equal to G(i,j). Spatial filtering of a signal by a Gaussian filter generates a lowpass signal. The lowpass signal output from the frequency splitter  70  is input into an avoidance signal generator  80  for forming an avoidance signal, as will be described in detail below. A texture signal generator  90  receives both the high pass signal and the avoidance signal and both signals are multiplied therein for producing a texture signal. 
     The pedestal generator  100  receives the original digital image channel and the texture signal and subtracts the texture signal from the digital image channel, which produces a pedestal signal. 
     Referring to FIG. 4, there is illustrated an exploded block diagram of the avoidance signal generator  80 . In this regard, the pre-avoidance channel is input into a frequency splitter  75  in order to generate a lowpass signal; the operation of a frequency splitter is described above. In the preferred embodiment, the standard deviation of the spatial filter used in this frequency splitter  75  is equal to the standard deviation of the spatial filter of the frequency splitter  70  (sigma=2.5 pixels.) However, those skilled in the art will recognize that this condition is not a requirement. The lowpass signal output from the frequency splitter  75  is input to the non-directional squared gradient calculator  110  for producing a non-directional squared gradient signal. This calculation is performed by first calculating the difference between the pixel and its upper vertical neighbor, as well as the difference between the pixel and its horizontal neighbor to the right. The non-directional squared gradient is the sum of the squares of these two differences. The non-directional squared gradient signal is then mapped by an artifact avoidance function applicator  120  to produce an artifact avoidance signal. 
     Referring to FIG. 5, this mapping function is performed by passing the non-directional squared gradient value through an artifact avoidance LUT formed in the following manner. For non-directional squared gradient values less than or equal to constant K 1 , then the artifact avoidance signal returned is 1.0. For non-directional squared gradient values greater than K 1  but less than constant K 2 , the artifact avoidance signal returned is equal to: 
     
       
         ½*(1+COS(π*(non-directional squared gradient− K   1 )/( K   2   −K   1 )). 
       
     
     For non-directional squared gradient values greater than K 2 , the artifact avoidance signal returned by the LUT is equal to 0.0. The constants K 1  and K 2  are dependent upon the data type of the digital image. The present invention utilizes digital images with a 12 bit description of each digital image channel at each pixel location. In the preferred embodiment, K 1 =148 and K 2 =1300. Alternatively, this mapping function may be performed by mapping the non-directional squared gradient value through any monotonically decreasing LUT ranging from 1.0 to 0.0 with increasing non-directional squared gradient. The artifact avoidance signal output from the artifact signal generator  80  is an example of a control signal that is used to modulate the highpass signal output from the frequency splitter  70  in order to generate the texture signal output from the texture signal generator  90 . 
     The invention has been described with reference to a preferred embodiment. However, it will be appreciated that variations and modifications can be effected by a person of ordinary skill in the art without departing from the scope of the invention. For instance, in the preferred embodiment, the 3D LUT anticipated inputs of the red, green, and blue signals. However, a 3D LUT may be devised which anticipates inputs of a single luminance signal and two chrominance signals. Those skilled in the art will realize that with a small amount of obvious modification, the present invention may be utilized in the described situation.