Patent Publication Number: US-7221807-B2

Title: Methods and systems for digital image characteristic adjustment using a neural network

Description:
BACKGROUND 
   Digital images can express a poor reproduction of a target scene due to various limitations in the sensing technology. Whether captured with a digital camera directly, by scanning a photograph or by some other method, sensor limitations can affect image quality. Aperture, exposure time, lighting conditions, sensor sensitivity and calibration and other factors can affect image characteristics. These variables can affect image sensing and recording such that the recorded image differs significantly from an observer&#39;s perception of the actual image target scene. These variations may comprise under-exposure, over-exposure, flatness, bi-modality and others. 
   An individual&#39;s perception of image quality and the accuracy with which an image reproduces it&#39;s target scene is also influenced by the subjective preferences of the viewer. While some viewer preferences can be modeled by mathematical functions, other preferences are not easily represented mathematically. 
   Under-exposed or over-exposed images can be adjusted by reshaping the luminance channel histogram. Generally, peaks and valleys are normalized to obtain a more constant luminance level throughout the image. However, in some scenarios, this technique can produce images that are more objectionable to a viewer than the un-adjusted image. 
   Human perception of lightness has been well studied. These studies are well documented in works such as: Adelson, E. H. (1993). “Perceptual Organization and the Judgment of Brightness.”  Science  262(24): 2042–2044; Adelson, E. H. (2000). Lightness Perception and Lightness Illusions.  The New Cognitive Neurosciences.  M. Gazzaniga. Cambridge, Mass., MIT Press: 339–351; Gilchrist (2002), A. Unified Luminance Anchoring Theory. 2002; and Pentland, E. H. A. a. A. P. (1996). The perception of shading and reflectance.  Perception as Bayesian Inference.  D. K. a. W. Richards. New York, Cambridge University Press: 409–423. 
   However, the human preference for various tone renderings of natural images is not well understood. A model for the prediction of human preferences for tone parameters is not presently available. Accordingly, automatic enhancement of image defects to more closely conform to viewer preferences can be difficult or impossible to perform using known techniques. 
   SUMMARY 
   The systems and methods of embodiments of the present invention comprise the application of a neural net to perform automatic image enhancements. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     In order that the manner in which the above-recited and other advantages and objects of the invention are obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which: 
       FIG. 1  is a diagram depicting an exemplary adjustment curve; 
       FIG. 2  is a diagram depicting an image adjustment tool of embodiments of the present invention; 
       FIG. 3  is a diagram depicting an exemplary tone enhancement system of embodiments of the present invention; 
       FIG. 4  is a diagram depicting an enhancement curve with interpolation points; 
       FIG. 5  is a diagram depicting neuron architecture; 
       FIG. 6  is a diagram depicting neural network architecture; 
       FIG. 7  is a diagram depicting a matrix form of a feed-forward neural network; 
       FIG. 8  is a diagram depicting a plot of a typical tangent sigmoid transfer function; 
       FIG. 9  is a diagram depicting embodiments of the present invention comprising a neural net and image enhancer; 
       FIG. 10  is a diagram depicting embodiments of the present invention comprising luminance enhancement data; 
       FIG. 11  is a diagram depicting embodiments of the present invention comprising observer preference collection; 
       FIG. 12  is a diagram depicting embodiments of the present invention comprising luminance histogram data and luminance channel enhancement; 
       FIG. 13  is a diagram depicting embodiments of the present invention comprising luminance histogram input for a neural net; and 
       FIG. 14  is a diagram depicting embodiments of the present invention comprising observer preference collection and image enhancement. 
   

   DETAILED DESCRIPTION 
   The figures listed above are expressly incorporated as part of this detailed description. It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the systems and methods of the present invention, as represented in  FIGS. 1 through 8  is not intended to limit the scope of the invention, as claimed, but is merely representative of the presently preferred embodiments of the invention. Some embodiments of the present invention will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout. 
   Data Collection Tools 
   The systems and methods of embodiments of the present invention typically comprise techniques and data that represent image enhancements preferred by human observers. In some embodiments, a representative set of images may be generated that comprises images without perceivable defects and images with various defects such as tone problems, for example. These representative image sets may be chosen from existing images with known characteristics or they may be generated using image adjustment tools such as, for example tone adjustment tools. In some embodiments, interactive adjustment tools may be employed. Interactive tools allow a user to make subjective decisions during the adjustment process and create a preferred image in a single step. Image characteristics and adjustments to each image are recorded. 
   Image preferences may be recorded during the image creation or adjustment process when the subjective preferences of the image creator or adjuster are to be used. Image preferences may also be obtained by presenting an observer or group of observers with a set of images with known characteristics and recording the preferences of each observer. 
   Some embodiments of the present invention employ image adjustment tools to create and identify image characteristics. Some of these tools may employ a curve utility, which allows a user to adjust the mapping of original image data onto new image data by manipulating the shape of a two-dimensional curve. In some embodiments, a luminance image channel is mapped through the curve function to determine new image channel values. In some embodiments, the luminance channel of a La*b* image is used in this manner. With reference to  FIG. 1 , an original image luminance value, Lin  2  is mapped through a function represented by curve  4  to produce an adjusted luminance value, Lout  6 . 
   These curve tools may use splines to define and change the shape of the curve. Splines are generally a convenient and intuitive representation for user interactive design of smooth curves. Typically, these image editing programs allow a user to change the shape of a curve and preview the change in the appearance of the image. In this manner, a user can interactively modify an image until the image satisfies the user&#39;s preferences and save the modified image data as a representation of user preference. 
   Some image adjustment tools used in embodiments of the present invention may comprise luminance channel histogram displays. These histograms can provide a user with valuable information about the relationships and distributions of various image characteristics such as tone. In reference to  FIG. 2 , some embodiments of the present invention may comprise image adjustment tools, which comprise a program with a Graphical User Interface (GUI) that is tailored for efficient adjustment of image characteristics by a user. Some embodiments of this tool, as shown in  FIG. 2 , may comprise a display comprising a graphical representation of an original image  10  and a graphical representation of an adjusted image  12 . 
   Embodiments may also comprise histograms  14 ,  16  that represent image characteristics. In the exemplary embodiment shown in  FIG. 2 , an original image luminance histogram  14  is displayed adjacent to the original image graphical representation  10 . An adjusted image luminance histogram  16 , may also be displayed adjacent to the adjusted image graphical representation  12 . In this manner, a user may view the effects of the adjustments being performed in real time and quickly adjust the image to match the user&#39;s preferences. 
   An image adjustment tool  11  may comprise a graphical display of the function  18  through which the luminance (or other data) is mapped. In interactive embodiments, a user may use a pointing device or other apparatus to drag or otherwise adjust the shape of the displayed curve  22 . As a curve is adjusted in this manner, the adjusted histogram  16  and adjusted image representation  12  may be updated in real-time, at appropriate intervals or otherwise thereby allowing a user to adjust an image to their preferences with a simple stroke of an input device. In some embodiments, a slide bar may be used for convenient adjustment of image characteristics. In some embodiments a reset button  20  may be used to clear user input data to allow a user to restart an enhancement data input procedure. 
   Image Adjustment 
   In reference to  FIG. 3 , the system architecture of some exemplary embodiments of the present invention may be described. These particular embodiments utilize an input image conforming to an sRGB standard color space. Other embodiments may utilize other color spaces and formats for input images. 
   The systems illustrated in  FIG. 3 , may be characterized as having two branches. In the top branch  30 , an image is analyzed and an curve, such as a tone enhancement curve is determined. In the bottom branch  32 , the enhancement curve is applied to the image. 
   In these embodiments, a tone enhancement is effected, however, other characteristics and combinations thereof may be adjusted in other embodiments. In the bottom branch  32 , a change an image characteristics, such as tone, is applied by first transforming the image into a luminance-segregated format  34  in which luminance may be adjusted independent of chrominance values. In these examples, an La*b* color space is used. Once luminance has been separated, the luminance channel may be modified  36  in many ways. In the illustrated embodiments, one-dimensional look-up table (1D LUT) is used. The 1D LUT is a discrete representation of the enhancement curve. 
   Image processing can be a calculation intensive procedure that can require significant processor and other resources as well as occupy significant amounts of time. When the methods and systems of embodiments of the present invention are to be integrated into apparatus with limited resources, such as some printers and other devices, a simplification of these processes can yield significant performance benefits, particularly in printing environments where printer output speed is critical. Thus, the application of a 1D LUT decreases computational and resource requirements and/or decreases processing time thereby boosting performance. 
   Once the image has been adjusted, the image may be transformed back to its original format, such as the sRGB standard color space  38 . 
   The top branch  30 , shown in  FIG. 3 , the input image is first analyzed. This process may begin by decimating  40  the image. Decimation is performed to reduce the amount of data to be analyzed. In embodiments where computational resources are high, decimation  40  may be skipped, however, in most embodiments, resources will be limited and decimation will provide computational efficiency. 
   After decimation  40 , is performed, an input image is converted to a luminance-specific format  42 . In these embodiments, an La*b* space is used. Use of a luminance-specific format allows the luminance channel to be manipulated independent of the chrominance characteristics. The La*b* color space may be referred to as a perceptually-uniform color space. 
   After conversion to a luminance-specific format, a luminance channel histogram may be computed  44 . The luminance channel histogram may provide useful information toward the design of the enhancement curve. This histogram contains information about the distribution of lightness values in the entire image. Other histograms may also be used for image adjustment purposes. 
   The luminance channel histogram, in the illustrated embodiments of  FIG. 3 , serves as the input to a neural network  46 . The neural network  46  analyzes the histogram data and produces an enhancement curve shape. This enhancement curve is then used to construct a 1D LUT  48 , which may be used to perform image tone enhancement and adjustment  36 . 
   As a successful model has not been created to reflect the human preferences recorded in the image data, embodiments of the present invention may use the observer image preference data to train a neural net to reproduce the desired image enhancements. This can be done without explicitly modeling the human visual system or subjective observer preferences. 
   Neural Network 
   Embodiments of the present invention utilize a neural network to approximate the subjective, human-generated preference data. In some embodiments, this data is summarized as a curve shape using a small set of parameters. As an example, cubic spline interpolation may be used to describe a curve. A good approximation of an arbitrary curve would include parameters that define the location, slope and curvature of select point along the curve. To simplify this representation we may describe a curve using interpolation points. The remainder of the curve can be reconstructed using Bézier cubic spline interpolation with some simple assumptions to define the slope and curvature of the interpolation points. In practice, these approximation methods have efficiently produced reasonable results. Other methods may be used to interpolate between curve points and select points along a curve. 
   In reference to  FIG. 4 , a tone enhancement curve  50  is illustrated. The enhancement curve is used to map original image luminance data, Lin, into output luminance data, Lout, thereby correcting or adjusting the image luminance to account for preferences. The curve may be approximated by a series of points  52  along the curve. Portions of the curve between the selected points  52  may be approximated using interpolation techniques. 
   In embodiments of the present invention, a neural network outputs interpolation points, as described with reference to  FIG. 4 , These values determine the curve height at specified location along the X axis. Generally, any reasonable number of points may be used. 
   A more detailed description of the structure of the network of embodiments of the present invention is given with reference to  FIGS. 5 and 6 . The basic unit of a neural network is a neuron as shown in  FIG. 5 . Inputs  60  are accepted into the neuron and input values are multiplied by neuron weights  62 , which are constants set by network training procedures. After inputs  60  are weighted, the weighted inputs are summed  64 . The neuron bias value  66  is then added to this sum  64  to yield a preliminary neuron output value, a k . This preliminary value is a biased, weighted sum of neuron inputs that may be expressed mathematically as equation 1: 
   
     
       
         
           
             
               
                 
                   a 
                   k 
                 
                 = 
                 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       N 
                     
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       
                         w 
                         
                           k 
                           , 
                           i 
                         
                       
                       ⁢ 
                       
                         h 
                         i 
                       
                     
                   
                   + 
                   
                     b 
                     k 
                   
                 
               
             
             
               
                 Eq 
                 . 
                 
                     
                 
                 ⁢ 
                 1 
               
             
           
         
       
     
   
   Where h 1−i  are the inputs to the k th  neuron, w k,1−N  are trained weights for the k th  neuron, and b k  is the trained bias value for the k th  neuron. 
   Before being output from the neuron, a transfer function  68  may be applied to the preliminary value, a k , in some embodiments. Applying this transfer function  68  results in a neuron output, o k . In some embodiments and certain layers of neural nets of some embodiments, a transfer function may be omitted. In these cases a k  may be equal to o k . 
   A neural network, such as those used in embodiments of the present invention, may comprise multiple layers  70 ,  72 . While each layer comprises an array of neurons with architectures similar to that described with reference to  FIG. 5 , the transfer functions may vary between layers. Output from any neuron may leave the network as final output or enter another layer of the network as an input. 
   In some embodiments of the present invention, a two layer network is utilized. For the neurons in the first layer of the network the output value o k  is calculated by applying a non-linear transfer function  74  to a k . In some embodiments, a Tangent Sigmoid (Tansig) transfer function is used. A transfer function used in some embodiments may be expressed mathematically as Equation 2. 
   
     
       
         
           
             
               
                 
                   o 
                   k 
                 
                 = 
                 
                   
                     2 
                     
                       1 
                       + 
                       
                         e 
                         
                           
                             - 
                             2 
                           
                           * 
                           
                             a 
                             k 
                           
                         
                       
                     
                   
                   - 
                   1 
                 
               
             
             
               
                 Eq 
                 . 
                 
                     
                 
                 ⁢ 
                 2 
               
             
           
         
       
     
   
   The neurons of some embodiments are connected to form a two-layer network as shown in  FIG. 6 . The first layer  70  comprises S neurons. For this layer, every histogram bin count serves as an input  76  to the neurons. The second layer&#39;s input  78  is the output from the first layer  70 . In these illustrated embodiments, every output from layer one  70  is an input to every neuron in layer two  72 . In layer two  72 , the transfer function may be linear and in some embodiments, a transfer function may be omitted. 
   The response of a neural network to a given input can be calculated using matrix operations and the transfer functions. This process is illustrated in  FIG. 7 . Consider for a moment the calculation of the response of a single neuron as described with reference to  FIG. 7 . Equation 1 gives the form for calculating the neurons response without including the transfer function. This equation can be reduced to a k ={right arrow over (w)} k ·{right arrow over (h)}+b k , where the weights and input values are concatenated into a-vectors. For a layer, all the weight vectors for the neurons can be used to form a weight matrix 
           W   =       [             w   →     1   T             ⋮               w   →     S   T           ]     .           
Then the preliminary output of every neuron in the layer (not including the transfer function) can be represented as {right arrow over (a)}=W·{right arrow over (h)}+{right arrow over (b)}, where the bias values and the preliminary output values (before transfer function) are represented as vectors. To calculate the final output values for the layer each element of {right arrow over (a)} is passed through the transfer function to obtain {right arrow over (o)}. The output vector of the first layer can then be used as the input vector for the next layer. Then the process repeats until there are no-more layers in the network.
 
   For the simple network architecture of the illustrated embodiments, the network response may be calculated using the following three steps:
         {right arrow over (a)} 1 ={right arrow over (W)} 1 ·{right arrow over (h)}+{right arrow over (b)} 1  Multiply the histogram with the first layer&#39;s weight matrix, then vector add the first layer&#39;s bias values. The result is an S element vector {right arrow over (a)} 1 .   Apply the first layer&#39;s transfer function to each element of {right arrow over (a)} 1 . In some embodiments, a tangent sigmoid function (Equation 2) is used. Denote the result as {right arrow over (o)} 1 .   {right arrow over (a)} 2 =W 2 ·{right arrow over (o)} 1 +{right arrow over (b)} 2  Multiply the output of first layer with the second layer&#39;s weight matrix, then vector add the second layers bias values. The result is a P element vector {right arrow over (a)} 2 .       

   In some embodiments, there is no transfer function on the second layer so the output of the network is {right arrow over (a)} 2 . The output layer could, however, use a non-linear transfer function or some other function in other embodiments. 
   Network Training 
   The networks of some embodiments of the present invention were tested and trained using a database of 525 images. In these embodiments, each image is associated with an enhancement data file comprising data that relates to the image adjustments required to meet the preferences of one or more observers. In some embodiments, images created using different devices, such as a variety of digital cameras, scanners or other devices and images with commonly occurring problems are used in network training to provide a robust data set that will yield a network that will be functional in a wide variety of applications. 
   In the training process, a portion of the available enhanced images may be selected for training the network and another portion of the images may be selected for testing the network. In these cases, a training set of images and their associated enhancement data may be used to select the weight and bias values for the neural network configurations being developed while a testing set of images is reserved to evaluate the network&#39;s ability to generalize the solution. 
   In some embodiments, the weight and bias values are selected to minimize the error between the network solution and the human-generated enhancement. This may be done through a Bayesian regularization backpropogation algorithm or by other methods. In particular embodiments, a Bayesian algorithm described in MacKay, D. “Bayesian Interpolation”, Neural Computation 4(3), pp. 415–447 (1992) M.I.T. Press and MacKay, D. “A Practical Bayesian Framework for Backprop Networks,” Neural Computation 4(3), pp. 448–472 (1992) M.I.T. Press., hereby incorporated by reference, may be used. 
   Network Implementations 
   The neural networks of some embodiments of the present invention may be implemented on processors with or without floating point arithmetic units. A fixed point (integer) implementation for a single neuron is described as an example. 
   Embedded environments typically have limited resources and many embedded microprocessors do not contain floating point units or micro-code operators for complex math functions like exponential. To overcome these limitations, floating point operations may be converted to fixed point operations and complex math functions may be mapped as 1D LUTs. 
   The following equations summarize the calculations required to simulate a feed-forward neural network utilized in some embodiments of the present invention.
 
 {right arrow over (a)} 1= W 1· {right arrow over (h)} +{right arrow over ( b )}1  Eq. 3
 
                   o1   ⁡     (   k   )       =       2     1   +     e       -   2     *     a1   ⁡     (   k   )               -   1             Eq   .           ⁢   4                 {right arrow over (o)} 2= W 2· {right arrow over (o)} 1+ {right arrow over (b)} 2  Eq. 5 
   In these embodiments, tailored for limited resource environments, we can assume that the architecture will not be able to handle integer division except by powers of two (right shift), but other configurations and embodiments may use standard integer division. 
   In these embodiments, the input histogram vector&#39;s elements are first quantized to an unsigned 8-bit number. The elements of the network&#39;s output vector {right arrow over (o)} 2  are also represented as unsigned 8-bit numbers. 
   One of the more complicated parts in converting the feed-forward network to integer arithmetic is the transfer function. The tangent sigmoid function contains an exponential (Equations 2 &amp; 4). An embedded programming environment typically does not have the ability to compute the exponent of a number. Consequently, the solution is to replace the tangent sigmoid function calculation with a LUT.  FIG. 8  shows a plot of the tangent sigmoid function. It is important to note that the function saturates at one and minus one, so the range of o 1 (k) is [−1,1]. This saturation occurs for abs(a 1 (k))&gt;3. The function is also symmetrical. Because of these properties, the LUT will be generated to approximate the tangent sigmoid function for the following range 0≦a 1 (k)≦4. The sign of a 1 (k) will be preserved in a separate sign bit and restored after the LUT has been applied. Equation 8 gives a formula that may be used for the conversion of the tangent sigmoid function to a LUT.
 
 SF 1 *{right arrow over (a)} 1= SF 1 *W 1· {right arrow over (h)}+SF 1 *{right arrow over (b)} 1 SF 1=2 Pow1   Eq.6
 
                   LUT   ⁡     [   k   ]       =       (   int   )     ⁢     (       SFLUT   *     (       2     1   +     e         -   2     *   4   *   k     SFLUT           -   1     )       +   0.5     )     ⁢           ⁢           SFLUT   =     2   PowLUT                 k   ∈     [     0   ,   SFLUT     ]                       Eq   .           ⁢   7                 SFLUT   *     o1   ⁡     (   k   )         =       sign   (     S   ⁢           ⁢   F   ⁢           ⁢   1   *     a   →     ⁢   1   ⁢     (   k   )       )     *     LUT   [       SFLUT   *     (     S   ⁢           ⁢   F   ⁢           ⁢   1   *     a   →     ⁢   1   ⁢     (   k   )       )         4   *   S   ⁢           ⁢   F   ⁢           ⁢   1       ]               Eq   .           ⁢   8                 SF 2 *SFLUT* {right arrow over (o)} 2= SF 2 *W 2·( SFLUT*{right arrow over (o)} 1)+ SF 2 *SFLUT*{right arrow over (b)} 2 SF 2=2 Pow2   Eq.9 
Equations 6–9 give illustrate formulae that may be used in the design process of the conversion. Equations 10–13 indicate the actual implementation of these exemplary formulae in C-code after some simplification to minimize the number of multiplications and divisions.
 
   Other parts of these processes not shown in these equations are the clipping operations performed before the LUT is applied and performed on the output {right arrow over (o)} 2 . Previously, we mentioned that the output saturates for inputs greater than four for the tangent sigmoid function. For this reason we clip any input a 1 (k)&gt;4 to 4. In some embodiments, we clip any index to the LUT greater than SFLUT to SFLUT. In practice we have also found that the output o 2 (k) values can exceed 255. To solve this problem, we can again clip o 2 (k) values that are greater than 255 to 255 in some embodiments. 
   
     
       
         
             
             
             
           
             
                 
             
           
          
             
               
                 
                   
                     
                       
                         Int 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           a 
                           → 
                         
                         ⁢ 
                         1 
                       
                       = 
                       
                         
                           IntW1 
                           · 
                           
                             h 
                             → 
                           
                         
                         + 
                         
                           Int 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             b 
                             → 
                           
                           ⁢ 
                           1 
                         
                       
                     
                   
                 
               
               
                 
                   
                     
                       
                         
                           
                             
                               
                                 Int 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   a 
                                   → 
                                 
                                 ⁢ 
                                 1 
                               
                               = 
                               
                                 SF1 
                                 * 
                                 
                                   · 
                                   
                                     a 
                                     → 
                                   
                                 
                                 ⁢ 
                                 1 
                               
                             
                           
                         
                         
                           
                             
                               IntW1 
                               = 
                               
                                 SF1 
                                 * 
                                 W1 
                               
                             
                           
                         
                         
                           
                             
                               
                                 Int 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   b 
                                   → 
                                 
                                 ⁢ 
                                 1 
                               
                               = 
                               
                                 SF1 
                                 * 
                                 
                                   b 
                                   → 
                                 
                                 ⁢ 
                                 1 
                               
                             
                           
                         
                       
                         
                     
                   
                 
               
               Eq.10 
             
             
                 
             
             
               
                 
                   
                     
                       
                         Into1 
                         ⁡ 
                         
                           ( 
                           k 
                           ) 
                         
                       
                       = 
                       
                         
                           sign 
                           ⁡ 
                           
                             ( 
                             
                               Int 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 a 
                                 → 
                               
                               ⁢ 
                               1 
                               ⁢ 
                               
                                 ( 
                                 k 
                                 ) 
                               
                             
                             ) 
                           
                         
                         * 
                         
                           LUT 
                           ⁡ 
                           
                             [ 
                             
                               
                                 Int 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   a 
                                   → 
                                 
                                 ⁢ 
                                 1 
                                 ⁢ 
                                 
                                   ( 
                                   k 
                                   ) 
                                 
                               
                               SF 
                             
                             ] 
                           
                         
                       
                     
                   
                 
               
               
                 
                   
                     
                       
                         
                           
                             
                               SF 
                               = 
                               
                                 
                                   4 
                                   * 
                                   SF1 
                                 
                                 SFLUT 
                               
                             
                           
                         
                         
                           
                             
                               
                                 Into1 
                                 ⁡ 
                                 
                                   ( 
                                   k 
                                   ) 
                                 
                               
                               = 
                               
                                 SFLUT 
                                 * 
                                 
                                   o1 
                                   ⁡ 
                                   
                                     ( 
                                     k 
                                     ) 
                                   
                                 
                               
                             
                           
                         
                       
                         
                     
                   
                 
               
               Eq.11 
             
             
                 
             
             
               
                 
                   
                     
                       
                         Int 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           o 
                           → 
                         
                         ⁢ 
                         2 
                       
                       = 
                       
                         
                           IntW2 
                           · 
                           
                             ( 
                             
                               Int 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 o 
                                 → 
                               
                               ⁢ 
                               1 
                             
                             ) 
                           
                         
                         + 
                         
                           Int 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             b 
                             → 
                           
                           ⁢ 
                           2 
                         
                       
                     
                   
                 
               
               
                 
                   
                     
                       
                         
                           
                             
                               
                                 Int 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   o 
                                   → 
                                 
                                 ⁢ 
                                 2 
                               
                               = 
                               
                                 SF2 
                                 * 
                                 SFLUT 
                                 * 
                                 
                                   o 
                                   → 
                                 
                                 ⁢ 
                                 2 
                               
                             
                           
                         
                         
                           
                             
                               IntW2 
                               = 
                               
                                 SF2 
                                 * 
                                 W2 
                               
                             
                           
                         
                         
                           
                             
                               
                                 Int 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   b 
                                   → 
                                 
                               
                               = 
                               
                                 SF2 
                                 * 
                                 SFLUT 
                                 * 
                                 
                                   b 
                                   → 
                                 
                                 ⁢ 
                                 2 
                               
                             
                           
                         
                       
                         
                     
                   
                 
               
               Eq.12 
             
             
                 
             
             
               
                 
                   
                     
                       
                         
                           o 
                           → 
                         
                         ⁢ 
                         2 
                       
                       = 
                       
                         
                           Int 
                           ⁢ 
                           
                               
                           
                           ⁢ 
                           
                             o 
                             → 
                           
                           ⁢ 
                           2 
                         
                         SFout 
                       
                     
                   
                 
               
               SFout = SF2 * SFLUT 
               Eq.13 
             
             
                 
             
          
         
       
     
   
   It should be noted that these exemplary embodiments are designed specifically for embedded environments and other environments with limited resources. Accordingly, steps have been taken to enable and optimize processes for these environments. Other embodiments, used in alternative environments may use other transfer functions and may employ alternative methods to the clipping techniques described herein. 
   In reference to  FIG. 9 , embodiments of the present invention may comprise methods and systems that include linking image characteristic data to image adjustment data  80 . In some embodiments, image characteristic data may comprise an image bitmap, an image luminance or color channel, a histogram of an image characteristic, or other data that represents an image. Image adjustment data may comprise any data that communicates a change in one or more image characteristics. Image adjustment data may comprise a change in an image characteristic, an adjustment curve or function with which an image characteristic may be modified or other data that may communicate a change in one or more image characteristics. 
   Linked image characteristic data and image adjustment data may be used to train a neural net  82  to reproduce the preference choices of an observer. In some embodiments, image characteristic data and image adjustment data are used to train a neural net to produce an enhancement curve that can be used to enhance or adjust  84  any arbitrary image to better conform to observer preferences. 
   In reference to  FIG. 10 , some embodiments of the present invention may link image characteristic data and image adjustment data that comprises preferred luminance enhancement data  86 . Preferred luminance enhancement data may comprise data that communicates changes in luminance values preferred by an observer. 
   This linked data may then be used to train a neural net to generate generalized image luminance enhancement data  88 . This generalized data may then be used to enhance  90  an arbitrary subject image to better conform to the luminance preferences of an observer. 
   In reference to  FIG. 11 , observer preference data may be obtained by presenting an observer with images  92 . In some embodiments, one or more base images may be displayed with one or more adjusted images that represent various adjustments to the base image. Some embodiments may comprise an interactive image display tool that displays a base image and an adjusted image and allows a user to edit the adjusted image until it conforms to the user&#39;s preferences. The characteristics of each base image and each adjusted image are known and the differences may be recorded. When an observer has designated a preference  94 , the preference is recorded  96  along with the data that identifies the differences between the base image and the preferred adjusted image. 
   This process may be repeated until sufficient data is obtained to represent one or more observer preferences  98 . This amount of information may vary depending on the type and number of adjustments used in the process. The sufficiency or substantiality of the data may be determined through statistical methods or my other means. Once a substantial amount of data has been obtained from one or more observers, the preference data may be used to train a neural net to generate data that can be used to reproduce the adjustments and enhancements preferred by the observer  100 . 
   In reference to  FIG. 12 , some embodiments of the present invention may comprise linking image characteristic data to image enhancement data  102 . Image enhancement data may comprise data that reflects differences between a base image and one or more adjusted images. Image enhancement data may comprise histogram data that represent image characteristics. Once data is presented to an observer, an observer&#39;s preferences regarding image characteristic changes may be collected and recorded  104 . 
   The collected data may then be used to train a neural net to generate generalized image enhancement data  106  that can be used to enhance arbitrary images to better conform to observer preferences. 
   Image enhancement may comprise the conversion of a subject image to a luminance-specific format  108 , if the image is not initially in this type of format. In these embodiments, luminance data may be modified independently of other image data. 
   A luminance histogram may be created  110  to represent image luminance characteristics. In some embodiments, luminance histogram data may be used as input to a neural net that has been trained  106  with image characteristic data comprising luminance histograms. In these embodiments, the neural net processes the histogram data  114  and creates an enhancement curve Look-Up Table (LUT) with point data that approximates an enhancement curve. The LUT may then be applied to an image  116  to enhance the image to more closely conform to observer luminance preferences. 
   In reference to  FIG. 13 , in some embodiments of the present invention, image characteristics may be represented by an image luminance histogram and observer luminance preferences may be represented by histograms or histogram changes. This data may be linked to the image luminance histogram  120 . A neural net may be trained  122  using the linked data. The neural net may be trained to generate generalized image enhancement data that may be used to adjust arbitrary images. 
   In these embodiments, a subject image is typically converted to a luminance-segregated format  124 , when necessary, to allow for independent luminance modification. A subject image luminance histogram is then created  126  and input to the neural net  128  for analysis. Once analysis is complete, the neural net may create an enhancement curve LUT  130  that can be used to enhance the subject image so that it more closely complies with observer luminance preferences  132 . The end result is an image  134  that is preferred by an observer. 
   In reference to  FIG. 14 , systems of embodiments of the present invention may comprise an image display  140  for display of images to an observer. These embodiments may further comprise an observer preference data collector  142  for collecting data related to observer preferences. Collected data may be recorded in an image characteristic recorder  144  that may be used to record image characteristic data, image adjustment data, observer preference data and other information. 
   The recorded data is sent to a neural net trainer  146  so that a neural net  148  may be trained to generate data that can be used to enhance images so that the images more closely conform to observer preferences. 
   Subject image data  152  may be input to the trained neural net  148  and processed by the net  148  to produce subject image enhancement data  150  that reflects observer preferences. Subject image enhancement data  150  and the initial subject image data  152  are sent to an image enhancer  154  where the subject image data  152  may be modified and enhanced to produce a preferred image  156  that more closely complies with observer preferences. 
   The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.