Abstract:
In one embodiment of the invention, there is provided a method of correcting a captured image for lens shading artifacts, comprising for a given lens determining a function L(x,y) being a lens shading correction function to be applied to images captured by the lens in order to correct for lens shading artifacts; applying a sampling technique to sample the function L(x,y) at selected points; and storing the sampled function L(x,y) in memory.

Description:
FIELD 
       [0001]    Embodiments of the invention relate to a method and system to correct an image for artifacts due to lens shading. 
       BACKGROUND OF THE INVENTION 
       [0002]      FIG. 1  of the drawings shows a typical image capturing system comprising a lens system  100  and a sensor (or film)  102  to capture an image illuminated by a light source  104 . 
         [0003]    In image capturing systems involving the lenses such as the one shown in  FIG. 1  of the drawings, the light distribution by the lens system is non-uniform, causing a captured image to have some light intensity fall-off toward the edges of the image. This artifact is called the lens shading artifact or vignette. 
         [0004]      FIG. 2  of the drawings illustrates the effect of the lens shading artifact on a captured image. Referring to  FIG. 2 , it will be seen that a captured image  200  includes a central circular region  202  where the light intensity is the brightest. The light intensity fails off radially away from the center region  202 . Edges  204  at the corners at the image  200  show the greatest fall off in light intensity. 
         [0005]    The lens shading artifact may be removed by multiplying the light intensity l(x,y) at the image location (x,y) by a lens shading correction amount L(x,y) 
         [0000]        I   corrected ( x,y )= l ( x,y )× L ( x,y )  (1)
 
         [0006]    In reality, for image capturing systems that include an image sensor, the sensor might contribute offset terms that would make the Equation (1) more complex. The offset terms are not considered so as not to obscure the invention. However, one skilled in the art will know how to modify Equation (1) with the offset terms. 
         [0007]    The lens shading artifact may be modeled and the function L(x,y) may be computed from the model. Because the function L(x,y) is a two-dimensional (2D) function with two variables x and y, its model tends to be complicated and expensive to implement in hardware. 
       SUMMARY 
       [0008]    In one embodiment of the invention, there is provided a method of correcting a captured image for lens shading artifacts, comprising:
       for a given lens determining a function L(x,y) being a lens shading correction function to be applied to images captured by the lens in order to correct for lens shading artifacts;   applying a sampling technique to sample the function L(x,y) at selected points; and   storing the sampled function L(x,y) in memory.       
 
         [0012]    In another embodiment of the invention, there is provided a method of correcting a captured image for lens shading artifacts due to a lens used in generating the captured image, comprising:
       reading image data corresponding to the captured image;   selecting stored sample points for a two-dimensional lens shading correction function for the lens from memory, and   in respect of image data falling between the sample points then generating missing values for the two-dimensional lens shading correction function based on the stored sample points.       
 
         [0016]    In another embodiment of the invention, there in provided an image processor, comprising:
       data reconstruction logic to reconstruct missing data points for a two-dimensional lens shading correction function; and   an image correction block based on the reconstructed missing data.       
 
         [0019]    In yet another embodiment of the invention there is provided an image capture system, comprising:
       camera optics;   an image sensor; and   an image processor to process an image captured by the image sensor using the camera optics, the image processor comprising:   data reconstruction logic to reconstruct missing data points for a two-dimensional lens shading correction function; and   an image correction block based on the reconstructed missing data.       
 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0025]      FIG. 1  shows an image capturing system. 
           [0026]      FIG. 2  illustrates the lens shading effect. 
           [0027]      FIG. 3A  shows a high-level block of an imaging system comprising an image processor that includes lens shading correction logic in accordance with one embodiment of the invention. 
           [0028]      FIG. 3B  shows a block diagram of the components the lens shading correction logic of  FIG. 3A . 
           [0029]      FIG. 4  shows a lens shading correction function L(x,y) for the image capturing system of  FIG. 1 . 
           [0030]      FIG. 5  illustrates a uniform sampling grid used to sample the lens shading correction function L(x,y), in accordance with one embodiment of the invention. 
           [0031]      FIG. 6  shows a flowchart of operations performed to generate sample points corresponding to the lens shading correction function L(x,y) of  FIG. 4 , in accordance with one embodiment of the invention. 
           [0032]      FIG. 7  shows a flowchart of operations to correct a captured image for the lens shading artifact based on the lens shading correction function L(x,y), in accordance with one embodiment of the invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0033]    In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these specific details. In other instances, structures and devices are shown only in block diagram form in order to avoid obscuring the invention. 
         [0034]    Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearance of the phrases “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments. 
         [0035]    In one embodiment, the present invention disclose a method of correcting for lens shading artifacts in a captured image based on a two-dimensional (2D) lens shading correction function for the lens used to capture the image. 
         [0036]    The method will be described with reference to  FIG. 1  of the drawings. 
         [0037]    Referring to  FIG. 1 , there is shown a camera system  300  in accordance with one embodiment of the invention. The camera system  300  includes camera optics  302  coupled to an image sensor  304 . The image sensor  304  captures images through a lens  303  of the camera optics  302  and stores the images in an image buffer  306 . Image processor  308  processes images captured by the image sensor  304 . The image processor  308  includes lens shading correction logic  310 . The camera system  300  has been greatly simplified by leaving out a great many components so as not to obscure the present invention. However, one of ordinary skill in the art would understand these components to form the camera system  300 . 
         [0038]    It is to be understood that the camera system  300  is representative of any device that includes a camera capable of capturing images. Thus, digital cameras, camera-equipped mobile phones, etc. would be representative examples the camera system  300 . 
         [0039]    The lens shading correction logic  310  comprises the logic required to perform the method. In accordance with embodiments of the invention, the lens shading correction logic  310  may be implemented in software, hardware, or a combination of both software and hardware. 
         [0040]    Referring to  FIG. 3 , in one embodiment the lens shading correction logic  310  includes lens shading correction parameters  312 , data reconstruction logic  314 , and an image correction block  316 . The function and operation of these components will be clear from the description below. 
         [0041]    The method comprises the following broad steps: construct the surface for the function L(x,y), subsample the surface to build a two-dimensional (2D) table, and reconstruct the function L(x,y) from the table data. 
       1. Construct L(x,y) 
       [0042]    In accordance with one embodiment of the invention, in order to construct the surface L(x,y), a uniform white or gray image is captured through the lens whose lens shading surface L(x,y) is to be determined, i.e. the lens  303  of the system  300 . If the captured image is I(x,y) then based on equation (1): 
         [0000]    
       
         
           
             
               
                 
                   
                     L 
                      
                     
                       ( 
                       
                         x 
                         , 
                         y 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         I 
                         corrected 
                       
                        
                       
                         ( 
                         
                           x 
                           , 
                           y 
                         
                         ) 
                       
                     
                     
                       I 
                        
                       
                         ( 
                         
                           x 
                           , 
                           y 
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0043]    The image data captured at the center of the lens  303  does not have the lens shading artifact. Thus, in one embodiment, the corrected image is normalized to the brightness of image data captured through the center of the lens  303 . 
         [0044]    In one embodiment, the brightest spot in the captured image is deemed to have been captured through the center of the lens, so 
         [0000]        I   corrected ( x,y )=Max( l ( x,y ))  (3)
 
         [0045]    Combining (3) and (2), yields the following equation: 
         [0000]    
       
         
           
             
               
                 
                   
                     L 
                      
                     
                       ( 
                       
                         x 
                         , 
                         y 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       Max 
                        
                       
                         ( 
                         
                           I 
                            
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         ) 
                       
                     
                     
                       I 
                        
                       
                         ( 
                         
                           x 
                           , 
                           y 
                         
                         ) 
                       
                     
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0046]    In one embodiment, to improve the data for L(x,y), the image data l(x,y) may be computed as an average over multiple image captures. On the final l(x,y), a local smoothing filter such as moving average or curve fitting may be used to improve the data. 
         [0047]    Referring to  FIG. 4  of the drawings, reference numeral  400  shows surface L(x,y) constructed for the lens  303 , in accordance with embodiment of the invention. 
       2. Build 2D Table 
       [0048]    If an image to be corrected has width W and height H, then the camera system  300  must store W×H points of data for the lens shading correction function L(x,y). For example, if the system  300  were a three megapixel camera then the system  300  would be required to store 3 million data points in its memory. 
         [0049]    In one embodiment the method of the present invention advantageously cuts down on the data points that the system  300  would need to store in memory for the function L(x,y) in order to correct an image for the lens shading artifact. 
         [0050]    The method includes building a 2D table of data points of the function L(x,y). In building the table most of the data for the function L(x,y) is discarded while only a few sample points for the function L(x,y) are kept. In one embodiment, the missing data for the function L(x,y) corresponding to the discarded data points may be calculated based on the stored data points, thus effectively reconstructing the lens shading correction function L(x,y). The calculation of the missing data is performed by the lens shading correction logic  310  at the time of processing a captured image. 
         [0051]    Since the function L(x,y) for most lenses is very smooth, in reality, more than 99.91 of the data points for the function L(x,y) may be discarded. 
         [0052]    To build the 2D table, in one embodiment data for the function L(x,y) is sampled and stored in a memory as the lens shading correction parameters  312  of the system  300 . The non-sampled data is discarded. The sampling points are chosen carefully so that the missing data can be reconstructed with minimum errors. 
         [0053]    The invention is not limited to any particular sampling method. In one embodiment, the sample paints are fixed and equally spaced. For this type of sampling more storage but less processing power would be required. On the other hand, a variable sample points would take less storage but might need a lot of processing power 
         [0054]      FIG. 5  illustrates a uniform sampling grid  500  used to sample the lens shading correction function L(x,y), in accordance with one embodiment of the invention. 
         [0055]    The fixed and equally spaced sample points are located at the grid intersections. 
         [0056]      FIG. 6  of the drawings shows the operations performed in determining L(x,y) building the 2D table, in accordance with one embodiment. Referring to  FIG. 5 , at block  600  a reference image is captured using the lens  303 . As noted above, in some embodiments more than one reference image may be captured and the multiple images then averaged and soothed into a single reference image. 
         [0057]    At block  602 , the function/surface L(x,y) is computed for the lens  303 . At block  604  a sampling operation is performed in order to select samples points for the function L(x,y). At block  606 , the selected sample points are stored in memory as the parameters  312 . 
       3. Reconstruct L(x,y) 
       [0058]    The function of the data reconstruction logic  310  is to reconstruct the missing data for the lens shading correction function L(x,y). In one embodiment the data reconstruction logic  310  implements interpolation methods for this purpose. Since the lens shading surfaces are very smooth, in one embodiment, bilinear or bicubic interpolation may used for the reconstruction. 
         [0059]      FIG. 7  shows a flowchart of operations to correct a captured image for the lens shading artifact based on the lens shading correction function L(x,y), in accordance with one embodiment of the invention. Referring to  FIG. 7 , at block  700  image data is read from the image buffer  306 . The image data corresponds to the captured image. 
         [0060]    At block  702 , the missing data for the function L(x,y) is reconstructed using data reconstruction logic  314 . At block  704 , a corrected image is generated based on the function L(x,y). The block  704  is performed by the image correction block  316 . 
         [0061]    Although the present invention has been described with reference to specific example embodiments, it will be evident that various modifications and changes can be made to these embodiments without departing from the broader spirit of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than in a restrictive sense.