Abstract:
Disclosed is a method for sharpening a digital image captured by a digital imaging device, the method comprising, for each pixel X m,n  of the digital image: a) at an edge detector determining an edge parameter E, the edge parameter E providing an indication of whether the pixel X m,n  is at an edge of the digital image; b) at a noose detector determining a noise parameter σ, the noise parameter σ providing an indication of whether the pixel X m,n  is a noise in the digital image; c) at a scaling module determining a scaling factor α m,n  based on a combination of the edge parameter E and the noise parameter σ; and d) at an image sharpening module applying the scaling factor α m,n  on a pixel by pixel basis to sharpen the digital image to get a sharpened pixel Y m,n .

Description:
TECHNICAL FIELD OF THE DISCLOSURE 
       [0001]    The present disclosure generally relates to digital images, and more particularly to sharpening of digital images. 
       BACKGROUND OF THE DISCLOSURE 
       [0002]    A digital image capturing system, such as a digital camera, is well known for recording either motion or still images. The digital image capturing system may be used to capture raw image data from an image sensor through a lens system, and subsequently processes the raw image data into output images using a digital image pipeline as shown in  FIG. 1 . Image sharpness is an important criterion on subjective quality of the output images because human observers often prefer images with crisp edges and nicely rendered details. The raw image data may not have a desired level of sharpness due to a variety of factors including but not limited to a quality of optics, focus setting, camera shake, object motion, and so on. Furthermore, in the digital image pipeline as shown in  FIG. 1 , some image processing algorithms such as demosaicing, noise filtering, and scaling can make the output images more blurry than the raw image data. As a result, an image sharpening algorithm is usually used in the digital image pipeline to increase the sharpness of the output images. 
         [0003]    Many image sharpening algorithms are known in the art. Popular ones include unsharp masking, high pass filtering, and the like. Generally it is possible to perform image sharpening either in a Red Green Blue (RGB) domain or in a luminance-chrominance domain such as a YUV. 
         [0004]    Pixels in a digital image may be arranged in rows and columns. Consider an input image pixel where X m,n , denotes a value of a pixel at an mth row and nth column. The input image may be fed into the image sensor as shown in  FIG. 1 . An unsharp masking method, known in the art, may be used for image sharpening and may generate an output image pixel Y m,n  using the following equation: 
         [0000]        Y   m,n   =X   m,n +α( X   m,n   −S   m,n )  (1)
 
         [0000]    where S m,n  is a smoothed version of X m,n , and α is a scaling factor. The smoothed value S m,n  is often generated by processing X m,n  with a low pass filter. Typical low pass filters used in image processing for this purpose can have a kernel of size 3×3, 3×5, 5×5, and the like. The scaling factor controls the sharpness of the output image. Generally, a larger value of α results in a sharper output image. One common characteristic of image sharpening algorithms is that they always enhance high frequency content of the images. Although this makes the output images sharper, it also amplifies the noise content of the images. 
       SUMMARY OF THE DISCLOSURE 
       [0005]    In view of the foregoing disadvantages inherent in the prior art, the general purpose of the present disclosure is to provide a method and apparatus for sharpening digital images to include all advantages of the prior art, and to overcome the drawbacks inherent therein. 
         [0006]    It is therefore an object of the present disclosure to provide a method and apparatus for increasing the sharpness of digital images without increasing the noise in the digital images. 
         [0007]    To achieve the above objects, in one aspect, the present disclosure provides a method for sharpening a digital image captured by a digital imaging device, the method comprising, for each pixel X m,n  of the digital image: a) at an edge detector determining an edge parameter E, the edge parameter E providing an indication of whether the pixel X m,n  is at an edge of the digital image; b) at a noise detector determining a noise parameter σ, the noise parameter σ providing an indication of whether the pixel X m,n  represents noise in the digital image; c) at a scaling module determining a scaling factor α m,n  based on a combination of the edge parameter E and the noise parameter σ; and d) at an image sharpening module applying the scaling factor α m,n  on a pixel by pixel basis to sharpen the digital image to get a sharpened pixel Y m,n . 
         [0008]    In another aspect, the present disclosure provides an image processor for sharpening a digital image captured by a image processing system, the image processor comprising: for each pixel X m,n  of the digital image: a) an edge detector capable of determining an edge parameter E, the edge parameter E provides an indication of whether the pixel X m,n  is at an edge of the digital image; b) a noise detector capable of determining a noise parameter σ, the noise parameter σ provides an indication of whether the pixel X m,n  represents noise in the digital image; c) a scaling module coupled to the edge detector and to the noise detector, the scaling module capable of determining a scaling factor α m,n  based on a combination of the edge parameter E and the noise parameter σ; and d) an image sharpening module coupled to the scaling module, the image sharpening module capable of utilizing the scaling factor α m,n  on a pixel by pixel basis to sharpen the digital image to get a sharpened pixel Y m,n . 
         [0009]    In yet another aspect of the present disclosure, the present disclosure provides computer-implemented methods, computer systems and a computer readable medium containing a computer program product for sharpening a digital image captured by a digital imaging device, the computer program product comprising: for each pixel X m,n  of the digital image: a) program code for determining an edge parameter E, the edge parameter E providing an indication of whether the pixel X m,n  is at an edge of the digital image; b) program code for determining a noise parameter σ, the noise parameter σ providing an indication of whether the pixel X m,n  represents noise in the digital image; c) program code for determining a scaling factor α m,n  based on a combination of the edge parameter E and the noise parameter σ; and d) program code for applying the scaling factor α m,n  on a pixel by pixel basis to sharpen the digital image to get a sharpened pixel Y m,n . 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0010]    The accompanying drawings, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate embodiments of concepts that include the claimed disclosure, and explain various principles and advantages of those embodiments. 
           [0011]      FIG. 1  is a block diagram of a prior art showing processing of a digital image in a conventional digital image pipeline; 
           [0012]      FIG. 2  shows a schematic block diagram of an image processing system, in accordance with an embodiment of the invention. 
           [0013]      FIG. 3  is a block diagram of an image buffer and an image processor used by the image processing system for sharpening a digital image, in accordance with an embodiment of the present disclosure; and 
           [0014]      FIG. 4  is a flow chart representing a method for sharpening an image, in accordance with an embodiment of the present disclosure. 
       
    
    
       [0015]    The method and apparatus have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. 
       DETAILED DESCRIPTION 
       [0016]    Before describing in detail embodiments that are in accordance with the present disclosure, it should be observed that the embodiments reside primarily in combinations of method steps and system components related to sharpening a digital image without amplifying a noise level in the digital image. 
         [0017]    In this document, relational terms such as first and second, and the like may be used solely to distinguish one module or action from another module or action without necessarily requiring or implying any actual such relationship or order between such modules or actions. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements that does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. 
         [0018]    Any embodiment described herein is not necessarily to be construed as preferred or advantageous over other embodiments. All of the embodiments described in this detailed description are illustrative, and provided to enable persons skilled in the art to make or use the disclosure and not to limit the scope of the disclosure, which is defined by the claims. 
         [0019]    The present disclosure provides a method and system for increasing the sharpness of digital images without or at least minimally amplifying a noise level in the digital images. Specifically, a digital image is processed or sharpened by the system pixel by pixel, as will be explained in conjunction with  FIGS. 2 ,  3 , and  4 . 
         [0020]    Referring to  FIG. 2 , a block diagram of an image processing system is shown, in accordance with an embodiment of the present disclosure. The image processing system comprises camera optics  100 , an image sensor  200 , an image buffer  300 , and an image processor  400 . The camera optics  100  and the image sensor  200  may enable the system to capture a digital image  500  (as is shown in  FIG. 3 ). The digital image  500  may be stored in the image buffer  300  which is coupled to the image processor  400 . Further, the digital image  500  may be read by the image processor  400  from the image buffer  300  for further processing and sharpening of the digital image  500 . In one embodiment, the system may be in the form of a digital camera, in which case it may include other components dictated by the function of the digital camera such camera optics  100 . 
         [0021]    Referring to now  FIG. 3 , a more detailed block diagram of the image buffer  300  and the image processor  400  is shown, in accordance with an embodiment of the present disclosure. The image buffer  300  is a memory device which may be used by the image processor  400  for extracting digital image  500  for sharpening and processing. 
         [0022]    The image buffer  300  may store the digital image  500  as shown in the  FIG. 3 . The digital image  500  may comprise millions of pixels depending on the resolution of the image sensor  200 . For the sake of brevity of this description, a 3×3 pixel matrix  510  of the digital image  500  is used for explaining the various embodiments of the present invention. Specifically, the 3×3 pixel matrix  510  has three columns and three rows having nine pixels in totality. 
         [0023]    The image processor  400  includes an edge detector  10 , a noise detector  20 , a scaling module  30 , an image sharpening module  40 , an image darkening module  50 , and an adder  60 . The image processor  400  may read the digital image  500  in the form of blocks of pixels in order to sharpen the digital image  500 . Specifically, a block of pixels may be read by the edge detector  10  from the image buffer  300 . The edge detector  10  executes an edge detection algorithm to determine whether a pixel X m,n  is at an edge. Specifically, the edge detector  10  is capable of determining an edge parameter E of the pixel X m,n  of the digital image  500 . The edge parameter E provides an indication of whether the pixel X m,n  is at an edge in the image. Further, the edge parameter E may determine whether the pixel X m,n  is on a dark side of the edge or on a light side of the edge. In embodiment of the present disclosure, a Sobel type edge detector may be used as the edge detection algorithm. However, it will be evident to a person skilled in the art that any other type of edge detector may be used without deviating from the scope of present disclosure. 
         [0024]    In the present embodiment, the blocks of pixels may also be read by the noise detector  20  from the image buffer  300 . The noise detector  20  is capable of determining a noise parameter σ which provides an indication of whether the pixel X m,n  represents noise in the digital image  500 . Specifically, the noise parameter σ may compare the pixel X m,n  with a neighborhood R of pixels centered on the pixel X m,n . In one embodiment, a 3×3 neighborhood R of pixels centered at X m,n  may be considered. The 3×3 neighborhood R of pixels defines an 8-pixel neighborhood comprising the pixel set {X m−1,n−1 , X m−1,n , X m−1,n+1 , X m,n−1 , X m,n+1 , X m+1,n−1 , X m+1,n , X m+1,n+1 }. 
         [0025]    The pixels in the neighborhood R are examined to determine the maximum value and minimum pixel values in the neighborhood R as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     A 
                     
                       m 
                       , 
                       n 
                     
                   
                   = 
                   
                     
                       max 
                       
                         R 
                         
                           m 
                           , 
                           n 
                         
                       
                     
                      
                     
                       X 
                       
                         m 
                         , 
                         n 
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
             
               
                 
                   
                     B 
                     
                       m 
                       , 
                       n 
                     
                   
                   = 
                   
                     
                       min 
                       
                         R 
                         
                           m 
                           , 
                           n 
                         
                       
                     
                      
                     
                       X 
                       
                         m 
                         , 
                         n 
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0000]    In one embodiment, if the pixel X m,n  is either larger than A m,n  or lower than B m,n , then the pixel X m,n  is considered to be a high level noise. Further, when the difference between A m,n  and B m,n  is large or when pixel X m,n  is close to either A m,n  or B m,n  pixel Xm,n would be considered to be a medium level noise. 
         [0026]    In one embodiment of the present disclosure, the edge parameter E and the noise parameter σ may be fed to the scaling module  30  for calculation of a scaling factor α m,n . Specifically, the edge detector  10  and the noise detector  20  are coupled to the scaling module  30  which is capable of determining the scaling factor α m,n  based on a combination of the edge parameter E and the noise parameter σ. The scaling factor α m,n  is used to sharpen the digital image  500 . In a one embodiment, the scaling module  30  controls the value of the scaling factor α m,n  for every pixel location by combining the results of edge parameter E and noise parameter σ. An objective of the combining the results of edge parameter E and noise parameter σ for calculating the scaling factor α m,n  is to avoid sharpening at either non-edge pixels or noise pixels. 
         [0027]    In one embodiment of the present disclosure, when the pixel X m,n  is considered to be a high level noise, the scaling factor will be set to zero and there will be no sharpening. Similarly, when the pixel X m,n  is considered to be a medium level noise, the scaling factor will be reduced thereby reducing the sharpening level. In other words, the scaling factor α m,n  may be discarded when the value of the pixel X m,n  is outside a range defined by the neighborhood pixels. 
         [0028]    In one embodiment of the present disclosure, the scaling factor α m,n  may be fed to the image sharpening module  40  which is coupled to the scaling module  30 . The image sharpening module  40  is capable of utilizing the scaling factor α m,n  in as a parameter of an image sharpening algorithm for sharpening the pixel X m,n  to get a sharpened pixel Y m,n  in an output image. In one embodiment, the image sharpening module  40  implements an unsharp masking algorithm of the form 
         [0000]        Y   m,n   =X   m,n +α m,n ( X   m,n   −S   m,n )  (4)
 
         [0000]    where the scaling factor α m,n  varies from pixel to pixel, and Y m,n  is a pixel an output/sharpened image. In one embodiment, to perform edge controlled sharpening, the scaling factor α m,n  is set to zero for the pixel locations where an edge is not detected. Further, for edge pixel locations, scaling factor α m,n  is set to a non-zero value to perform edge sharpening. 
         [0029]    In one embodiment of the present disclosure, the edge parameter E may be fed to the image darkening module  50  which is coupled to the edge detector  10 . The image darkening module  50  is capable of determining an offset parameter U m,n  when the pixel X m,n  is determined to be on the dark side of the edge. When the pixel X m,n  is determined to be on the dark side of the edge, the image darkening module  50  darkens the pixel X m,n  in order to sharpen the pixel X m,n  using the offset parameter U m,n . 
         [0030]    In an embodiment of the present disclosure, an adder may generate an output image pixel T m,n  based on a combination of the sharpened pixel Y m,n  and the offset parameter U m,n  as shown in the following equation: 
         [0000]        T   m,n   =Y   m,n   −U   m,n   (5)
 
         [0031]    Referring now to  FIG. 4 , a flow chart representing a method for sharpening the digital image  500  is shown, in accordance with an embodiment of the present disclosure. The method starts at  402  where an image, such as the digital image  500 , is captured using the image processing system described. At  404 , a pixel X m,n  of the digital image  500  is read by the edge detector  10 . Further, at  406 , the edge parameter E of the pixel X m,n  is determined using the edge detector  10 . At  408 , it is determined whether the pixel X m,n  is at an edge of the digital image  500 . When the pixel X m,n  is determined to be at the edge of the digital image  500 , then at  410  it is determined whether the pixel X m,n  is on a dark side of the edge. If yes, then the noise parameter σ of the pixel X m,n  is determined using the noise detector  20  at  412 . At  414 , the scaling factor α m,n  is determined at a scaling module  30  based on a combination of the edge parameter E and the noise parameter σ. At  416 , the image darkening module  50  determines an offset parameter. At  418 , the image darkening module  50  darkens in order to sharpen the pixel X m,n  using the offset parameter U m,n . At  420 , the adder generates an output image pixel T m,n  based on a combination of the sharpened pixel Y m,n  and the offset parameter U m,n . 
         [0032]    It will be appreciated that embodiments of the disclosure described herein may comprise one or more conventional processors and unique stored program instructions that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all functions of sharpening a digital image without amplifying a noise level in the digital image. Alternatively, some or all functions of sharpening a digital image could be implemented by a state machine that has not stored program instructions, or in one or more Application Specific Integrated Circuits (ASICs), in which each function or some combinations of certain of the functions are implemented as custom logic. Of course, a combination of the two approaches could be used. Thus, methods and means for these functions have been described herein. Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation. 
         [0033]    As will be understood by those familiar with the art, the disclosure may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Likewise, the particular naming and division of the modules, agents, managers, functions, procedures, actions, methods, classes, objects, layers, features, attributes, methodologies and other aspects are not mandatory or significant, and the mechanisms that implement the disclosure or its features may have different names, divisions and/or formats. Furthermore, as will be apparent to one of ordinary skill in the relevant art, the modules, agents, managers, functions, procedures, actions, methods, classes, objects, layers, features, attributes, methodologies and other aspects of the disclosure can be implemented as software, hardware, firmware or any combination of the three. Of course, wherever a component of the present disclosure is implemented as software, the component can be implemented as a script, as a standalone program, as part of a larger program, as a plurality of separate scripts or programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and/or in every and any other way known now or in the future to those of skill in the art of computer programming. Additionally, the present disclosure is in no way limited to implementation in any specific programming language, or for any specific operating system or environment. Accordingly, the disclosure of the present disclosure is intended to be illustrative, but not limiting, of the scope of the disclosure, which is set forth in the following claims.