System and method for estimating image sharpness

A method for determining image sharpness estimation of an image. Image data for at least one column of pixels and at least one row of pixels of the image is sampled. A first filter is applied on sequential differences of sampled image data resulting in a first filtered sequence of first filtered values. A second filter is applied on the sequential differences resulting in a second filtered sequence of second filtered values. A feature set of pixels of the image is selected. Image sharpness estimation is determined based on the first filtered values and the second filtered values of the feature set of pixels.

TECHNICAL FIELD

Embodiments of the present invention relate to the fields of image analysis and image processing.

BACKGROUND ART

Image analysis is a process by which an imaging application determines a qualitative or quantitative feature based on input image data. For example, image analysis may be used determine whether an image is in or out of focus, if it is too bright or too dark, or how much sharpening should be applied to it. Image analysis may be implemented in conjunction with an image acquisition or image-printing device. In an image acquisition device it might be implemented for focus determination and visualization applications. In an image-printing device it might be implemented in the context of an image processing system.

Image processing is a process by which an imaging application alters input image data. For example, image processing may be used to change the color space of a digital image. Image processing may be implemented in conjunction with a printing device in order to adjust the color appearance or perceived sharpness of an image according to the specifications of the printing device. Current imaging applications receive image data from a wide array of diverse sources. For example, the image data may be a disposable camera image developed in an automatic film developing treatment machine, scanned at home, and compressed, or may be a manually enhanced high-resolution image from a professional digital image archive. A printer or any other imaging device is typically unable to operate in diverse conditions without first estimating the quality of its input images in order to process them accordingly.

Adaptive image processing is typically used to address problems associated with enhancing images having variable image quality (IQ). Currently, there are several methods of adaptive image processing with respect to image sharpness enhancement. However, the current available methods each present certain operational drawbacks or limitations.

Feature-based models are one category of current adaptive image sharpening methods. Feature-based models require the existence of identifiable features of the image. As such, feature-based models are only useful in situations where certain information is known about an image, and are not generally applicable to all images.

Model-based zero analysis methods are another category of adaptive image sharpening. However, model-based zero analysis assumes a particular parametric blur model and no image noise. Since a blur model for an image is often not known, this class of adaptive image sharpening is only useful in limited situations.

Autoregressive-moving average (ARMA) filter models can also be used for adaptive image sharpening. However, ARMA filter models are computationally intensive, requiring a substantial contribution of computing resources, and are occasionally unstable, thereby providing limited applicability. Similarly, single stage algorithms may be used for adaptive image processing, but are also computationally intensive.

Other current methods for adaptive image enhancement are problematic because they mix sharpness enhancement with contrast enhancement. Because an image can be sharp but have low contrast, or can be blurred but have a high contrast, it may be necessary to differentiate between sharpness and contrast problems, which these methods do no provide. Furthermore, other adaptive image enhancement methods assume a fractal image model, which is not true in the general case.

SUMMARY OF THE INVENTION

Various embodiments of the present invention, a method and system for determining image sharpness estimation of an image, are described herein. In one embodiment, image data for at least one column of pixels and at least one row of pixels of the image is sampled. A first filter is applied on sequential differences of sampled image data resulting in a first filtered sequence of first filtered values. A second filter is applied on the sequential differences resulting in a second filtered sequence of second filtered values. A feature set of pixels of the image is selected. Image sharpness estimation is determined based on the first filtered values and the second filtered values of the feature set of pixels.

BEST MODE FOR CARRYING OUT THE INVENTION

Aspects of the present invention may be implemented in a computer system that includes, in general, a processor for processing information and instructions, random access (volatile) memory (RAM) for storing information and instructions, read-only (non-volatile) memory (ROM) for storing static information and instructions, a data storage device such as a magnetic or optical disk and disk drive for storing information and instructions, an optional user output device such as a display device (e.g., a monitor) for displaying information to the computer user, an optional user input device including alphanumeric and function keys (e.g., a keyboard) for communicating information and command selections to the processor, and an optional user input device such as a cursor control device (e.g., a mouse) for communicating user input information and command selections to the processor.

FIG. 1is a block diagram of system100for determining the sharpness of an image, in accordance with an embodiment of the present invention. System100utilizes sampled data of an image to determine an estimation of image sharpness. System100comprises image data sampler module110, filtering module120, phase shift correction module130, feature set selection module140, and image sharpness estimation module150. It should be appreciated that system100may be implemented within a computer system as software or as hardware. For example, a module may be a piece of software code, a hardware device, or a portion of a hardware device.

System100receives input image data105at image data sampler module110. In one embodiment, input image data105is a Joint Photographic Experts Group (JPEG) image. It should be appreciated that any form of image data, such as Tagged Image File Format (TIFF), Graphics Interchange Format (GIF), a bitmap, and other form may be used. Input image data105includes luminance information and color information. It should be appreciated that image data105may also include other image information. In one embodiment, the color information includes Red-Green-Blue (RGB) color scheme data. In another embodiment, the color information includes Cyan-Magenta-Yellow-Black (CMYK) color scheme data. Input image data105includes a pixel grid, in which a particular pixel has associated image data. For example, a pixel of the pixel grid includes associated luminance value and color information.

Image data sampler module110is configured to sample image data from a portion of pixels of input image data105. In one embodiment, the sampled image data is a luminance value for a pixel. In another embodiment, the sampled image data is a color value for a pixel. In one embodiment, image data from at least one column of pixels and one row of pixels is sampled. In one embodiment, image data from pixels of every Mth row and every Nth column is sampled, wherein M and N are positive integers. In one embodiment, M and N are equal. The sequences of sampled image data (e.g., sampled sequences115) are associated with a particular column or row of pixels. Sampled sequences115are then forwarded to filtering module120.

Filtering module120is operable to filter sequential differences of sampled sequences115. It should be appreciated that the sequential differences can be determined at sampler module110, filtering module120, or in a separate module. For example, the sequential difference determination may be factored in to individual filters of filtering module120. In one embodiment, filtering module120includes a low pass filter for filtering sampled sequences115and a high pass filter for filtering sampled sequences115. However, it should be appreciated that filtering module120may use more or different filters. For example, if an individual filter of filtering module120is configured to determine the sequential difference values of sampled sequences115, the low pass filter may be replaced with a band pass filter. In one embodiment, the filters of filtering module120are 6-tap Infinite Impulse Response (IIR) filters. However, it should be appreciated that any filter may be used.

Filtering module120applies the filters to sampled sequences115to generate filtered sequences125. For example, where filtering module120includes two filters, at least one first filtered sequence of first filtered values associated with a first filter and at least one second filtered sequence of second filtered values associated with a second filter are generated. In one embodiment, where filtering module120includes a low pass filter and a high pass filter, at least one low pass sequence of low pass values and at least one high pass sequence of high pass values, respectively, are generated.

In one embodiment, a phase shift between filtered sequences125occurs as a result of the filtering. In one embodiment, filtered sequences125are transmitted to phase shift correction module130in order to compensate for the phase shift. Phase shift correction module130is operable to align filtered sequences125, resulting in aligned filtered sequences135. Aligned filtered sequences135are then transmitted to feature set selection module140. It should be appreciated that, in various embodiments, no phase shift occurs between filtered sequences125. In these embodiments, filtered sequences125are already aligned, thus obviating the need for phase shift correction module130. Therefore, phase shift correction module130is optional.

Feature set selection module140is operable to select a feature set of pixels of the image. In one embodiment, feature set145is selected from pixels corresponding to a portion of aligned filtered sequences135. In one embodiment, feature set selection module140receives a feature set threshold value142. It should be appreciated that feature set threshold value142may be automatically generated or user-defined. Feature set selection module140compares feature set threshold value142to the filtered values of one of aligned filtered sequences135. If the filtered value is greater than feature set threshold value142, the corresponding pixel is selected as a pixel of feature set145. It should be appreciated that a value based on the filtered value may be compared to feature set threshold value142. For example, filtered value squared may be compared to feature set threshold value142. Feature set145includes the aligned filtered values for the selected pixels. It should be appreciated that a value based on the corresponding values of two or more different aligned filters may be compared to feature set threshold value142.

Edge pixels are useful in determining image sharpness estimation. As such, it may be desirable to restrict feature set145to edge pixels perpendicular to the scanning direction (e.g., a vertical edge pixel in a row sequence or a horizontal edge pixel in a column sequence). In one embodiment, feature set selection module140is operable to further constrain feature set145by excluding pixels that are not displaced substantially perpendicular to the scanning direction of a sequence. In one embodiment, feature set selection module140compares pixels to sampled pixels of neighboring rows in determining whether the pixel is an edge pixel. Feature set145is then transmitted to image sharpness estimation module150.

Image sharpness estimation module150is operable to estimate image sharpness155based on the aligned filtered values135of feature set145. In one embodiment, the ratio of the second aligned filtered value to the first aligned filtered value135for pixels of feature set145is calculated. The square function of the ratios are then averaged, resulting in a single value, image sharpness estimation155. In another embodiment, the ratios are compared to a sharpness threshold value152prior to averaging. It should be appreciated that sharpness threshold value152may be automatically generated or user-defined. If the square function of the ratio exceeds sharpness threshold value152, it is replaced with sharpness threshold value152. In other words, the square function of the ratio is clipped at sharpness threshold value152. The square function of the ratios and clipped ratios are then averaged, resulting in a single value, image sharpness estimation155.

It should be appreciated that image sharpness estimation155can be calculated in many other ways. For example, in one embodiment, a ratio of the averages of the square function of aligned filtered values is calculated, resulting in image sharpness estimation155. In another embodiment, weighting factor is used to calculate a ratio of the weighted square function of averages of aligned filtered values, resulting in image sharpness estimation155. In one embodiment, the image sharpness estimation155is used in automatic image sharpness enhancement.

FIG. 2is a block diagram of system200for determining image sharpness estimation of an image, in accordance with an embodiment of the present invention. System200is a detailed embodiment of system100ofFIG. 1, and operates in a similar manner. System200utilizes sampled data of an image to estimate image sharpness. System200comprises image data sampler module210, sequential difference module220, filtering module230, phase shift correction module240, feature set selection module250, and image sharpness estimation module260. It should be appreciated that system200may be implemented within a computer system as software or as hardware. For example, a module may be a piece of software code, a hardware device, or a portion of a hardware device.

Image data sampler module210is configured to sample image data from a portion of pixels of input image data205. In one embodiment, the sampled image data is a luminance value for a pixel. In another embodiment, the sampled image data is a color value for a pixel. In one embodiment, image data from at least one column of pixels and one row of pixels is sampled. In one embodiment, image data from pixels of every Mth row and every Nth column is sampled, wherein M and N are positive integers. In one embodiment, M and N are equal. The sequences of sampled image data (e.g., sampled sequences215) are associated with a particular column or row of pixels. Sampled sequences215are then forwarded to sequential difference module220. A sampled sequence Xjof sampled sequences215may be denoted as Xj, j=1, 2, 3, 4, . . . , N.

Sequential difference module220is operable to determine the sequential differences between the sampled values of sampled sequences215. In one embodiment, a sequential difference Yiof sequential difference sequences225is calculated according to Yi=X(i+1)−Xi, i=1, 2, . . . N−1. As described inFIG. 1, it should be appreciated that the sequential differences determination can be combined into filtering module230.

Filtering module230is operable to filter sequential difference sequences225. In one embodiment, filtering module230includes low pass filter232and high pass filter234for filtering sequential difference sequences225. However, it should be appreciated that filtering module230may use more or different filters. For example, if an individual filter of filtering module230is configured to determine the sequential difference values of sampled sequences215(e.g., there is no sequential difference module220), the low pass filter may be replaced with a band pass filter. In one embodiment, low pass filter232and high pass filter234are 6-tap Infinite Impulse Response (IIR) filters. However, it should be appreciated that any filter may be used. In one embodiment, low pass value Liis calculated according to Equation 1 and high pass value Hiis calculated according to Equation 2:
Li=a3*Y(i−3)+a2*Y(i−2)+a1*Y(i−1)+a0*Yi−[b3*L(i−3)+b2*L(i−2)+b1*L(i−1)]  (1)
wherein a3, a2, a1, a0, b3, b2 and b1 are design parameters of a 6-tap IIR filter. In one embodiment, a3=a0=0.0029, a1=a2=0.0087, b3=−0.5321, b2=1.9294, and b1=−2.3741.
Hi=c3*Y(i−3)+c2*Y(i−2)+c1*Y(i−1)+c0*Yi−[d3*H(i−3)+d2*H(i−2)+d1*H(i−1)]  (2)
wherein c3, c2, c1, c0, d3, d2 and d1 are design parameters of a 6-tap IIR filter. In one embodiment, c3=0.0317, c2=−0.0951, c1=0.0951, c0=−0.0317, d3=1.4590, d2=0.9104, and d1=0.1978.

Filtering module230is operable to generate low pass filtered sequence236including low pass values (e.g., Li) and high pass filtered sequence238including high pass values (e.g., Hi).

In one embodiment, a phase shift between low pass filtered sequence236and high pass filtered sequence238occurs as a result of the filtering. In one embodiment, low pass filtered sequence236and high pass filtered sequence238are transmitted to phase shift correction module240in order to compensate for the phase shift. Phase shift correction module240is operable to align low pass filtered sequence236and high pass filtered sequence238, resulting in aligned low pass filtered sequence246and aligned high pass filtered sequence248. In one embodiment, low pass value Liis aligned according to Li=L(I+dl) and high pass value Hiis aligned according to Hi=H(I+dh). In one embodiment, dl=3 and dh=0.

The aligned filtered sequences are then transmitted to feature set selection module250. It should be appreciated that, in various embodiments, no phase shift occurs between low pass filtered sequence236and high pass filtered sequence238. In these embodiments, low pass filtered sequence236and high pass filtered sequence238are already aligned, thus obviating the need for phase shift correction module240. Therefore, phase shift correction module240is optional.

Feature set selection module250is operable to select a feature set of pixels of the image. In one embodiment, feature set250is selected from pixels corresponding to a portion of aligned low pass filtered sequence246and aligned high pass filtered sequence248. In one embodiment, feature set selection module250receives a feature set threshold value252. It should be appreciated that feature set threshold value252may be automatically generated or user-defined. Feature set selection module250compares feature set threshold value252to the filtered values of one of aligned low pass filtered sequence246and aligned high pass filtered sequence248. If the filtered value is greater than feature set threshold value252, the corresponding pixel is selected as a pixel of feature set255. It should be appreciated that a value based on the filtered value may be compared to feature set threshold value252. It should be appreciated that a value based on the corresponding values of two or more different aligned filters may be compared to feature set threshold value252. For example, filtered value squared may be compared to feature set threshold value252. Feature set255includes the aligned filtered values for the selected pixels. In one embodiment, a pixel Xtis a feature of feature set252if (Lt)2>T, wherein Ltis a low pass value and T is a feature set threshold value. In one embodiment, T=200.

Edge pixels are useful in determining image sharpness estimation. As such, it may be desirable to restrict feature set255to edge pixels perpendicular to the scanning direction (e.g., a vertical edge pixel in a row sequence or a horizontal edge pixel in a column sequence). In one embodiment, feature set selection module250is operable to further constrain feature set255by excluding pixels that are not displaced substantially perpendicular to the scanning direction of a sequence. In other words, feature pixels can be further constrained to have a gradient aligned to the 1D direction. In one embodiment, alignment gradient GradAlignis defined as GradAlign=X(i+1)−X(i-1)and perpendicular gradient GradPerpis defined as GradPerp=X+1i−X−1i, where Xdiis a pixel displaced d places perpendicular to the row or column. A pixel is excluded from feature set252if

GradAlignGradPerp
is greater than Td. In one embodiment, Td=3. In one embodiment, feature set selection module250compares pixels to sampled pixels of neighboring rows in determining whether the pixel is an edge pixel. Feature set255is then transmitted to image sharpness estimation module260.

Image sharpness estimation module260is operable to estimate image sharpness estimation265based on the aligned filtered values246and248of feature set255. In one embodiment, the square function of the ratio of the aligned high pass value248to the aligned low pass value246for pixels of feature set255is calculated. The squares of the ratios are then averaged, resulting in a single value, image sharpness estimation265. In another embodiment, the squares of the ratios are compared to a sharpness threshold value262prior to averaging. It should be appreciated that sharpness threshold value262may be automatically generated or user-defined. If the ratio exceeds sharpness threshold value262, it is replaced with sharpness threshold value262. In other words, the ratio is clipped at sharpness threshold value262. In one embodiment, sharpness threshold value262is 2. The squares of the ratios and clipped ratios are then averaged, resulting in a single value, image sharpness estimation265. In one embodiment, the image sharpness estimation265is used in automatic image sharpness enhancement.

It should be appreciated that image sharpness estimation265can be calculated in many other ways. For example, in one embodiment, a ratio of the square function of the averages of aligned filtered values is calculated, resulting in image sharpness estimation265. In another embodiment, weighting factor is used to calculate a ratio of the weighted squared averages of aligned filtered values, resulting in image sharpness estimation265.

FIG. 3is a flow chart of a process300for determining image sharpness estimation of an image, in accordance with an embodiment of the present invention. In one embodiment, process300is carried out by processors and electrical components under the control of computer readable and computer executable instructions. The computer readable and computer executable instructions reside, for example, in data storage features such as computer usable volatile and non-volatile memory. However, the computer readable and computer executable instructions may reside in any type of computer readable medium. Although specific steps are disclosed in process300, such steps are exemplary. That is, the embodiments of the present invention are well suited to performing various other steps or variations of the steps recited inFIG. 3.

At step305of process300, image data for at least one column of pixels or at least one row of pixels of an image is sampled, resulting in at least one sequence of image data. In one embodiment, the sampled image data includes luminance values. In another embodiment, the sampled image data includes color values. At step310, a sequential difference of the sampled image data for pixels of the sequence of image data is determined, resulting in a sequential difference sequence. It should be appreciated that step310is optional.

At step312, at least one filter is applied on the sampled image data, resulting in at least one sequence of filtered values. In one embodiment, the filter is operable to directly determine the sequential differences of the sampled image data. In one embodiment, the filter is a band pass filter. In another embodiment, where a sequential difference sequence is determined at step310, a low pass filter and a high pass filter are applied to the sequential difference sequence. At step315, a low pass filter is applied to the sequential difference sequence, resulting in a low pass sequence of low pass values. At step320, a high pass filter is applied to the sequential difference sequence, resulting in a high pass sequence of high pass values. In one embodiment, the low pass filter and the high pass filter are 6-tap IIR filters.

At step325, phase shift for the low pass sequence and the high pass sequence is corrected. It should be appreciated that the low pass sequence and the high pass sequence may already be aligned; thus, step325is optional.

At step330, a feature set of pixels of the image is selected. In one embodiment, pixels of the feature set are selected from the sampled image data if the low pass value for a pixel exceeds a threshold value. In one embodiment, a pixel is excluded from the feature set if the gradient in the pixel is not substantially perpendicular to the linear direction of image data.

At step335, the image sharpness estimation is determined based on the low pass values and the high pass values of the feature set of pixels. In one embodiment, the image sharpness estimation is calculated as the average of the square function of ratios of high pass values to corresponding low pass values for pixels of the feature set. In another embodiment, the image sharpness estimation is calculated as the average of the clipped square ratios of high pass values to low pass values for pixels of the feature set. In another embodiment, the image sharpness estimation is calculated as the ratio of the average of square function of high pass values to the average of square function of low pass values for pixels of the feature set. In another embodiment, the image sharpness estimation is calculated as the ratio of the weighted average of square high pass values to the weighted average of square low pass values for pixels of the feature set. In one embodiment, the image sharpness estimation is used in automatic image sharpness enhancement.

Embodiments of the present invention provide a method for determining image sharpness estimation of an image. Specifically, embodiments of the present invention provide for a generally applicable method for determining an estimation of image sharpness. The described embodiments do not require certain information to be known about the image. Furthermore, by sparsely sampling the image data, the described embodiments do not require substantial computations. Moreover, the described embodiments can determine an image sharpness estimation independent of other image features, such as contrast.

Various embodiments of the present invention, a method for determining image sharpness estimation of an image, are thus described. While the present invention has been described in particular embodiments, it should be appreciated that the present invention should not be construed as limited by such embodiments, but rather construed according to the following claims.