Image processing method and apparatus for correcting an image

The present disclosure relates to method and apparatus for image recovery and image correction. In an embodiment, the present disclosure relates to recovering images that are occluded due to some material in the optical path while capturing the images. The detection of noisy region is performed by taking the residual image between the images yielded by large sized and small sized Gaussian window smoothening filters on red, green and blue channels. The recovery of the image that alleviates the noise is obtained by point-wise multiplication of the above said residue image with a suitably synthesized Gaussian distributed co-efficient of same size.

TECHNICAL FIELD

The present disclosure relates to image processing. In particular, the present disclosure relates to a method and an apparatus for alleviating image artifacts caused by dirty camera lenses.

BACKGROUND

In photography or image acquisition and/or processing techniques, impurities can cause imperfections in any images produced. The impurities can include dust, dirt or any type of thin occlude material which attenuates the incoming light and scatters the portion of light towards the camera capturing the image. For example, if dust is introduced into a camera or image processing system, such dust can cause obstructions, shadows or other extraneous or undesired artifacts in any images that are ultimately produced.

Conventionally, when a foreign substance such as dust sticks to the image capturing optical system, the captured images from the system may contain a foreign substance. Hence, the quality of the image degrades. This type of image degradation not only amounts to annoying effect but also hinders the analysis that are aimed to either some pathological defects detection in medical image based expert system or any other image based image systems.

Dust or other impurities or the effects caused by the presence of such impurities may be decreased or removed from an image processing method. However, the conventional dust correction may degrade the image quality depending on the object state around the dust. In some cases, smearing may occur in the object upon dust correction, and a sense of incongruity may be generated between the dust region and its periphery.

The present disclosure has been made in consideration of the above-described problems, and relates to a method and an apparatus for correcting an image.

SUMMARY OF THE DISCLOSURE

The shortcomings of the prior art are overcome and additional advantages are provided through the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the claimed disclosure.

The present disclosure relates to removal of effects of impurities, such as dust particles, from an image.

In an embodiment, the present disclosure optimizes a dust correction condition and increases the quality of an image to be output.

In an embodiment, the image processing of the present disclosure aims in recovering the images that are occlude or corrupted due to some material in the optical path while capturing the images.

In one embodiment, the present disclosure relates to an image processing method for correcting an image. The method comprises obtaining a high smoothened image IHand a low smoothened image ILby applying large size and small size median window filter on the image IC. Further, a noise alleviated image using images obtained using the above step. An intermediate image is generated by performing point wise multiplication between corresponding pixels of the image ICand the low smoothened image IL. Further, the method comprises obtaining a residual image by subtracting each pixel of high smoothened image IHfrom corresponding pixels of the intermediate image and generating a binary image from the residual image. Then, morphological erosion is applied on the binary image for a predetermined number of times and a rim portion of the received image is obtained by performing pixel wise subtraction between the generated binary image and the image obtained after morphological erosion. Further, the method comprises obtaining portions of the image ICto be blurred by placing rim portion of the image as mask over the noise alleviated image and performing blurring on the portions of the image ICby using window of a predetermined size for a predetermined color model and obtaining a corrected image.

In one embodiment, the present disclosure relates to an image processing apparatus for correcting an image. The apparatus comprises an image receiving unit configured to receive the image to be corrected from an image capturing unit and a processor coupled to the image receiving unit. The processor is configured to obtain a high smoothened image IHand a low smoothened image ILby applying large size and small size median window filter on the image ICand obtain a noise alleviated image from above step. Then the processor generates an intermediate image by performing point wise multiplication between corresponding pixels of the image ICand the low smoothened image IL. Further the processor generate a binary image from a residual image, said residual image is obtained by subtracting each pixel of highly smoothened image IHfrom corresponding pixels of the intermediate image. The processor is also configured to apply morphological erosion on the binary image for a predetermined number of times and obtain rim portion of the received image by performing pixel wise subtraction between the generated binary image and the image obtained after morphological erosion. Further, the processor obtains portions of the image ICto be blurred by placing rim portion of the image as mask over the noise alleviated image and performs blurring on the portions of the image ICby using window of a predetermined size for a predetermined color model and obtaining a corrected image. The apparatus further comprises a memory unit coupled with the processor.

The aforementioned and other features and advantages of the disclosure will become further apparent from the following detailed description of the presently preferred embodiments, read in conjunction with the accompanying drawings. The detailed description and drawings are merely illustrative of the disclosure rather than limiting, the scope of the disclosure being defined by the appended claims and equivalents thereof.

DETAILED DESCRIPTION

FIG. 1illustrates an image processing apparatus for correcting an image in accordance with an embodiment of the present disclosure. The image processing apparatus100comprises an image capturing unit102, image receiving unit104, a processor106and a memory unit108. The image receiving unit104is configured to receive an image to be corrected ICfrom the image capturing unit102. The image receiving unit104is communicatively connected to the processor106. The memory unit108is connected to the processor106to store images.

FIG. 2illustrates a flowchart explaining an image processing method in accordance with an embodiment of the present disclosure.

FIG. 3illustrates an original image to be corrected in accordance with an embodiment of the present disclosure. The image in the illustrated figure is a retinal image. However, any image that is occluded or corrupted due to some material in the optical path while capturing the image can be taken as original input image.

Referring back toFIG. 2, the model behind the formation of artefact caused due to the contaminant deposition on the path of light while capturing an image is described below. The contaminant deposition can be due to materials including but not limited to oil or dust or any other semi-transparent occluding material. The model sets a foundation for a methodology to identify and alleviate this kind of artefacts caused in the image. In an exemplary embodiment, it is assumed that above said semi-transparent contaminant particles are causing light attenuators that ultimately alter the image according to the below form
I(i,j)=α(i,j)×J(i,j)  (1)

Where I(i,j) is a pixel of an image with lens artefact caused due to light attenuator, α(i,j) on pixel of a lens artefact-free image(i,j). In this embodiment, it is assumed that the causing of these artefacts diffraction of light is negligible, since regions of these semi-transparent occlude do not exhibit interference effects associated with diffraction around the edges of a barrier. In the above model, considered scattering effect due to occluding object is not considered since it is negligibly small.

First, an image to be corrected ICis received from the image receiving unit102at step202. Then, the corrupted region in the image containing the image artifact is identified. The corrupted region identification process can be done manually or automatically.

The manual selection procedure for identifying corrupted region in the image comprises:

i) Identifying the region that is corrupted due to semi-transparent occluding object by visual inspection in the image at step204.ii) Selecting a rectangular block image with horizontal width and vertical width as ‘Horizontal Width’ and ‘Vertical Width’ respectively at step206. The rectangular block image is selected such that lens artifact with maximum occlusion is approximately located in the mid portion and rest of the occluded image is also covered as illustrated inFIG. 5. The rectangular block comprising selected image artifact is marked as502.
The automatic selection procedure for identifying corrupted region in the image comprises:i) Identifying the thin semi-transparent occluding model upon assembling of camera optical lens into an image capturing unit102.ii) Identifying the region that is corrupted due to semi-transparent occluding object by cross correlation method using an occluding model.

The occluding model uses the equation (1) as described above. In an embodiment of the occluding model, the image pixels is point wise multiplied by α(i,j). In another embodiment of the occluding model, a convolution is performed between α(i,j) and Image pixels. A person skilled in the art would understand that any other occluding model can be used with the present disclosure.

In an embodiment, the term α(i,j) can be a single exponential function as taken in the embodiment as shown inFIG. 6. In an alternative embodiment, the term α(i,j) can be weighted linear combinations of the exponential functions. The number of terms in the linear combinations depends on number of layers of semi-transparent contaminant depositions and number spots contaminants with varying deposition that creates non-uniform obstructions of light rays.iii) Selecting a rectangular block image with horizontal width and vertical width as ‘Horizontal Width’ and ‘Vertical Width’ respectively. The rectangular block image is selected such that lens artifact with maximum occlusion is approximately located in the mid portion and rest of the occluded image is also covered. In an exemplary embodiment, the ‘Horizontal Width’ and ‘Vertical Width’ of the artifact image blocks can be selected by taking respective width as 10 percent of the original image widths with highest loss due to artifact in the middle as illustrated inFIG. 5.

Following the image artifact identification, the following processing is performed for all RGB channels. At step208, a low smoothened image ILis obtained by applying small size Gaussian kernel window median filter on the identified image artifact. In an exemplary embodiment, the window size is taken as 7×7. Further at step210, a high smoothened image IHis obtained by applying large size median window smoothing filter on the identified image artifact. In an exemplary embodiment, the window size is taken as 105×105. Then a Gaussian image matrix G is formed by each pixel, g(i,j) using the relationship given below:

Any artifact caused due to thin semi-transparent contaminant material on the lens, which is in the path of incident or reflected light rays could be viewed as a loss in the irradiance function at each pixel location (i,j) of the captured image and same could be modelled as weighted intensity or weighted luminance function. The weighting function α(i,j) can be modelled as any exponentially decaying function such that maximum loss in intensity occurs at the centre (ic,jc). One such α(i,j) is viewed as 2-D special weighting function (i,j), where its co-ordinate (i,j) is modelled as out of Gaussian surface on the 2D plane as defined below (as illustrated inFIG. 6):

Where (ic,jc) represents center point of the image block chosen with window size (Horizontal Width×Vertical Width) that possess artifact as described earlier and parameters wi2and wj2are related to block size as mentioned below:
2wi2=val×Horizontal Width  (3a)
2wj2=val×Vertical Width  (3b)

Where, val is dependent on the chosen Horizontal Width and Vertical Width of image block. The parameter val helps to control the surface curvature of the chosen Gaussian surface which enables to adjust the intensity variation of the artifact on the image. g(i,j) takes maximum value, which is unity, at the center (ic,jc) of the block of size(Horizontal Width×Vertical Width). In an exemplary embodiment, the parameter val takes value of 0.33, 0.25 respectively.

In the above model, the Gaussian weighting function g(i,j) or any generic weighting function α(i,j) is point operation. Hence a point wise inverse operation yields estimate of J(i,j), a lens artefact alleviated image or recovered image.FIG. 6illustrates thin semi-transparent model as Gaussian surface.FIG. 7illustrates generated Gaussian image as per the manual and automatic selection procedure for identifying corrupted region in the image.

Further, at step214, an intermediate image IF′ is obtained by performing point wise multiplication between low smoothened image ILand Gaussian image G, such that intermediate image
IF′IL·G,
where · is point wise multiplication between corresponding pixels of ILand G.

At step216, a residual image IRis calculated by subtracting each pixels of high smoothened image IHfrom corresponding pixels of the intermediate image IF′, such that
IRIF′−IH

Then, a binary image is formed from the residual image at step218by performing the binarization process as below:

In binarization process, a gray scale image is converted into a binary image. The binary images are formed by using threshold. If pixel value is greater than threshold then binary pixel value is selected as 1, otherwise the binary pixel value is selected as 0. If the image corresponding to intended object takes 0, then background will be 1 or vice versa.

At step220, morphological erosion is applied for plurality of iterations, for example 3 iterations on the binary image and a lens artifact corrupt region as biggest white patch is identified in the above obtained binary image B(i,j). The erosion is a morphological process in which one or more pixels that is touching one or more background pixels is changed into the one or more background pixels. Any morphological operation refers to certain operation like erosion, where an object is hit by a structuring element. Erosion is a shrinking operation and is dependent on structuring element. Additionally, erosion operation is accompanied by enlargement of background.

In binary morphology, an image is viewed as a subset of a Euclidean space Rdor the integer grid Zd, for some dimension d. The basic idea in binary morphology is to probe an image with a simple, pre-defined shape, drawing conclusions on how this shape fits or misses the shapes in the image.

Further, an image recovery procedure or lens artifact removal procedure is as described below:

First, a region of rim is obtained at step220by performing pixel wise subtraction between B(i,j) before and after erosion process. Then, the lens artifact alleviated image J(i,j) is obtained at step222by computing:
J(i,j)=(IF−IL)×G×(reduction_value),
where reduction_value for red, green and blue channel is chosen as 0.7, 0.7 and 0.5, respectively.

Further, the rim portion of the image is used as mask in the lens artifact image computed above and the following processing is performed as illustrated inFIGS. 8a,8b,8c,9a,9band9c.i) Performing blurring operation on the portion of the image pixels mentioned in the above step using windows as 51×51, 51×51 and 9×9 for green, blue and red channel, respectively.ii) Obtain a refined lens artifact alleviated output image J(i,j) by replacing pixels lens artifact alleviated image J(i,j) in rim portion with pixel obtained by blurring operation and retaining remaining pixels as it is.

FIG. 4illustrates the corrected original image by image processing method of the present disclosure. Once the original image is treated with image processing method of the present disclosure, the final image obtained is illustrated inFIG. 4.

In an embodiment, the present disclosure alleviated the effect of artifact noise in images. The method of the present disclosure does not restrict to the relationship mentioned with exponentials. The method can be generalized with one or more linear or non-linear combination of exponentials.

The method of identification and recovery of image artifact noise is not only limited to image artifact caused by a single occluding semi transparent material. An image with any number of such artifacts can be used as input image for the method of the present disclosure. This written description uses examples to disclose embodiments of the present disclosure, including the best mode, and also to enable any person skilled in the art to practice the technology disclosed in the present disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the present disclosure is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.