Image processing apparatus and systems using estimated point spread function

An image processing apparatus includes a point spread function (PSF) pattern generation unit for generating a PSF pattern in which a plurality of PSFs are located in a plurality of lines of the PSF pattern; a PSF estimation unit for estimating PSFs of an out-of-focus input image from step responses of the plurality of lines of the PSF pattern with respect to an edge of the out-of-focus input image; and an image restoration unit for restoring the out-of-focus input image to a focused restored image using the estimated PSF.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Korean Patent Application No. 10-2010-0028087, filed on Mar. 29, 2010, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.

BACKGROUND

The inventive concept relates to an image processing apparatus and image processing methods, and more particularly, to an image processing apparatus and image processing method for estimating a point spread function (PSF) and performing image restoration using the estimated PSF, thereby increasing image quality. If an image is determined to be out-of-focus or blurred by estimating a PSF, digital auto-focusing is performed by compensating for the out-of-focus or blurring problem.

SUMMARY

The inventive concept provides an image processing apparatus and image processing method for estimating a point spread function (PSF) and performing image restoration using the estimated PSF, thereby increasing image quality.

According to some embodiments of the inventive concept, there is provided a image processing apparatus that includes a point spread function (PSF) pattern generation unit that is configured to generate a PSF pattern in which multiple PSFs are located in multiple lines of the PSF pattern, a PSF estimation unit that is configured to estimate at least one of the PSFs of an out-of-focus input image from at least one of multiple step responses of respective ones of the lines of the PSF pattern with respect to an edge of the out-of-focus input image and an image restoration unit that is configured to restore the out-of-focus input image to a focused restored image using the estimated PSF.

In some embodiments, the PSF pattern includes a center point and multiple center circles and the PSFs are located at a lattice point where multiple horizontal lines and multiple vertical lines are located inside an outermost center circle among the center circles. Some embodiments provide that the PSF pattern includes the PSFs arranged diagonal to each other in four directions with respect to a centered horizontal line among the horizontal lines and a centered vertical line among the vertical lines.

In some embodiments, the PSF estimation unit is configured to determine step responses of vertical lines of the PSF pattern with respect to a vertically oriented edge of the out-of-focus input image. Some embodiments provide that the PSF estimation unit includes a step response calculation unit that is configured to performing image convolution on the edge of the out-of-focus input image and each line of the PSF pattern and to calculate the step responses including an intensity of the center point and/or intensities of the center circles as factors. Some embodiments include an intensity calculation unit that is configured to determine the intensities of the center point and the center circles of the PSF pattern from the step responses and a PSF estimation value calculation unit that is configured to calculate estimation values of the PSFs belonging to the PSF pattern from the intensities of the center point and the center circles calculated by the intensity calculation unit.

In some embodiments, the step responses are determined by using the intensity of the center point and/or intensities of the center circles as factors if the PSFs are located in the center point and/or the center circles, and by using an intensity of an adjacent center point and/or intensities of multiple adjacent center circles as factors, if the PSFs are not located in the center point and/or the center circles. Some embodiments provide that responsive to a matrix A having factors corresponding to the intensity of the center point and/or the intensities of the center circles of each of the step responses for lines of the PSF pattern in a direction, a vector for the intensities of the center point and the center circles is r, and a vector for the step responses is s, the intensity calculation unit calculates the intensities of the center point and the center circles according to an equation Ar=s. In some embodiments, the intensity calculation unit determines a pseudo inverse matrix A+with respect to the matrix A, and calculates the intensities of the center point and the center circles according to an equation r=A+s.

Some embodiments provide that if the PSFs are located in the center point or the center circles, the PSF estimation calculation unit estimates the intensity of the center point and/or the intensities of the center circles as the corresponding PSF, and, if the PSFs are not located in the center point or the plurality of center circles, the PSF estimation calculation unit estimates the average value of the intensities of an adjacent center point or adjacent center circles as the corresponding PSF.

Some embodiments provide that the image restoration unit de-convolutes the out-of-focus input image and the estimated PSF and performs digital focus correction on the out-of-focus input image.

Some embodiments of the present invention include an image system that includes an imaging device that is configured to capture an image of an object through a lens, and generate an image signal of the captured image, an image processing apparatus that is configured to receive the image signal from the imaging device and to process the image signal, and a display unit that is configured to display the processed image signal. In some embodiments, the image processing apparatus includes a PSF pattern generation unit that is configured to generate a PSF pattern in which multiple PSFs are located in multiple lines of the PSF pattern, a PSF estimation unit that is configured to estimate PSFs of an out-of-focus input image from step responses of the lines of the PSF pattern with respect to an edge of the out-of-focus input image, and an image restoration unit that is configured to restore the out-of-focus input image to a focused restored image using the estimated PSF.

Some embodiments of the present invention include an image processing method performed in an image processing apparatus. Operations according to such methods may include generating a point spread function (PSF) pattern that includes multiple PSFs that correspond to intersections of multiple vertical lines and multiple horizontal lines, estimating multiple step responses that correspond to the vertical lines of the PSF pattern with respect to a vertically oriented edge of an out of focus image, and estimating the intensities of the center point and the center circles of the PSF pattern from the step responses. Operations may include calculating a PSF estimate value of the out-of-focus input image from the step responses of the vertical lines of the PSF pattern with respect to an edge of the out-of-focus input image. Some embodiments provide that the generating, estimating and/or calculating operations are performed using at least one data processor.

In some embodiments, the PSF pattern includes a center point and multiple center circles and the PSFs are located inside an outermost center circle among the center circles. Some embodiments provide include, before estimating ones of the step responses, performing image convolution on the edge of the out-of-focus input image and each vertical line of the PSF pattern. Some embodiments provide that, if the PSFs are located at the center point or in the center circles, the step responses are determined using an intensity of the center point and/or intensities of ones of the center circles as factors. Some embodiments provide that, if the PSFs are not located at the center point or in the center circles, the step responses are determined using an intensity of an adjacent center point and/or intensities of ones of multiple adjacent center circles as factors.

In some embodiments, calculating the PSF estimate includes performing image convolution on the edge of the out-of-focus input image and each line of the PSF pattern and calculating the step responses including an intensity of the center point or intensities of the center circles as factors, and determining the intensities of the center point and the center circles of the PSF pattern from the step responses.

Some embodiments provide that a matrix A includes factors corresponding to the intensity of the center point or the intensities of the center circles of each of the step responses for lines of the PSF pattern, a vector r corresponds to the intensities of the center point and the center circles, a vector s corresponds to the step responses, and the intensities are determined using an equation Ar=s. In some embodiments, determining the intensities includes obtaining a pseudo inverse matrix A+with respect to the matrix A, and calculating the intensities of the center point and the center circles according to an equation r=A+s.

Some embodiments include restoring the out-of-focus input image to a focused image using the estimated PSF estimate. In some embodiments, restoring the out-of-focus input image includes de-convoluting the out-of-focus input image and the estimated PSF and performing digital focus correction on the out-of-focus input image.

Some embodiments of the present invention include an image processing method performed in an image processing apparatus. Operations according to such methods include generating a PSF pattern in which multiple PSFs are located in multiple lines, estimating a PSF of an out-of-focus input image from step responses of the lines of the PSF pattern with respect to an edge of the out-of-focus input image, and restoring the out-of-focus input image to a focused image by using the estimated PSF.

In some embodiments, the PSF pattern includes a center point and multiple center circles, and the plurality of PSFs are located at a lattice point where multiple horizontal lines and multiple vertical lines are located inside an outermost center circle among the center circles. Some embodiments provide that estimating the PSF includes determining step responses of vertical lines of the PSF pattern with respect to a vertically oriented edge of the out-of-focus input image. In some embodiments, estimating the PSF includes performing image convolution on the edge of the out-of-focus input image and each line of the PSF pattern and calculating the step responses including an intensity of the center point or intensities of the center circles as factors. Operations may also include determining the intensities of the center point and the center circles of the PSF pattern from the step responses and calculating estimation values of the PSFs belonging to the PSF pattern from the intensities of the center point and the center circles. Some embodiments provide that the step responses are determined by having the intensity of the center point or the intensities of the center circles as factors if the PSFs are located at the center point or in the center circles, and by having an intensity of an adjacent center point or intensities of multiple adjacent center circles as factors, if the PSFs are not located at the center point or in the center circles.

In some embodiments, determining the intensities includes, when a matrix A includes factors with respect to the intensity of the center point or the intensities of the center circles of each of the step responses for lines of the PSF pattern, and a vector r corresponds to the intensities of the center point and the center circles, and a vector s corresponds to the step responses, calculating the intensities of the center point and the center circles according to an equation Ar=s.

Some embodiments provide that determining the intensities includes obtaining a pseudo inverse matrix A+with respect to the matrix A, and calculating the intensities of the center point and the center circles according to an equation r=A+s. Some embodiments provide that calculating the estimation values includes, if the plurality of PSFs are located in the center point or the center circles, estimating the intensity of the center point or the intensities of the center circles as the corresponding PSF, and, if the PSFs are not located in the center point or the center circles, estimating the average value of the intensities of the adjacent center point or the adjacent center circles as the corresponding PSF.

In some embodiments, restoring the out-of-focus input image includes de-convoluting the out-of-focus input image and the estimated PSF and performing digital focus correction on the out-of-focus input image.

It is noted that aspects of the invention described with respect to one embodiment, may be incorporated in a different embodiment although not specifically described relative thereto. That is, all embodiments and/or features of any embodiment can be combined in any way and/or combination. These and other objects and/or aspects of the present invention are explained in detail in the specification set forth below.

DETAILED DESCRIPTION OF THE EMBODIMENTS

It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. Thus, a first element discussed below could be termed a second element without departing from the scope of the present invention. In addition, as used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It also will be understood that, as used herein, the term “comprising” or “comprises” is open-ended, and includes one or more stated elements, steps and/or functions without precluding one or more unstated elements, steps and/or functions. The term “and/or” includes any and all combinations of one or more of the associated listed items.

It will also be understood that when an element is referred to as being “connected” to another element, it can be directly connected to the other element or intervening elements may be present. In contrast, when an element is referred to as being “directly connected” to another element, there are no intervening elements present. It will also be understood that the sizes and relative orientations of the illustrated elements are not shown to scale, and in some instances they have been exaggerated for purposes of explanation.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and this specification and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein. The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention, however, may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

It should be construed that forgoing general illustrations and following detailed descriptions are exemplified and an additional explanation of claimed inventions is provided.

Reference numerals are indicated in detail in some embodiments of the present invention, and their examples are represented in reference drawings. Throughout the drawings, like reference numerals are used for referring to the same or similar elements in the description and drawings.

FIG. 1is a block diagram of an image processing apparatus100according to some embodiments of the inventive concept.

Referring toFIG. 1, the image processing apparatus100may include a point spread function (PSF) pattern generation unit120, a PSF estimation unit140, and an image restoration unit160. The PSF pattern generation unit120generates a PSF pattern P_PSF for estimating a PSF used to restore an out-of-focus input image InIMG. The PSF is a function of a distribution of the amount of light obtained from a surface on which an image is formed when an input of a point passes through an optical system (lenses, etc.) The PSF estimation unit140estimates the PSF indicating a degree of out-focusing or blurring of the out-of-focus input image InIMG. The image restoration unit160restores the out-of-focus input image InIMG using the estimated PSF E_PSF. That is, the image restoration unit160restores the out-of-focus input image InIMG to the focused image ReIMG.

FIG. 2is a flowchart of an image processing method200according to some embodiments of the inventive concept.

Referring toFIG. 2, the image processing method200of the present embodiment includes an operation220of generating a PSF, an operation240of estimating the PSF, and an operation260of restoring an image.

The image processing apparatus100and the image processing method200of the present embodiment will now be described in more detail. The input image InIMG of the image processing apparatus100may be out-of-focus or blurred since a light reflected (emitted) from a point (a point light source) of a subject is not captured as a point of an image sensor (not shown). The degree of out-focusing or blurring is uniformly distributed according to a shape of an isotropy circle of confusion (COC) about a focus.

Therefore, in order to focus (restore) the out-of-focus input image InIMG, the degree of out-focusing is determined. As described above, the degree of out-focusing of the input image InIMG is determined by estimating the PSF.

FIG. 3is a conceptual diagram of a PSF h.

Referring toFIG. 3, an estimated value of the PSF h with respect to a point (pixel) of the input image InIMG is obtained by calculating intensities of a center point r0and center circles r1˜r5with respect to pixels of the input image InIMG. The intensities of a center point r0and center circles r1˜r5may be represented by an amount of light.

FIG. 4is a diagram for explaining a PSF pattern P_PSF generated by the PSF pattern generation unit120ofFIG. 1according to some embodiments of the inventive concept.

Referring toFIG. 4, the PSF pattern generation unit120of the present embodiment establishes the PSF pattern P_PSF so that a plurality of PSFs (shown in dots) are located on or within an outermost center circle r5, and on lattice points of vertical lines C1˜C11and horizontal lines R1˜R11. The PSF pattern generation unit120establishes the PSF pattern P_PSF so that the PSFs are symmetrical to each other in four directions. In particular, the PSF pattern P-PSF are characterized as the center point r0and the center circles (circles of confusion) r1˜r5. A single square blank pix may correspond to a single pixel of the input image InIMG.

According to some embodiments, if the PSF is located on the center point r0or the center circles r1˜r5of the PSF pattern P_PSF, the PSF is estimated as intensities of the center circles r1˜r5. Meanwhile, if the PSF is not located on the center point r0or the center circles r1˜r5of the PSF pattern P_PSF, the PSF is estimated as an intensity of an adjacent center point or an average of the intensities of the center circles r1˜r5.

It is assumed that r(0) denotes an intensity of a center point, and r(k) denotes intensities of center circles. In the PSF pattern P_PSF, k may be 1, 2, 3, 4, or 5. The PSF located on a center circle5r5is r(5). The PSF located between the center circle5r5and a center circle4r4is (r(5)+r(4))/2. The PSF may be expressed as (h(i,j)) based on an algorithm which is shown below, using a vertical direction location i and a horizontal direction location j according to the image processing apparatus100and the image processing methods200as disclosed herein,

[Algorithm]For⁢⁢i,j=-R,…⁢,0,…⁢,R⁢For⁢⁢k=0,…⁢,R⁢If⁢⁢i2+j2=k2,then⁢⁢h⁡(i,j)=r⁡(k)⁢Else⁢⁢if⁢⁢(k-1)2<i2+j2<k2,⁢⁢then⁢⁢h⁡(i,j)=12⁢{r⁡(k)+r⁡(k-1)}
r(k) denotes an intensity of a center point or intensities of center circles. R denotes the number of center circles (R is an integer number greater than 1). If a vertical direction location i and a horizontal direction location j of the center point is (0,0), the vertical direction location i and the horizontal direction location have j values between −R and R.

As shown in the algorithm above, according to the image processing apparatus100and the image processing methods200, if the PSF is located on the center point or the center circles (i2+j2=k2) the PSF is estimated as r(0) or r(k) that is the intensity of the center point or intensities of center circles. Meanwhile, if the PSF is not located on the center point or the center circles ((K−1)2<i2+j2<k2), the PSF is estimated as the average value of the intensities of the adjacent center point r0or the adjacent center circles ({r(k)+r(k−1)}/2).

According to the image processing apparatus100and the image processing method200of embodiments described herein, the intensity of the center point r0or the intensities of the center circles r1˜r5may be calculated in order to estimate the PSF. A method of calculating the intensity of the center point r0or the intensities of the center circles r1˜r5will now be described in more detail.

A step response calculation unit142calculates a step response sr with respect to an edge of the input image InIMG. Referring toFIGS. 5 and 6, the step response calculation unit142scans (in an arrow direction ofFIG. 6) the PSF pattern P_PSF from a dark area A of the input image InIMG including a vertical direction edge EG to a bright area B thereof, and performs image convolution on the PSF pattern P_PSF and the input image InIMG including the vertical direction edge EG. That is, when the horizontal lines C1˜C11of the PSF patterns P_PSF (as illustrated inFIG. 4) are located in the vertical direction edge EG of the input image InIMG, the step response (S(J)˜S(J+D−1(=11))) is calculated. Although the dark area A may be darker than shown in section (a) ofFIG. 5, the dark area A is shown slightly lighter in order to show the content included therein.

The enumerated circles illustrated inFIG. 5represent the PSF pattern P_PSF through the sequence of periods in the image convolution. When the PSF pattern P_PSF is located in the dark area A (period1ofFIG. 5andFIG. 6A), a convolution value is maintained as “0”. The convolution value starts increasing after the vertical line1C1of the PSF pattern P_PSF first meets the vertical direction edge EG (period2ofFIG. 5andFIG. 6B). The convolution value increases during periods3through6when the PSF pattern P_PSF moves from the vertical direction edge EG to the bright area B. If the PSF pattern P_PSF is completely located in the bright area B (period7ofFIG. 5AandFIG. 6D), the convolution value is maintained at a predetermined value x.

A method of calculating a step response sr(j) will now be described with reference toFIG. 6B.

Referring toFIG. 4andFIG. 6B, a PSF at which the vertical direction edge EG first meets the PSF pattern P_PSF is located in a lattice point (intersection point) of the vertical line C1and the horizontal line R6of the PSF pattern P_PSF. If J ofFIG. 5denotes a horizontal direction location where the vertical direction edge EG first meets the PSF pattern P_PSF, the step response sr(j) with respect to the vertical line C1is expressed below.
Sr(J)=r(5)  [Equation 1]

Referring toFIG. 6C, after the vertical line C1of the PSF pattern P_PSF meets the vertical direction edge EG of the input image InIMG, if the vertical line C1moves by a single pixel in an arrow direction, the vertical line2C2of the PSF pattern P_PSF meets the vertical direction edge EG of the input image InIMG. A step response sr(J+1) with respect to the vertical line C2of the PSF pattern P_PSF is expressed below.
sr(J+1)=s(J)+[2r(5)+4(r(5)+r(4))/2+r(4)]=3r(4)+5r(5)  [Equation 2]
In Equation 2, two of seven PSFs located at the vertical line C2are located in a center circle r5, four thereof are located between the center circle r5and the center circle r4, and the other one is located in the center circle r4. As described above, if the PSF is located on a center point or center circles (i2+j2=k2), the PSF is estimated as r(0) or r(k), that is, the intensity of the center point or intensities of center circles. Meanwhile, if the PSF is not located on the center point or the center circles ((K−1)2<i2+j2<k2), the PSF is estimated as the average value of the intensities of the adjacent center point r0or the adjacent center circles ({r(k)+r(k−1)}/2).

Therefore, step responses with respect to the vertical line C3through the vertical line C11of the PSF pattern P_PSF may be obtained with reference to Equations 1 and 2. For example, the vertical line C3may be obtained by adding values of two PSFs located in the center circle r5, two PSFs located between the center circle r5and the center circle r4, four PSFs located between the center circle r4and the center circle r3, and one PSF located in the center circle r3in the step response sr(J+1) with respect to the vertical line C2of the PSF pattern P_PSF.

In the manner stated above, referring toFIG. 6D, the PSF pattern P_PSF moves by a single pixel in the arrow direction, and performs image convolution on the input image InIMG according to a left outermost vertical line (the vertical line C11). As a result, eleven step responses sr(J)˜sr(J+10) with respect to the vertical lines C1˜C11of the PSF pattern P_PSF are calculated as listed in a table ofFIG. 7.

For example, referring toFIG. 7, the step response sr(J) is for the vertical line C1of the PSF pattern P_PSF, the step response sr(J+1) is for the vertical line C2of the PSF pattern P_PSF, and the step response sr(J+2) is for the vertical line C3of the PSF pattern P_PSF.

If the table regarding the step responses sr(J)˜sr(J+10) having an intensity of a center point or intensities of center circles as factors is determined as shown inFIG. 7, an intensity detection unit144of the PSF estimation unit140may obtain the intensity of the center point r0and the intensities of the center circles r1˜r5of the PSF pattern P_PSF by calculating a value of the following linear equation. That is, in Equation 3, since a vector s with respect to a step response and a matrix A with respect to the table ofFIG. 7are determined, a vector r with respect to the intensities of the center point r0and the center circles r1˜r5can be calculated.
Ar=s  [Equation 3]
D denotes a diameter of the PSF pattern P_PSF, and R denotes the number of center circles included in the PSF pattern P_PSF, A denotes a matrix Dx(R+1) including elements having the same value as shown in the table ofFIG. 7, r denotes a vector (R+1)×1 with respect to intensities of center circles, and s denotes a vector Dx1with respect to step responses. In the PSF pattern P_PSF, D(˜2R+1,R=5) is 11.

The intensity detection unit144may factorize the matrix Dx(R+1) A using a singular value decomposition (SVD) according to the following Equation 4,
A=USVT[Equation 4]
U denotes an orthogonal matrix D×D, S denotes a rectangular matrix Dx(R+1) in which elements located in a diagonal line are not “0”, and V denotes an orthogonal matrix (R+1)×(R+1). The SVD may be extended to a complex field in which a term of an orthogonal matrix is substituted as a unit matrix. A matrix transposeTmay be substituted as a conjugate transpose. Therefore, a pseudo inverse matrix A+of A is calculated according to Equation 5,
A+=VS+UT[Equation 5]
S+denotes a matrix transpose of S having entries other than all 0.

By using Equations 3 and 5, a vector r=[r(0), . . . , r(k))]Twith respect to the intensities of the center point and the center circles is calculated according to Equation 6 below,
r=A+s  [Equation 6]
Referring toFIG. 1again, the PSF estimation calculation unit146of the PSF estimation unit140estimates the PSF by substituting r of Equation 6 with the above algorithm. That is, if the PSF is located on the center point or the center circles (i2+j2=k2), the PSF is estimated as r(0) or r(k), that is, the intensity of the center point or intensities of center circles.) Meanwhile, if the PSF is not located on the center point or the center circles ((K−1)2<i2+j2<k2), the PSF estimation calculation unit146estimates the PSF as the average value of the intensities of the adjacent center point r0or the adjacent center circles ({r(k)+r(k−1)}/2).

The estimated PSF E_PSF is shown inFIG. 8A.

The image restoration unit160restores the input image InIMG using the estimated PSF E_PSF. For example, the image restoration unit160may de-convolute the estimated PSF E_PSF shown inFIG. 8Awith the input image InIMG, and output a restored image ReIMG shown inFIG. 8B. Upon comparing the restored image ReIMG with the out-of-focus input image InIMG and a step response thereof shown inFIG. 8E, the restored image ReIMG may have an ideal step response as shown inFIG. 8C.

As described above, the image processing apparatus100and the image processing method200of the present embodiment may perform image convolution on line units of a special PSF pattern with an input image and obtain an ideal step response. The ideal step response is used to estimate a PSF, thereby easily and accurately estimating the PSF.

FIG. 9is a block diagram of an image system900including the image processing apparatus100, according to some embodiments of the inventive concept.

Referring toFIG. 9, the imaging system900may include a lens910, an imaging device920, and the image signal processing apparatus100. The imaging system900may further include a storage unit930and a display unit940.

The imaging device920may capture an image of an object (not shown), generate image signals of the captured image, and output the generated image signals to the image signal processing apparatus100. The imaging device920may be embodied as a complementary metal-oxide semiconductor (CMOS) image sensor (CIS), a charge-coupled device (CCD), etc. The pixel array of the imaging device920may be, e.g., an active pixel sensor (APS) array in which a million or more pixels are arranged. The imaging device920may generate image signals including R, G, and B by capturing an image of an object (not shown) using photodiodes included in each of pixels of the APS array. Three types of color filters, e.g., R, G, B, may be arranged, e.g., in the form of a Bayer pattern, on the top of each pixel constituting the APS array, so that light of predetermined colors may be incident thereto. In such cases, image signals output by the imaging device920may have a Bayer pattern.

The image signal processing apparatus100shown inFIG. 9may be a general image signal processor (ISP), and may include the PSF pattern generation unit120, the PSF estimation unit140, and the image restoration unit160.

The image processing apparatus100may perform at least one of an interpolation process, a gamma correction process, an edge enhancement process, auto exposure (AE) control, auto white balance (AWB) control, and digital zoom, in addition to the function of the image restoration unit160.

The lens910may be arranged between an object (not shown) and the imaging device920. The lens910may refract an image of the object transmitted to the imaging device920. In some embodiments, the lens910of the imaging system900may be an extended depth of focus (EdoF) lens.

The storage unit930may receive an image signal output by the image processing apparatus100and may store the received image signal. The image signal output by the image processing apparatus100may be compressed in a format, such as the joint photographic expert group (JPEG) format or the motion picture experts group (MPEG) format, before being stored in the storage unit930. The storage unit930of the imaging system300may be, e.g., a general memory device, such as a dynamic random access memory (DRAM), a flash memory, etc.

The display unit940may receive the image signal output by the image processing apparatus100and may display the image signal on a screen. An image signal output by the image processing apparatus100may be output to the display unit940after resolution conversion. The display unit940of the imaging system900may be a general display device, e.g., a liquid crystal display (LCD).

An imaging system employing one or more aspects described herein, e.g., the imaging system900, may be included, e.g., in a camera module for a mobile phone camera, a digital still camera, etc.