Imaging device and focusing-verification display method

An imaging device comprising an image generation unit, a difference-emphasis processing unit, a display unit, a display controller that displays the first display image on the display unit and displays the second display image having been subjected to the difference-emphasis processing by the difference-emphasis processing unit in a displayed area of the first display image, and a calculation unit that calculates a parallax between the first pixel in the first image and the second pixel in the second image corresponding to the first pixel, wherein the difference-emphasis processing unit determines whether the parallax between the first image and the second image is large or small on the basis of the parallax calculated by the calculation unit, and performs the difference-emphasis processing on the basis of a result of determination whether the parallax is large or small.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging device displaying a focusing-verification image for manual focus and a focusing-verification display method.

2. Description of the Related Art

As a digital camera, those having a so-called manual focus mode have been well known in which a user can manually adjust a focus, in addition to those of auto focus using a phase difference detection method and a contrast detection method.

As a digital camera having the manual focus mode, there have been well known those using a method of using a split microprism screen in which a reflex mirror is provided such that focus adjustment can be carried out while confirming an imaged subject to display a visual phase difference and a method of visually confirming contrast.

A digital camera with the reflex mirror being omitted which has spread in recent years has no method for confirming a subject image while displaying the phase difference because of the lack of the reflex mirror, and has had to depend on the contrast detection method. In this case, however, contrast cannot be displayed at a resolution beyond that of a display device such as a LCD or the like, and thus, there has been no choice but to take a method of displaying with a part being enlarged.

Therefore, in recent years, a split image (also referred to as a “focusing-verification image” in the description) is displayed in a live view image in order to facilitate an operation of an operator bringing the subject into focus in the manual focus mode. Here, the split image is obtained by compositing two subject images (phase difference image) acquired by pupil division picking up, and represents the phase difference of the subject image. In other words, the split image is displayed in which an upper half of one subject image and a lower half of the other subject image are arranged vertically adjacent to each other. The vertically adjacent two subject images are displayed with being horizontally displaced from each other in an out-of-focus state, and the vertically adjacent two subject images are displayed with no horizontal displacement in an in-focus state. The operator manipulates a focus ring for focusing such that the horizontal displacement between two subject images in the split image disappears.

A digital camera described in Japanese Patent Application Laid-Open No. 2004-40740 (hereinafter referred to as PTL 1) vertically moves a diaphragm and an optical axis to image a subject picture at each of two distance measurement positions and uses these two subject pictures to display a split image in a live view image.

A digital camera described in Japanese Patent Application Laid-Open No. 2001-309210 (hereinafter referred to as PTL 2) finds as a displacement amount a value corresponding to a distance between an image plane of a subject image and a light receiving surface of an image pickup device to display in a live view image a split image in which the subject images are displaced in horizontally opposing directions depending on this displacement amount.

Digital cameras described in Japanese Patent Application Laid-Open No. 2009-147665 (hereinafter referred to as PTL 3) and Japanese Patent Application Laid-Open No. 2009-163220 (hereinafter referred to as PTL 4) include an image pickup device having a plurality of normal pixels for imaging and a plurality of two kinds of phase difference pixels for focus detection arrayed on an image pickup surface, the phase difference pixel receiving a pupil-divided subject light. This digital camera generates a picked-up image on the basis of an output signal from the normal pixel to display a live view image and generates a split image on the basis of output from each of two kinds of phase difference pixels to display in the live view image.

SUMMARY OF THE INVENTION

However, the digital camera described in PTL 1 requires a mechanical configuration for moving the diaphragm, which causes a problem of securing a space for housing this configuration, increase of the number of parts and the like. The digital camera described in PTL 2 does not have a configuration for pupil-dividing and picking up of the subject light, which makes it difficult to achieve an accurate and unbroken split image (focusing-verification image).

In the digital camera described in PTLs 3 and 4 which use two kinds of phase difference pixels to generate the split image, the displacement amount between two subject images vertically displayed as the split image is small in a slightly out-of-focus state, which makes it less easy to visually recognize the split image, disadvantageously causing difficulty in sufficient focusing by a user using a manual focus manipulation.

There has been devised a parallax emphasis technology for emphasizing a parallax between two subject images, but such a parallax emphasis technology has a large processing load and parallax emphasis has been difficult to perform while displaying a live view. For example, the processing load for a matching processing matching between two subject images is large.

An object of the present invention is to provide an imaging device capable of making it easy to see an image for focusing-verification even in a slightly out-of-focus state and a focusing-verification display method.

In order to achieve the above object, the invention provides an imaging device including an image generation unit that generates a first display image on the basis of an image signal output from an image pickup device having first and second pixel groups, subject lights passed through first and second regions in an imaging lens being pupil-divided and incident on the first and second pixel groups, respectively, and generates a second display image used for focusing-verification on the basis of a first image and a second image output from the first pixel group and the second pixel group, respectively, a difference-emphasis processing unit that performs difference-emphasis processing for enlarging a difference of pixel-values between a first pixel in the first image and a second pixel in the second image corresponding to the first pixel, a display unit, and a display controller that displays the first display image on the display unit and displays the second display image having been subjected to the difference-emphasis processing by the difference-emphasis processing unit in a displayed area of the first display image.

This makes a border line to be interposed between the first image and second image constituting the second display image (focusing-verification image) to emphasize the difference of the pixel-value, which can make the second display image (focusing-verification image) easily seen even if the parallax is small and can display the second display image (focusing-verification image) that is emphasized with low load even while displaying a live view as compared with a case where the parallax emphasis is carried out.

According to an embodiment, the image pickup device further has a third pixel group on which the subject light not pupil-divided is incident, and the first display image is generated on the basis of a third image (normal image) output from the third pixel group.

According to an embodiment, the difference-emphasis processing unit calculates an average value between the pixel-value of the first pixel and the pixel-value of the second pixel corresponding to the first pixel, and enlarges, using the average value as a reference, a difference between the pixel-value of the first pixel and the average value and a difference between the pixel-value of the second pixel corresponding to the first pixel and average value. This not only allows a phase difference between the first image and the second image to become easier to visually recognize, but also allows an image configuration of the first image and the second image to scarcely change.

According to an embodiment, a calculation unit that calculates a parallax between the first pixel in the first image and the second pixel in the second image corresponding to the first pixel is included, in which the difference-emphasis processing unit determines whether the parallax between the first image and the second image is large or small on the basis of the parallax calculated by the calculation unit, and performs the difference-emphasis processing on the basis of a result of determination whether the parallax is large or small. According to an embodiment, the difference-emphasis processing unit, as a result of determining whether the parallax is large or small, sets, in a case where the parallax is determined to be large, an increase amount for the difference of the pixel-value to be smaller than that in a case where the parallax is determined to be small. According to an embodiment, the difference-emphasis processing unit determines how many pixels continuously exist in which the difference between the pixel-value of the first pixel and the pixel-value of the second pixel corresponding to the first pixel is larger than a threshold, and processes such that the more the continuous pixels, the less the increase amount for the difference of the pixel-value is made. This makes it possible that a portion having sufficient parallax is not subjected to the excessive difference-emphasis to allow the second display image (image for focusing-verification) to be kept in an original visibility.

According to an embodiment, the difference-emphasis processing unit determines whether the difference between the pixel-value of the first pixel and the pixel-value of the second pixel corresponding to the first pixel is large or small, and performs the difference-emphasis processing on the basis of a result of determination whether the difference of the pixel-value is large or small. According to an embodiment, the difference-emphasis processing unit, as a result of determining whether the difference of the pixel-value is large or small, sets, in a case where the difference of the pixel-value is determined to be large, the increase amount for the difference of the pixel-value to be smaller than that in a case where the difference of the pixel-value is determined to be small. This makes the excessive difference-emphasis not carried out in the high contrast condition to allow the second display image (image for focusing-verification) to be kept in an original visibility.

According to an embodiment, in the difference-emphasis processing unit, assuming that the pixel-values of the first pixel and the second pixel corresponding to the first pixel are L and R, respectively, the average value between the pixel-value of the first pixel and the pixel-value of the second pixel corresponding to the first pixel is ave=(L+R)/2, and the difference-emphasis coefficient is K (where, K>0), a pixel-value L′ of the first pixel and a pixel-value R′ of the second pixel after the difference-emphasis processing are L′=L+(L−ave)×K and R′=R+(R−ave)×K, respectively.

According to an embodiment, the difference-emphasis processing unit processes such that the larger a difference between the pixel-value L of the first pixel and the pixel-value R of the second pixel corresponding to the first pixel, the smaller the difference-emphasis coefficient K is made.

According to an embodiment, the difference-emphasis processing unit uses a shading correction coefficient for correcting shading due to pupil-division of the first image and the second image to perform the difference-emphasis processing. This allows the difference-emphasis to be carried out without the shading component being emphasized.

According to an embodiment, assuming that the pixel-values of the first pixel and the second pixel corresponding to the first pixel are L and R, respectively, the shading correction coefficient with respect to the first image is α, the shading correction coefficient with respect to the second image is β, an average value obtained by carrying out an arithmetic with respect to the shading correction coefficient is ave=(α×L+β×R)/2, and the difference-emphasis coefficient is K (where, K>0), a pixel-value a′ of the first pixel and a pixel-value b′ of the second pixel after the difference-emphasis processing are L′=L+(α×L−ave)×K and R′=R+(β×R−ave)×K, respectively.

According to an embodiment, the difference-emphasis processing unit processes such that the larger a difference between the pixel-value L of the first pixel and the pixel-value R of the second pixel corresponding to the first pixel, the smaller the difference-emphasis coefficient K is made.

According to an embodiment, the difference-emphasis processing unit performs the difference-emphasis processing on, of the second display image, only an area in the vicinity of a border line between the first image and the second image.

The present invention provides a focusing-verification display method, using an image pickup device that has first and second pixel groups which subject lights passed through first and second regions in an imaging lens are pupil-divided and incident on, an image generation unit that generates a first display image on the basis of an image signal output from the image pickup device and generates a second display image used for focusing-verification on the basis of a first image and a second image output from the first pixel group and the second pixel group, respectively, and a display unit, the method including a difference-emphasis processing step performing difference-emphasis processing for enlarging a difference of pixel-values between a first pixel in the first image and a second pixel in the second image corresponding to the first pixel, a display image generating step generating the second display image by the image generation unit on the basis of the first image and the second image having been subjected to the difference-emphasis processing, and a displaying step displaying the first display image on the display unit and displaying the second display image having been subjected to the difference-emphasis processing in the difference-emphasis processing step in a displayed area of the first display image.

According to the invention, an image for focusing-verification (second display image) can be made easily seen even in a slightly out-of-focus state.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, a description is given in detail of embodiments according the present invention with reference to the drawings.

[Exemplary Configuration of Digital Camera]

As shown inFIG. 1, provided on a front face of a camera main body2aof a digital camera2which is an example of an imaging device according to the invention are a lens barrel3configured to include an image pickup optical system, a stroboscopic light emitting part5and the like. The camera main body2ahas a shutter button6, a power switch7and the like provided on a top face thereof. The lens barrel3has a focus ring (lens movement mechanism)3a, used for a manual focus (hereinafter, simply referred to as “MF”) manipulation, rotatably attached to an outer periphery thereof.

As shown inFIG. 2, the camera main body2ahas a display unit8and an operation unit9provided on a rear face thereof. The display unit8serves as an electronic viewfinder in an imaging standby state to display a live view image (also referred to as through image). In image reproduction, an image is reproduced and displayed on the display unit8on the basis of image data recorded in a memory card10.

The operation unit9includes various switches. The operation unit9in this example includes a mode selector switch, cross-shaped key, execution key and the like. The mode selector switch is operated when two operating modes of the digital camera2is switched. The digital camera2has an imaging mode for imaging a subject to obtain a picked-up image, a reproducing mode for reproducing and displaying the picked-up image and the like. The imaging mode includes an AF mode for performing an auto focus (hereinafter, simply referred to as “AF”) operation, and an MF mode for performing the MF manipulation.

The cross-shaped key and the execution key are used for various operations. The cross-shaped key and execution key in this example are used in the MF mode as an input device (position input part, number input part) for instructing change of a position, the number or the like of a split line (hereinafter, sometimes referred to as “border line”) when displaying a focusing-verification image described later. The cross-shaped key and the execution key are operated when a menu screen or a setting screen is displayed on the display unit8, a cursor displayed in the menu screen or the setting screen is moved, and various settings for the digital camera2are fixed, and so on.

The camera main body2ahas a card slot into which the memory card10is loaded and a loading lid to open and close an opening of the card slot provided on a bottom face thereof, illustration thereof being omitted.

As shown inFIG. 3, a CPU11of the digital camera2sequentially executes various programs and data read out from a memory13on the basis of a control signal from the operation unit9to overall control various sections of the digital camera2. A RAM area of the memory13serves as a transient storage for a work memory where the CPU11performs processing or for various pieces of data. The lens barrel3has an imaging lens17including a zoom lens15and a focus lens16, a mechanical shutter18and the like incorporated therein. The zoom lens15and the focus lens16are driven by a zoom mechanism19and a focus mechanism20, respectively, and moved back and forth along an optical axis O1of the imaging lens17. The zoom mechanism19and the focus mechanism20include a gear, a motor and the like. The focus mechanism20is connected with the focus ring3a(focus manipulation part) via a gear not shown. Therefore, the focus mechanism20moves the focus lens16along a direction of the optical axis O1(hereinafter, referred to as “optical axis direction”) as the focus ring3ais rotationally manipulated in the MF mode. In other words, a focus manipulation for changing a position of a focus lens of the imaging lens17(focus position) is carried out by way of the focus ring3a.

The mechanical shutter18has a movable unit (illustration thereof omitted) moving between a closing position where a subject light incident on an image pickup device23is blocked and an opening position where the subject light incident is permitted. The mechanical shutter18moves the movable unit to the respective positions to open/block a path of light from the imaging lens17to the image pickup device23. The mechanical shutter18includes a diaphragm controlling a light amount of the subject light incident on the image pickup device23. The mechanical shutter18, the zoom mechanism19, and the focus mechanism20undergo a motion control by the CPU11via a lens driver25.

The color image pickup device23(hereinafter, simply referred to as “image pickup device”) is arranged behind the mechanical shutter18. The image pickup device23converts the subject light passed through the imaging lens17and the like into an electrical signal to output. The image pickup device23may be various kinds of image pickup devices including a CCD (Charge Coupled Device) type image pickup device, a CMOS (Complementary Metal Oxide Semiconductor) type image pickup device and the like. An image pickup device driver27controls the image pickup device23to be driven under control by the CPU11.

The image processing circuit29subjects the output signal (output) from the image pickup device23to various processings such as gradation conversion, white balance correction, γ correction processing and the like to generate picked-up image data. The image processing circuit29generates, in addition to the picked-up image data, split image data for MF manipulation (hereinafter, also referred to as “focusing-verification image”) in the MF mode. The picked-up image data and the split image data are transiently stored in the VRAM area of the memory13(a VRAM may be separately provided). The VRAM area has a memory area for live view image storing images of continuous two fields to sequentially store the picked-up image data and the split image data to be overwritten.

A compression-decompression processing circuit31subjects the picked-up image data stored in the VRAM area to compression processing when the shutter button6is operated to be pressed down. The compression-decompression processing circuit31subjects the compressed image data obtained via a media I/F32from the memory card10to decompressed processing. The media I/F32records and reads out the picked-up image data in and from the memory card10.

A display controller33reads out the picked-up image data and split image data stored in the VRAM area in the imaging mode to sequentially output to the display unit8. The display controller33outputs the picked-up image data decompressed by the compression-decompression processing circuit31to the display unit8in the reproducing mode.

<Configuration of Color Image Pickup Device>

As shown inFIG. 4, a R pixel35of red (R) color, a G pixel36of green (G) color, and a B pixel37of blue (B) color are arrayed in a matrix on an image pickup surface23aof the image pickup device23(seeFIG. 3). Each of the pixels35,36, and37is a normal pixel which is to be formed into an image with the subject image being subjected to the pupil-division and configured to include a photoelectric conversion device39(seeFIG. 5) and a color filter40of any of three primary colors arranged above the photoelectric conversion device39(seeFIG. 5). In other words, each of the pixels35to37may be considered as a non pupil-division photoelectric conversion device accompanied by the color filter40. Note that “on” and “above” refer to a direction from a semiconductor substrate45toward a microlens49inFIG. 5(upper direction in the figure).

The color filter40of each of R color, G color, and B color is provided respectively above the photoelectric conversion device39for each of the R pixel35, the G pixel36and the B pixel37.

A color filter array (pixel array) of the image pickup device23has the following features (1), (2), (3), (4), (5), and (6).

The color filter array includes a basic array pattern P of non Bayer array having a square array pattern corresponding to 6×6 pixels, and this basic array pattern P is repeatedly arranged in a horizontal direction and a vertical direction in the figure.

In this way, since the RGB color filters40are arrayed with a predetermined period, when the R, G, and B signals read out from the image pickup device23are subjected to synchronization (interpolation) processing (demosaicing processing) and the like, the processing can be performed according to the repeated pattern, as compared with a random array known in the related art. In a case where an image is subjected to thinning processing to be reduced in units of the basic array pattern P, the color filter array after the thinning processing can be made the same as the color filter array before the thinning processing, allowing a common processing circuit to be used.

In the color filter array, one or more color filters of G color which corresponds to a color most contributing for obtaining a luminance signal (color of G in the embodiment) are arranged in a line in each of a horizontal direction, a vertical direction and a diagonal direction (diagonal upper right and diagonal lower left directions, and diagonal lower right and diagonal upper left directions) of the color filter array.

In this way, the color filter of G color is arranged in the line in each of the horizontal, vertical and diagonal directions of the color filter array, improving reproduction accuracy in the image interpolation processing (synchronization processing or the like) in a high-frequency area not limited to in a direction of high-frequency.

In the basic array pattern P, the pixel-numbers of the R pixels35, the G pixels36and the B pixels37are eight, twenty, and eight, respectively. In other words, a ratio between the pixel-numbers of the pixels35to37respectively of R, G, and B colors is 2:10:2, a ratio of the pixel-number of the G pixels36is larger than ratios of the pixel-numbers of the R pixels35and B pixels37of other colors.

In this way, the pixel-number of the G pixels36is different in the ratio from the ratios for the pixel-numbers of the R and B pixels35and37, and particularly the ratio of the pixel-number of the G pixels36most contributing for obtaining the luminance signal is set to be larger than the ratios of the pixel-numbers of the R and B pixels35and37, which can suppress aliasing in the image interpolation processing (synchronization processing or the like) and also improve high-frequency reproducibility.

In the color filter array, one or more color filters40of R color and B color which correspond to two or more other colors than the G color (colors of R and B in the embodiment) arranged in a line of each of the horizontal and vertical directions of the color filter array in the basic array pattern P.

The color filters40of R color and B color are arranged in the line of each of the horizontal and vertical directions of the color filter array, reducing generation of color moire (false color). This can eliminate the arrangement of an optical low-pass filter for suppressing generation of the false color in the light path from an incident plane to the image pickup surface in the optical system, or even in a case of applying the optical low-pass filter, those weak in a function of cutting a high-frequency component for preventing the false color from being generated can be applied so as not to deteriorate resolution.

The color filter array includes a square array corresponding to 2×2 G pixels36each provided with the color filter40of G color. Such 2×2 G pixels36are extracted to find a difference absolute value between pixel-values of the G pixels36in the horizontal direction, a difference absolute value between pixel-values of the G pixels36in the vertical direction, and difference absolute value between pixel-values of the G pixels36in the diagonal direction such that it can determined that there is a correlation in a direction in which the difference absolute value is small, of the horizontal direction, the vertical direction, and the diagonal direction.

In other words, according to this color filter array, information on the G pixels36having the minimum pixel interval is used to be able to determine a direction in which the correlation is high, of the horizontal direction, the vertical direction, and the diagonal direction. A result of this direction determination can be used for the interpolation processing (synchronization processing or the like) in which interpolation is performed from surrounding pixels.

The basic array pattern P is point-symmetric with respect to the center thereof (the center of four color filters40of G color). Four 3×3 sub-arrays in the basic array pattern P are point-symmetric with respect to the respective color filters40of G color at the center of the pattern. Such symmetry makes it possible to reduce or simplify a circuit size of a processing circuit at a subsequent stage.

The image pickup surface23ahas part areas (e.g., central area) on which a first phase difference pixel36a(represented by “G1in a circle” in the figure) and a second phase difference pixel36b(represented by “in a circle G2” in the figure) are provided instead of a part of the G pixels36. The first phase difference pixel36aand the second phase difference pixel36beach are alternately provided at intervals on plural vertical columns (second pixel rows)42and plural horizontal rows (first pixel rows)43in the pixel array of the image pickup device23(one vertical column42and one horizontal row43are representatively designated by the reference numeral in the figure). In the description, the vertical column and horizontal row provided with the phase difference pixel, of the vertical columns and the horizontal rows of the image pickup device23, are designated by the reference numerals “42” and “43”, respectively.

In the embodiment, the first phase difference pixel36aand the second phase difference pixel36beach are arranged along the horizontal direction and the vertical direction on positions where the vertical columns42and the horizontal rows43intersect. The intervals between the same kind phase difference pixels (i.e., first phase difference pixel-first phase difference pixel, second phase difference pixel-second phase difference pixel) are 12-pixel pitch in the vertical direction and 6-pixel pitch in the horizontal direction.

InFIG. 5showing a cross-sectional view of the horizontal row43, the semiconductor substrate (sub)45has the photoelectric conversion devices39formed in a matrix on a surface layer thereof. The semiconductor substrate45is provided with various circuits used to drive the pixels or output the signals, illustration thereof being omitted.

A light transmissive insulation film46is provided on the semiconductor substrate45. A light shielding film47is provided on the insulation film46. The light shielding film47has a normal opening47a, first eccentric opening47b, and second eccentric opening47c. The first to second eccentric openings47band47ceach are formed to have an opening diameter smaller than the normal opening47a.

The normal opening47ais formed on the photoelectric conversion device39of each of the RGB pixels35to37. The center of the normal opening47ais positioned on the center of the photoelectric conversion device39.

The first eccentric opening47bis formed on the photoelectric conversion device39aof the first phase difference pixel36a. The center of the first eccentric opening47bis deviated rightward in the figure with respect to the center of the photoelectric conversion device39abelow itself. This makes a region of a substantially left half of the photoelectric conversion device39aof the first phase difference pixel36a(hereinafter, simply referred to as a left region) be covered by the light shielding film47, whereas a center part of a region of a substantially right half (hereinafter, simply referred to as a right region) is exposed.

The second eccentric opening47cis formed on the photoelectric conversion device39bof the second phase difference pixel36b. The center of the second eccentric opening47cis deviated leftward in the figure with respect to the center of the photoelectric conversion device39bbelow itself. This makes a right region of the photoelectric conversion device39bof the second phase difference pixel36bbe covered by the light shielding film47, whereas a center part of a left region thereof is exposed.

A light transmissive planarizing layer48having a flat surface is provide on the light shielding film47. The color filters40of R, G, and B colors are provided on the planarizing layer48at positions respectively corresponding to the pixels35to37of respective R, G, and B colors. The color filter40of G color is provided at each of positions corresponding to the first and second phase difference pixels36aand36b.

The microlens49is provided on the color filter40of each color and above each of photoelectric conversion devices39,39a, and39b. There may be provided various layers including a light transmissive flat layer also between the color filter40and the microlens49.

A subject light SOL incident from a left oblique direction in the figure on the microlens49above each of the RGB pixels35to37is collected to the right region of the photoelectric conversion device39by the microlens49. In contrast, a subject light50R incident from a right oblique direction in the figure on the microlens49is to the left region off the photoelectric conversion device39by the microlens49. For this reason, the RGB pixels35to37are made to have high sensitivity to both the subject light SOL and the subject light50R.

The subject light SOL incident on the microlens49above the first phase difference pixel36ais collected by the microlens49through the first eccentric opening47bto the right region of the photoelectric conversion device39a. In contrast, the subject light50R incident on the microlens49is blocked by the light shielding film47, and thus is not collected to left region of the photoelectric conversion device39.

The subject light50R incident on the microlens49above the second phase difference pixel36bis collected by the microlens49through the second eccentric opening47cto the left region of the photoelectric conversion device39b. In contrast, the subject light50R incident on the microlens49is blocked by the light shielding film47, and thus is not collected to the left region of the photoelectric conversion device39. Therefore, the light shielding film47functions as a pupil-division part performing the pupil division. Note that the microlens49may be made eccentric instead of allowing the light shielding film47(eccentric openings47band47c) to function as the pupil-division part.

The subject lights50L and50R are subject lights passed through a left region17L and a right region17R, respectively, of the imaging lens17(zoom lens15and focus lens16). For the purpose of preventing complexity of the figure, both lenses15and16are displayed with being unified, and scales of the imaging lens17and the image pickup device23also actually differ.

The subject light incident on the image pickup device23is pupil-divided by the light shielding film47such that the first phase difference pixel36ais made to have high sensitivity to the subject light SOL, whereas the second phase difference pixel36bis made to have high sensitivity to the subject light50R.

[Kind of Pixel of Image Pickup Device (Photoelectric Conversion Device Accompanied by Color Filter)]

The image pickup device23in the example includes a plurality of the first phase difference pixels36a(hereinafter, referred to as “first pixel”) and a plurality of the second phase difference pixels36b(hereinafter, referred to as “second pixel”) where the subject lights (subject light SOL and subject light50R) passed through the different regions (left region17L and right region17R) in the imaging lens17are pupil-divided to be formed into an image, and the plural normal pixels35,36, and37(hereinafter, referred to as “third pixel”) where a subject image is formed without being pupil-divided.

Hereinafter, in the description, for the convenience of explanation, a description is given of the image processing in the image processing circuit29using the above terms “first pixel”, “second pixel” and “third pixel”.

<Configuration of Image Processing Circuit>

As shown inFIG. 6, the image processing circuit29is configured to include a normal processing unit52and a split image processing unit54. The split image processing unit54is configured to include a selection unit102, image generation unit104, and difference-emphasis processing unit106.

The normal processing unit52subjects a third image58cas output signals (output) of a third pixel group57cincluding a plurality of the third pixels (normal pixels35,36, and37) to the image processing to output the third image58c(third image) having been subjected to the image processing as a color picked-up image55.

The split image processing unit54generates a monochrome split image61(focusing-verification image) on the basis of a first image58aas output signals (output) of a first pixel group57aincluding a plurality of the first pixels (first phase difference pixels36a) and a second image58bas output signals (output) of a second pixel group57bincluding a plurality of the second pixels (second phase difference pixels36b).

The selection unit102extracts displayed portions (first division image61L and second division image61R) constituting the split image61from the first image58aas the output signals (output) of the first pixel group57aand the second image58bas the output signals (output) of the second pixel group57b.

The selection unit102, as specifically shown inFIG. 7andFIG. 8, concerning the first image58aand the second image58brespectively output from the first pixel group57aand second pixel group57bof the image pickup device23, selects which pixel (image pixel) of a pixel59ain the first image58aand a pixel59bin the second image58bis used for generating the split image61, the pixel59aand the pixel59bcorresponding to each other. For example, one portion of the split image61(e.g., upper half) is extracted from the first image58aand the other portion of the split image61(e.g., lower half) is extracted from the second image58b.

The difference-emphasis processing unit106uses the first division image61L on the basis of the pixel-value of the first pixel36aand the second division image61R on the basis of the pixel-value of the second pixel36bto perform difference-emphasis processing emphasizing a difference between the pixel-values. The difference-emphasis processing unit106specifically enlarges the difference between the pixel-values of the first pixel (59ainFIG. 8) and the second pixel (59binFIG. 8), where the first pixel and the second pixel correspond to each other in the split image61(focusing-verification image), with a split line63being used as a reference, in a direction perpendicular to the split line63with the split line63interposed therebetween.

The image generation unit104, as shown inFIG. 9andFIG. 10, uses the displayed portion of the first image58aextracted from the first image58aby the selection unit102(first division image61L) and the displayed portion of the second image58bextracted from the second image58bby the selection unit102(second division image61R) to generate the split image61.

As shown inFIG. 9, the split image processing unit54generates, on the basis of a luminance component of the output signal (output) from the first pixel group57a, the monochrome first division image61L (displayed portion of the first image58a) obtained in a case where a central area of a subject in an upper half area thereof in the figure is seen from an L (left) viewpoint side. The split image processing unit54generates, on the basis of a luminance component of the output signal (output) from the second pixel group57b, the monochrome second division image61R (displayed portion of the second image58b) obtained in a case where the central area of the subject in a lower half area thereof in the figure is seen from a R (right) viewpoint side. This allows the monochrome split image61including the first division image61L and the second division image61R to be obtained. The first division image61L and the second division image61R are arranged in the split image61to be adjacent to each other with the split line63(also referred as “border line”) parallel to the horizontal direction being as a border. The split image61is composited into the color picked-up image55such that the split image61is easily grasped, which composition is performed by the display controller33.

The picked-up image55(the third image58chaving been subjected to the image processing) and the split image61are transiently stored in the VRAM area of the memory13. The display controller33reads out the picked-up image55and the split image61from the memory13and composites the split image61into the picked-up image55to be output to the display unit8thereafter. This makes it possible for a user to see a live view image where the monochrome split image61is displayed in a displayed area of the full-color picked-up image55.

The first division image61L as the displayed portion of the first image58aand the second division image61R as the displayed portion of the second image58bare shifted, depending on a focusing state of the focus lens16, in a right and left direction [horizontal direction (first direction)] in the figure. The displacement amount between the first division image61L and the second division image61R at this time corresponds to a displacement amount of a focus of the focus lens16. In other words, the right and left direction in the figure is a phase difference direction corresponding to a displacement direction of the subject lights formed by the imaging lens17into an image on the image pickup surface23a. The displacement amount between the first division image61L and the second division image61R becomes zero (including substantially zero) when the focus lens16is focusing.

As shown inFIG. 10, as the focus of the focus lens16becomes out of focus, the displacement amount between the first division image61L and the second division image61R becomes larger. This allows the user to carry out the focus adjustment while verifying the live view image. In the figure, the subject out of focus is expressed by a two-dot chain line.

FIG. 9andFIG. 10show an example in which the split image61(focusing-verification image) is displayed on the displayed area of the picked-up image55, the picked-up image55being an image obtained after subjecting the third image58c(normal image) output from the third pixel group57cto the image processing, but the configuration may be such that only the split image61is displayed on the display unit8. In other words, cases may be accepted where all pixels of the image pickup device23are the phase difference pixels (the first phase difference pixel and the second phase difference pixel), or where the phase difference pixels (the first phase difference pixel and the second phase difference pixel) are arranged all areas of the image pickup device23at a certain ratio to display only the split image61on the display unit8.

The digital camera2is provided with an AF detection circuit for auto focus or the like, illustration thereof being omitted. The AF detection circuit analyzes an image constituted by the output signal of the first pixel36aand an image constituted by the output signal of the second pixel36band detects the displacement direction of both images and the displacement amount between both images to find a focus adjustment amount (also referred to as defocus amount) of the imaging lens17. On the basis of the focus adjustment amount, the CPU11controls the lens driver25to drive the focus lens16by the focus mechanism20for adjusting the focus. Such phase difference type AF processing has been well known, a specific description thereof being omitted.

Additionally, the digital camera2is provided with an AE detection circuit or the like, illustration thereof being omitted. The CPU11, on the basis of a result of the AE detection circuit, drives the mechanical shutter18via the lens driver25to perform AE processing.

<General Flow of Imaging Process>

Next, a description is given of working of the digital camera2having the above configuration with reference toFIG. 10. When the digital camera2is set to the AF mode or MF mode (step S2) of the imaging mode (step S1) by way of the operation unit9, the CPU11controls the motion of the mechanical shutter18via the lens driver25and drives the image pickup device23via the image pickup device driver27(step S3). The operation of the digital camera2in the case where the AF mode is set has been well known, a specific description thereof being omitted.

When the MF mode (step S2) is set, the output signals from the third pixels35,36, and37(normal pixel) of the image pickup device23are input to the normal processing unit52of the image processing circuit29. The normal processing unit52subjects the third image58cas the output signals from the third pixels35to37to the image processing to store as the full color picked-up image55in the VRAM area of the memory13(step S4).

The selection unit102extracts the first division image61L and the second division image61R used for the focusing-verification image from the first image58aand the second image58b, respectively. The difference-emphasis processing unit106uses the first division image61L and the second division image61R to perform the difference-emphasis processing emphasizing a difference between the pixel-values. The difference-emphasis processing unit106specifically enlarges the difference between the first pixel36aand the second pixel36bwhich correspond to each other in a direction perpendicular to the split line63with the split line63of the split image61(focusing-verification image) being uses as a reference. The image generation unit104generates the split image61including the monochrome first division image61L and the monochrome second division image61R (step S5). The difference-emphasis processing for the first division image61L and the second division image61R is performed at step S5. The generated split image61is stored in the VRAM area of the memory13.

The display controller33reads out the picked-up image55and the split image61from the memory13and composites the split image61into the displayed area of the picked-up image55to be output to the display unit8thereafter. This allows to a live view image including the monochrome split image61be displayed in the full-color picked-up image55(step S6).

Since the first image66L and second image66R of the split image61is shifted in the right and left direction in the figure depending on the focusing state of the focus lens16, the user rotationally manipulates the focus ring3ato move the focus lens16along the optical axis direction. As the focus lens16comes closer to the focusing position where the subject is focused on, the displacement amount between the first image66L and the second image66R becomes smaller. This allows the user to carry out the focus adjustment while verifying the live view image.

When the focus lens16is set to the focusing position, the displacement amount between the first image66L and the second image66R becomes zero, as shown inFIG. 9. This causes the focus lens16to focus on the subject, completing the focus adjustment. Hereinafter, the above processing is repeatedly performed until the shutter button6is pressed down.

Whether or not an imaging instruction is input is determined by way of pressing down the shutter button6(Yes at step S7), when the imaging instruction is input (Yes at step S7), the normal processing unit52generates the picked-up image55of one frame to transiently store in the VRAM area of the memory13. This picked-up image55is compressed by the compression-decompression processing circuit31, and thereafter, recorded in memory card10via the media I/F32(step S8). Hereinafter, whether or not imaging is ended is determined by way of completion of the MF mode (step S9), and the above processing is repeatedly performed until the MF mode is completed.

Hereinafter, a description is given in detail of the difference-emphasis for the first division image61L and the second division image61R in the split image61(focusing-verification image) using separately various embodiments.

<Digital Camera in First Embodiment>

In the digital camera2in a first embodiment, the difference-emphasis processing unit (106inFIG. 6) performs the difference-emphasis processing that enlarges the difference (|L−R|) between a pixel-value L of the first pixel36aand a pixel-value R of the second pixel36bfor each pair pixel (PR inFIG. 8) with the split line (63inFIG. 9andFIG. 10) being interposed therebetween, as shown by arrows inFIG. 12. This emphasizes the difference of the pixel-value between the first division image61L on the basis of the pixel-values L of a plurality of the first pixels36aof the image pickup device23and the second division image61R on the basis of the pixel-values R of a plurality of the second pixels36bof the image pickup device23.

InFIG. 12, L represents the pixel-value of the first pixel36abefore the difference-emphasis processing, R represents the pixel-value of the second pixel36bbefore the difference-emphasis processing, ave represents an average value ((L+R)/2) between the pixel-value L of the first pixel36aand the pixel-value R of the second pixel36bbefore the difference-emphasis processing, L′ represents the pixel-value of the first pixel36aafter the difference-emphasis processing, and R′ represents the pixel-value of the second pixel36bafter the difference-emphasis processing.

First, the difference-emphasis processing unit106in this example calculates the average value ave (=(L+R)/2) between the pixel-value L of the first pixel36aand the pixel-value R of the second pixel36b. Next, the difference-emphasis processing unit106in this example enlarges a difference (|L−ave|) between the pixel-value L of the first pixel36aand the average value ave, with the average value ave being used as a reference, and enlarges a difference (|R−ave|) between the pixel-value R of the second pixel36band the average value ave. For example, the difference-emphasis is performed as shown in a formula below.
L′=L+(L−ave)×K
R′=R+(R−ave)×K[Formula 1]

Here, K represents a difference-emphasis coefficient. For the pair pixel undergoing the difference-emphasis, K>0, and for the pair pixel not undergoing the difference-emphasis, K=0.

The difference-emphasis processing unit106in this example ensmalls the difference-emphasis coefficient K as the difference between the pixel-value L of the first pixel36aand the pixel-value R of the second pixel36bis larger.

<Working Effect of Digital Camera in First Embodiment>

If the split image61in non-focusing show in (A) portion ofFIG. 13is subjected to the difference-emphasis processing, a split image61′ shown in (B) portion ofFIG. 13is obtained.

For the purpose of easy understanding of the present invention, split images61aand61bare shown which are obtained in a case where a subject having uniform color in a y direction is picked up. Therefore, each of the first division image61L and the second division image61R constituting the split images61aand61bhas the pixel-value constant in the direction y perpendicular to the split line63(up-and-down direction).

When comparing the split image61in (A) portion ofFIG. 13before the difference-emphasis processing and the split image61′ in (B) portion ofFIG. 13after the difference-emphasis processing like these, it can be found that, in an area66where the pixel-values are different between the first division image61L and the second division mage61R, the difference of the pixel-value between the first division image61L and the second division image61R is enlarged and emphasized, which results in that a parallax d (phase difference) in an x direction (right and left direction) between a the first division image61L and the second division image61R has become easier to visually recognize.

The average value ave between the pixel-value L of the pixel in the first division image61L (first pixel) and the pixel-value R of the pixel in the second division image61R (the second pixel) is calculated, the pixels opposite to each other with the split line63being interposed therebetween, and this average value ave is used as a reference to enlarge a difference between the pixel-value L of the pixel in the first division image61L (first pixel) and the average value ave, and a difference between the pixel-value R of the pixel in the second division image61R (the second pixel) and the average value ave, which scarcely changes an image configuration in the first division image61L and the second division image61R in the case where the difference of the pixel-value is emphasized.

Therefore, according to the digital camera in the embodiment, the displacement amount in the focusing-verification image (split image) becomes easy to visually recognize.

<Digital Camera in Second Embodiment>

A description is given of the digital camera2in a second embodiment.

The image processing circuit (29inFIG. 6) is as already described, and the difference-emphasis processing unit (106inFIG. 6) included in the image processing circuit29is in common with the first embodiment in that, as the difference-emphasis processing, concerning the first pixel59aand the second pixel59bcorresponding to each other in the up-and-down direction y with the split line63being interposed therebetween, the difference |L−R| between the pixel-value L of the first pixel59aand the pixel-value R of the second pixel59bis enlarged. Hereinafter, a description is given of a point of the difference-emphasis processing unit106in the embodiment which is different from the first embodiment.

The CPU11in the embodiment (calculation unit) calculates a parallax between the pixel (first pixel59a) in the first division image61L as the displayed portion of the first image58aand the pixel (second pixel59b) in the second division image61R as the displayed portion of the second image58b. Here, the parallax indicates a blur amount of the third image58c(normal image), and corresponds to the displacement amount between the first division image61L and the second division image61R in the split image61in the right and left direction x (horizontal direction along the split line63).

The difference-emphasis processing unit106of the image processing circuit29in the embodiment determines whether a magnitude of the parallax (blur amount) between the first division image61L and the second division image61R is large or small on the basis of the parallax calculated by the CPU11in the split image61to set an increase amount for the difference of the pixel-value |L−R| in the case where the parallax is determined to be larger to be smaller than an increase amount for the difference of the pixel-value |L−R| in the case where the parallax is determined to be small.

In (A) portion ofFIG. 14, the split image61before the difference-emphasis processing is shown in the case where the parallax d (displacement amount between the first division image61L and the second division image61R in the horizontal direction x along the split line63) is large. In the case of the large parallax d like this, if the difference-emphasis coefficient the same as in the case of the small parallax d is used to perform the difference-emphasis processing, a portion having the parallax d is made into fully white and black portions with no shading gradation, like the split image61′ after the difference-emphasis processing as shown in (B) portion ofFIG. 14, which makes is difficult for the user to visually recognize what subject is the portion having the parallax, even if he/she can verify the magnitude of the parallax. In a portion having sufficient parallax, the parallax is naturally easy to visually recognize with no difference-emphasis being performed, but if the difference-emphasis is performed, the shading gradation disappears to make it difficult to visually recognize what the subject is.

Therefore, the difference-emphasis processing unit106in the embodiment determines whether the parallax d between the first division image61L and the second division image61R in the split image61(focusing-verification image) is large or small to switch the difference-emphasis coefficient K depending on the parallax d such that the increase amount for the difference of the pixel-value |L−R| in the case where the parallax d is determined to be large is set to be smaller than the increase amount for the difference of the pixel-value |L−R| in the case when parallax d determined to be small.

FIG. 15is a characteristic graph showing parallax emphasis in the case where the parallax d is small andFIG. 16a characteristic graph showing parallax emphasis in the case where the parallax d is large. In these figures, an abscissa axis represents a position of a pixel in the horizontal direction (right and left direction x) along the split line63and an ordinate axis represents a pixel-value.

A specific example for easily determining whether the parallax is large or small and for appropriately performing the difference-emphasis may include a following method.

First, an area in the split image61having the difference of the pixel-value |L−R| in the up-and-down direction y which is larger than a threshold Tp is detected, a width of the detected area in the right and left direction x (direction along the split line) is detected, and the difference-emphasis coefficient K is changed over according to a detection result. For example, the larger the detected width, the smaller the difference-emphasis coefficient K is made. Here, the threshold Tp≧0, and in a case of Tp=0, the width=parallax d. However, the width to be determined may be limited with Tp>0 to avoid error determination and reduce the processing load. In this way, the width found with Tp>0 (a value not the same as the parallax d but corresponding to the parallax d) is also referred to as the “parallax” herein.

Secondly, it is determined how many pixels (the first pixel59aor the second pixel59b) continuously exist in which the difference |L−R| between the pixel-value of the first pixel59aand the pixel-value of the second pixel59bcorresponding to the first pixel59ais larger than the threshold, and the difference-emphasis coefficient K is changed over according to a result of the determination. For example, the larger the pixel-number of pixels in which the difference |L−R| exceeds the threshold, the smaller the difference-emphasis coefficient K is made. As for the pixel-number, the number only of pixels (the first pixel59aor the second pixel59b) continuously larger than the threshold Tp in the right and left direction x is detected to avoid the error detection and reduce the processing load.

FIG. 17is a flowchart showing a flow of a main part of an exemplary difference-emphasis processing in the embodiment. This processing is executed by the difference-emphasis processing unit106according to a program.

This processing changes over the difference-emphasis coefficient K by the above second method (detecting the pixel-number per unit of length).

Firstly, the difference-emphasis processing unit106, as shown inFIG. 18, determines whether or not 5 or more pixels, in which the difference of the pixel-value satisfies |L−R|>40 (the first pixel59aor the second pixel59b), continuously exist in a range of a near pixel row including a target pixel PL(x) in the right and left direction x (i.e., PL(x−3), PL(x−2), PL(x−1), PL(x), PL(x+1), PL(x+2), and PL(x+3), added up to a total of 7 pixels, in this example) (step S22), and sets the difference-emphasis coefficient K to “0.5” if 5 or more pixels continuously exist (step S24). For example, in a case where the difference of the pixel-value |L−R|>40 is satisfied in 5 pixels of PL(x−2) to PL(x+2) continuous in the right and left direction x inFIG. 18, the difference-emphasis coefficient K is set to “0.5” which is the minimum value in this example.

Additionally, the difference-emphasis processing unit106determines whether or not 5 or more pixels, in which the difference of the pixel-value satisfies |L−R|>20, continuously exist in a range of a near pixel row including the target pixel PL(x) in the right and left direction x (i.e., PL(x−3) to PL(x+3), added up to a total of 7 pixels) (step S26), and sets the difference-emphasis coefficient K to “1” if 5 or more pixels continuously exist (step S28). For example, in a case where the difference of the pixel-value |L−R|>20 is satisfied in 5 pixels of PL(x−2) to PL(x+2) continuous in the right and left direction x inFIG. 18, the difference-emphasis coefficient K is set to “1”.

In other words, the difference-emphasis processing unit106in this example determines how many pixels continuously exist in which the difference of the pixel-value |L−R| is larger than a first threshold for each area of a certain width (near ±3 pixels including the target pixel, in this example) in the split image61along the split line63, compares the pixel-number of the pixels with a second threshold, and sets, in a case where the pixel-number in the area of a certain width is determined to be larger, the increase amount for the difference of the pixel-value |L−R| to be smaller than that in a case where the pixel-number in the area of a certain width is determined to be smaller. In this example, it is determined how many pixels continuously exist in which the difference of the pixel-value |L−R| is larger than the first threshold, and the more the continuous pixels, the smaller the increase amount for the difference of the pixel-value is made.

In a case where existence of 5 or more pixels continuous in the near pixel row in which |L−R| is larger than the threshold cannot be detected (No at step S22and No at step S26), the difference-emphasis processing unit106sets the difference-emphasis coefficient K to “5” (step S30).

Note that the threshold (“40”, 20”) for |L−R| and the threshold (5 pixels) for the pixel-number described above are examples, and not limited to these thresholds.

<Working Effect of Digital Camera in Second Embodiment>

According to the digital camera2including the difference-emphasis processing unit106in the embodiment, it is determined whether the magnitude of the parallax between the pixel in the first division image61L and the pixel in the second division image61R is large or small, the pixels corresponding to each other in the split image61(focusing-verification image) with the split line63(border line) being used as a reference, and the increase amount for the difference of the pixel-value |L−R| in a case where the parallax is determined to be large is set to be smaller than the increase amount for the difference of the pixel-value |L−R| in a case where the parallax is determined to be small, which allows the difference only in a portion having a fine parallax to be emphasized such that a portion having a parallax in the split image61is made to be easily seen.

<Digital Camera in Third Embodiment>

A description is given of the digital camera2in a third embodiment.

the image processing circuit (29inFIG. 6) and the difference-emphasis processing unit106included in the image processing circuit29are as already described, and are in common with the first embodiment in that, as the difference-emphasis processing, concerning the first pixel59aand the second pixel59bcorresponding to each other in the up-and-down direction y (direction perpendicular to the split line63) with the split line63being interposed therebetween, the difference |L−R| between the pixel-value L of the first pixel59aand the pixel-value R of the second pixel59bis enlarged. Hereinafter, a description is given of a point of the difference-emphasis processing unit106in the embodiment which is different from the first embodiment.

The difference-emphasis processing unit106in the embodiment determines whether the difference |L−R| between the pixel-value L of the first pixel59aand the pixel-value R of the second pixel59bis large or small, and sets the increase amount for the difference of the pixel-value |L−R| in a case where the difference |L−R| is determined to be large to be smaller than the increase amount for the difference of the pixel-value |L−R| in a case where the difference |L−R| is determined to be small.

FIG. 19is a characteristic graph showing the difference-emphasis in a case where the difference |L−R| is determined to be large (in the high contrast condition), andFIG. 20a characteristic graph showing the difference-emphasis in a case where the difference |L−R| is determined to be small (in the low contrast condition).

The difference-emphasis processing unit106in the embodiment weakens the difference-emphasis in the high contrast condition shown inFIG. 19and strengthens the difference-emphasis in the low contrast condition shown inFIG. 20.

The difference-emphasis processing unit106changes over a magnitude of the difference-emphasis coefficient K on the basis of the difference |L−R| between the pixel-value L of the first pixel36aand the pixel-value R of the second pixel36bso as to change the increase amount for the difference |L−R| between the pixel-values L and R. In other words, the difference-emphasis coefficient K is made smaller in the high contrast condition when the difference of the pixel-value |L−R| is large than in the low contrast condition when the difference of the pixel-value |L−R| is small.

A value of difference-emphasis coefficient K may be found by means of a function using the difference of the pixel-value |L−R| as a parameter. For example, such function is suitable in which the larger the difference of the pixel-value, the smaller the value of K and the smaller a change amount for the value of K, and K=5×exp (0.01×(−|L(x)−R(x)|)) can be applied, for example. Here, x represents a position in the split image61in the right and left direction (e.g., direction along the split line63).

<Working Effect of Digital Camera in Third Embodiment>

According to the digital camera2including the difference-emphasis processing unit106in the embodiment, the difference-emphasis coefficient K is set to be smaller in the high contrast condition when the difference of the pixel-value |L−R| between the first pixel59aand the second pixel59bis large, the pixels opposite to each other with the split line63being interposed therebetween than in the low contrast condition when the difference of the pixel-value |L−R| between the first pixel59aand the second pixel59bis small, which can avoid excessive difference-emphasis in the high contrast condition and carry out sufficient difference-emphasis in the low contrast condition.

<Digital Camera in Fourth Embodiment>

A description is given of the digital camera2in a fourth embodiment.

The image processing circuit (29inFIG. 6) and the difference-emphasis processing unit106included in the image processing circuit29are as already described, and hereinafter, a description is given of a point of the difference-emphasis processing unit106in the embodiment which is different from the first embodiment.

The difference-emphasis processing unit106in the embodiment uses a shading correction coefficient for correcting shading due to the pupil-division by the image pickup device23to perform the difference-emphasis processing so as to prevent a shading component due to the pupil-division from being emphasized.

In (A) portion ofFIG. 21, shown are shading characteristics L(x) caused in the first division image61L and shading characteristics R(x) caused in the second division image61R due to the pupil-division by the image pickup device23. Here, x represents a position in the right and left direction x. These characteristics L(x) and R(x) are obtained by picking up with uniform luminance. As shown (A) portion of inFIG. 21, shadings are generated which are different between the first division image61L and the second division image61R, and if an image including such a shading component is subjected to the difference-emphasis, the shading component is emphasized as shown in (B) portion ofFIG. 21.

Therefore, the difference-emphasis processing unit106in the embodiment uses the shading correction coefficient such that the shading component is not included in a difference-emphasis component (increase in |L−R|).

The shading correction coefficient are coefficients α(x) and β(x) from which the shading is deleted as shown inFIG. 22by carrying out an arithmetic with respect to the shading characteristics L(x) and R(x) shown in (A) portion ofFIG. 21. Here, x represents a position in a right and left direction x.

The difference-emphasis processing unit106specifically sets the shading correction coefficient with respect to the first division image61L to α(x), the shading correction coefficient with respect to the second division image61R to β(x), an average value obtained by carrying out an arithmetic with respect to the shading correction coefficient to ave(x)=(α(x)×L(x)+β(x)×R(x))/2, and the difference-emphasis coefficient to K(x) (however, when K(x)>0), the pixel-value of the first pixel36aa′(x) and pixel-value of the second pixel36bb′(x) after the difference-emphasis processing are L′(x)=L(x)+(α(x)×L(x)−ave)×K and R′(x)=R(x)+(β(x)×R(x)−ave(x))×K(x), respectively).

<Working Effect of Digital Camera in Fourth Embodiment>

In a case where the difference-emphasis is not carried out by use of the shading correction coefficient, that is, the difference-emphasis is carried out by way of arithmetics L′(x)=L(x)+(L(x)−ave)×K and R′(x)=R(x)+(R(x)−ave(x))×K(x), the shading component shown in (A) portion ofFIG. 21is emphasized as shown in (B) portion ofFIG. 21.

In contrast, in a case where the difference-emphasis is carried out by use of the shading correction coefficient similarly to the embodiment, that is, the arithmetics L′(x)=L(x)+(α(x)×L(x)−ave)×K and R′(x)=R(x)+(β(x)×R(x)−ave(x))×K(x) are carried out, the shading component shown in (A) portion ofFIG. 21is not emphasized and is remained with no change and the difference-emphasis can be carried out.

<Variation of Target Area Subjected to Difference-Emphasis>

In the first embodiment to fourth embodiment above, for the purpose of easy understanding of the invention, the case is described as an example where the difference-emphasis processing is performed to overall the first division image61L and second division image61R in the split image61, but as shown inFIG. 23the difference-emphasis processing may be performed to only an area67in the vicinity of the split line63(border line) of the first division image61L and the second division image61R.

Here, the “area in the vicinity” of the split line63is an area of the pixel-number within ±10% from the division border line (split line63) with respect to the number of all pixels in a division direction of the split image61(focusing-verification image) (y direction perpendicular to the split line63), for example.

In the first embodiment to third embodiment, the case is described as an example where the number of the split line63is one, but a plurality of split lines63may be provided in the split image61and the difference-emphasis processing may be performed on only an area in the vicinity of each of such a plurality of split lines63.

The split line63shaped in a lattice may be provided in the split image61and the difference-emphasis processing may be performed on only an area in the vicinity of such a split line63shaped in a lattice.

<Variation of Pixel Array of Image Pickup Device>

[Basic Array Pattern of Non Bayer Array]

The pixel array of the image pickup device23(color filter array) in each embodiment above includes a basic array pattern P corresponding to 6×6 pixels which are repeatedly arranged in the horizontal and vertical directions, but may include a basic array pattern of an array pattern corresponding to N×N pixels (N is three or more).

A color of the filter is not limited to RGB three primary colors. For example, the color filter array of color filters of four colors RGB three primary colors+another color (for example, emerald (E)) may be used. The color filter array of color filters of C (cyan), M (magenta), and Y (yellow) as complementary colors of the primary colors RGB.

The pixel array of the image pickup device23(color filter array) may be a pixel array of Bayer array. The image pickup device23in the example has a plurality of the first phase difference pixel (first pixels) and a plurality of the second phase difference pixels (second pixels) arranged in a part of the pixel array constituted by Bayer array.

The pixel array of the image pickup device23(color filter array) may be a pixel array constituted by two planes of array in which the pixels of the same color are arranged to be displaced. The image pickup device23in the example has a plurality of the first phase difference pixel (first pixel) and a plurality of the second phase difference pixel (second pixel) arranged, and has the pair pixel of the first phase difference pixel and the second phase difference pixel to be adjacent to each other, in a part of the pixel array constituted by two planes of array in which the pixel of the same color arranged to be displaced.

In each embodiment above, a description is given using the digital camera as the imaging device according to the invention, but the invention may be applied to a mobile phone having a camera function, smartphone, PDA (Personal Digital Assistants), and portable game console, for example. Hereinafter, a description is given in detail using the smartphone as an example with reference to the drawings.

FIG. 24shows an outer appearance of a smartphone500. The smartphone500shown inFIG. 24having a housing502shaped in a flat plate includes on one face of the housing502a display and input unit520(also referred to as “touch panel type display unit”) in which a display panel521and an operation panel522(touch panel) as an input unit are integrated. The housing502includes a speaker531, microphone532, operation unit540, and camera unit541. A configuration of the housing502is not limited thereto, and a configuration in which the display unit and the input unit are independent of each other, and a configuration having a clamshell structure or a slide mechanism may be used, for example.

FIG. 25is a block diagram showing the configuration of the smartphone500shown inFIG. 24. As shown inFIG. 25, included are as main components of the smartphone a radio communication unit510, display and input unit520, telephoning unit530, operation unit540, camera unit541, storage unit550, external input/output unit560, GPS (Global Positioning System) reception unit570, motion sensor unit580, power supply unit590, and main controller501. The smartphone500has, as a main function, a radio communication function for carrying out mobile radio communication with a base station device BS via a mobile communication network NW.

The radio communication unit510carries out radio communication with the base station device BS included in the mobile communication network NW according to an instruction from the main controller501. This radio communication is used to transmit and receive various pieces of file data such as audio data, image data and the like, and e-mail data and the like and receive Web data, streaming data and the like.

The display and input unit520is a so-called touch panel which, by way of control by the main controller501, displays and visually delivers to the user an image (still image and moving image) and text information, and detects a user's operation to the displayed information, and includes the display panel521and the operation panel522. In a case where a generated 3D image is viewed, the display panel521is preferably a 3D display panel.

The display panel521uses a LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display) and the like as a display device. The operation panel522placed such that an image displayed on a display surface of the display panel521can be visually recognized is a device for detecting one or more coordinates operated by a user's finger or a stylus. If this device is operated by a user's finger or a stylus, a detection signal generated due to the operation is output to the main controller501. Subsequently, the main controller501detects an operated position (coordinates) on the display panel521on the basis of the received detection signal.

As shown inFIG. 24, the display panel521and operation panel522in the smartphone500are integrated to constitute the display and input unit520, and the operation panel522is arranged in a manner to fully cover the display panel521. In a case of using this arrangement, the operation panel522may have a function to detect the user's operation on also an area outside the display panel521. In other words, the operation panel522may have a detection area for an overlapping portion overlapped with the display panel521(hereinafter, referred to as a displayed area) and a detection area for a peripheral portion not overlapped with the display panel521other than the overlapping portion (hereinafter, referred to as a non-displayed area).

Note that a size of the displayed area and a size of the display panel521may completely match each other, but both sizes may not necessarily match. The operation panel522may have two sensitive areas of the peripheral portion and an inside portion other than that. Further, a width of the peripheral portion is appropriately designed depending on a size of the housing502and the like. A position detection method used for the operation panel522includes a matrix switch method, resistance film method, surface acoustic wave method, infrared ray method, electromagnetic induction method, electrostatic capacitance method and the like, any method of which may be used.

The telephoning unit530having the speaker531and the microphone532converts user voice input through the microphone532into the audio data processable by the main controller501to output to the main controller501, and decodes the audio data received by the radio communication unit510or the external input/output unit560to output from the speaker531. As shown inFIG. 24, for example, the speaker531and the microphone532are mounted on a face the same as a face provided with the display and input unit520. The microphone532may be mounted also on a lateral face of the housing502.

The operation unit540which is a hardware key using a key switch and the like accept an instruction from the user. For example, as shown inFIG. 24, the operation unit540is mounted on a lower portion of the display unit of the housing502of the smartphone500, lower side face, and is a press-button type switch which is turned on when pressed down by a finger or the like and is brought into a turned-off state by a restoring force of a spring or the like when the finger is released.

The storage unit550stores a control program and control data for the main controller501, application software including an image processing program for generating a left-eye image and a right-eye image according to the invention, the first and second digital filter group used for generating stereoscopic image, parallax map, address data having a name, telephone number and the like of the telephoning other end associated with each other, data of transmitted and received e-mail, Web data downloaded by way of Web browsing, and downloaded content data, and transiently stores streaming data or the like. The storage unit550includes an internal storage unit551built in the smartphone and an external storage unit552having a detachable external memory slot. Each of the internal storage unit551and the external storage unit552included in the storage unit550is achieved by use of a storage medium such as a flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., Micro SD (trademark) memory or the like), RAM (Random Access Memory), and ROM (Read Only Memory).

The external input/output unit560serves as an interface with all external devices coupled to the smartphone500to allow other external devices to be directly or indirectly connected via a communication or the like (e.g., USB (Universal Serial Bus), IEEE1394, etc.) or network (e.g., Internet, wireless LAN, Bluetooth (trademark), RFID (Radio Frequency Identification), Infrared Data Association: IrDA) (trademark), UWB (Ultra Wideband) (trademark), ZigBee (trademark), etc.).

Examples of the external device coupled to the smartphone500include, for example, a wired/wireless head set, wired/wireless external charger, wired/wireless data port, memory card or SIM (Subscriber Identity Module Card)/UIM (User Identity Module Card) card connected via a card socket, external audio and video device connected via an audio and video I/O(Input/Output) terminal, external audio and video device wirelessly connected, smartphone via a wired/wireless connection, personal computer via a wired/wireless connection, PDA via a wired/wireless connection, personal computer via a wired/wireless connection, earphone, and the like. The external input/output unit can deliver data received by way of transmission from the external device above to the respective components in the smartphone500and transmit the data in the smartphone500to the external devices.

The GPS reception unit570receives GPS signals transmitted from GPS satellites ST1to STn to perform positioning arithmetic processing on the basis of the received plural GPS signals according to an instruction from the main controller501, and detects a position including latitude, longitude, and altitude of the smartphone500. When positional information can be acquired from the radio communication unit510or the external input/output unit560(e.g., wireless LAN), the GPS reception unit570may use the positional information to detect the position. The motion sensor unit580which includes, for example, a triaxial acceleration sensor or the like detects physical motion of the smartphone500according to an instruction from the main controller501. Detection of the physical motion of the smartphone500allows a direction or acceleration of motion of the smartphone500to be detected. A result of this detection is to be output to the main controller501.

The power supply unit590supplies electrical power stored in a battery (not shown) to each part of the smartphone500according to an instruction from the main controller501.

The main controller501which includes a microprocessor operates according to the control program or control data stored in the storage unit550and collectively controls the respective parts of the smartphone500. The main controller501has a mobile communication controlling function to control each part in a communication system and an application processing function for performing audio communication or data communication via the radio communication unit510.

The application processing function is attained by the main controller501operating according to the application software stored by the storage unit550. Examples of the application processing function include, for example, an IrDA function to control the external input/output unit560to perform the data communication with an opposite device, e-mail function to transmit and receive an e-mail, Web browsing function to view a Web page, function to generate a 3D image from a 2D image according to the invention, and the like.

The main controller501has an image processing function to display a video on the display and input unit520on the basis of the image data such as the received data or the downloaded streaming data (data of still image and moving image). The image processing function refers to a function that the main controller501decodes the above image data and subjects a result of this decoding to the image processing to display the image on the display and input unit520.

Further, the main controller501executes display control of the display panel521and operation detecting control to detect the user's operation via the operation unit540and the operation panel522.

The main controller501executes the display control to display an icon for starting the application software or a software key such as a scroll bar, or display a window for creating an e-mail. Note the scroll bar refers to a software key for accepting an instruction to move a displayed portion of an image such as a large image not entirely accommodated within a displayed area of the display panel521.

The main controller501executes the operation detecting control to detect the user's operation input via the operation unit540, accepts via the operation panel522an operation on the above icon or input of a character string to an input field in the above window, or accepts a request input via the scroll bar for scrolling of the displayed image.

Further, the main controller501has a touch panel controlling function to execute the operation detecting control to determine whether an operated position on the operation panel522is the overlapping portion (displayed area) overlapped with the display panel521or the peripheral portion (non-displayed area) not overlapped with the display panel521other than the overlapping portion, and control the sensitive area of the operation panel522or a displayed position of the software key.

The main controller501can also detect a gesture operation on the operation panel522and executed a predetermined function depending on the detected gesture operation. The gesture operation means not a simple touch operation of related art, but an operation including tracking by a finger or the like, simultaneously specifying a plurality of positions, or combining these to track from at least one of a plurality of positions.

The camera unit (imaging device)541is a digital camera electronically imaging by use of the image pickup device such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device), and has a configuration the basically same as the digital camera according to the above embodiments.

The camera unit541can under the control of the main controller501converts the image data obtained by picking-up into a compressed image data such as JPEG (Joint Photographic coding Experts Group), for example, to store in the storage unit550and output via the external input/output unit560or the radio communication unit510. In the smartphone500shown inFIG. 24, the camera unit541is mounted on the same face as the display and input unit520, but, a mounted position of the camera unit541being not limited thereto, may be mounted on a rear face of the display and input unit520, or a plurality of camera units541may be mounted. In the case where a plurality of camera units541are mounted, the camera unit541for imaging may be changed over for singularly imaging or a plurality of camera units541may be simultaneously used for imaging.

The camera unit541can be used for the various functions of the smartphone500. For example, an image obtained by the camera unit541may be displayed on the display panel521, or an image by the camera unit541may be used as one of operation input on the operation panel522. When the GPS reception unit570detects a position, the position can be detected by referring an image from the camera unit541. Further, by referring an image from the camera unit541, without using the triaxial acceleration sensor or in combination with the triaxial acceleration sensor, an optical axis direction of the camera unit541of the smartphone500can be determined, and also a current usage environment can be determined. Of course, an image from the camera unit541may be used in the application software.

Besides, the image data of a still image or moving image may be added with the positional information obtained by the GPS reception unit570, voice information obtained by the microphone532(which may be voice-text converted by the main controller into text information), attitude information obtained by the motion sensor unit580and the like to be stored in the storage unit550and be output via the external input/output unit560or the radio communication unit510.

The smartphone500shown inFIG. 24andFIG. 25has the function similar to the digital camera2described above. The main controller501inFIG. 25has the function of the image processing circuit29shown inFIG. 6orFIG. 22. The display and input unit520(touch panel type display unit) includes the “display unit”, “position input part”, and “number input part” according to the invention.

The smartphone500in this example accepts a drag operation for dragging the split line63shown inFIG. 10by the display and input unit520. When the drag operation dragging the split line63(border line) is performed on the display and input unit520with the split image61(focusing-verification image) being displayed on the display and input unit520, the main controller501changes the position of the split line63in the split image61in concert with the drag operation.

Hereinabove, for the purpose of easy understanding of the present invention, various embodiments separately described, but various embodiment may be adequately combined and carried out.

Note that the present invention is not limited to the examples described herein and the examples shown in the drawings, and various design changes and modifications may be made of course within a scope not departing from a gist of the present invention.