Patent Description:
Conventionally, there is known an imaging apparatus that performs a focus detection of an imaging lens by a phase difference detection method using a two-dimensional image sensor in which a micro lens is formed on each pixel.

Patent Document <NUM> discloses an imaging apparatus in which a pair of focus detection pixels are disposed on part of a two-dimensional image sensor including a plurality of pixels. The pair of focus detection pixels are configured to receive light from different areas on an exit pupil of an imaging lens by a light shielding layer having an opening, and provide a pupil division. An imaging signal is acquired by imaging pixels that are disposed on most part of the two-dimensional image sensor, and an image shift amount is obtained from focus detection signals from the partially disposed focus detection pixels to perform the focus detection.

Further cited prior art documents are <CIT> showing a depth detection apparatus, imaging apparatus and depth detection method, <CIT> showing a focus adjustment device, method for controlling the same, and image capture apparatus, <CIT> showing a control apparatus, image pickup apparatus, control method, and non-transitory computer-readable storage medium for performing focus detection, <CIT> A showing an imaging apparatus capable of adjusting a focus position and <CIT> showing focus detection apparatus and method, and image capturing apparatus.

However, in the imaging apparatus disclosed in Patent Document <NUM>, each partial pupil area divided by a micro lens has a different shape, and thus a shape difference occurs between the focus detection signals. As a result, an effective baseline length for each spatial frequency band changes, and the focus detection accuracy lowers.

With the foregoing problems in mind, it is an object of the present invention to provide a focus detection apparatus, a focus detection method, and a focus detection program, each of which can correct a focus detection error caused by a shape difference between the focus detection signals, and perform a focus detection with high accuracy.

It is provided a focus detection apparatus, a focus detection method, and a focus detection program according to the respective claims.

The present invention can provide a focus detection apparatus, a focus detection method, and a focus detection program, each of which can correct a focus detection error caused by a shape difference between the focus detection signals, and perform a focus detection with high accuracy.

Referring now to the accompanying drawings, a description will be given of embodiments according to the present invention. Corresponding elements in respective figures are designated by the same reference numerals, and a description thereof will be omitted.

While this embodiment describes that the present invention is applied to an imaging apparatus such as a digital camera, the present invention is broadly applicable to an apparatus different from an imaging apparatus such as a focus detection apparatus, an information processing apparatus, and an electronic apparatus.

<FIG> is a block diagram showing the configuration of an imaging apparatus <NUM> having a focus detection apparatus according to this embodiment. The imaging apparatus <NUM> is a digital camera system including a camera body and an interchangeable lens (imaging optical system or image capturing optical system) that is detachably attached to the camera body. However, the present invention is not limited to this embodiment, and is also applicable to an imaging apparatus in which a camera body and a lens are integrated with each other.

The imaging optical system (image capturing optical system) generates an object image (optical image) of an object. A first lens unit <NUM> is disposed on the frontmost side (object side) among the plurality of lens units constituting the imaging optical system, and held by the lens barrel so as to move forward and backward along the optical axis OA. A diaphragm/shutter (diaphragm) <NUM> adjusts a light amount in the imaging by adjusting an aperture diameter, and serves as an exposure time adjusting shutter in still image capturing. A second lens unit <NUM> moves forward and backward along the optical axis OA integrally with the aperture/shutter <NUM>, and has a zoom function that performs a zooming operation in association with the moving forward and backward operation of the first lens unit <NUM>. A third lens unit <NUM> is a focus lens unit that performs focusing (focus operation) by moving forward and backward along the optical axis OA. An optical low-pass filter <NUM> is an optical element that reduces a false color and moiré in a captured image.

An image sensor <NUM> includes, for example, a CMOS sensor or a CCD sensor, and a peripheral circuit thereof, and performs a photoelectric conversion of the object image. The image sensor <NUM> uses, for example, a two-dimensional single-plate color sensor in which on-chip primary color mosaic filters are formed in Bayer array on a light receiving pixel having m pixels in the horizontal direction and n pixels in the vertical direction.

In a zooming operation, a zoom actuator <NUM> moves (drives) a cam cylinder (not shown) to move the first lens unit <NUM> and the second lens unit <NUM> along the optical axis OA. An diaphragm/shutter actuator <NUM> adjusts the aperture diameter of the aperture/shutter <NUM> in adjusting the light amount (imaging light amount). A focus actuator <NUM> moves the third lens unit <NUM> along the optical axis OA during focusing.

An electronic flash <NUM> is an illumination apparatus used to illuminate an object. The electronic flash <NUM> uses a flash illumination apparatus with a xenon tube or an illumination apparatus with a continuously emitting LED (Light Emitting Diode). An AF auxiliary light source <NUM> projects an image of a mask having a predetermined aperture pattern onto an object via a projection lens. This configuration can improve the focus detecting capability for a dark object or a low-contrast object.

A CPU <NUM> is a control apparatus (control means) that governs a variety of controls of the imaging apparatus <NUM>. The CPU <NUM> includes a calculator, a ROM, a RAM, an A/D converter, a D/A converter, a communication interface circuit, and the like. The CPU <NUM> reads out and executes a predetermined program stored in the ROM to drive a variety of circuits of the imaging apparatus <NUM> and controls a series of operations such as a focus detection (AF), imaging, image processing, and recording.

The CPU <NUM> further includes a pixel signal acquisition means (acquisition means) 121a, a signal generation means 121b, a focus detection means 121c, and a lens information acquisition means 121d.

The electronic flash control circuit <NUM> performs a lighting control of the electronic flash <NUM> in synchronization with the imaging operation. The auxiliary light source drive circuit <NUM> performs a lighting control of the AF auxiliary light source <NUM> in synchronization with the focus detection operation. The image sensor drive circuit <NUM> controls the imaging operation of the image sensor <NUM>, A/D-converts the acquired image signal, and transmits it to the CPU <NUM>. The image processing circuit (image processing apparatus) <NUM> performs processing such as a gamma conversion, a color interpolation, or a JPEG (Joint Photographic Experts Group) compression, for image data output from the image sensor <NUM>.

A focus drive circuit <NUM> drives the focus actuator <NUM> based on the focus detection result, and performs focusing by moving the third lens unit <NUM> along the optical axis OA. A diaphragm/shutter drive circuit <NUM> drives the diaphragm/shutter actuator <NUM> to control the aperture diameter of the diaphragm/shutter <NUM> and also controls the exposure time in still image capturing. A zoom drive circuit <NUM> drives the zoom actuator <NUM> according to the zoom operation of the photographer to move the first lens unit <NUM> and the second lens unit <NUM> along the optical axis OA for the magnification variation operation.

A lens communication circuit <NUM> communicates with the interchangeable lens attached to the camera body to acquire the lens information of the interchangeable lens. The acquired lens information is output to the lens information acquisition means 121d in the CPU <NUM>.

The display unit <NUM> includes, for example, an LCD (Liquid Crystal Display). The display unit <NUM> displays information on an imaging mode of the imaging apparatus <NUM>, a preview image prior to imaging, a confirmation image after the imaging, or an in-focus state display image in the focus detection. The operation unit <NUM> includes a power switch, a release (imaging trigger) switch, a zoom operation switch, an imaging mode selection switch, and the like. The release switch has a two-step switch of a half-pressed state (SW1 is on) and a fully pressed state (SW2 is on). A recording medium <NUM> is, for example, a flash memory that is removable from the imaging apparatus <NUM>, and records a captured image (image data). A memory <NUM> stores a captured image and the like in a predetermined format.

Referring now to <FIG>, a description will be given of a pixel array and a pixel structure of the image sensor (two-dimensional CMOS sensor) <NUM> according to this embodiment. <FIG> is a diagram showing a pixel (imaging pixel) array of the image sensor <NUM>. <FIG> illustrates the pixel structure of the image sensor <NUM>, <FIG> is a plan view of a pixel <NUM> of the image sensor <NUM> (viewed from the +z direction), and <FIG> is a sectional view taken along a line a-a in <FIG> (viewed from the -y direction).

<FIG> illustrates the pixel array of the image sensor <NUM> in a range of <NUM> columns × <NUM> rows. In this embodiment, each pixel (pixels 200R, <NUM>, and 200B) has two subpixels <NUM> and <NUM>. Thus, <FIG> illustrates a subpixel array in the range of <NUM> columns × <NUM> rows.

As illustrated in <FIG>, in the pixel unit <NUM> of <NUM> columns × <NUM> rows, pixels 200R, <NUM>, and 200B are arranged in a Bayer array. In other words, the pixel unit <NUM> includes a pixel 200R having a spectral sensitivity of R (red) located at the upper left, the pixels <NUM> having a spectral sensitivity of G (green) located at the upper right and lower left, and the pixel 200B having a spectral sensitivity of B (blue) located at the lower right. The pixels 200R, <NUM>, and 200B have subpixels (focus detection pixels) <NUM> and <NUM> arranged in <NUM> columns × <NUM> row. The subpixel (first subpixel) <NUM> is a pixel that receives a light flux that has passed through the first pupil area in the imaging optical system. The subpixel (second subpixel) <NUM> is a pixel that receives a light flux that has passed through the second pupil area of the imaging optical system.

As illustrated in <FIG>, the image sensor <NUM> has a large number of <NUM> columns × <NUM> rows of pixels (<NUM> columns × <NUM> rows of subpixels) on the surface, and outputs an imaging signal (subpixel signal). The image sensor <NUM> according to this embodiment has a pixel cycle P of <NUM>, and a pixel number N is <NUM>,<NUM> rows × <NUM> rows = <NUM> million pixels. Further, the image sensor <NUM> has a period PSUB of <NUM> in the column direction of the subpixels, and a subpixel number NSUB of <NUM> columns × <NUM> rows = <NUM> million pixels. The number of pixels is not limited to this example, and <NUM> or more columns may be horizontally provided to realize the <NUM> motion image. Further, the pixel having the subpixel and the pixel having no subpixel (non-divided pixel) may be mixed in the pixel array.

As illustrated in <FIG>, the pixel <NUM> according to this embodiment has a micro lens <NUM> for condensing incident light on the light receiving plane side of the pixel. A plurality of micro lenses <NUM> are two-dimensionally arrayed, and separated from the light receiving plane by a predetermined distance in the z-axis direction (direction of the optical axis OA). Further, the pixel <NUM> has photoelectric converters <NUM> and <NUM> of a division number NLF=Nx×Ny (division number <NUM>) divided by Nx (two divisions) in the x direction and divided by Ny (one division) in the y direction. The photoelectric converters <NUM> and <NUM> correspond to the subpixels <NUM> and <NUM>, respectively.

Each of the photoelectric converters <NUM> and <NUM> is configured as a photodiode having a pin structure in which an intrinsic layer is sandwiched between a p-type layer and an n-type layer. If necessary, the intrinsic layer may be omitted and configured as a pn junction photodiode. The pixel <NUM> (each pixel) includes a color filter <NUM> between the micro lens <NUM> and the photoelectric converters <NUM> and <NUM>. If necessary, a spectral transmittance of the color filter <NUM> can be changed for each pixel or each photoelectric converter, or the color filter may be omitted. Where the color filter is omitted, the spectral transmittance of white having a high transmittance in the visible light region may be provided, or the spectral transmittance may be provided in the infrared light region.

The light incident on the pixel <NUM> is collected by the micro lens <NUM>, separated by the color filter <NUM>, and then received by the photoelectric converters <NUM> and <NUM>. In the photoelectric converters <NUM> and <NUM>, pairs of electrons and holes are generated according to the received light amount, and after they are separated by the depletion layer, the electrons of a negative charge are stored in the n-type layer. On the other hand, holes are discharged to the outside of the image sensor <NUM> through a p-type layer connected to a constant voltage source (not shown). The electrons accumulated in the n-type layers of the photoelectric converters <NUM> and <NUM> are transferred to an electrostatic capacitance unit (FD) through the transfer gate and converted into a voltage signal. Although it is preferable that the depth of the photoelectric converter <NUM> be common to each pixel, the depth may be changed (shallowed) in some pixels (such as the pixel 200B having a spectral sensitivity of B).

Referring now to <FIG>, a description will be given of the pupil division function of the image sensor <NUM>. <FIG> is a diagram for explaining the correspondence between the pixel <NUM> of the image sensor <NUM> and the partial pupil area. <FIG> is a sectional view of the a-a section of the pixel structure illustrated in <FIG> viewed from the +y side, and illustrates an exit pupil plane of the imaging optical system. In <FIG>, in order to correspond to the coordinate axis of the exit pupil plane, the x-axis and y-axis of the sectional view are respectively inverted with respect to the x-axis and y-axis in <FIG>.

A light flux from the object passes through an exit pupil <NUM> in the imaging optical system and enters each pixel. A pupil area <NUM> is a pupil area in the pupil area of the imaging optical system that can be received by the entire pixel <NUM> when all the photoelectric converters <NUM> and <NUM> (subpixels <NUM> and <NUM>) are combined. A partial pupil area (first partial pupil area) <NUM> has a substantially conjugate relationship via the micro lens <NUM> with a light receiving plane of the photoelectric converter <NUM> whose center of gravity is decentered in the -x direction. Thus, the partial pupil area <NUM> represents a pupil area that can be received by the subpixel <NUM>. The center of gravity of the partial pupil area <NUM> is decentered on the pupil plane toward the +x side. A partial pupil area (second partial pupil area) <NUM> is in a substantially conjugate relationship via the micro lens <NUM> with the light receiving plane of the photoelectric converter <NUM> whose center of gravity is decentered in the +x direction. Thus, the partial pupil area <NUM> represents a pupil area that can be received by the subpixel <NUM>. The center of gravity of the partial pupil area <NUM> of the subpixel <NUM> is decentered on the pupil plane toward the -x side.

The subpixel <NUM> actually receives light from an AF pupil (first AF pupil) <NUM>, which is an area where the exit pupil <NUM> and the partial pupil area <NUM> overlap each other. The subpixel <NUM> actually receives the light from an AF pupil (second AF pupil) <NUM> which is an area where the exit pupil <NUM> and the partial pupil area <NUM> overlap each other.

<FIG> is a diagram for explaining a pupil division in the imaging optical system and the image sensor <NUM>. The light fluxes having passed through the AF pupils <NUM> and <NUM> enter an imaging plane <NUM> of the image sensor <NUM> at different angles relative to each pixel of the image sensor <NUM> and are received by the <NUM>×<NUM> divided subpixels <NUM> and <NUM>. This embodiment describes an example in which the pupil area is divided into two in the horizontal direction, but the present invention is not limited to this embodiment and the pupil division may be performed in the vertical direction if necessary.

In this embodiment, the image sensor <NUM> includes a plurality of subpixels which share one micro lens and receive a plurality of light fluxes passing through different pupil areas in the imaging optical system (imaging lens). The image sensor <NUM> includes, as a plurality of subpixels, a first subpixel (a plurality of subpixels <NUM>) and a second subpixel (a plurality of subpixels <NUM>). In addition, the imaging optical system may have an array of pixels that receive light fluxes passing through the combined area of the AF pupils <NUM> and <NUM>. In the image sensor <NUM>, each pixel has first and second subpixels. However, if necessary, the imaging pixels and the first and second subpixels may be separate pixel configurations, and the first and second subpixels may be partially disposed in part of the imaging pixel array.

This embodiment generates a first focus detection signal based on the pixel signal of the subpixel <NUM> of each pixel of the image sensor <NUM>, and a second focus detection signal based on the pixel signal of the subpixel <NUM> of each pixel, and performs a focus detection. Further, this embodiment can generate an imaging signal (captured image) having a resolution of the effective pixel number N by adding and reading the signals of the subpixels <NUM> and <NUM> for each pixel of the image sensor <NUM>.

Referring to <FIG>, a description will be given of a relationship between the defocus amount and the image shift amount of the first focus detection signal acquired from the subpixel <NUM> and the second focus detection signal acquired from the subpixel <NUM> in the image sensor <NUM>. <FIG> is a diagram showing the relationship between the defocus amount and the image shift amount. <FIG> illustrates that the image sensor <NUM> is disposed on the imaging plane <NUM>, and similar to <FIG> and <FIG>, the exit pupil <NUM> of the imaging optical system is divided into two AF pupils <NUM> and <NUM>.

A defocus amount d is defined, such that Idl is a distance from the imaging position of the object to the imaging plane <NUM>, a front focus state where the imaging position is closer to the object than the imaging plane <NUM> is expressed with a negative sign (d<<NUM>), and a back focus state where the imaging position is located on the side opposite to the object of the imaging plane <NUM> is expressed with a positive code (d><NUM>). In the in-focus state where the imaging position of the object is located at the imaging plane <NUM> (in-focus position), the defocus amount d=<NUM> is established. <FIG> illustrates an object <NUM> corresponding to the in-focus state (d=<NUM>), and an object <NUM> corresponding to the front focus state (d<<NUM>). The front focus state (d<<NUM>) and the back focus state (d><NUM>) will be collectively referred to as a defocus state (|d|><NUM>).

In the front focus state (d<<NUM>), among the light fluxes from the object <NUM>, the light fluxes that have passed through the AF pupil <NUM> (or the AF pupil <NUM>) are condensed once. Then, the light flux spreads with a width Γ1 (Γ2) centered on a gravity center position G1 (G2) of the light flux, and provided an image blurred on the imaging plane <NUM>. The blurred image is received by the subpixels <NUM> (subpixels <NUM>) constituting the respective pixels arranged in the image sensor <NUM>, and a first focus detection signal (second focus detection signal) is generated. Hence, the first focus detection signal (second focus detection signal) is recorded as an object image in which the object <NUM> is blurred with the width Γ1 (Γ2) at the gravity center position G1 (G2) on the imaging plane <NUM>. The blur width Γ1 (Γ2) of the object image generally increases in proportion to the increase of the magnitude |d| of the defocus amount d. Similarly, a magnitude |p| of an image shift amount p of the object image between the first focus detection signal and the second focus detection signal (= difference G1-G2 between the gravity center positions of the light fluxes) generally proportionally increases as the magnitude |d| of defocus amount d increases. The same applies to the back focus state (d><NUM>), but the image shift direction of the object image between the first focus detection signal and the second focus detection signal is opposite to that in the front focus state.

Thus, in this embodiment, the magnitude of the image shift amount between the first focus detection signal and the second focus detection signal increases, as the magnitude of the first focus detection signal and the second focus detection signal or the defocus amount of the imaging signal obtained by adding the first and second focus detection signals to each other increases.

This embodiment provides phase difference type focusing using the relationship between the defocus amount and the image shift amount between the first focus detection signal and the second focus detection signal.

The phase difference type focusing shifts the first focus detection signal and the second focus detection signal relative to each other, calculates a correlation amount representing the signal coincidence degree, and detects the image shift amount based on the shift amount that improves the correlation (signal coincidence degree). Since the magnitude of the image shift amount increases between the first focus detection signal and the second focus detection signal, as the magnitude of the defocus amount of the imaging signal increases, the focus detection is performed by converting the image shift amount into the defocus amount.

Referring now to <FIG>, a description will be given of a pupil shift at the peripheral image height of the image sensor <NUM>. <FIG> is an explanatory diagram of an effective F-number (effective aperture value) caused by the pupil shift, and the relationship between the AF pupils <NUM> and <NUM> respectively corresponding to the subpixels <NUM> and <NUM> of respective pixels arranged at the peripheral image heights of the image sensor <NUM>, and the exit pupil <NUM> in the imaging optical system.

<FIG> illustrates that an exit pupil distance D1 of the imaging optical system (distance between the exit pupil <NUM> and the imaging plane <NUM>) and a set pupil distance Ds of the image sensor <NUM> are substantially equal to each other. In this case, similar to the central image height, the exit pupil <NUM> in the imaging optical system is divided substantially uniformly by the AF pupils <NUM> and <NUM> at the peripheral image height.

As illustrated in <FIG>, when the exit pupil distance D1 of the imaging optical system is shorter than the set pupil distance Ds of the image sensor <NUM>, there is a pupil shift between the exit pupil <NUM> of the imaging optical system and the entrance pupil of the image sensor <NUM>, at the peripheral image height of the image sensor <NUM>. Hence, the exit pupil <NUM> of the imaging optical system is unevenly divided. In <FIG>, the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM> is smaller (brighter) than the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM>. On the other hand, at the image height on the opposite side, the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM> is larger (darker) than the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM>.

As illustrated in <FIG>, when the exit pupil distance D1 of the imaging optical system is longer than the set pupil distance Ds of the image sensor <NUM>, there is a pupil shift between the exit pupil <NUM> of the imaging optical system and the entrance pupil of the image sensor <NUM>, at the peripheral image height of the image sensor <NUM>. Thus, the exit pupil <NUM> of the imaging optical system is unevenly divided. In <FIG>, the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM> is larger (darker) than the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM>. On the other hand, at the image height on the opposite side, the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM> is smaller (brighter) than the effective F-number of the subpixel <NUM> corresponding to the AF pupil <NUM>.

As the pupil division becomes uneven at the peripheral image height due to the pupil shift, the effective F-numbers of the subpixels <NUM> and <NUM> also become uneven. Thus, the blur spread of either one of the first focus detection signal and the second focus detection signal becomes wider, and the other blur spread becomes narrower. It is thus preferable that in a predetermined area of the image, among the plurality of focus detection signals, the weight coefficient of the focus detection signal output by the subpixel having the smallest effective F-number can be minimized or the weight coefficient of the focus detection signal output by the subpixel having the largest effective F-number can be maximized.

<FIG> is an explanatory view of the light intensity distribution when the light enters the micro lens <NUM> formed in each pixel. <FIG> illustrates a calculation example of the light intensity distribution inside the image sensor <NUM> when a plane wave of the right circularly polarized light with a wavelength λ=<NUM> is incident from the micro lens <NUM> parallel to the optical axis. The numerical calculation used an FDTD (Finite Difference Time Domain) method.

<FIG> illustrates the light intensity distribution on a section parallel to the optical axis of the micro lens. The micro lens optical system in each pixel includes a micro lens <NUM>, a planarization layer, a sealing layer, an insulating layer, and the like. The micro lens optical system may include a plurality of micro lenses. Assume that the pixel period is 2a, the focal length of the micro lens optical system is f, and the aperture angle of the micro lens optical system is 2ϕ. Further, the refractive index at the focal position of the micro lens optical system is n. A coordinate along the optical axis is set to z. In the coordinate z, the focal position is set to an origin (z=<NUM>), the micro lens side is set to a negative sign, and the opposite side to the micro lens is set to a positive sign. H is a main point.

An numerical aperture NA of the micro lens optical system is defined by the following expression (<NUM>). [Expression <NUM>] <MAT>.

Further, the F-number F of the micro lens optical system is defined by the following expression (<NUM>). [Expression <NUM>] <MAT>.

The incident light is condensed on a focal position by the micro lens optical system. However, due to the influence of the diffraction of the light wave nature, the diameter of the condensed spot cannot be made smaller than the diffraction limit Δ, and has a finite size. Assume that the intensity distribution of the condensed spot is close to the Airy pattern. Then, the diffraction limit Δ can be approximately obtained by the following expression (<NUM>), where λ is the wavelength of the incident light. [Expression <NUM>] <MAT>.

The size of the light receiving plane of the photoelectric converter is about <NUM> to <NUM>, whereas the condensed spot of the micro lens is about <NUM>. Thus, the AF pupils <NUM> and <NUM> in <FIG> that are in a conjugate relationship with the light receiving plane of the photoelectric converter via the micro lens are not clearly divided due to the diffraction blurs, and form the light receiving rate distribution (pupil intensity distribution) dependent on the incident angle of light.

<FIG> illustrates the light intensity distribution on a section perpendicular to the optical axis of the micro lens at the focal position of the micro lens. At the point position (z=<NUM>), the diameter of the condensed spot becomes the diffraction limit Δ and the smallest.

The back focus depth +zD and the front focus depth -zD of the micro lens optical system can be obtained by the following expression (<NUM>) with the diffraction limit Δ being a permissible circle of confusion. The range of depth of focus is -zD<z<+zD. [Expression <NUM>] <MAT>.

Assume that the intensity distribution of the condensed spot is close to a Gaussian distribution. Then, a diameter w of the condensed spot substantially satisfies the following expression (<NUM>) as a function of the coordinate z. [Expression <NUM>] <MAT>.

Herein, zR is a Rayleigh length, and is defined as zR=αRzD by setting a coefficient αR to <NUM>.

In the calculation example illustrated in <FIG>, the wavelength λ is <NUM>, the pixel period 2a is <NUM>, the focal length f of the micro lens optical system f is <NUM>, and the refractive index n is <NUM> at the focal position. The F-number of the micro lens optical system is F=<NUM>, the diffraction limit is Δ=<NUM>, and the depth of focus is zD=<NUM>.

<FIG> is a diagram showing a pupil intensity distribution, and the abscissa axis represents the X axis of the exit pupil plane of the imaging optical system, and the ordinate axis represents the light receiving rate. A solid line represents a pupil intensity distribution hA(x) obtained by a projection conversion of the AF pupil <NUM> in <FIG> into the y-axis direction and by forming a one-dimension in the x-axis direction. A broken line represents a pupil intensity distribution hB(x) obtained by a projection conversion of the AF pupil <NUM> into the y-axis direction and by forming a one-dimension in the x-axis direction.

The +x side of the pupil intensity distribution hA(x) shows a sharp curve because it is shielded by the exit pupil defined by the lens frame and the diaphragm frame in the imaging optical system. Further, on the -x side, the boundary of the partial pupil area is blurred due to the pupil division by the micro lens and the influence of the diffraction, and a gentle curve is formed. On the other hand, the pupil intensity distribution hB(x) has a form obtained by inverting the positive and negative of the x axis of the pupil intensity distribution hA(x). Thus, the pupil intensity distribution hA(x) and the pupil intensity distribution hB(x) do not have the same shape, and the coincidence degree lowers in the parallel movement (shifting) and superposition. Further, the shape of the pupil intensity distribution is also determined by the spectral sensitivity characteristic of the image sensor <NUM>. Furthermore, the shape of the pupil intensity distribution is also determined by the state of the pupil shift and the frame shielding that depend on an image height between the exit pupil <NUM> of the imaging optical system and the entrance pupil (partial pupil areas <NUM> and <NUM>) of the image sensor <NUM> described above with reference to <FIG>.

<FIG> is a diagram in which the pupil intensity distribution hA(x) and the pupil intensity distribution hB(x) are Fourier transformed. The Fourier transform HA(k) of the pupil intensity distribution hA(x) and the Fourier transform HB(k) of the pupil intensity distribution hB(x) are represented by the following expression (<NUM>). The Fourier transforms HA(k) and HB(k) are collectively expressed as HA, B(k). Further, the components µA(k) and µB(k) that constitute the phase component are collectively expressed as µA,B(k). Herein, k is a wave number. [Expression <NUM>] <MAT>.

<FIG> illustrates the amplitude component (|HA, B(k)|) of the Fourier transforms HA(k) and HB(k). Since the pupil intensity distributions hA(x) and hB(x) illustrated in <FIG> are in a substantially mirror-inverted relationship, the absolute values of the Fourier transforms substantially accord with each other. In addition, it may change depending on the states of pupil shift and frame shielding.

<FIG> illustrates phase components (phase transfer functions) of the Fourier transforms HA(k) and HB(k). A solid line represents a phase component (-k×µA(k)) of the Fourier transform HA(k). Since the component µA(k) constituting the phase component increases substantially monotonously according to the expression (<NUM>), the solid line substantially monotonically decreases. A broken line represents the phase component (-k×µB(k)) of the Fourier transform HB(k). Since the component µB(k) constituting the phase component decreases substantially monotonously according to expression (<NUM>), the solid line substantially monotonically increases.

<FIG> illustrates the phase difference µAB(k) of Fourier transforms HA(k) and Fourier transform HA(k). The phase difference µAB(k) is expressed by the following expression (<NUM>). [Expression <NUM>] <MAT>.

<FIG> is a diagram showing components of the pupil intensity distributions hA(x) and hB(x) for each spatial frequency. <FIG> illustrates components where the wave number k represented by the solid lines in <FIG> is <NUM>. <FIG> illustrates components where the wave number k represented by the alternate long and short dash lines in <FIG> is <NUM>. <FIG> represents the components where the wave number k represented by the dotted lines in <FIG> is <NUM>. As illustrated in <FIG>, the phase difference µAB becomes smaller as the wave number k becomes smaller. Therefore, the phase difference µAB(k) changes in accordance with the wave number k, as illustrated in <FIG>. This is because the shapes of the pupil intensity distributions hA(x) and hB(x) are different as illustrated in <FIG>.

A line image hA(x|d) of the first focus detection signal and a line image hB(x|d) of the second focus detection signal are obtained by scaling the pupil intensity distributions hA(x) and hB(x) according to the exit pupil distance of the imaging optical system and the defocus amount. Since the line images hA(x|d) and hB(x|d) have substantially similar relationships to the pupil intensity distributions hA(x) and hB(x) respectively, they have the shapes illustrated in <FIG>. In other words, the shapes of the pupil intensity distributions hA(x) and hB(x) are determined based on at least one of the spectral sensitivity characteristic of the image sensor <NUM>, the lens frame of the imaging optical system, and the diaphragm frame of the imaging optical system.

Assume that Z is the exit pupil distance (distance between the exit pupil plane and the imaging plane <NUM>) of the imaging optical system illustrated in <FIG> and d is the defocus amount. Then, the line images hA(x|d) and hB(x|d) are represented by the following expression (<NUM>). The line images hA(x|d) and hB(x|d) are collectively expressed as hA, B(x|d). The exit pupil distance Z is assumed to be sufficiently larger than the defocus amount d. [Expression <NUM>] <MAT>.

A Fourier transform HA(k|d) of the line image hA(x|d) and a Fourier transform HB(k|d) of the line image hB(x|d) are represented by the following expression (<NUM>), and the symmetry function is invariant to the replacement of the wave number k and the defocus amount d. The Fourier transforms HA(k|d) and HB(k|d) are collectively expressed as HA, B(k|d). [Expression <NUM>] <MAT>.

Assume that the light amount distribution of the object is f (x). Then, the first focus detection signal gA(x|d) and the second focus detection signal gB(x|d) in the defocus state of the defocus amount d are represented by the following expression (<NUM>) based on the relational expressions of the convolution and the Fourier transform. The first focus detection signal gA(x|d) and the second focus detection signal gB(x|d) are collectively expressed as gA, B(x|d). [Expression <NUM>] <MAT>.

In the focus detection processing, in order to stably perform a focus detection of a low-contrast object or the like, the DC component and high frequency noise are cut by a band-pass filter, the first and second focus detection signals gA(x|d) and gB(x|d) are limited to the vicinity of a specific wave number kAF component, and a focus detection is performed. The first focus detection signal gA(x|d, kAF) and the second focus detection signal gB(x|d, kAF) limited to the wave number kAF component are expressed by the following expression (<NUM>). [Expression <NUM>] <MAT>.

Thus, the phase difference between the first focus detection signal gA(x|d, kAF) and the second focus detection signal gB(x|d, kAF) limited to the wave number kAF component at the defocus amount d is the image shift amount q at the wave number kAF, and expressed by the following expression (<NUM>). [Expression <NUM>] <MAT>.

Herein, the conversion coefficient K<NUM> for the image shift amount q<NUM> of the defocus amount d<NUM> is expressed by the following expression (<NUM>). [Expression <NUM>] <MAT>.

The detected defocus amount ddet is expressed by the following expression (<NUM>) using the conversion coefficient K<NUM>. [Expression <NUM>] <MAT>.

As described above, the phase difference µAB(k) (=µA(k)-µB(k)) in the expression (<NUM>) changes depending on the wave number k. On the other hand, when the wave number k is fixed to the wave number kAF, the phase difference µAB (dkAF/Z) in the fourth term in the expression (<NUM>) changes depending on the defocus amount d. When the abscissa axis in <FIG> is replaced by the wave number k with the defocus amount d, the phase difference µAB(dkAF/Z) changes as illustrated in <FIG> according to the defocus amount d.

<FIG> is a diagram showing a relationship between the set defocus amount d and the detected defocus amount ddet. The detected defocus amount ddet ideally changes in proportion to the set defocus amount d, as shown by the solid line in <FIG>. However, in practice, the detected defocus amount ddet changes as shown by the broken line in <FIG> because the phase difference µAB(dkAF/Z) depends on the magnitude of the set defocus amount d.

Accordingly, this embodiment calculates the set defocus amount (corrected defocus amount) d by correcting the detected defocus amount ddet, and performs the focus detection processing based on the corrected defocus amount.

This embodiment calculates the set defocus amount (corrected defocus amount) d by the following expression (<NUM>) using the detected defocus amount ddet(K<NUM>q) and the correction coefficient S. [Expression <NUM>] <MAT>.

In the expression (<NUM>), in the focus detection, since the set defocus amount d is unknown, the set defocus amount d is replaced with the detected defocus amount ddet (=K<NUM>q), and the correction factor is calculated.

<FIG> is a diagram showing a relationship between the detected defocus amount ddet (=K<NUM>q) and the correction value S(ddet) for each F-number. The correction coefficient is set to be smaller as the absolute value of the detected defocus amount is larger, as illustrated in <FIG>. As the absolute value of the detected defocus amount is larger, the phase difference µAB becomes larger from the relationship of <FIG> and the correction coefficient becomes smaller from the expression (<NUM>). Further, as illustrated in <FIG>, the correction coefficient is set to be larger as the absolute value of the detected defocus amount is smaller. As the absolute value of the detected defocus amount is smaller, the phase difference µAB becomes smaller from the relationship of <FIG> and the absolute value of the correction coefficient becomes larger from the expression (<NUM>).

This embodiment calculates the correction coefficient based on the F-number (aperture value), but the present invention is not limited to this embodiment. The correction coefficient may be calculated based on the focus detection position (image height coordinate) on the image sensor <NUM>. Further, the correction coefficient may be calculated based on the spatial frequency band of the focus detection signal. The correction coefficient may be calculated based on the color (R/G/B) of the focus detection signal. In addition, the correction coefficient may be calculated based on the lens information (the sign of the detected defocus amount (front focus/back focus)) of the interchangeable lens acquired by the lens information acquisition means 121d. The table relating to the relationship illustrated in <FIG> may be stored in a memory such as a storage medium <NUM> and a memory <NUM>.

Referring now to <FIG>, a description will be given of a focus detection method according to this embodiment executed by the CPU <NUM>. <FIG> is a flowchart showing the focus detection method according to this embodiment. The focus detection method according to this embodiment is implemented according to a focus detection program as a computer program operating on the software and hardware. The focus detection program may be stored, for example, in a memory (not shown) in the imaging apparatus, or may be recorded on a computer-readable recording medium. While the CPU <NUM> executes the focus detection method according to this embodiment, a personal computer (PC) or a dedicated apparatus as a focus detection apparatus may execute the focus detection method according to this embodiment. In addition, a circuit corresponding to the focus detection program according to this embodiment may be provided and the focus detection method according to this embodiment may be implemented by operating the circuit.

In the step S101, a pixel signal acquisition means 121a acquires pixel signals received by the subpixels <NUM> and <NUM> of each pixel of the image sensor <NUM>. The image signal acquisition means 121a may acquire pixel signals that are captured in advance by the image sensor <NUM> according to this embodiment and stored in a recording medium.

In the step S102, the signal generation means 121b generates, based on the pixel signal, a first focus detection signal according to different first partial pupil areas in the imaging optical system, and a second focus detection signal according to the second partial pupil areas. A pixel signal captured by the image sensor <NUM> will be referred to as LF. In addition, assume that the subpixel signal in the is-th (<NUM>≤iS≤Nx) order in the column direction and the js-th (<NUM>≤jS≤Ny) in the row direction in each pixel signal of the pixel signal LF is set to an k-th subpixel signal where k=N(jS-<NUM>)+iS (<NUM>≤k≤NLF). The k-th focus detection signal Ik(j, i) in the i-th order in the column direction and the j-th order in the row direction which corresponds to the k-th partial pupil area in the imaging optical system is generated by the following expression (<NUM>). [Expression <NUM>] <MAT>.

This embodiment shows an example with k=<NUM> and k=<NUM> divided into two in the x-direction in which Nx=<NUM>, Ny=<NUM>, and NLF=<NUM>. The signals from the first subpixels <NUM> divided into two in the x direction are selected for each pixel based on the pixel signals corresponding to the pixel array illustrated in <FIG>. Hence, a first focus detection signal I<NUM>(j, i) is generated as an RGB signal of the Bayer array having a resolution of the pixel number N (horizontal pixel number NH × vertical pixel number NV) corresponding to the first partial pupil area <NUM> in the imaging optical system. Similarly, a second focus detection signal I<NUM>(j, i) is generated which corresponds to the second partial pupil area <NUM> in the imaging optical system.

In this embodiment, the first focus detection signal I<NUM>(j, i) and the second focus detection signal I<NUM>(j, i) are the first focus detection signal gA(x|d) and the second focus detection signal gB(x|d) in the expression (<NUM>).

Next, from the k-th focus detection signal Ik(k=<NUM>, <NUM>) as the RGB signal of the Bayer array, the color centers of gravity of the respective color RGB are made to coincide with one another for each position (j, i), and the k-th focus detection luminance signal Yk (k=<NUM>, <NUM>) is generated by the following expression (<NUM>). If necessary, the shading (light amount) correction processing may be performed for the k-th focus detection luminance signal Yk in order to improve the focus detection accuracy. [Expression <NUM>] <MAT>.

Next, one-dimensional band pass filtering is performed for the k-th focus detection luminance signal Yk (k = <NUM>, <NUM>) in the pupil division direction (column direction), and a first focus detection signal dYA is generated which is limited to substantially the wave number kAF component. Further, the one-dimensional band pass filtering is performed for the second focus detection luminance signal Y<NUM> in the pupil division direction (column direction) to generate a second focus detection signal dYB approximately limited to the wave number kAF component. As the one-dimensional band-pass filter can use, for example, first order differential filters [<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM><NUM>, -<NUM>, -<NUM>, -<NUM>, -<NUM>, -<NUM>, -<NUM>, -<NUM>, -<NUM>]. The pass band of the one-dimensional band-pass filter may be adjusted, if necessary.

In this embodiment, the first focus detection signal dYA and the second focus detection signal dYB approximately limited to the wave number kAF component are the first focus detection signal gA(x|d, KAF) and the second focus detection signal gB(x|d, kAF) limited to the wave number kAF component of the expression (<NUM>).

In the step S103, the focus detection means 121c calculates a detected defocus amount based on the focus detection signal.

Assume a first focus detection signal is dYA(jAF+j<NUM>, iAF+i<NUM>) and a second focus detection signal is dYB(jAF+j<NUM>, iAF+i<NUM>) which are limited to substantially the wave number kAF component in the J<NUM>-th (-n<NUM>≤j<NUM>≤n<NUM>) order in the row direction and in the i<NUM>-th (-m<NUM>≤i<NUM>≤m<NUM>) order in the column direction as the pupil division direction around the focus detection position (jAF, iAF) as the center. Where a shift amount is set to s (-ns≤s≤ns), a correlation amount COREVEN(jAF, iAF, s) is calculated at each position (jAF, iAF) by the expression (18A), and a correlation amount CORODD(jAF, iAF, s) is calculated at each position (jAF, iAF) by the expression (18B).

The correlation amount CORODD(jAF, iAF, s) is a correlation amount made by shifting to the correlation amount COREVEN(jAF, iAF, s), by a half phase, shift amounts of the first focus detection signal dYA and the second focus detection signal dYB approximately limited to the wave number kAF component.

Next, from the correlation amount COREVEN(jAF, iAF, s) and the correlation amount CORODD(jAF, iAF, s), an average is calculated by calculating the shift amount of the real value that minimizes the correlation amount by the subpixel calculation, and an image shift amount q is detected at the focus detection position (jAF, iAF). The detected defocus amount ddet is detected by the expression (<NUM>) using the conversion coefficient K<NUM> of the expression (<NUM>) for the image shift amount q.

For each image height position of the focus detection area, a conversion coefficient K from the image shift amount to the defocus amount is multiplied according to lens information such as an optical characteristic of the image sensor (pupil intensity distribution for each k-th subpixel), the F-number F of the imaging lens (imaging optical system), and the exit pupil distance Dl, and a defocus amount MDef(jAF, iAF) is calculated at the focus detection position (jAF, iAF).

In the step S104, the focus detection means 121c calculates the corrected defocus amount by correcting the detected defocus amount calculated in the step S103 using the correction coefficient.

As described above, the present invention can correct the focus detection error caused by the shape difference between the focus detection signals and execute the focus detection with high accuracy.

Referring now to <FIG> and <FIG>, a description will be given of an imaging apparatus according to this embodiment. The imaging apparatus according to this embodiment is different from that of the first embodiment in the configuration of the image sensor. The other configuration is the same as that of the first embodiment, and thus a detailed description thereof will be omitted. In the imaging apparatus (two-dimensional CMOS sensor) <NUM> according to this embodiment, each pixel includes the first to fourth subpixels, and the signals of the first to fourth subpixels are added and read out to generate an imaging signal (captured image).

<FIG> is a diagram showing a pixel (imaging pixel) array of the image sensor <NUM>. <FIG> is a view showing a pixel structure of the image sensor <NUM>, <FIG> is a plan view (viewed from the +z direction) of the pixel <NUM> of the image sensor <NUM>, and <FIG> is a sectional view (viewed from the -y direction) taken along a line a-a in <FIG>.

<FIG> illustrates a pixel array of the image sensor <NUM> in a range of <NUM> columns × <NUM> rows. In this embodiment, each pixel (pixels 200R, <NUM>, and 200B) has four subpixels <NUM>, <NUM>, <NUM>, and <NUM>. Hence, <FIG> illustrates an array of the subpixels in the range of <NUM> columns × <NUM> rows.

As illustrated in <FIG>, in the <NUM> column × <NUM> row pixel unit <NUM>, the pixels 200R, <NUM>, and 200B are arranged in a Bayer array. In other words, in the pixel unit <NUM>, a pixel 200R having a spectral sensitivity of R (red) is located at the upper left, and pixels <NUM> each having a spectral sensitivity of G (green) are located at the upper right and the lower left, and a pixel 200B having a spectral sensitivity of B (blue) is located at the lower right. The pixels 200R, <NUM>, and 200B have subpixels (focus detection pixels) <NUM>, <NUM>, <NUM>, and <NUM> arranged in <NUM> columns × <NUM> rows. A subpixel (first subpixel) <NUM> is a pixel that receives the light flux that has passed through a first pupil area in the imaging optical system. A subpixel (second subpixel) <NUM> is a pixel that receives the light flux that has passed through a second pupil area in the imaging optical system. A subpixel (third subpixel) <NUM> is a pixel that receives the light flux that has passed through a third pupil area in the imaging optical system. A subpixel (fourth subpixel) <NUM> is a pixel that receives the light flux that has passed through a fourth pupil area of the imaging optical system.

As illustrated in <FIG>, the image sensor <NUM> has a large number of <NUM> columns × <NUM> rows of pixels (<NUM> columns × <NUM> rows of subpixels) arranged on the surface, and outputs an imaging signal (subpixel signal). The image sensor <NUM> according to this embodiment has a pixel period P of <NUM>, and a pixel number N of <NUM>,<NUM> rows × <NUM> rows = <NUM> million pixels. Further, the image sensor <NUM> has a period PSUB of <NUM> in the column direction of the subpixels, and a subpixel number NSUB of <NUM> horizontal rows × <NUM> vertical rows = <NUM> million pixels. The pixel number is not limited to this embodiment, and <NUM> or more columns may be horizontally provided to realize the <NUM> motion image. Further, the pixel having the subpixel and the pixel having no subpixel (non-divided pixel) may be mixed in the pixel array.

As illustrated in <FIG>, the pixel <NUM> according to this embodiment is provided with a micro lens <NUM> for condensing the incident light on the light receiving plane side of the pixel. A plurality of micro lenses <NUM> are two-dimensionally arrayed, and arranged at a position separated from the light receiving plane by a predetermined distance in the z-axis direction (the direction of the optical axis OA). Further, the pixel <NUM> has photoelectric converters <NUM>, <NUM>, <NUM>, and <NUM> divided into NH in the x direction (divided into <NUM>) and into Nv in the y direction (divided into <NUM>). The photoelectric converters <NUM> to <NUM> correspond to the subpixels <NUM> to <NUM>, respectively.

This embodiment generates a first focus detection signal based on the pixel signals of the subpixels <NUM> and <NUM> of each pixel of the image sensor <NUM>, and a second focus detection signal based on the pixel signals of the subpixels <NUM> and <NUM> of each pixel, and performs the focus detection. Further, an imaging signal (captured image) having a resolution of an effective pixel number N can be generated by adding and reading out the signals of the subpixels <NUM>, <NUM>, <NUM>, and <NUM> for each pixel of the image sensor <NUM>.

The first focus detection signal may be generated based on the pixel signals of the subpixels <NUM> and <NUM>. At this time, the second focus detection signal is generated based on the pixel signals of the subpixels <NUM> and <NUM>. In addition, the first focus detection signal may be generated based on the pixel signals of the subpixels <NUM> and <NUM>. At this time, the second focus detection signal is generated based on the pixel signals of the subpixels <NUM> and <NUM>. This embodiment divides the pixel into two in the x direction and the y direction, but the present invention is not limited to this embodiment. For example, it may be divided into two or more, or the number of divisions may be different between the x direction and the y direction.

The present invention can supply a program that implements one or more functions of the above embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus can read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

Claim 1:
A focus detection apparatus configured to perform a focus detection using a pixel signal obtained by photoelectrically converting light passing through different pupil areas in an imaging optical system, the focus detection apparatus comprising:
an acquisition means (121a) configured to acquire the pixel signal;
a signal generation means (121b) configured to generate a first focus detection signal and a second focus detection signal corresponding to the different pupil areas using the pixel signal and
a focus detection means (121c) configured to calculate an image shift amount between the first focus detection signal and the second focus detection signal and to calculate a defocus amount from the image shift amount and a conversion coefficient;
characterized in that
the focus detection means is configured to calculate a first defocus amount from the image shift amount and a conversion coefficient and to calculate a second defocus amount by correcting the first defocus amount using a correction coefficient based on a phase transfer function corresponding to the different pupil areas,
wherein the correction coefficient is used for converting the first defocus amount into the second defocus amount,
wherein the correction coefficient is set to be small when an absolute value of the first defocus amount is large and to be large when the absolute value is small; wherein the apparatus is configured to execute the focus detection based on the second defocus amount.