Patent Publication Number: US-2020296274-A1

Title: Image capturing apparatus and method for controlling same, and storage medium

Description:
BACKGROUND OF THE INVENTION 
     Field of the Invention 
     The present invention relates to a focal point detection technique in an image capturing apparatus. 
     Description of the Related Art 
     Conventionally, image capturing apparatuses are required to have automatic focal point detection functions, and image capturing apparatuses provided with an automatic focal point detection function of the phase difference detection method are popular. The phase difference detection method is a method in which light from an object is divided into two images (a pair of images) by a separator lens (spectacle lens), and the focus state is detected from the phase difference between the two images. 
     Japanese Patent Laid-Open No. 2018-29342 discloses a focal point detection apparatus in which each pixel constituting an area sensor includes four photoelectric conversion units, and which is capable of detecting the focus state of an object in a plurality of correlation calculation directions such as the vertical, horizontal, and oblique directions by forming pairs of images by changing combinations of the photoelectric conversion units. 
     However, with the prior art disclosed in Japanese Patent Laid-Open No. 2018-29342, when focusing on a combination of photoelectric conversion units (referred to hereinafter as pixels) having an oblique correlation calculation direction, there are cases in which a large focal point detection error occurs due to the pixels having rhombus shapes and being adjacent to one another. This will be described in further detail. 
       FIG. 10  is a diagram illustrating the positional relationship of pixels in an oblique direction in the prior art. Here, an A image and a B image constitute a pair of images, and there is a half-pixel shift between the phase of the A image and the phase of the B image. When there is a thin line at a position adjacent to a corner of a pixel in the B image as illustrated in  FIG. 10 , a pixel in the A image receives light from the thin line near the center of the pixel, and thus, can detect the contrast of the thin line. On the other hand, in the pixel in the B image, the thin line is located at an edge of the pixel, and thus, the pixel in the B image cannot detect the contrast of the thin line. Even if the thin line is located at a position other than the position illustrated in  FIG. 10 , the light receiving amounts of the A image and the B image vary considerably depending on the position of the object with contrast, i.e., the thin line, and a large phase difference detection error occurs. 
     SUMMARY OF THE INVENTION 
     The present invention has been made in view of the above-described problem, and provides an image capturing apparatus that can perform focal point detection with high accuracy even for an object with contrast in an oblique direction. 
     According to a first aspect of the present invention, there is provided an image capturing apparatus comprising: a two-dimensional imaging sensor having a first imaging area and a second imaging area respectively receiving light having passed through a first pupil area and a second pupil area that are obtained by dividing a pupil area of an imaging lens in a correlation calculation direction; and at least one processor or circuit configured to function as a calculation unit configured to add pixels in the first imaging area in a predetermined direction that differs from two directions, which are the correlation calculation direction and a direction perpendicular to the correlation calculation direction, to form a first line signal in which the added pixels line up in the correlation calculation direction, to add pixels in the second imaging area in the predetermined direction to form a second line signal in which the added pixels line up in the correlation calculation direction, and to perform a correlation calculation using the first line signal and the second line signal. 
     According to a second aspect of the present invention, there is provided a method for controlling an image capturing apparatus including a two-dimensional imaging sensor having a first imaging area and a second imaging area respectively receiving light having passed through a first pupil area and a second pupil area that are obtained by dividing a pupil area of an imaging lens in a correlation calculation direction, the method comprising: adding pixels in the first imaging area in a predetermined direction that differs from two directions, which are the correlation calculation direction and a direction perpendicular to the correlation calculation direction, to form a first line signal in which the added pixels line up in the correlation calculation direction; adding pixels in the second imaging area in the predetermined direction to form a second line signal in which the added pixels line up in the correlation calculation direction; and performing a correlation calculation using the first line signal and the second line signal. 
     Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a lateral view of a digital camera that is a first embodiment of the image capturing apparatus of the present invention. 
         FIG. 2  is a perspective view schematically illustrating the configuration of a focal point detection optical system in the first embodiment. 
         FIG. 3  is a circuit diagram of a focal point detection sensor in the first embodiment. 
         FIG. 4  is a diagram illustrating drive timings of the focal point detection sensor in the first embodiment. 
         FIG. 5A  is an enlarged view of an imaging area on the focal point detection sensor in the first embodiment. 
         FIG. 5B  is an enlarged view of an imaging area on the focal point detection sensor in the first embodiment. 
         FIG. 5C  is an enlarged view of an imaging area on the focal point detection sensor in the first embodiment. 
         FIG. 6  is a flowchart for describing operations of the camera in the first embodiment. 
         FIG. 7  is a flowchart for describing operations in focal point adjustment processing in the first embodiment. 
         FIG. 8  is a perspective view schematically illustrating the configuration of a focal point detection optical system in a second embodiment. 
         FIG. 9  is a plan view illustrating some pixels of an image sensor in the second embodiment in isolation. 
         FIG. 10  is a diagram illustrating the positional relationship of pixels in an oblique direction in the prior art. 
     
    
    
     DESCRIPTION OF THE EMBODIMENTS 
     Hereinafter, embodiments will be described in detail with reference to the attached drawings. Note, the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but limitation is not made an invention that requires all such features, and multiple such features may be combined as appropriate. Furthermore, in the attached drawings, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted. 
     First Embodiment 
       FIG. 1  is a lateral view of a digital camera that is a first embodiment of the image capturing apparatus of the present invention. 
     In  FIG. 1 , a digital camera  100  includes a camera main body  101  and a lens (imaging lens)  150 . Note that the internal configuration is illustrated in perspective in  FIG. 1  to facilitate understanding of the description. The camera main body  101  includes a CPU  102 , a memory  103 , an image sensor  104 , a shutter  105 , a half mirror  106 , a focus plate  107 , a photometric sensor  108 , a pentaprism  109 , an optical finder  110 , and a sub mirror  111 . Furthermore, the camera main body  101  includes a focal point detection unit  120 , which includes a visual field mask  112 , an infrared cut filter  113 , a field lens  114 , an aperture  115 , a secondary imaging lens  116 , and a focal point detection sensor (sensor for focal point detection)  117 . The lens  150  includes an LPU  151  and a lens group  152 . 
     The CPU  102  is constituted by a microcomputer, and executes the different types of control performed in the camera main body  101 . The memory  103  is a memory such as a RAM or ROM connected to the CPU  102 , and stores data and programs to be executed by the CPU  102 . The image sensor  104  is constituted by a CCD, CMOS sensor, or the like that includes an infrared cut filter, a low pass filter, or the like, and images light entering through the lens  150  as an object image. The shutter  105  can be driven to open and close. The shutter  105  closes and blocks light to the image sensor  104  when shooting is not performed, and opens and exposes the image sensor  104  to light when shooting is performed. The half mirror  106 , when shooting is not performed, reflects part of the light entering through the lens  150  and images the light on the focus plate  107 . The photometric sensor  108  includes an image sensor such as a CCD or CMOS sensor, and performs photographic-subject recognition processing such as a photometric operation, a face detection operation, a tracking operation, and light source detection. The pentaprism  109  reflects light passing through the focus plate  107  toward the photometric sensor  108  and the optical finder  110 . 
     Furthermore, the half mirror  106  transmits part of the light entering through the lens  150 . The transmitted light is bent downward by the sub mirror  111  located on the rear side of the half mirror  106 , and, after passing through the visual field mask  112 , the infrared cut filter  113 , the field lens  114 , the aperture  115 , and the secondary imaging lens  116 , the light is imaged on the focal point detection sensor  117 , in which photoelectric conversion elements are two-dimensionally disposed. The focal point detection unit  120  detects the focus state of the lens  150  based on image signals obtained by photoelectrically converting this image. 
     The LPU  151  is constituted by a microcomputer, and executes control for moving the lens group  152  in the lens  150 . For example, the LPU  151 , upon receiving a defocus amount indicating the amount of divergence of focus from the CPU  102 , moves the lens group  152  to an in-focus position (referred to hereinafter as “focusing position”) based on the defocus amount. 
       FIG. 2  is a perspective view schematically describing light beams in the focal point detection system. 
     In  FIG. 2 , light beams from an object OBJ pass through a plurality of pupil areas of the lens  150 , and are imaged on a focus plane P (primary imaging surface) near the visual field mask  112 . The object image imaged on the primary imaging surface is divided into a plurality of pairs of images by the secondary imaging lens  116 , which is constituted by a plurality of separator lenses (spectacle lenses), and is re-imaged on the focal point detection sensor  117 . A defocus amount can be calculated by performing a correlation calculation on pairs of images photoelectrically converted by this focal point detection sensor  117 . 
     Out of the plurality of light beams from the object OBJ, the light beams passing through pupil areas  201   a  and  201   b  are imaged on two imaging areas of the focal point detection sensor  117  which have a horizontal-direction correlation, namely the imaging areas  501   a  and  501   b . Out of the plurality of light beams from the object OBJ, the light beams passing through pupil areas  202   a  and  202   b  are imaged on two imaging areas of the focal point detection sensor  117  which have a vertical-direction correlation, namely the imaging areas  502   a  and  502   b.    
     Out of the plurality of light beams from the object OBJ, the light beams passing through pupil areas  203   a  and  203   b  are imaged on two imaging areas, namely the imaging areas  503   a  and  503   b , and the light beams passing through pupil areas  204   a  and  204   b  are imaged on two imaging areas, namely the imaging areas  504   a  and  504   b . These imaging areas have an oblique-direction correlation (correlation in a diagonal direction of pixels). 
       FIG. 3  is a diagram illustrating the configuration of the focal point detection sensor  117  in the first embodiment. 
     The focal point detection sensor  117  is a two-dimensional C-MOS area sensor, and some of the pixels (an area corresponding to 2 columns×4 rows of pixels) of the focal point detection sensor  117  are illustrated in  FIG. 3  to facilitate understanding of the description. In actuality, a large number of the pixels illustrated in  FIG. 3  are disposed to enable the acquisition of high-resolution images. 
     In  FIG. 3 , each pixel  30  of the focal point detection sensor  117  includes a photoelectric conversion unit  1  constituted by a MOS transistor gate and a depletion layer below the gate, a photogate  2 , and a transfer switch MOS transistor  3 . Furthermore, a reset MOS transistor  4 , a source-follower amplifier MOS transistor  5 , and a horizontal selection switch MOS transistor  6  are included in every other pixel. Furthermore, in each pixel column, a source-follower load MOS transistor  7 , an output transfer MOS transistor  9 , and a column AD circuit (column AD conversion circuit)  13  are disposed. A DFE circuit  14  is connected to the column AD circuit  13 . Readout rows are selected by a vertical scanning circuit  15 . 
     Next,  FIG. 4  is a timing chart illustrating the operations of the focal point detection sensor  117 . The operations of the focal point detection sensor  117  will be described using  FIGS. 3 and 4 . 
     First, due to output from the vertical scanning circuit  15 , a control pulse φL, is switched to high and vertical output lines are reset. Furthermore, control pulses φR 0 , φPG 00 , and φPGe 0  are switched to high, and reset MOS transistors  4  are switched on and the electric charge in photogates  2  is also reset. 
     At time PT 0 , a control pulse φS 0  is switched to high to switch on selection switch MOS transistors  6  and select FD parts  21  of the first and second rows. Next, the control pulse φR 0  is switched to low to stop the resetting of the FD parts  21  and place the FD parts  21  in a floating state, and the state between the gate and source of the source-follower amplifier MOS transistors  5  is switched to a through state. Subsequently, at time PT 1 , a control pulse φTS is switched to high to switch on the output transfer MOS transistors  9  and to make the output transfer MOS transistors  9  output the dark voltages of the FD parts  21  to the column AD circuit  13  according to a source follower operation. Then, the dark outputs of the FD parts  21  are converted into digital signals by the column AD circuit  13 , and data N of dark voltage values converted into digital signals is temporarily stored by the DFE circuit  14 . 
     Next, in order to perform photoelectric conversion on output from pixel  30 - 11  and pixel  30 - 12  in the first row, a control pulse φTX 00  for the first row is switched to high to switch on transfer switch MOS transistors  3 , and then, the control pulse φPG 00  is switched to low at time PT 2 . Here, it is preferable to adopt a voltage relationship such as a voltage relationship that raises the potential wells spreading beneath the photogates  2  and causes the light generation carriers to be completely transferred to the FD parts  21 . 
     At time PT 2 , as a result of the electric charge from photoelectric conversion units  1 , which are constituted by photodiodes, being transferred to the FD parts  21 , the electric potentials of the FD parts  21  change in accordance with the light-receiving amounts of the photoelectric conversion units  1 . Here, because the source-follower amplifier MOS transistors  5  are in the floating state, the control pulse φTS is switched on at time PT 3  to output the electric potentials of the FD parts  21  to the column AD circuit  13 , and the bright outputs are converted into digital signals. Data S of bright output voltage values converted into digital signals is processed by the DFE circuit  14  performing an S/N ratio calculation, and pixel signals with reduced random noise and fixed pattern noise are obtained. 
     Furthermore, the bright output data S and the dark output data N from the pixels  30 - 11  and  30 - 12  are digitally converted simultaneously by the column AD circuit  13 , and an S/N ratio is calculated by the DFE circuit  14 . Digital data obtained by subtracting the converted dark output from the converted bright output is output to the CPU  102 , with the pulse timing controlled by the DFE circuit  14 . 
     After the bright output S is output to the column AD circuit  13 , the control pulse φR 0  is switched to high to place the reset MOS transistors  4  in a conducting state, and the FD parts  21  are reset to a power source voltage VDD. After the output of the digital data from the first row is finished, reading out of the line in the second row is performed. The reading out for pixel  30 - 21  and pixel  30 - 22  in the second row is performed by simultaneously driving a control pulse φTXe 0  and a control pulse φPGe 0 , supplying a high pulse to the control pulse φTS, and taking out dark output data N and bright output data S. As a result of driving being performed as described above, reading out of the first row and the reading out of the second row can be performed independently from one another. 
     Following this, independent output from all of the pixels can be performed by subsequently causing the vertical scanning circuit  15  to perform a scan and similarly performing reading out of the (2n+1)th and (2n+2)th (n=1, 2, . . . ) rows. That is, if n=1, pixel signals of pixels  30 - 31  and  30 - 32  are read out by first switching a control pulse φS 1  to high and then switching a control pulse φR 1  to low, subsequently switching control pulses φTS and φTX 01  to high, and switching a control pulse φPG 01  to low and switching the control pulse φTS to high. Subsequently, pixel signals of pixels  30 - 41  and  30 - 42  are read out by applying control pulses φTXe 1  and φPGe 1  and control pulses similar to those described above. 
     Here, the column AD circuit  13  is a known ramp circuit-type AD converter that uses a comparator to compare voltages output from pixels via a column output line with a ramp voltage. A pixel signal is converted into a digital signal by activating an unillustrated counter when the comparison with the ramp voltage is started and measuring the time taken until the comparison result is reversed. If this configuration is adopted, there are cases where shot noise (referred to hereinafter as “lateral-stripe random noise”) is generated in the horizontal direction due to an error in the inclination of a ramp voltage and an error in the start timing of a counter, for example. 
     By performing the S/N ratio calculation using the DFE circuit  14  in  FIG. 3 , pixel signals with reduced random noise and reduced fixed-pattern noise can be obtained, but the S/N ratio decreases under a low luminance environment. In view of this, it is effective to perform pixel addition. Pixel addition methods will be described using  FIGS. 5A to 5C . 
       FIGS. 5A to 5C  are enlarged diagrams of imaging areas on the focal point detection sensor  117 . In  FIG. 5A , a pixel region of the imaging area  501   a  illustrated in  FIG. 2  is illustrated in isolation. Here, one pixel for the correlation calculation, which is indicated in each pixel range enclosed in bold lines, is formed by adding four pixels in the vertical direction (pixel column direction). Furthermore, as A image line signals for the correlation calculation, which are enclosed by broken lines, signals of three lines, namely the lines  505 ,  506 , and  507  are used. Similarly, three lines from the pixels in the imaging area  501   b  are used as B image line signals. First, the correlation calculation is performed for one line, and a correlation amount is calculated. Here, the correlation calculation disclosed in Japanese Patent No. 6254780 is performed. The amount of difference between the A image and the B image is calculated pixel by pixel while shifting the pixels, and the total of the amounts is adopted as the correlation amount. A correlation amount is similarly calculated for the other lines (i.e., the second and third lines). After the correlation amounts for three lines have been added up, the pixel shift amount for which the correlation amount is smallest is adopted as the calculation result of the phase difference. A defocus amount is calculated from this phase difference result. 
     In the case of the imaging areas  501   a  and  501   b  illustrated in  FIG. 2 , for which the correlation calculation direction is the horizontal direction, the addition of four pixels in the vertical direction can improve the S/N ratio not only with respect to pixel random noise but also with respect to lateral-stripe random noise generated in the horizontal direction. That is, by using the direction of the column output lines (vertical output lines), or that is, the direction in which pixel signals are transferred to the column AD circuit  13 , as the direction in which pixels are added, the S/N ratio can be improved not only with respect to pixel random noise but also with respect to lateral-stripe random noise generated in the horizontal direction. Furthermore, the S/N ratio is also improved even if one line of units of twelve added pixels is used rather than three lines of units of four added pixels, but the contrast resolution decreases if the contrast of an object is obliquely distributed with respect to the direction in which pixels are added when one line of units of twelve added pixels is used. Thus, the detection accuracy is higher with three lines of units of four added pixels than with one line of units of twelve added pixels. 
     In  FIG. 5B , a pixel region of the imaging area  502   a  illustrated in  FIG. 2  is illustrated in isolation. Here, one pixel for the correlation calculation, which is indicated in each pixel range enclosed in bold lines, is formed by adding four pixels in the horizontal direction. Furthermore, as A image line signals for the correlation calculation, which are enclosed by broken lines, signals of three lines, namely the lines  508 ,  509 , and  510  are used. Similarly, three lines from the pixels in the imaging area  502   b  are used as B image line signals. 
     In the case of the imaging areas  502   a  and  502   b  illustrated in  FIG. 2 , for which the correlation calculation direction is the vertical direction, the addition of four pixels in the horizontal direction can improve the S/N ratio with respect to pixel random noise. However, the addition of pixels does not have any effect with respect to lateral-stripe random noise generated in the horizontal direction. Thus, the focal point detection accuracy is lower compared to the case of the imaging areas  501   a  and  501   b , for which the correlation calculation direction is the horizontal direction. 
     In  FIG. 5C , a pixel region of the imaging area  503   a  illustrated in  FIG. 2 . for which the correlation calculation direction is an oblique direction, is illustrated in isolation. Here, one pixel for the correlation calculation, which is indicated in each pixel range enclosed in bold lines, is formed by adding four pixels in the vertical direction (predetermined direction), which differs from the above-described oblique direction and the direction perpendicular to the oblique direction. Furthermore, as A image line signals for the correlation calculation, which are enclosed by broken lines, signals of three lines, namely the lines  511 ,  512 , and  513  are used. Similarly, three lines from the pixels in the imaging area  503   b  are used as B image line signals. Here, by adding four pixels in the vertical direction and not in the horizontal direction, a situation in which adjacent pixels come into contact with one another at one point is prevented. 
     Furthermore, similarly to the case of the imaging areas  501   a  and  501   b  illustrated in  FIG. 5A , for which the correlation calculation direction is the horizontal direction, the addition of pixels in the vertical direction, or that is, the direction in which pixel signals are transferred to the column AD circuit  13 , can improve the S/N ratio with respect to pixel random noise and also with respect to lateral-stripe random noise generated in the horizontal direction. 
     Furthermore, by adopting a parallelogram as the shape in which pixels are arranged if the first to third lines are combined, the line length L can be maximized with respect to the imaging areas. Accordingly, the amount of pixel shift when the calculation of the phase difference is performed is increased, and thus, the defocus detection range can be increased. 
     What is described in  FIG. 5C  similarly applies to the other pair of oblique-direction imaging areas  504   a  and  504   b , and it suffices to add pixels in the vertical direction and to form lines in the correlation calculation direction (oblique direction). 
       FIG. 6  is a flowchart illustrating the procedure of shooting control processing executed by the digital camera  100  of the present embodiment. The processing in  FIG. 6  is executed by the CPU  102  executing one or more programs stored in the memory  103 , and assumption is made of a case in which the digital camera  100  is already activated. 
     In step S 101 , the CPU  102  determines whether or not a switch SW 1 , which is switched on by a half-press of a shutter switch, has been switched on or not. If the switch SW 1  is switched on, the CPU  102  proceeds to step S 102 , and if not, the CPU  102  waits without performing any processing. 
     In step S 102 , the CPU  102  controls the photometric sensor  108  and performs AE processing. Accordingly, photometric values including luminance information of an object in stationary light (referred to hereinafter as “photometric values in stationary light”) can be acquired. Furthermore, based on the photometric values in stationary light, exposure control values, such as the ISO sensitivity and the aperture value during shooting, and the accumulation time in the focal point detection sensor  117  are determined. 
     In step S 103 , the CPU  102  controls the focal point detection sensor  117  and performs phase difference autofocus (AF) processing. The CPU  102  transmits a calculated defocus amount to the LPU  151 . As a result of this, the LPU  151  moves the lens group  152  to the focusing position based on the received defocus amount. Note that the details of the AF processing will be described later using the flowchart in  FIG. 7 . 
     In step S 104 , the CPU  102  determines whether or not a switch SW 2 , which is switched on by a full-press of the shutter switch, has been switched on or not. If the switch SW 2  is switched on, the CPU  102  proceeds to step S 105 , and if not, the CPU  102  returns to step S 101 . 
     In step S 105 , the CPU  102  performs actual shooting, and the processing in the present flow is terminated. 
       FIG. 7  is a flowchart illustrating the procedure of the AF processing in step S 103  in  FIG. 6 . 
     In step S 201 , the CPU  102  causes the focal point detection sensor  117  to perform an accumulation operation for the accumulation time determined based on the photometric values including the object luminance information acquired in step S 102  in  FIG. 6 , and receives digital pixel data output from the focal point detection sensor  117 . The operations of the focal point detection sensor  117  are as described above using  FIGS. 3 and 4 . 
     In step S 202 , the CPU  102  calculates the phase difference from pixel signals for individual imaging areas acquired in step S 201 , and calculates a defocus amount. The directions in which pixels in imaging areas are added, the method for forming the lines for correlation calculation, etc., are as described above in  FIGS. 5A to 5C . 
     Here, defocus amounts for a plurality of correlation calculation directions are calculated, and a final defocus amount is acquired by performing averaging, weighted averaging, etc. Alternatively, a defocus amount for one of the plurality of correlation calculation directions may be selected. There is no particular limitation regarding the method of selection, but one correlation calculation direction can be selected for which it can be considered that the reliability of the defocus amount is high. The reliability of a defocus amount is considered as being high when the correlation between the waveforms of the pair of images is high, the contrast is high, etc. 
     In step S 203 , the CPU  102  determines whether or not the lens  150  is in an in-focus state. Specifically, the lens  150  is determined as being in-focus if the defocus amount calculated in step S 202  is within a predetermined range, e.g., within 1/4Fδ (where F is the lens aperture value and δ is a constant (e.g., 20 μm)), for example. For example, in a case in which the lens aperture value F equals 2.0, the lens  150  is determined as being in-focus and the AF processing is terminated if the defocus amount is 10 μm or less. 
     On the other hand, if all defocus amounts are greater than 1/4Fδ in step S 203 , the CPU  102 , in step S 204 , calculates a lens driving amount based on the defocus amount calculated in step S 202  and instructs the lens  150  to drive the lens group  152 . Then, the CPU  102  returns to the processing in step S 201 , and repeats the operations in steps S 201  to S 204  until the lens  150  is determined as being in the in-focus state. 
     As described above, the AF accuracy can be improved by adding pixels in the optimal direction for each imaging area having different correlation calculation directions and performing calculation. 
     Second Embodiment 
     In the following, a second embodiment of the present invention will be described. The configuration of the digital camera of the second embodiment is similar to the configuration of the digital camera of the first embodiment, and thus, description of the configuration of the digital camera of the second embodiment is omitted. The second embodiment differs from the first embodiment in that focal point detection is performed by the image sensor  104 . The image sensor  104  is a two-dimensional C-MOS area sensor, and has a circuit configuration similar to that of the focal point detection sensor  117 . 
       FIG. 8  is a schematic diagram illustrating a state in which light beams emitted from an emission pupil of the lens  150  enter a unit pixel of the image sensor  104 . 
     In  FIG. 8 , a unit pixel  1100  includes 2×2 photodiodes, namely photodiodes  1101 ,  1102 ,  1103 , and  1104 . A color filter  1002  and a microlens  1003  are disposed in front of the unit pixel  1100 . The lens  150  includes an emission pupil  1010 . Given that the optical axis  1001  is at the center of the light beams emitted from the emission pupil  1010 , the light passing through the emission pupil  1010  enters the unit pixel  1100  with the optical axis  1001  acting as the center. 
     The emission pupil  1010  of the lens  150  is divided by the 2×2 photodiodes, namely the photodiodes  1101 ,  1102 ,  1103 , and  1104 . Focal point detection is made possible by forming pairs of images while changing the combination of the photodiodes  1101 ,  1102 ,  1103 , and  1104  receiving light entering from different pupil areas. 
       FIG. 9  is a plan view in which some of the plurality of pixels disposed in the image sensor  104  are illustrated in isolation. The 2×2 pixels enclosed in each round frame indicated by a broken line correspond to the unit pixel  1100 .  FIG. 9  illustrates the addition of pixels in a case in which the correlation calculation direction is an oblique direction, and corresponds to  FIG. 5C  in the first embodiment. 
     In  FIG. 9 , one pixel of the A image is formed by adding four pixels lining up in the vertical direction, which are enclosed in bold frames. Furthermore, one line of an A image line signal for the correlation calculation is formed by adding four pixels in the vertical direction in a similar manner while shifting pixels in an oblique direction that is the correlation calculation direction. One line of a B image line signal for the correlation calculation is formed by using hatched pixels as B images and adding the pixels four pixels at a time in the vertical direction while shifting pixels in the oblique direction that is the correlation calculation direction, similarly to the case of A images. 
     By forming the A image line signal and the B image line signal for the correlation calculation in such a manner, effects similar to those achieved in the case illustrated in  FIG. 5C  can also be achieved in an apparatus that performs focal point detection using the image sensor  104 . 
     Other Embodiments 
     Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like. 
     While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions. 
     This application claims the benefit of Japanese Patent Application No. 2019-047338, filed Mar. 14, 2019, which is hereby incorporated by reference herein in its entirety.