Image sensor with symmetric multi-pixel phase-difference detectors, and associated methods

An imaging system with on-chip phase-detection includes an image sensor with symmetric multi-pixel phase-difference detectors. Each symmetric multi-pixel phase-difference detector includes (a) a plurality of pixels forming an array and each having a respective color filter thereon, each color filter having a transmission spectrum and (b) a microlens at least partially above each of the plurality of pixels and having an optical axis intersecting the array. The array, by virtue of each transmission spectrum, has reflection symmetry with respect to both (a) a first plane that includes the optical axis and (b) a second plane that is orthogonal to the first plane. The imaging system includes a phase-detection row pair, which includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel rows and a pair, and an analogous phase-detection column pair.

BACKGROUND

The vast majority of electronic cameras have autofocus capability. The autofocus function automatically focuses the camera on objects in the scene viewed by the camera. Autofocus may be fully automatic such that the camera identifies objects in the scene and focuses on the objects. In some cases, the camera may even decide which objects are more important than other objects and subsequently focus on the more important objects. Alternatively, autofocus may utilize user input specifying which portion or portions of the scene are of interest. Based thereupon, the autofocus function identifies objects within the portion or portions of the scene, specified by the user, and focuses the camera on such objects.

To achieve market adoption, the autofocus function must be reliable and fast such that every time a user captures an image, the camera quickly brings the desired portion, or portions, of the scene into focus. Preferably, the autofocus function is sufficiently fast that the user does not notice any delay between pressing the trigger button and image capture. The autofocus is particularly important for cameras having no means for manual focus, such as compact digital cameras and camera phones.

Many electronic cameras use contrast autofocus, wherein the autofocus function adjusts the imaging objective to maximize contrast in at least a portion of the scene, thus bringing the portion of the scene into focus. More recently, phase-detection autofocus has gained popularity because it is faster than contrast autofocus. Phase-detection autofocus directly measures the degree of misfocus by comparing light passing through one portion of the imaging objective, e.g., the left portion, with light passing through another portion of the imaging objective, e.g., the right portion. Some digital single-lens reflex cameras include a dedicated phase-detection sensor in addition to the image sensor that captures images.

However, this solution is not feasible for more compact and/or less expensive cameras. Therefore, camera manufacturers are developing image sensors with on-chip phase detection, i.e., image sensors with integrated phase detection capability. A variety of pixel layouts have been proposed for this purpose. These include pixel layouts that include phase-detection pixels that have an opaque mask (or equivalently a shield) that blocks light from reaching one side of the pixel. Such partially-masked (partially-shielded) pixels enable distinguishing light passing through the imaging objective from different directions. Drawbacks of this approach are that the light-blocking masks decrease pixel sensitivity in low-light conditions, shadow neighboring pixels, and reflect light that results in cross-talk in neighboring pixels.

SUMMARY OF THE INVENTION

The embodiments disclosed herein demonstrate on-chip phase detection without the use of a light-blocking mask in phase-detection pixels. This absence of a light-blocking mask, e.g., between a pixel's microlens and photosensitive region, results in several advantages over prior-art phase-detection pixels that include a light-blocking mask. These include better light sensitivity and, with respect to neighboring pixels, reduced shadowing and reduced cross-talk.

An image sensor with symmetric multi-pixel phase-difference detectors is disclosed. Each symmetric multi-pixel phase-difference detector includes (a) a plurality of pixels forming an array and each having a respective color filter thereon, each color filter having a transmission spectrum and (b) a microlens at least partially above each of the plurality of pixels and having an optical axis intersecting the array. The array, by virtue of each transmission spectrum, has reflection symmetry with respect to at least one of (a) a first plane that includes the optical axis and (b) a second plane that is orthogonal to the first plane.

An imaging system with on-chip phase-detection is also disclosed. The imaging system includes a phase-detection row pair, a phase-detection column pair, and a phase-processing module. The phase-detection row pair is capable of measuring a pair of horizontal line profiles for light incident from left and right directions and includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel rows. The phase-detection column pair is capable of measuring a pair of vertical line profiles for light incident from top and bottom directions and includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel columns. The phase-processing module is capable of processing the at least one pair of horizontal line profiles and the at least one pair of vertical line profiles to measure phase shift associated with an arbitrarily-oriented and arbitrarily-located edge in the scene.

A method for phase detection using an image sensor with symmetric multi-pixel phase-difference detectors is disclosed. The method includes generating a first line profile and a second line profile, and determining a first phase shift from a spatial separation between the first line profile and the second line profile. The first line profile is generated from an object edge imaged on a first pixel subset in each of a plurality of mutually collinear symmetric multi-pixel phase-difference detectors of the image sensor. The second line profile is generated from the object edge imaged on a second pixel subset in each of the plurality of mutually collinear symmetric multi-pixel phase-difference detectors.

DETAILED DESCRIPTION

FIG. 1illustrates one exemplary image sensor100, with symmetric multi-pixel phase-difference detectors, in an exemplary use scenario190. Image sensor100is implemented in an electronic camera110for imaging of a scene150. Electronic camera110is, for example, a camera phone or a compact digital camera. Electronic camera110utilizes the on-chip phase detection capability of image sensor100to focus on scene150. When focused, electronic camera110utilizes image sensor100to capture a focused image120, instead of a defocused image130, of scene150.

Image sensor100is configured to provide on-chip symmetric multi-pixel phase-difference detection capable of detecting edges, within scene150, of arbitrary orientation and location. Image sensor100thereby enables robust autofocus for electronic camera110. For example, through use of image sensor100, electronic camera110is able to reliably autofocus on sparsely populated scenes150. Image sensor100also enables a very flexible autofocus function, which may be utilized by electronic camera110to autofocus on objects of arbitrary location within scene150, and/or on an arbitrary selection of portions of scene150that are associated with one or more edges. Herein, an “edge” in a scene refers to a spatial difference such as spatial brightness difference or a spatial color difference.

In an embodiment, image sensor100is a complementary metal-oxide-semiconductor (CMOS) image sensor. Image sensor100may be a color image sensor or a monochrome image sensor. Image sensor100includes a pixel array102that may include color filters arranged in a color filter array, such as a Bayer pattern or others known in the art.

FIGS. 2A, 2B, 3A, 3B, 4A, and 4B, discussed below, illustrate how on-chip phase detection of image sensor100(FIG. 1) may be used to determine the degree of misfocus of one exemplary imaging system composed of image sensor100and an imaging objective210.

FIGS. 2A and 2Bshow an imaging scenario200that illustrates imaging of an object edge230by imaging objective210onto image sensor100, when object edge230is in focus of the imaging system. Object edge230may be a physical edge or a boundary between two differently-colored regions of an object, such as adjacent stripes or text on a background.FIG. 2Ashows imaging scenario200in perspective view, whileFIG. 2Bshows imaging scenario200in cross-sectional view.FIGS. 2A and 2Bare best viewed together. With respect to coordinate system298, object edge230is parallel to the x-axis, imaging objective210has an optical axis213parallel to the z-axis, and image sensor100is parallel to the x-y plane.

Exemplary portions211and212of an imaging objective210are located on opposite sides of and are equidistant from optical axis213. Portions211and212define two rays, or ray bundles,251and252propagating from an object edge230towards an image sensor100. Rays251propagate from object edge230to image sensor100through portion211of imaging objective210. Similarly, rays252propagate from object edge230to image sensor100through portion212of imaging objective210. WhileFIGS. 2A and 2Billustrate object edge230as being located on optical axis213, object edge230may be located away from optical axis213, without departing from the scope hereof.

Imaging objective210has a focal length f. Assuming that imaging objective210is a thin lens, the thin lens equation dictates that

1f=1DO+1DI,(EQ.⁢1)
where DOis the distance202from an object to imaging objective210and DIis the distance203from imaging objective210to a focused image of the object. In imaging scenario200, imaging objective210is at a distance201, denoted by L, from image sensor100, where L=DI. Therefore, object edge230is in focus of the imaging system formed by imaging objective210and image sensor100, and the images formed on image sensor100by portions211and212coincide to yield a single image235.

FIGS. 3A and 3Bshow an imaging scenario300that illustrates imaging of an object edge330by the imaging system ofFIGS. 2A and 2B, where object edge330is more distant than being in focus of the imaging system. Object edge330is similar to object edge230.FIG. 3Ashows imaging scenario300in perspective view, whileFIG. 3Bshows imaging scenario300in cross-sectional view.FIGS. 3A and 3Bare best viewed together. With respect to coordinate system298, object edge330is parallel to the x-axis.

Object edge330is at a distance302from imaging objective210, where distance302is greater than distance202. WhileFIGS. 3A and 3Billustrate object edge330as being located on optical axis213, object edge330may be located away from optical axis213, without departing from the scope hereof. Rays351and352propagating from object edge330through imaging objective portions211and212(FIGS. 3A and 3B), respectively, to image sensor100(FIGS. 3A and 3B) intersect at a point331. According to EQ. 1, since distance302(DO) is greater than distance202, distance303(DI) is less than distance203. Hence, point331is located between imaging objective210and image sensor100at a distance304(denoted byΔD) from image sensor100. Consequently, as illustrated by rays351and352, imaging objective portions211and212form respective images332and333on image sensor100. Images332and333are apart from each other by a distance311. Distance311corresponds to the misfocus-induced phase shift ΔS between images332and333and is indicative of the amount of blur in imaging scenario300.

FIGS. 4A and 4Bshow an imaging scenario400that illustrates imaging of an object edge430by the imaging system ofFIGS. 2A and 2B, where object edge430is less distant than being in focus of the imaging system. Object edge430is similar to object edge230.FIG. 4Ashows imaging scenario400in perspective view, whileFIG. 4Bshows imaging scenario400in cross-sectional view.FIGS. 4A and 4Bare best viewed together. With respect to coordinate system298, object edge430is parallel to the x-axis.

Object edge430is at a distance402from imaging objective210, where distance402is greater than distance202. WhileFIGS. 4A and 4Billustrate object edge430as being located on optical axis213, object edge430may be located away from optical axis213, without departing from the scope hereof. Rays451and452propagate from object edge430through imaging objective portions211and212, respectively, to image sensor100and intersect at a point431. According to EQ. 1, since distance402(DO) is less than distance202, distance403(DI) is greater than distance203. Hence, point431is located beyond image sensor100by a distance404, denoted by ΔD, from the photosensitive surface of image sensor100. Consequently, as illustrated by rays451and452, imaging objective portions211and212form respective images432and433on image sensor100. Images432and433are apart from each other by a distance409. Distance409corresponds to the misfocus-induced phase shift ΔS between images432and433and is indicative of the amount of blur in imaging scenario400.

Imaging scenario200(FIGS. 2A and 2B), imaging scenario300(FIGS. 3A and 3B), and imaging scenario400(FIGS. 4A and 4B) illustrate that misfocus, of the imaging system composed of imaging objective210and image sensor100, results in a phase shift between light propagating to image sensor100through different portions of imaging objective210. Image sensor100is configured to measure this phase shift. An associated autofocus function may adjust imaging objective210to minimize or reduce the phase shift, and thereby focus the imaging system on an object.

WhileFIGS. 2A, 2B, 3A, 3B, 4A, and 4Bshow imaging objective210as being a thin lens, imaging objective210may be a thick lens or a multi-lens objective without departing from the scope hereof.

Horizontal dual-pixel phase-difference detector540includes two horizontally-adjacent phase-detection pixels541and542, color filters543and544, and a microlens532. Microlens532is above phase-detection pixels541and542, which respectively have color filter543and544thereon. Microlens532has an optical axis533. In an embodiment, pixels541and542may form a planar array to which optical axis intersects at a 90-degree angle.

As oriented inFIG. 5A, phase-detection pixels541and542may be denoted as a left-pixel and a right-pixel respectively. Pixels541and542are referred to as phase-detection pixels because they each lack a dedicated microlens above them; rather, each pixel541and542is beneath a common microlens532. For clarity of illustration, the dashed boxes denoting pixels541and542are smaller than boxes denoting respective color filters543. Color filters543and544may have a same transmission spectrum, and may be formed of a single continuous piece of material.

While microlens532is shown to have an oval cross-section in the plan view ofFIGS. 5A and 5B, it may have a differently-shaped cross-section without departing from the scope hereof. For example, microlens532may have a rectangular cross-section in a plane parallel to the x-y plane of coordinate system298such that it completely covers both pixels541and542. Microlens532may include a portion of a spherical surface, an ellipsoidal surface, or an aspheric surface.

Vertical dual-pixel phase-difference detector550is horizontal dual-pixel phase-difference detector540rotated by ninety degrees such that it is oriented parallel to the x-axis of coordinate system298and phase-detection pixels541and542are vertically-adjacent. As oriented inFIG. 5B, phase-detection pixels541and542may be denoted as a bottom-pixel and a top-pixel respectively.

In an embodiment, phase-detection pixels541and542lack masking elements designed to prevent light from reaching photosensitive regions thereof. That is, phase-detection pixels541and542have no additional masking elements relative to non-phase-detection pixels of image sensor100.

FIG. 5CandFIG. 6show a plan view and a cross-sectional view, respectively, of one symmetric multi-pixel phase-difference detector500.FIGS. 5A and 6are best viewed together in the following description. A microlens530is above phase-detection pixels511,512,513, and514each having either color filter521or522thereon. For clarity of illustration, the dashed boxes denoting pixels511-514are smaller than boxes denoting respective color filters521and522.

Microlens530is positioned above phase-detection pixels511-514such that its optical axis531is centered therebetween. Pixels511-514are referred to as phase-detection pixels because they each lack a dedicated microlens above them; rather, each pixel511-514is beneath a common microlens530.

Color filters543,521, and522each transmit a specified range or ranges of visible electromagnetic radiation to its associated underlying pixel. For example, visible color filters based on primary colors have pass bands corresponding to the red, green, or blue (RGB) region of the electromagnetic spectrum, and are referred to as red filters, green filters, and blue filters respectively. Visible color filters based on secondary colors have pass bands corresponding to combinations of primary colors, resulting in filters that transmit either cyan, magenta, or yellow (CMY) light, and are referred to as cyan filters, magenta filters, and yellow filters, respectively. A panchromatic color filter (Cl) transmits all colors of visible light equally. Since the transmission spectrum of a pixel's color filter distinguishes it from its neighboring pixels, a pixel is referred to by its filter type, for example, a “red pixel” includes a red filter. Herein, the transmission of a pixel refers to the transmission spectrum of its color filter.

Symmetry planes501and502may be perpendicular to each other, contain optical axis531, and intersect each other at optical axis531. Phase-detector pixels511-514may have a common back-plane510such that they form a planar array. Optical axis531may intersect back-plane510at a 90-degree angle such that optical axis531is perpendicular to pixel array102. Symmetric multi-pixel phase-difference detectors500have reflection symmetry with respect to both symmetry planes501and502. Symmetric multi-pixel phase-difference detector500also has two-fold rotational symmetry. Table 1 shows fourteen exemplary color filter configurations of symmetric multi-pixel phase-difference detectors500, where R, G, B, C, M, Y, and Cl denote red, green, blue, cyan, magenta, yellow, and panchromatic color filters respectively. In any of the fourteen configurations, the two color filters may be switched without departing from the scope hereof. For example, in configuration (c), color filter521is a green filter and color filter522is a red filter.

Phase-detection pixels511and512may each be viewed as left pixels and together may be denoted as a left-pixel pair. Phase-detection pixels513and514may each be viewed as right pixels and together may be denoted as a right-pixel pair. Phase-detection pixels511and513may each be viewed as top pixels and together may be denoted as a top-pixel pair. Phase-detection pixels512and514may each be viewed as bottom pixels and together may be denoted as a bottom-pixel pair.

In symmetric multi-pixel phase-difference detectors500, pixels511-514and their associated color filters521and522form a two-dimensional two-by-two pixel array. In an embodiment, symmetric multi-pixel phase-difference detectors500may include more than four pixels, e.g., eight pixels in a two-by-four array or sixteen pixels in a four-by-four array.

In an embodiment, phase-detection pixels511-514lack masking elements designed to prevent light from reaching photosensitive regions thereof. That is, phase-detection pixels511-514have no additional masking elements relative to non-phase-detection pixels of image sensor100.

FIG. 7is a cross-sectional view of imaging scenario200with an image sensor700shown at three positions relative to the focal plane. Image sensor700is an embodiment of image sensor100. With respect to coordinate system298, each pixel row of image sensor700is in a respective plane parallel to the y-z plane, while each pixel column of image sensor700is in a respective plane perpendicular to the y-z plane.

Image sensor700(1) is at the focal plane, image sensor700(2) is behind the focal plane, and image sensor700(3) is in front of the focal plane. The cross-sectional view ofFIG. 7is such that y-z plane of coordinate system298intersects phase-detection column pairs732of image sensor700. A phase-detection column pair732is a pair of adjacent pixel columns that include one or more symmetric multi-pixel phase-difference detectors500. An x-y plane cross-sectional view of imaging scenario200would be analogous toFIG. 7, but with the cross-section intersecting phase-detection row pairs.

For image sensor700(1), rays251and252are both chief rays incident on symmetric multi-pixel phase-difference detectors500(2).

Pixel511and pixel512may be denoted as a top pixel and a bottom pixel respectively, where top and bottom refer to the positive and negative y directions, respectively, in coordinate system298. Alternatively, pixel511and pixel512may be denoted as a left pixel and a right pixel respectively, where left and right refer to the positive and negative y directions, respectively, in coordinate system298. Whether pixels511and512are viewed as respectively top and bottom pixels or left and right pixels may depend on the orientation of imaging scenario, and hence coordinate system298, with respect to an image horizon.

For image sensor700(2), rays251and252are a chief rays incident on symmetric multi-pixel phase-difference detectors500(1) and500(3) respectively. In symmetric multi-pixel phase-difference detector500(1), ray251is detected by pixel511. In symmetric multi-pixel phase-difference detectors500(3), ray252is detected by pixel512. Symmetric multi-pixel phase-difference detectors500(3) and500(1) are separated by a distance711, which is equivalent to distance311ofFIG. 3Bbetween images332and333. In the example of image sensor700, distance711equals distance709, as both distances correspond to the separation between the same two symmetric multi-pixel phase-difference detectors500(1) and500(3).

InFIG. 7and the above discussion thereof, each multi-pixel phase-difference detector500(1-3), each with pixels511and512, may be replaced with a horizontal dual-pixel phase-difference detector540(1-3), each with pixels541and542.

FIG. 8is a plan view of a portion of a pixel array802that includes symmetric multi-pixel phase-difference detectors. Pixel array802is an embodiment of pixel array102(FIG. 1). Pixel array802is formed of pixel rows830and pixel columns890that are parallel to directions y and x of coordinate system298, respectively. Pixel array802includes an array of color filters arranged in a Bayer pattern, as shown inFIG. 8. Each color filter covers a respective pixel of the pixel array. Red, green, and blue color filters are denoted by R, G, and B, respectively.

Pixel array802also includes multi-pixel phase-difference detectors800periodically interspersed as a square grid within the Bayer pattern. Each multi-pixel phase-difference detectors800occupies a phase-detection row pair, such as phase-detection row pairs831(1-3), and a phase-detection column pair, such as phase-detection column pairs891(1-4). Each phase-detection row pair831may be perpendicular to each phase-detection column pair891. Multi-pixel phase-difference detectors800interspersed within the Bayer pattern in a different manner than shown inFIG. 8without departing from the scope hereof. For example, multi-pixel phase-difference detectors800may form a triangular grid, a rectangular grid, or combinations thereof.

Each multi-pixel phase-difference detector800includes four phase-detection pixels with symmetric color filters thereon and a common microlens530.FIG. 8labels two pixel columns890(N) and840(N+1) that include a symmetric multi-pixel phase-difference detectors800. Integers N and N+1 are pixel column indices.

Each multi-pixel phase-difference detector800is an embodiment of symmetric multi-pixel phase-difference detectors500. Each multi-pixel phase-difference detector800has filter configuration (a) of Table 1 such that it includes four green filters 2×2 array such that filters of the same color are positioned diagonal from each other. Multi-pixel phase-difference detector800may have other filter configurations, such as those listed in Table 1, without departing from the scope hereof. For clarity of illustration, not all multi-pixel phase-difference detectors800and microlenses530are labeled inFIG. 8.

FIG. 10shows a schematic graph of pixel values vs. pixel column index of multi-pixel phase-difference detectors800in a common phase-detection row pair (831(1) for example), in response to imaging an object1050with vertically-oriented edges1051and1052on pixel array802.

Dashed horizontal line profiles1061,1071, and1081of plots1060,1070, and1080respectively, may represent the pixel response of the “left-side” pixels of multi-pixel phase-difference detector800. Pixels511and512constitute a first vertically-oriented pixel subset of multi-pixel phase-difference detector800. Solid horizontal line profiles1062,1072, and1082of plots1060,1070, and1080respectively represent the pixel response of the “right-side” pixels of multi-pixel phase-difference detector800. Pixels513and514constitute a second vertically-oriented pixel subset of multi-pixel phase-difference detector800.

Alternatively, dashed horizontal line profiles1061,1071, and1081of plots1060,1070, and1080respectively, may represent the pixel response of one vertical dual-pixel phase-difference detector851(1) of one dual-pixel phase-difference detector pair851. Similarly, solid horizontal line profiles1062,1072, and1082of plots1060,1070, and1080respectively may represent the pixel response of one dual-pixel phase-difference detector851(r) of one dual-pixel phase-difference detector pair851.

Plot1060is an image of object1050with both edges1051and1052in focus, as shown in regions1063and1065where line profiles1061and1062overlap. In plot1070, edges1051and1052are out of focus (front-focused), as indicated by distance1009between line profiles1071and1072. Distance1009is an example of distance409. In plot1080, edges1051and1052are out of focus (back-focused), as indicated by distance1011between line profiles1081and1082. Distance1011is an example of distance311.

FIG. 11shows a schematic graph of pixel values vs. pixel column index of symmetric multi-pixel phase-difference detectors800in a common phase-detection column pair (891(1) for example), in response to imaging object1150with horizontally-oriented edges1151and1152on pixel array802.

Dashed vertical line profiles1161,1171, and1181of plots1160,1170, and180respectively, represent the pixel response of the “top” pixels of multi-pixel phase-difference detector800. Pixels511and513constitute a first vertically -oriented pixel subset of multi-pixel phase-difference detector800. Solid vertical line profiles1162,1172, and1182of plots1160,1170, and1180represent the pixel response of the “bottom” pixels of multi-pixel phase-difference detector800. Pixels512and514constitute a second vertically-oriented pixel subset of multi-pixel phase-difference detector800.

Plot1160is an image of object1150with both edges1151and1152in focus, as shown in regions1163and1165where line profiles1161and1162overlap. In plot1170, edges1151and1152are out of focus (front-focused), as indicated by distance1109between line profiles1171and1172. Distance1109is an example of distance409. In plot1180, edges1151and1152are out of focus (back-focused), as indicated by distance1111between line profiles1081and1082. Distance1111is an example of distance311.

FIG. 12shows a schematic graph of pixel values vs. pixel column index of symmetric multi-pixel phase-difference detectors800in a common phase-detection diagonal833(FIG. 8), in response to imaging a diagonally-oriented object1250on pixel array802along a cross-section1255. Without departing from the scope hereof, diagonally-oriented object1250may be oriented at an arbitrary angles with respect to the y-axis of coordinate system298.

Plot1260is an image of object1250with both edges1251and1252in focus, as shown in regions1263and1265where profiles1261and1262overlap. In plot1270, edges1251and1252are out of focus (front-focused), as indicated by distance1209between profiles1271and1272. Distance1209is an example of distance409. In plot1280, edges1251and1252are out of focus (back-focused), as indicated by distance1211between profiles1281and1282. Distance1211is an example of distance311.

FIG. 13illustrates detection and phase-shift measurement of one exemplary arbitrarily oriented edge1360by image sensor100(FIG. 1). Edge1360has a width1361and is an image of a transition between two areas of different brightness and/or color in scene150. The extent of edge1360is determined by (a) the actual extent of the transition in scene150and (b) the degree of misfocus of the image of the transition.

Bottom pixels512and514(FIGS. 5 and 9) and top pixels511and513of a phase-detection column pair891generate electrical signals indicating vertical line profiles1322and1332for edge1360along phase-detection column pair891. Line profiles1322and1332are plotted as brightness and/or color measurement1390versus vertical position1382. Bottom pixels512and514produce one of vertical line profiles1322and1332, while top pixels511and513produce the other one of vertical line profiles1322and1332. Edge1360is apparent in each of line profiles1322and1332as a change in brightness and/or color measurement1390. Each of line profiles1322and1332provide a measurement of the extent1312of edge1360along phase-detection column pair891. Together, line profiles1322and1332provide a measurement of the misfocus-induced phase shift1302between line profiles1322and1332.

Left pixels511and512and right pixels513and514of a phase-detection row pair831generate electrical signals indicating horizontal line profiles1324and1334for edge1360along phase-detection row pair831. Line profiles1324and1334are plotted as brightness and/or color measurement1390versus horizontal position1384. Left pixels511and512produce one of horizontal line profiles1324and1334, while right pixels513and514produce the other one of horizontal line profiles1324and1334. Edge1360is apparent in each of line profiles1324and1334as a change in brightness and/or color measurement1390. Each of line profiles1324and1334provide a measurement of the extent1314of edge1360along phase-detection row pair831. Together, line profiles1324and1334provide a measurement of misfocus-induced phase shift1304between line profiles1324and1334.

If the optical system that images scene150onto image sensor100is free of astigmatism, misfocus-induced phase shift1304is the same as misfocus-induced phase shift1302. If, on the other hand, the optical system is astigmatic, misfocus-induced phase shift1304may be different from misfocus-induced phase shift1302.

The accuracy of misfocus-induced phase shift1302increases as extent1312decreases toward its minimum value, which is width1361. Similarly, the accuracy of misfocus-induced phase shift1304decreases as extent1314decreases toward its minimum value, which is also width1361. In the example ofFIG. 13, edge1360has a greater horizontal component than vertical component. Therefore, extent1312is significantly smaller than extent1314. Assuming no or negligible astigmatism, misfocus-induced phase shift1302is the same as misfocus-induced phase shift1304. Accordingly, phase-detection column pair891provides a better phase-shift measurement than phase-detection row pair831.

The example ofFIG. 13is for an ideal situation. If further accounting for non-idealities, such as noise and/or interfering features in the scene, aberrations of the optical system, and electronic noise of image sensor100, line profiles1322,1332,1324, and1334may be substantially noisier than what is shown inFIG. 13. In such situations, misfocus-induced phase shift1304may be undetectable, and only phase-detection column pair891is capable of providing a measurement of the misfocus-induced phase shift associated with edge1360.

It follows from the above discussion that phase-detection column pair891provides a better phase-shift measurement for near-horizontal edges than phase-detection row pair831, while phase-detection row pair831provides a better phase-shift measurement for near-vertical edges than phase-detection column pair891. It also follows that phase-detection column pair891is unable to enable measurement of the phase shift for vertical edges, and depending on non-ideal properties discussed above, may be unable to enable measurement of the phase shift for near-vertical edges. Likewise, phase-detection row pair831is unable to enable measurement of the phase shift for horizontal edges, and depending on non-ideal properties discussed above, may be unable to enable measurement of the phase shift for near-horizontal edges. Consequently, accuracy of an image sensor100improves when it includes both phase-detection row pairs831and phase-detection column pairs891.

Interface1460is an interface that handles communication between imaging system1400and a user and/or an external system such as a computer. Interface1460may include user interface devices such as a display, a touch screen, and/or a keyboard. Interface1460may include wired (such as Ethernet, USB, FireWire, or Thunderbolt) and/or wireless (such as Wi-Fi or Bluetooth) connections for communicating images to a user or an external system.

For each phase-detection row pair831, or each one of several portions of each phase-detection row pair831, considered by phase-processing module1420, phase-processing module1420processes electrical signals generated by left pixels511and512and right pixels513and514to determine a horizontal line profile pair1424that includes a horizontal line profile1425and a horizontal line profile1426. Phase-processing module1420determines horizontal line profile1425and horizontal line profile1426based upon electrical signals received from left pixels511and512and right pixels513and514, respectively. Examples of horizontal line profiles1425and1426include line profiles1061,1062,1071,1072,1081, and1082ofFIG. 10.

For each phase-detection column pair891(FIG. 8), or each one of several portions of each phase-detection column pair891, considered by phase-processing module1420, phase-processing module1420processes electrical signals generated by bottom pixels512and514(FIG. 6) and top pixels511and513to determine a vertical line profile pair1421that includes of a vertical line profile1422and a vertical line profile1423. Phase-processing module1420determines vertical line profile1422and vertical line profile1423based upon electrical signals received from bottom pixels512and514and top pixels511and513, respectively. Examples of vertical line profiles1422and1423include line profiles1161,1162,1171,1172,1181, and1182ofFIG. 11.

Based upon at least one of vertical line profile pair1421and horizontal line profile pair1424, phase-processing module1420detects an edge (such as edge1360) in an image formed on pixel array102and determines associated phase shifts1427. The edge, thus detected by phase-processing module1420, may have arbitrary orientation relative to pixel array102.

Although image sensor100is shown inFIG. 14as having three of each of phase-detection column pairs891, phase-detection rows pairs831, bottom pixels512and514, top pixels511and513, left pixels511and512, and right pixels513and514, actual numbers may be different, without departing from the scope hereof.

In an embodiment, imaging system1400includes an autofocus module1440and an imaging objective1410. Imaging objective1410is for example imaging objective210ofFIGS. 2, 3, 4, and 7. Based upon phase shifts1427received from phase-processing module1420, autofocus module1440adjusts imaging objective1410to form an image of scene150(FIG. 1) on image sensor100, from which image sensor100generates image data1480. For example, autofocus module1440may adjust imaging objective1410to minimize phase shifts1427. Scene150may include object edges such as edges1051,1052,1151,1152, and1360.

In an embodiment, imaging system1400includes a region-of-interest (ROI) selection module1430that selects an ROI, within pixel array102, to be processed by phase-processing module1420. ROI selection module1430may receive ROI specification from one of interface1460and an object-detection module1431. Object-detection module1431is for example a face-detection module. Alternatively, or in combination therewith, ROI selection module1430receives, from phase-processing module1420, locations of edge(s) with respect to pixel array102and, based thereupon, determines an ROI specification.

Imaging system1400may further include one or both of an enclosure1490and a power supply1470.

FIG. 15is a flowchart illustrating a method1500for phase detection using an image sensor with symmetric multi-pixel phase-difference detectors.

Step1510is optional. In step1510, method1500images an object edge onto the image sensor with an imaging objective. In an example of step1510, scene150is imaged onto image sensor100with imaging objective1410of imaging system1400(FIG. 14).

In step1520, method1500generates a first line profile from the object edge imaged on a first pixel subset in each of a plurality of mutually collinear symmetric multi-pixel phase-difference detectors of the image sensor. In an example of step1520, horizontal line profile1425is imaged on one phase-detection column pair891of image sensor100. Horizontal line profile1425is for example first line profile1071(FIG. 10) generated from object edges1051and1052imaged on a first pixel subset—pixels511and512—in each mutually collinear symmetric multi-pixel phase-difference detectors500of image sensor100.

Step1520may include optional step1522. In step1522, method1500sums pixel responses of pixels in the first pixel subset. In an example of step1522, pixel responses of pixels511and512are summed to yield first line profile1071.

In step1530, method1500generates a second line profile from the object edge imaged on a second pixel subset in each of a plurality of mutually collinear symmetric multi-pixel phase-difference detectors of the image sensor. In an example of step1520, horizontal line profile1426is imaged on one phase-detection column pair891. Horizontal line profile1426is for example second line profile1072(FIG. 10) generated from object edges1051and1052imaged on a second pixel subset—pixels513and514—in each mutually collinear symmetric multi-pixel phase-difference detector500of image sensor100.

Step1530may include optional step1532. In step1532, method1500sums pixel responses of pixels in the second pixel subset. In an example of step1532, pixel responses of pixels513and514are summed to yield second line profile1072.

In step1540, method1500determines a first phase shift from a spatial separation between the first line profile and the second line profile. In an example of step1540, phase-processing module1420determines one phase shift1427, for example, distance1009between line profiles1071and1072.

Step1550is optional. In step1550, method1500reduces the first phase shift by changing a distance between the imaging objective and the image sensor. In an example of step1550, autofocus module1440reduces phase shift1427changing a distance between imaging objective1410and image sensor100.

FIG. 16is a plan view of exemplary symmetric multi-pixel phase-difference detectors1601-1606, which are embodiments of symmetric multi-pixel phase-difference detector500. Symmetric multi-pixel phase-difference detector1601is identical to symmetric multi-pixel phase-difference detector500where each color filter521and522is a panchromatic (clear) color filter1621. Symmetric multi-pixel phase-difference detector1602is identical to symmetric multi-pixel phase-difference detector500where each color filter521is a red color filter1622and each color filter522is a green color filter921. Symmetric multi-pixel phase-difference detector1603is identical to symmetric multi-pixel phase-difference detector500where each color filter521is a blue color filter1623and each color filter522is a green color filter921. Symmetric multi-pixel phase-difference detectors1604-1606are identical to symmetric multi-pixel phase-difference detector500where each color filter522is a clear color filter1621and each color filter521is a red color filter1621, green color filter921, and a blue color filter1623respectively.

Combinations of Features.

Features described above as well as those claimed below may be combined in various ways without departing from the scope hereof. The following examples illustrate some possible, non-limiting combinations:

(A1) An image sensor may include symmetric multi-pixel phase-difference detectors. Each symmetric multi-pixel phase-difference detector includes (a) a plurality of pixels forming an array and each having a respective color filter thereon, each color filter having a transmission spectrum and (b) a microlens at least partially above each of the plurality of pixels and having an optical axis intersecting the array. The array, by virtue of each transmission spectrum, has reflection symmetry with respect to at least one of (a) a first plane that includes the optical axis and (b) a second plane that is orthogonal to the first plane.

(A2) In the image sensor denoted as (A1), the array may be a planar array.

(A3) In either or both image sensors denoted as (A1) and (A2), the optical axis may intersect the array at a 90-degree angle.

(A4) Any of the image sensors denoted as (A1) through (A3), may further include a phase-detection row pair that includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel rows; and a phase-detection column pair that includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel columns.

(A5) In any of the image sensors denoted as (A1) through (A4), the plurality of pixels may be four in number and arranged as a 2×2 planar array.

(A6) In any of the image sensors denoted as (A1) through (A5), the array, by virtue of each transmission spectrum, having reflection symmetry with respect to both (a) a first plane that includes the optical axis and (b) a second plane that is orthogonal to the first plane.

(A7) In any of the image sensors denoted as (A5), the color filter on two of the plurality of pixels may each have a first transmission spectrum; the color filter on the remaining two of the plurality of pixels may each have a second transmission spectrum.

(A8) In any of the image sensors denoted as (A7), the first transmission spectrum and the second transmission spectrum may correspond to the transmission spectrum of one of a red color filter, a blue color filter, a green color filter, a cyan color filter, a magenta color filter, a yellow color filter, and a panchromatic color filter.

(B1) An imaging system with on-chip phase-detection may include a phase-detection row pair, a phase-detection column pair, and a phase-processing module. The phase-detection row pair is capable of measuring a pair of horizontal line profiles for light incident from left and right directions and includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel rows. The phase-detection column pair is capable of measuring a pair of vertical line profiles for light incident from top and bottom directions and includes a plurality of symmetric multi-pixel phase-difference detectors in a pair of adjacent pixel columns. The phase-processing module is capable of processing the pair of horizontal line profiles and the pair of vertical line profiles to measure phase shift associated with an arbitrarily-oriented and arbitrarily-located edge in the scene.

(B2) The imaging system denoted as (B1) may further include an autofocus module for adjusting focus of an imaging objective to reduce the phase shift.

(C1) A method for phase detection using an image sensor with symmetric multi-pixel phase-difference detectors may include generating a first line profile and a second line profile, and determining a first phase shift from a spatial separation between the first line profile and the second line profile. The first line profile is generated from an object edge imaged on a first pixel subset in each of a plurality of mutually collinear symmetric multi-pixel phase-difference detectors of the image sensor. The second line profile is generated from the object edge imaged on a second pixel subset in each of the plurality of mutually collinear symmetric multi-pixel phase-difference detectors.

(C2) In the method denoted as (C1), the step of generating the first line profile may include summing pixel responses of pixels of the first pixel subset. The step of generating the second line profile may include summing pixel responses of pixels of the second pixel subset.

(C3) In the method denoted as (C2), the step of summing pixel responses of pixels of the first pixel subset may include summing pixel responses of a pair of two adjacent pixels of the first pixel subset. The step of summing pixel responses of pixels of the second pixel subset may include summing pixel responses of a pair of two adjacent pixels of the second pixel subset not included in the first pixel subset.

(C4) In any of the methods denoted as (C1) through (C3), each multi-pixel phase-difference detectors may be mutually collinear in a first direction parallel to one of (i) pixel rows of the image sensor and (ii) pixel columns of the image sensor.

(C5) Any of the methods denoted as (C1) through (C4) may further include: (a) generating a third line profile from an object edge imaged on a first pixel subset in each of a second plurality of multi-pixel phase-difference detectors of the image sensor that are mutually collinear in a second direction that is perpendicular to the first direction, (b) generating a fourth line profile from the object edge imaged on a second pixel subset in each of the second plurality of mutually collinear symmetric multi-pixel phase-difference detectors, and (c) determining a second phase shift from a spatial separation between the first line profile and the second line profile.

(C6) Any of the methods denoted as (C1) through (C6) may further include imaging the object edge onto the image sensor with an imaging objective.

(C7) The method denoted as (C6) may further include reducing the first phase shift by changing a distance between the imaging objective and the image sensor.