Camera architecture having a repositionable color filter array

A camera system includes an array of image pixels disposed in or on a substrate and laid out in a multi-ring pattern. The array of image pixels is coupled to acquire image data of a color image in response to light incident on the array of image pixels. A color filter array (“CFA”) is positioned to color filter the light incident on the array of image pixels and includes at least two different color filter types that filter different color bands of the light. An actuator is coupled to the CFA to adjust the CFA in a sequence and a controller is coupled to the actuator to control the sequence such that each image pixel in the array of image pixels is temporarily optically subtended by each of the at least two different color filter types in the CFA while acquiring the image data associated with the color image.

TECHNICAL FIELD

This disclosure relates generally to camera systems, and in particular but not exclusively, relates to color filters and image pixel arrays.

BACKGROUND INFORMATION

Conventional cameras use image sensors with an integrated color filter array (“CFA”) to capture a color image. The CFA is typically designed to have a Bayer pattern (RGGB) with each pixel on the image sensor subtended by a single color filter element above with a one-to-one correspondence. For example, a 5 megapixel (MP) image sensor includes 1.25 M red pixels, 1.25 M blue pixels, and 2.5 M green pixels. To obtain a full 5 MP color image from this image sensor, a demosaicing process (color interpolation) is required to reconstruct the color image from the incomplete color sample output. The demosaicing process inherently compromises the image quality, particularly the sharpness.

Demosaicing algorithms exist, which strive to improve image sharpness. However, there is a fundamental limit of outcome due to the incomplete color sample from the image sensor. There is often a tradeoff between computing speed and image quality with these algorithms.

DETAILED DESCRIPTION

Embodiments of a system and method of operation of a camera system having a repositionable color filter array and in some embodiments a circular pixel array are described herein. In the following description numerous specific details are set forth to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

FIG. 1is a block diagram illustrating a camera system100, in accordance with an embodiment of the disclosure. The illustrated embodiment of camera system100includes a color filter array (“CFA”)105, an image sensor, an actuator110, a controller112, and a camera housing115. The illustrated embodiment of the image sensor includes metal layers120and a substrate125having a pixel array130and peripheral circuitry135.

Camera system100captures color images from light filtered through CFA105and incident onto pixel array130. In one embodiment, pixel array130is a circular pixel array having pixels that are laid out on substrate125in a multi-ring pattern, as opposed to a conventional image sensor having a pixel array laid out in rows and columns. Additionally, CFA105is adjustable by actuator110in a repeated sequence such that each image pixel within pixel array130is sequentially exposed to the multiple different color filter types within CFA105. By exposing each pixel within pixel array130to each of the different color types within CFA105and subsequently combining the different color frames into a single full color image, the demosaicing procedure necessary in conventional image sensors is avoided. In the case of a RGB (red, green, blue) implementation of CFA105, each pixel within pixel array130is exposed to R, G, and B light in succession. In the example of a 5 MP pixel array, three 5 MP color frame images are sequentially acquired and the color data combined into a true 5 MP color image without loss of sharpness due to demosaicing (i.e., color interpolating).

In the illustrated embodiment, the image sensor is a backside illuminated image sensor having metal layers120disposed on the frontside of substrate125while pixel array130is disposed in or on the backside of substrate125. Metal layers120may include multiple layers of signal lines separated by interlayer dielectric material. Metal layers120couple the in-pixel circuitry of each image pixel to peripheral circuitry135by routing the signals under pixel array130to the peripheral region of substrate125. Peripheral circuitry135may include amplifiers, sample and hold circuitry, analog-to-digital converters, buffers, and various other logic for combining color frames into full color images without need of demosaicing.

CFA105may be implemented as a primary color filter including three different types of color filters. For example, CFA105may include red, green, and blue color filter types, or cyan, yellow, and magenta color filter types, or otherwise. In other embodiments, CFA105may include just two color filter types or more than three color filter types that each filter different color bands of incident light.

CFA105is coupled to actuator110, which in turn is controlled by controller112. Actuator110is coupled to CFA105to adjust CFA105between sequential acquisition of color frames. In one embodiment, actuator110is a linear actuator that translates CFA105back and forth and/or side to side. In one embodiment, actuator110is a rotating actuator that rotates CFA105. Actuator110may be implemented using a variety of technologies such as a micro-electro-mechanical system (“MEMS”) actuator, a piezo-electric crystal actuator, an electrostatic actuator, a micro-motor, a servo, or otherwise. In other embodiments, actuator110may manipulate the incident light paths to selectively redirect the incident light through selected color filters via refractive/reflective optical techniques. Controller112is coupled to actuator110and includes the logic for controlling actuator110in a sequential and repeated manner. Controller112may be implemented in hardware (e.g., hardware logic gates), software or firmware executing on a general purpose controller, or a combination of both. In one embodiment, controller112is included within peripheral circuitry135and integrated within substrate125.

FIGS. 2A and 2Billustrate circular pixel arrays having multi-ring layout patterns, in accordance with an embodiment of the disclosure.FIG. 2Aillustrates a circular pixel array200having a regular pattern of truncated pie shaped image pixels205.FIG. 2Billustrates a circular pixel array201having an irregular pattern of truncated pie shaped image pixels205. Both circular pixel arrays200and201represent example implementations of pixel array130illustrated inFIG. 1.

Both circular pixel arrays200and201organize image pixels205into rings (e.g., concentric rings). Image pixels205fan out in a radial pattern about the center of the pixel array. As such, a perimeter shape of each image pixel205is a truncated pie shape. Circular pixel array201illustrates how the layout of image pixels205into a ring geometry need not be a regular pattern from one ring to the next and the number of image pixels205within each ring need not be consistent over the whole pixel array.

In the illustrated embodiments, image pixels205increase in size (e.g., increase in size of the active photo-sensitive surface area of the pixel) with increasing radial distance from the center of the pixel array. Accordingly, with each ring image pixels205get larger and larger. As such, circular pixel arrays200and201have a non-uniform pixel resolution and a non-uniform light sensitivity. The increased sized image pixels205near the outer perimeter of the pixel arrays have a lower pixel resolution and higher light sensitivity compared to image pixels205near the inner central region of the pixel arrays. However, these effects are complementary to the optical effects in a typical lens system. For example, when an object lens is positioned in front of either of pixel arrays200or201, for bringing the object image into focus on the pixel array, lens vignetting results in less brightness along the periphery of the lens. Thus, the increase in size of image pixels205with radial distance can be designed to compensate for lens vignetting to achieve substantially uniform image brightness in the acquired image. Correspondingly, since image resolution already drops off along the periphery region of lenses, the reduced pixel resolution at the periphery region should have a negligible impact on the overall acquired image quality.

AlthoughFIGS. 2A and 2Billustrate circular pixel arrays200and201as being perfect circles, it should be appreciated that the overall perimeter shape of pixel arrays200and201may assume other shapes such as an ellipse or other freeform curvatures. In these alternative embodiments, the shapes of the individual image pixels200may also deviate from a truncated pie shape. AlthoughFIG. 2Aillustrates pixel array200having 60 image pixels205andFIG. 2Billustrates pixel array201having 46 image pixels205, it should be appreciated that these figures are merely for illustration purposes and these pixel arrays may in fact be implemented with thousands or millions of image pixels205. Image pixels205may be implemented as a CMOS image sensor with light sensitive photo-diode regions or a charged coupled device (“CCD”) image sensor. In one embodiment, image pixels205are backside illuminated CMOS image sensors.

FIG. 3illustrates a CFA300having a pie shape structure, in accordance with an embodiment of the disclosure. CFA300represents one possible implementation of CFA105illustrated inFIG. 1. The illustrated embodiment of CFA300includes color filter elements305,310, and315each having a pie shape and laid out in a repeating fan-like pattern. Thus, CFA300includes three different color filter types (e.g., red, green, blue or cyan, yellow, magenta, or other primary color patterns) that each filter a different color band of light.

When CFA300is position in the optical path of pixel array130, each color filter element305,310, and315individually optically subtends multiple image pixels within pixel array130(one-to-many correspondence). In other words, the incident light that is filtered through a given color filter element is incident upon multiple image pixels. During operation, CFA300is rotated through three sequential positions to acquire three separate color frames of image data. Each position aligns a different color filter type over a given image pixel such that each image pixel is sequentially exposed to light incident from the same angle(s), but filtered through three different color filter types. These frames are subsequently combined to generate a full resolution, full color image without color interpolation (i.e., demosaicing).

FIGS. 4A and 4Billustrate radial readout line patterns400and401for reading out image data from circular pixel arrays200and201, respectively, in accordance with an embodiment of the disclosure. Radial readout line patterns400and401may be implemented in metal layers120under pixel array130to readout image data acquired by the image pixels to peripheral circuitry135. Whereas conventional image sensors use x-y grid style column readout or row readout lines to read from an x-y grid style pixel array, radial readout line patterns400and401use readout lines405that run substantially radial from the center of the pixel array out to the periphery. InFIG. 4A, each radial readout line405couples to the same number of image pixels and is time shared by the rings of image pixels during readout. InFIG. 4B, the radial readout lines405may couple to a variable number of image pixels according to the geometry of the pixel array.

FIG. 5is a flow chart illustrating a process500of operation of camera system100including a circular pixel array, in accordance with an embodiment of the disclosure. The order in which some or all of the process blocks appear in process500should not be deemed limiting. Rather, one of ordinary skill in the art having the benefit of the present disclosure will understand that some of the process blocks may be executed in a variety of orders not illustrated, or even in parallel.

In a process block505, CFA105is actuated by actuator110under control of controller112to adjust CFA105to acquire the image data for the first color frame Cx. Actuating CFA105may include translating CFA105, rotating CFA105, or otherwise. In one embodiment, actuating CFA105may include manipulating an optical path leading to CFA105to redirect incident light through different sections of CFA105. In yet another embodiment, actuating CFA105may include electronically changing the color filtering properties of CFA105itself. Depending upon the size and type of CFA105, the image data acquired for each color frame Cx may include image data for only a single color band (e.g., see CFA605illustrated inFIGS. 6A & 6B) or may include image data for multiple different color bands (e.g., CFA300).

In a process block510, pixel array130is reset. Resetting pixel array130includes charging/discharging the photo-sensitive regions to a reset potential in anticipation of acquiring the photo-generated charges associated with the next image frame. In one embodiment, the entire pixel array130is globally reset at the same time. In another embodiment, pixel array130is reset and read-out in rolling sections (e.g., one ring at a time or otherwise). In this rolling reset and read-out embodiment, reset and image acquisition is sequential in sections over pixel array130.

In a process block515, after the image pixels have been reset, the integration period or image acquisition window is commenced. The light incident on pixel array130is filtered through CFA105while in its current adjustment setting or position. Again, image acquisition may be controlled by a global shutter or a rolling shutter to correspond with the type of reset used.

Once the image data of color frame Cx is acquired, the image data is readout of pixel array130into peripheral circuitry135via metal layers120(process block520). In one embodiment, readout occurs along radial readout lines405one ring at a time. Of course, other readout schemes using other layout orientations of readout lines may be implemented (e.g., see grid readout lines illustrated inFIG. 8). During readout, the image data of the current color frame Cx is temporarily buffered (process block525). In one embodiment, peripheral circuitry135includes a buffer large enough to hold image data associated with three different color frames Cx, one for each adjustment setting of CFA105.

If additional color frames Cx still need to be acquired within the same exposure cycle of a given color image (decision block530), process500loops back to capture the next color frame Cx (loop535) and returns to process block505where CFA105is again adjusted to capture the next color frame Cx. In one embodiment, readjusting CFA105includes physically repositioning CFA105in an incremental, sequential, and repeating manner. Once the image data associated with all positions of CFA105have been captured and readout as separate color frames Cx (decision block530), process500continues to process block540.

In one embodiment, three separate color frames Cx are captured, one for each of the primary color filter types included within CFA105. In process block540, the image data of the separate color frames are combined into a single, full color image frame. The color frames Cx include image data collected for each image pixel when the incident light is filtered through each of the different color filter types. For example, for a given image pixel, color frame C1 may include image data captured when the given image pixel is subtended by a red color filter such that the light incident on that given image pixel is red filtered. The color frame C2 may include image data captured when the given image pixel is subtended by a green color filter such that the light incident on that given image pixel is green filtered. The color frame C3 may include image data captured when the given image pixel is subtended by a blue color filter such that the light incident on that given image pixel is blue filtered. By combining the image data acquired by the given pixel during color frames C1, C2, and C3, a full color image is generated for that single image pixel without need of color interpolation. In other words, image data for all three primary colors is acquired by each pixel at each location within pixel array130—not interpolating color from a repeating pattern of four different pixels within a Bayer Pattern group that are offset from each other. As such, image sharpness does not suffer due to the demosaicing process used in connection with Bayer Pattern CFAs. Finally, in a process block545, the full color image frame is output in a process block545.

FIGS. 6A and 6Bare block diagrams illustrating a camera system600having a repositionable color filter array that is translated, in accordance with an embodiment of the disclosure. The illustrated embodiment of camera system600includes a CFA605, an image sensor610having a pixel array, a lens assembly615having an aperture620(also referred to as lens entrance pupil), an actuator625, and a housing630.

Image sensor610may be implemented with embodiments of circular pixel array200or201and radial readout lines400or401. In other embodiments, image sensor610may be implemented with a conventional rectangular layout pixel array having columns and rows. In the illustrated embodiment, camera system600repositions CFA605via a translation motion instead of a rotation motion. CFA605is positioned external to lens assembly615above aperture620and is translated between capturing color frames of a color image. The illustrated embodiment of CFA605includes filter elements (e.g., three filter elements in the illustrated embodiment) that are large enough to cover aperture620such that all image pixels within image sensor610are optically subtended by a single filter element at a given time. As such, each color frame Cx captures image data for only a single color band, as opposed to CFA300. Actuators625translate CFA605between capturing color frames to sequentially align each filter element in front of aperture620. Thus, for a three element CFA embodiment, actuators625should be capable of physically translating and holding CFA605between three positions.

FIG. 7is a block diagram illustrating a camera system700having a CFA that is also translated, in accordance with an embodiment of the disclosure. Camera system700operates in a similar manner to camera system600, but integrates CFA705internal to the lens assembly715such that it is positioned below aperture720. Image sensor710may also be implemented with embodiments of circular pixel array200or201and radial readout lines400or401or by using a conventional rectangular pixel array having pixels laid out in columns and rows.

FIG. 8illustrates a straight readout line pattern800for reading out image data from a circular pixel array, in accordance with an embodiment of the disclosure. Straight readout line pattern800may be implemented in metal layers120under pixel array130to readout image data acquired by the image pixels to peripheral circuitry135. Straight readout line pattern800uses an x-y grid style of columns805or rows810to read from a circular pixel array. Since the circular pixel array has increased resolution towards the center, there is a non-uniform amount of data to read out from each column or row readout line805or810. Rather, the columns or rows that extend under the center of the circular pixel array would readout more image data than the lines extending under the peripheral region of the circular pixel array.