Method and system to implement a stacked chip high dynamic range image sensor

Method of implementing stacked chip HDR algorithm in image sensor starts with pixel array capturing first frame with first exposure time and second frame with a second exposure time that is longer or shorter than the first exposure time. Pixel array is disposed in first semiconductor die and is partitioned into pixel sub-arrays. Each pixel sub-array is arranged into pixel groups, and each pixel group is arranged into pixel cell array. Readout circuits disposed in second semiconductor die acquire image data of first and second frame. Each pixel sub-array is coupled to a corresponding readout circuit through a corresponding one of a plurality of conductors. ADC circuits convert image data from first and second frames to first and second ADC outputs. Function logic on the second semiconductor die adding first and second ADC outputs to generate a final ADC output. Other embodiments are also described.

FIELD

An example of the present invention relates generally to image sensors. More specifically, examples of the present invention are related to methods and systems to implement a stacked chip high dynamic range image sensor.

BACKGROUND

High speed image sensors have been widely used in many applications in different fields including the automotive field, the machine vision field, and the field of professional video photography. The technology used to manufacture image sensors, and in particular, complementary-metal-oxide-semiconductor (CMOS) image sensors, has continued to advance at great pace. For example, the demand of higher frame rates and lower power consumption has encouraged the further miniaturization and integration of these image sensors.

One way to increase the frame rate of a CMOS image sensor may be to increase the number of readout circuits operating in parallel. In conventional image sensors, one column of pixels in a pixel array may share one readout circuit. In other examples of the conventional art, one column of pixel cells in a pixel array may share a plurality of readout circuits. These solutions provide a higher frame rate, but require more silicon area, which is not helpful in the miniaturization of silicon image sensors.

Further, many applications require a high dynamic range (HDR) to capture the scene illuminations ranges from 10−1for night vision to 105lux for bright sunlight or direct headlights light condition. This high dynamic range corresponds to a dynamic range of at least 100 dB. Current Charge-coupled devices (CCD) and CMOS sensors cannot achieve this range due to the full well limitation and noise floor limitation, which is typically around 60˜70 dB. A high dynamic range sensor design is needed to extend the applications of CMOS image sensor into the high dynamic range areas.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it is understood that embodiments of the invention may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques have not been shown to avoid obscuring the understanding of this description.

As will be disclosed in various examples, an effective method to read out a pixel array with high dynamic range (HDR) that utilizes pixel sub-arrays that are arranged in a stacked CMOS chip solution in which pixel cells are included in a first semiconductor die, and in which readout circuitry is included in a second semiconductor die. For instance, in one example, the first semiconductor die may be a pixel die, and the second semiconductor die may be an application specific integrated circuit (ASIC) die. In one example, the pixel sub-arrays may be made up of clusters of n×m pixel groups. In the example, the amplifier output nodes of the pixel cells inside the n×m pixel groups are coupled together such that each one of the n×m pixel groups share a single readout circuit included in readout circuitry in accordance with the teachings of the present invention. In the example, the pixel sub-arrays are read out in parallel at high speed and/or with low power in accordance with the teachings of the present invention. In one example, the HDR of the stacked chip image sensor with shared pixel architecture where a cluster of pixels cells share a readout circuitry is increased.

FIG. 1is a block diagram illustrating an example imaging system that includes an image sensor having a pixel array with a plurality of pixels arranged in pixel sub-arrays with pixel architecture for high dynamic range (HDR) in a stacked CMOS image sensor scheme in accordance to one embodiment of the invention. As illustrated inFIG. 1, imaging system100includes an image sensor having a pixel array105partitioned into a plurality of pixel sub-arrays including a pixel architecture for HDR in a stacked image sensor scheme in accordance with the teachings of the present invention. In the illustrated example, imaging system100is realized with stacked CMOS chips, which include a pixel die170stacked with and coupled to an ASIC die108. For instance, in one example, pixel die170includes a pixel array105, and ASIC die180includes control circuitry120, readout circuitry130, and function logic140. In the depicted example, control circuitry120is coupled to control operation of pixel array105, which is coupled to be read out by readout circuitry130through bitlines260(inFIG. 2).

In particular, in the example depicted inFIG. 1, pixel array105is a two-dimensional (2D) array that is partitioned into a plurality of pixel sub-arrays110as shown. In one example, each pixel sub-array110includes a plurality of pixel groups, each of which includes a plurality of pixel cells (not shown inFIG. 1). In the example, each one of the plurality of pixel groups in a pixel sub-array is coupled to utilize the same bit line of bit lines260, and share the same readout circuit in readout circuitry130, more details of which will be described below in connection withFIG. 2.

Control circuitry120is coupled to pixel array105to control the operational characteristics of pixel array105. In one example, control circuitry120is coupled to generate a global shutter signal for controlling image acquisition for each pixel cell. In the example, the global shutter signal simultaneously enables particular pixel cells within all pixel sub-arrays110of pixel array105to simultaneously transfer the image charge from their respective photodetector during a single acquisition window. In one embodiment, the control circuitry120controls the pixel array to cause the pixel array105to capture a first frame with a first exposure time and a second frame with a second exposure time. The first exposure time (“Tlong”) may be longer than the second exposure time (“Tshort”). In other embodiments, the first time exposure time (“Tshort”) may be shorter than the second exposure time (“Tlong”). In one embodiment, an automatic exposure control logic is included in function logic140and determines a ratio of the first exposure time to the second exposure time. The automatic exposure control logic thus calculates the appropriate exposure values (e.g., the first and second exposure time) which are transmitted to the control circuitry120to implement the exposure values during capture and readout of the pixel array105. In this embodiment, a gain factor is determined by the ratio of the first exposure time to the second exposure time. The gain factor may be determined by control circuitry120or function logic140.

In one example, after each of the pixel cells in a pixel sub-array110has acquired or captured its image data or image charge, the image data is read out by readout circuitry130through a bit line of bit lines260. In one embodiment, a logic circuitry (not shown) can control readout circuitry130and output image data to function logic140. Function logic140may simply store the image data or even manipulate the image data by applying post image effects (e.g. crop, rotate, remove red eye, adjust brightness, adjust contrast, or otherwise).

FIG. 2is a schematic of one example of a portion of an image sensor including a pixel sub-array210, which may be one of a plurality of pixel sub-arrays included in a pixel array, such as for example pixel array105ofFIG. 1, in accordance with the teachings of the present invention. In the example depicted inFIG. 2, pixel sub-array210includes a plurality of pixel groups220,221,222, and223that arranged into n=2 columns and m=2 rows. Each of the four pixel groups220,221,222, and223that make up pixel sub-array210in the example depicted inFIG. 2includes four pixel cells230,231,232and233arranged into p=2 columns and q=2 rows, and pixel support circuitry240that is shared by all four pixel cells230,231,232and233, of each pixel group220,221,222, and223.

The pixel cell is the smallest repeating unit in pixel array105ofFIG. 1, and each of the pixel cells230,231,232, and233shown in the example illustrated inFIG. 2includes a photodetector251and transfer transistor252, which is coupled to be controlled by a transfer signal TG. Transfer transistors arranged in the same row in pixel array105, and in the same position within a respective pixel group may be controlled by the same transfer signal. For example, transfer transistor252of pixel cell230, arranged in the upper left corner of pixel group220is controlled by transfer signal TG1(i−1), and the corresponding pixel cell in pixel group221that is arranged in the same row as pixel cell230in pixel group220, also includes a transfer transistor that is controlled by transfer signal TG1(i−1) as shown.

Each of the four transfer transistors252in pixel cells230,231,232, and233of a particular pixel group, such as pixel group220, shares a single floating diffusion node241. Each of the pixel support circuitry240shown in the illustrated example is coupled to and is shared by the four transfer transistors252in pixel cells230,231,232, and233of each particular pixel group, and includes a reset transistor242, an amplifier transistor243, which in the illustrated example is a source follower (SF) coupled transistor243, a row select transistor244, and a capacitor245, which is coupled to a capacitor line270. Floating diffusion node241is coupled to be reset to a floating diffusion reset voltage via power supply RFD through a reset transistor242. Reset transistor242is coupled to be controlled in response to a reset signal RST. In the example, pixel groups that are arranged in the same row are controlled by the same reset signal. For instance, pixel groups220and221are controlled by reset signal RST(i−1), while pixel groups222and223are controlled by reset signal RST(i).

Floating diffusion node241is also coupled to the control terminal of an amplifier transistor, which inFIG. 2is the source follower transistor243having its gate terminal coupled floating diffusion node241, and drain terminal coupled to power supply VDD. In the depicted example, row select transistor244is controlled by a row select signal. In the example, pixel groups that are arranged in the same row are controlled by the same row select signal RS. For instance, pixel groups220and221are controlled by row select signal RS(i−1), while pixel groups222and223are controlled by row select signal RS(i). In one example, row select transistor244is coupled between bit line260and the source terminal of source follower transistor243. The drain terminal of source follower transistor243is coupled to the power supply VDD. Pixel cells in the same pixel sub-array are coupled to the same bit line.

Capacitor245is coupled between floating diffusion241and capacitor line270. In the depicted example, capacitor line270coupled to pixel groups220and222is coupled to receive a signal cap_line(j). Capacitor245may increase the capacitance of floating diffusion node241to increase the dynamic range of a pixel cell in response to cap_line(j). In the illustrated example, capacitor245of each pixel group220,221222, and223may be used to disable other pixel groups when a certain pixel group is being read. For instance, pixel groups220and222may be disabled during the read out of pixel groups221and223by applying a low voltage to capacitor line270in response to cap_line(j). Similarly, pixel groups221and223may be disabled during the read out of pixel groups220and222by apply a low voltage via cap_line(j+1).

In other examples, it is appreciated that capacitor245and capacitor line270may be omitted, and pixel groups that contain pixels cells that are not being read out may be disabled by applying a low voltage to RFD. In other examples, pixel groups which contain pixel cells that are not being read out may be disabled by coupling a pull down transistor between floating diffusion241and a low voltage such as ground, and enabling the pull down transistor to provide the low voltage to floating diffusion241.

As summarized above, it is noted that in the example depicted inFIG. 2that pixel sub-array210includes a plurality of pixel groups arranged in a n×m array, where n=2 and m=2. In addition, it is noted that each pixel group includes a plurality of pixel cells arranged in a p×q array, where p=2 and q=2, and where the pixel cells in each pixel group all share the same pixel support circuitry240. It is appreciated of course that the illustrated example utilizes n=2, m=2, p=2, and q=2, for explanation purposes, and that in other examples, other values may be utilized for n, m, p, and q, where n>1, m>1, p>1, and q>1, and where n, m, p, and q are integers.

As illustrated in the depicted example, all of the pixel cells of pixel sub-array210, are formed on a pixel die270, and share the same bit line260. In one example, bit line260may couple all of the pixel cells of pixel sub-array210to a single readout circuit285, which may be included as one of a plurality of readout circuits included in readout circuitry283formed on an ASIC die280that is stacked with and coupled to pixel die270. In one example, each single readout circuit285of the plurality of readout circuits included in readout circuitry283is coupled to a single one of the plurality of pixel sub arrays through a single bit line260. In one example, an interconnect layer290is disposed between the pixel die270and ASIC die280. In one example, interconnect layer290may include a plurality of conductors. In example, each one of the plurality of conductors may be utilized to couple the readout circuitry283to the circuitry included in pixel die270.

For instance, in the example depicted inFIG. 2, bit line260is realized using one of the plurality of conductors that are included in interconnect layer290. In other words, in one example, each single one of the plurality of pixel sub-arrays (e.g., pixel sub-array210) in pixel die270may be coupled to a corresponding single one of a plurality of readout circuits (e.g., readout circuit285) included in readout circuitry283in ASIC die280through a corresponding single one of the plurality of conductors (e.g., bit line260) included in interconnect layer290. As such, in one example, each single one of the plurality of pixel sub arrays may be read out in parallel by a corresponding single one of the plurality of readout circuits through a corresponding single one of the plurality of conductors, or single bit line, in accordance with the teachings of the present invention.

In one example, the interconnect layer290may include vias such as micro-through silicon vias (μTSVs) or through silicon vias (TSVs). In other examples, one pixel sub-array210may be coupled to more than one readout circuit285formed on ASIC die280. In yet other examples, two or more pixel sub-arrays210may share one readout circuit285formed on an ASIC die280. In one example, each of the plurality of readout circuits285may include analog-to-digital converter (ADC) circuits, adders, and memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM), that are formed on ASIC die280. In still other examples, each of the plurality of readout circuits285may include ADC circuits and adders formed on an ASIC die280, with memory such as SRAM and DRAM formed on a memory die, which may be coupled to ASIC die280through an interconnect layer.

Referring toFIG. 3, a block diagram illustrates the details of one of the plurality of readout circuits in readout circuitry130inFIG. 1in accordance to one embodiment of the invention. As shown inFIG. 3, readout circuitry130may include scanning circuit310, and an ADC circuitry320. Scanning circuit310may also include amplification circuitry, selection circuitry (e.g., multiplexers), etc. to readout a row of image data at a time along readout bit lines260or may readout the image data using a variety of other techniques, such as a serial readout or a full parallel readout of all pixels simultaneously. In one embodiment, readout circuitry130reads out image data from pixel array105that includes reading out the image data from two frames having set exposure times. The first frame may have an exposure time (“Tlong”) that is longer than the second frame's exposure time (“Tshort”). In other embodiments, the first frame may have an exposure time (“Tshort”) that is shorter than the second frame's exposure time (“Tlong”). Scanning circuit130acquires the image data of the first frame and the image data of the second frame. In one embodiment, the image data of the first frame may be stored in function logic140. The storing of the image data of the first frame may be performed prior to capturing the second frame with the shorter or longer exposure time by pixel array. ADC circuitry320may convert each of the image data from scanning circuit310from analog to digital. For example, ADC circuits320included in readout circuits, respectively, may convert the image data of the first frame from analog to digital to obtain a first ADC output and the image data of the second frame from analog to digital to obtain a second ADC output. Referring back toFIG. 1, function logic140may add the first and second ADC outputs to generate a final ADC output. An external host may then perform HDR combination and linearization. The HDR combination and linearization is performed on a per pixel, per pixel cluster, or per sub-array basis. Accordingly, the exposure ratio may be changed on a per pixel, per pixel cluster, or per sub-array basis. For example, each cluster (or sub-array) may dynamically determine (e.g., using a previous frame) the ratio of the longer exposure time (Tlong) to the short exposure time (Tshort).

In one embodiment, pixel array105may capture the first frame with the long exposure time. Readout circuitry130or function logic140may store the output of ADC circuitry320. The pixel array105may then capture the second frame with the shorter exposure time. ADC output for the second frame is readout and added to the ADC output of the first frame. In this embodiment, the ADC circuitry320is 9 bits in size. The result of the addition of the ADC outputs for the first and second frames is stored in a frame buffer. In one embodiment, the result of the addition is 10 bits of data and the frame buffer is 10 bits in size. An external host (off-chip) may then perform HDR combination and linearization of the result of the addition.

Moreover, the following embodiments of the invention may be described as a process, which is usually depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed. A process may correspond to a method, a procedure, etc.

FIG. 4is a flow chart showing an example process for reading out a pixel array partitioned into pixel sub-arrays in accordance to one embodiment of the invention. In the depicted example, it is appreciated that the process may be applied for example to the pixel sub-arrays as described in above with respect toFIG. 1and/orFIG. 2. For instance, as described above, each pixel sub-array includes a plurality of pixel groups, each of which includes a plurality of pixel cells as discussed in detail above with respectFIG. 1and/orFIG. 2. The method400starts at block410with pixel array capturing a first frame with a first exposure time. In one embodiment, pixel array is disposed in a first semiconductor die. At block402, a plurality of readout circuits acquire an image data of the first frame. Readout circuits are included in readout circuitry disposed in a second semiconductor die. Each one of the plurality of pixel sub-arrays is coupled to a corresponding one of the plurality of readout circuits through a corresponding one of a plurality of conductors. At block403, a plurality of ADC circuits respectively included in readout circuits convert the image data of the first frame from analog to digital to obtain a first ADC output. At block404, function logic stores the first ADC output. Function logic may be disposed in the second semiconductor die. In one embodiment, a frame buffer included in function logic stores the first ADC output. At block405, pixel array captures a second frame with a second exposure time. The first exposure time may be longer than the second exposure time. In other embodiments, the first time exposure time may be shorter than the second exposure time. At block406, readout circuits acquire an image data of the second frame. At block407, ADC circuits convert the image data of the second frame from analog to digital to obtain a second ADC output. At block408, the second ADC output is readout and function logic adds the first and the second ADC outputs to generate a final ADC output. At block409, the final ADC output is stored in the frame buffer included in function logic. At block410, an external host performs HDR combination and linearization.

In another embodiment, rather than function logic storing the first ADC output at block404, ADC circuits includes the frame buffer that stores the first ADC output. In this embodiment, ADC circuits also includes logic gates that are used to add the first and second ADC outputs at block408to generate the final ADC output that is stored in the ADC circuits' frame buffer at block409. In another embodiment, the final ADC output may also be stored in a frame buffer included in function logic at Block409.

FIGS. 5(a) and 5(b)are graphs illustrating the light level with respect to the least significant bit (LSB) outputs corresponding to a frame with a longer exposure time (Tlong) and a frame with a shorter exposure (Tshort) according to one embodiment of the invention. Specifically,FIGS. 5(a) and 5(b)show the possible 9 bit ADC outputs (e.g., first and second ADC outputs) corresponding to the first frame having the longer exposure time (Tlong) and the second frame having the shorter exposure (Tshort) as well as the 10 bit final ADC output that is the result of the addition of the first and second ADC outputs. In other embodiments, the first frame may have the shorter exposure time (“Tshort”) and the second frame may have the longer exposure time (“Tlong”).FIG. 5(b)further illustrates the result of HDR combination and linearization which may be performed off-chip by an external host. In bothFIGS. 5(a) and 5(b), the Full Well Capacity (FWC) is indicated.

With the imaging system that includes an image sensor having a pixel array with a plurality of pixels arranged in pixel sub-arrays with pixel architecture for high dynamic range (HDR) in a stacked CMOS image sensor scheme in accordance to one embodiment of the invention, the dynamic range may be increased without increasing the output data rate. Normally, two or more frame captures must be output unless HDR combination is performed on-chip, which adds complexity and cost. Further, the imaging system and the readout method described herein increases the dynamic range of an image sensor without increasing the resolution of ADC circuitry320. Finally, the imaging system and readout methods according to embodiments of the invention provides for adaptive exposure time and dynamic range across the pixel array, and specifically, provides cluster level control of the pixel array rather than on a frame-level.