OPTICAL SENSOR WITH SIMULTANEOUS IMAGE/VIDEO AND EVENT DRIVEN SENSING CAPABILITIES

An optical sensor includes a pixel array of pixel cells. Each pixel cell includes photodiodes to photogenerate charge in response to incident light and a source follower to generate an image data signal in response to the charge photogenerated from the photodiodes. An image readout circuit is coupled to the pixel cells to read out the image data signal generated from the source follower of at least one of the pixel cells of a row of the pixel array. An event driven circuit is coupled to the pixel cells to read out the event data signals generated in response to the charge from the photodiodes of another row of the pixel cells of the pixel array. The image readout circuit is coupled to read out the image data signal and the event driven circuit is coupled to read out the event data signals from pixel array simultaneously.

BACKGROUND INFORMATION

Field of the Disclosure

This disclosure relates generally to image sensors, and in particular but not exclusively, relates to image sensors that include event sensing circuitry.

Background

Image sensors have become ubiquitous and are now widely used in digital cameras, cellular phones, security cameras, as well as medical, automobile, and other applications. As image sensors are integrated into a broader range of electronic devices, it is desirable to enhance their functionality, performance metrics, and the like in as many ways as possible (e.g., resolution, power consumption, dynamic range, etc.) through both device architecture design as well as image acquisition processing.

A typical image sensor operates in response to image light from an external scene being incident upon the image sensor. The image sensor includes an array of pixels having photosensitive elements (e.g., photodiodes) that absorb a portion of the incident image light and generate image charge upon absorption of the image light. The image charge photogenerated by the pixels may be measured as analog output image signals on column bitlines that vary as a function of the incident image light. In other words, the amount of image charge generated is proportional to the intensity of the image light, which is read out as analog image signals from the column bitlines and converted to digital values to provide information that is representative of the external scene.

DETAILED DESCRIPTION

Various examples directed to a hybrid optical sensor with simultaneous image/video capturing and event driven sensing capabilities are described herein. In the following description, numerous specific details are set forth to provide a thorough understanding of the examples. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail in order to avoid obscuring certain aspects.

Spatially relative terms, such as “beneath,” “below,” “over,” “under,” “above,” “upper,” “top,” “bottom,” “left,” “right,” “center,” “middle,” and the like, may be used herein for ease of description to describe one element or feature's relationship relative to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is rotated or turned over, elements described as “below” or “beneath” or “under” other elements or features would then be oriented “above” the other elements or features. Thus, the exemplary terms “below” and “under” can encompass both an orientation of above and below. The device may be otherwise oriented (rotated ninety degrees or at other orientations) and the spatially relative descriptors used herein are interpreted accordingly. In addition, it will also be understood that when an element is referred to as being “between” two other elements, it can be the only element between the two other elements, or one or more intervening elements may also be present.

As will be discussed, various examples of stacked CIS system in which a hybrid optical sensor with simultaneous image/video capturing and event driven sensing capabilities are disclosed. Although normal image/video sensors offer great image and/or video capturing capabilities, one of the limitations with normal image/video sensors is that normal image sensors do not provide ultra-high frame rates and ultra-high speed capture capabilities that may be useful in a variety of applications such as machine vision, gaming, and artificial intelligence sensing areas. Attempts to provide typical image/video sensors to with such ultra-high frame rates and ultra-high speed capabilities have resulted in compromised solutions that provide poor quality image captures compared to their normal image sensor counterparts.

In the various examples disclosed herein, a hybrid optical sensor is provided in which the optical sensor includes a combination of a normal image/video sensing that provides great image and video capture capabilities, as well as simultaneously sense events at ultra-high frame rates and at ultra-high speeds from pixel cells in the same columns of the pixel array for a wide variety of event driven applications.

To illustrate,FIG. 1illustrates one example of stacked complementary metal oxide semiconductor (CMOS) image sensor (CIS) system100in accordance with the teachings of the present invention. As shown in the depicted example, stacked CIS system100includes a first die102, a second die,104, and a third die106that are stacked and coupled together in a stacked chip scheme. In various examples, the first die102, second die104, and third die106are semiconductor dice that include a suitable semiconductor material such as for example silicon. In the example, the first die102, which may also be referred to as the top die102of the stacked CIS100, includes a pixel array108. The third die106, which may also be referred to as the bottom die106of the stacked CIS100, includes an image readout circuit116, which may also be referred to as image readout mixed-signal circuitry, is coupled to the pixel array108of the top die102through column level connections for normal image readout110. In one example, the column level connections for normal image readout110may be implemented from column bitlines of the pixel array108with through silicon vias (TSVs) between the top die102and the bottom die106, which are routed through the second die104.

In one example, pixel array108is a two-dimensional (2D) array including a plurality of pixel cells that include photodiodes that are exposed to incident light. As will be discussed in greater detail below, in various examples, each of the pixel cells may include a plurality of photodiodes arranged for example in a 4C (2×2) arrangement that share pixel circuitry of the pixel cell. As illustrated in the depicted example, the pixel cells are arranged into rows and columns to acquire image data of a person, place, object, etc., which can then be used to render images and/or video of a person, place, object, etc. In the example, each pixel cell is configured to photogenerate image charge in response to the incident light. After each pixel cell has acquired its image charge, the corresponding analog image charge data is read out by the image readout circuit116in the bottom die106through column bit lines, which may be implemented with through silicon vias (TSVs) included in the column level connections for normal image readout110. In the various examples, the image charge from each row of pixel array108may be read out in parallel through column bit lines by image readout circuit116.

In the various examples, the image readout circuit116in the bottom die106includes amplifiers, analog to digital converter (ADC) circuitry, associated analog support circuitry, associated digital support circuitry, etc., for normal image readout and processing. In some examples, image readout circuit116may also include event driven readout circuitry, which will be described in greater detail below. In operation, the photogenerated analog image charge signals are read out from the pixel cells of pixel array108, amplified, and converted to digital values in image readout circuit116. In some examples, image readout circuit116may readout a row of image data at a time. In other examples, image readout circuit116may readout the image data using a variety of other techniques (not illustrated), such as a serial readout or a full parallel readout of all pixels simultaneously. The image data may be stored or even manipulated by applying post image effects (e.g., crop, rotate, remove red eye, adjust brightness, adjust contrast, or otherwise).

In the depicted example, the second die104, which may also be referred to as the middle die104of the stacked CIS100, includes an event driven sensing array112that is coupled to the pixel array108in the top die102. In the various examples, the event driven sensing array112is coupled to the pixel cells of pixel array108through hybrid bonds between the top die102and the middle die104. In one example, the event driven sensing array112includes an array of event driven circuits. As will be discussed, in one example, each one of the event driven circuits in event driven sensing array112is coupled to a plurality of pixel cells of the pixel cells in pixel array108through hybrid bonds between the top die102and the middle die104to asynchronously detect events that occur in the light that is incident upon the pixel array108in accordance with the teachings of the present invention.

In one example, each one of the event driven circuits in event driven sensing array112is coupled to 8 pixel cells included in two rows of the pixel array108. In the various examples, corresponding event detection signals are generated by the event driven circuits in the event driven sensing array112. The event detection signals may be coupled to be received and processed by event driven peripheral circuitry114, which in one example is arranged around the periphery of event driven sensing array112in the middle die104as shown inFIG. 1. The depicted example also illustrates the column level connections for normal image readout110that are routed through middle die104between the top die102and the bottom die106.

FIG. 2Aillustrates one example of a pixel array208in accordance with the teachings of the present invention. It is appreciated that the pixel array208ofFIG. 2Amay be an example implementation of pixel array108of the example stacked CIS system100as shown inFIG. 1, and that similarly named and numbered elements described above are coupled and function similarly below.

As shown in the depicted example, pixel array208includes a plurality of photodiodes218. As illustrated in the depicted example, photodiodes218are arranged into rows (e.g., 1, 2, 3, . . . , Ry) and columns (e.g., 1, 2, 3, . . . , Cx). In one example, pixel array208is a 64 megapixel array with X=9,248 columns and Y=6,944 rows. In other examples, it is appreciated the pixel array208may have different dimensions.

The example depicted inFIG. 2Aalso illustrates that pixel array208is a color pixel array with a Bayer color filter pattern including red (R), green (G), and blue (B) color filters. In the example, Bayer binning is provided with 4C (2×2) groupings of red (R) color filters, 4C (2×2) groupings of green (G) color filters, and 4C (2×2) groupings of blue (B) color filters, which are disposed over respective photodiodes218. In various examples, it is appreciated that pixel array208ofFIG. 2Amay be utilized for normal image/video capture in a variety of resolutions including for instance full resolution image captures up to 64 megapixels, as well as video formats including 4K, 1080p, 720p, etc.

For instance,FIG. 2Billustrates one example of a readout of a pixel array208with 4C (2×2) photodiode binning to capture a 16 megapixel image/video in accordance with the teachings of the present invention. As shown, each 4C (2×2) grouping of photodiodes218of the same color (e.g., red (R), green (G), blue (B)) may be read out together with Bayer binning. In particular, the charge information photogenerated from the four photodiodes218of each 4C (2×2) grouping of the same color are combined together when read out of the pixel array208. As will be illustrated below, in one example, the four photodiodes218of each 4C (2×2) grouping of photodiodes218share a floating diffusion, reset transistor, source follower, and row select transistor of the pixel circuitry of a pixel cell, which are then read out from pixel array208to capture a 16 megapixel image/video from pixel array208.

FIG. 2Cillustrates one example of a readout of a pixel array208with 2× fast vertical binning and 2× digital horizontal binning of 4C (2×2) photodiode groupings in accordance with the teachings of the present invention. As shown in the depicted example, four 4C (2×2) groupings of photodiodes218of the same color (e.g., red (R), green (G), blue (B)) may be read out together with Bayer binning. In the example, 2× fast vertical binning is performed on each pair of 4C (2×2) groupings of photodiodes218of the same color that are coupled to the same column bitline to perform the vertical binning. Furthermore, 2× digital horizontal binning is performed on each pair of 4C (2×2) groupings of photodiodes218of the same color in the same row. In one example, the 2× digital horizontal binning may be performed after the analog to digital conversion (ADC) is performed in the image readout circuit. With the 2× fast vertical binning and the 2× digital horizontal binning of the four 4C (2×2) groupings of photodiodes218of the same color, a 4 megapixel image/video may be captured from pixel array208.

FIG. 2Dillustrates one example of a simultaneous readout of a pixel array with 2× digital horizontal binning of 4C (2×2) photodiode groupings and multiple rows of the pixel array for event driven sensing in accordance with the teachings of the present invention. As shown in the depicted example, 2× digital horizontal binning is performed on a pair of 4C (2×2) groupings of photodiodes218of the same color in the same row in a first portion (e.g., the upper 4 rows of photodiodes or upper 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array shown inFIG. 2D.

However, in the depicted example, no 2× fast vertical binning is performed on pairs of 4C (2×2) groupings of photodiodes218of the same color that share the same column bitline. Instead, in the depicted example, every 4 rows of photodiodes, or every 2 rows of 4C (2×2) groupings of photodiodes, are disconnected from the image readout circuit. Rather, the plurality of pixel cells in a second portion (e.g., lower 4 rows of photodiodes or lower 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208, which are disconnected from the image readout circuit, are read out simultaneously by event driven circuitry for event driven sensing in accordance with the teachings of the present invention. In the lower portion or second portion from which event driven events are detected, the row control buffers are gated (e.g., enabled/disabled) to support this reconfiguration in accordance with the teachings of the present invention.

As will be discussed, in the depicted example, pixel cells that are read out for event driven sensing include pixel cells that are in the same columns as pixel cells that are read out simultaneously by the image readout circuit in accordance with the teachings of the present invention. In the example depicted inFIG. 2D, the second portion of the pixel array208that is read out by the event driven circuit for event driven sensing includes two rows or 8 pixel cells. The event driven sensing of the 4C (2×2) groupings of photodiodes from these two rows of the pixel array208is equivalent to providing a 1 megapixel sensor for ultra-high frame rate and ultra-high speed event driven sensing in accordance with the teachings of the present invention.

To illustrate,FIG. 3shows one example schematic of a stacked CIS system300with a pixel array including pixel cells308that can be read out with an image readout circuit316and an event driven circuit366to simultaneously capture images/video and detect events from the same column of the pixel array in accordance with the teachings of the present disclosure. It is appreciated the stacked CIS system300ofFIG. 3may be one example of the stacked CIS system100as shown inFIG. 1, and that similarly named and numbered elements described above are coupled and function similarly below.

In the example depicted inFIG. 3, a grouping of N pixel cells308<1>-308<N> of the pixel array that are included in top die302is illustrated. In one example, the grouping of N pixel cells308<1>-308<N> shown inFIG. 3may correspond to the first portion (e.g., upper portion including 4 rows of photodiodes or 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208ofFIG. 2D, or the second portion (e.g., lower portion including 4 rows of photodiodes or 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208shown inFIG. 2Dabove. In the example, N=8 pixel cells308<1>-308<N> across two rows of 4C (2×2) pixel cells the pixel array208ofFIG. 2D. It is appreciated that in other examples, N may be equal to a different number.

As shown in the example depicted inFIG. 3, each pixel cell308<1>-308<N> is a shared pixel design that includes photodiodes318-1,318-2,318-3,318-4that are configured to photogenerate charge in response to incident light362. Thus, in one example, the four photodiodes318-1,318-2,318-3,318-4may be arranged as a 4C (2×2) arrangement of photodiodes in a pixel array as shown inFIGS. 2A-2D. In other examples, it is appreciated that a different number of photodiodes may be included.

In the example illustrated inFIG. 3, transfer transistors320-1,320-2,320-3,320-4are coupled to the photodiodes318-1,318-2,318-3,318-4, respectively. A floating diffusion322is coupled to the photodiodes318-1,318-2,318-3,318-4through transfer transistors320-1,320-2,320-3,320-4, respectively, to receive the photogenerated charge from photodiodes318-1,318-2,318-3,318-4. A gate of source follower transistor326is coupled to the floating diffusion322to generate an image data signal in response to the charge in the floating diffusion322that is photogenerated by the photodiodes318-1,318-2,318-3,318-4. In one example, the image data signal includes a current IIMAGE334that is coupled to be read out through a row select transistor328and received by image readout circuitry316in the bottom die306through a hybrid bond365from top die302to second die304, which is then routed through a through silicon via (TSV)310through the second die304to the bottom die306as shown.

In the example depicted inFIG. 3, a reset transistor324is coupled to floating diffusion322. In one example, reset transistor324is coupled to reset the photogenerated charge in pixel cell308in a normal image readout mode. For instance, in one example, reset transistor324is coupled to reset the charge in the floating diffusion322. In one example, reset transistor324is also coupled to photodiodes318-1,318-2,318-3,318-4through transfer transistors320-1,320-2,320-3,320-4to reset the charge in photodiodes318-1,318-2,318-3,318-4.

It is noted that the drain of the reset transistor324is not coupled to a pixel supply voltage in top die302. Instead, as shown in the illustrated example, the drain of reset transistor324in top die302is coupled through a hybrid bond364to a mode select switch330disposed in the second die304in accordance with the teachings of the present invention. In the example, the mode select switch330is configured when turned ON to couple the drain of the reset transistor324to a voltage supply in response to a mode select signal332.

In one example, the three readout transistors including reset transistor324, source follower transistor326, and row select transistor328may be isolated from the other devices included in the pixel cell308with an isolation structure370, which may include for example a full deep trench isolation (DTI) structure or the like. In addition, in various examples, each pixel cell308is isolated from each adjacent neighboring pixel cell308with isolation structures such as full deep trench isolation (DTI) structures or the like.

As mentioned, the pixel cell308example illustrated inFIG. 3is one of the plurality of N pixel cells308<1>-308<N> that are included in the pixel array. Thus, in one example, when the mode select switch330is turned ON in a normal image readout mode, the mode select switch330is configured to couple the drain of the reset transistor324of each the plurality of N pixel cells308<1>-308<N> to the voltage supply through one or more hybrid bonds364in response to the mode select signal332in accordance with the teachings of the present invention.

It is appreciated that when the mode select switch330is configured to couple the voltage supply to the drain of the reset transistors324, the plurality of N pixel cells308<1>-308<N> are configured to operate in a normal imaging mode. When configured in the normal imaging mode, the image readout circuit316can be used to capture images or video from the plurality of N pixel cells308<1>-308<N> in accordance with the teachings of the present invention. This example is described with respect the first portion (e.g., the upper 4 rows of photodiodes or upper 2 rows of 4C (2×2) pixel cells) of the pixel array208inFIG. 2Dabove from which images or video may be captured from the pixel array208by the image readout circuit in accordance with the teachings of the present invention.

Continuing with the example depicted inFIG. 3, when the mode select switch330is turned OFF in response to the mode select signal332, the plurality of N pixel cells308<1>-308<N> are configured to operate in an event driven sensing mode. This example is described with respect the second portion (e.g., the lower 4 rows of photodiodes or lower 2 rows of 4C (2×2) pixel cells) of the pixel array208inFIG. 2Dabove from which event data signals may be read out from the pixel array208through the drain of the reset transistor324with an event driven circuit in accordance with the teachings of the present invention.

To illustrate, during the event driven mode, the mode select switch330is configured to be turned OFF in response to the mode select signal332. As such, the voltage supply included in second die304is disconnected from the drain of reset transistor324. Instead, an event driven circuit366included in second die304is coupled to read out the event data signals, which are included in a photocurrent IEVENT336generated by the photodiodes318-1,318-2,318-3,318-4, through hybrid bonds364and the reset transistor324.

In the example depicted inFIG. 3, the event driven circuit366includes a converter circuit340coupled to the drain of reset transistor324to convert a photocurrent IEVENT336of the event data signal to a voltage. The voltage buffer circuit341is coupled to the converter circuit340to buffer the output voltage signal of the converter circuit340. A comparator and handshake circuit343is coupled to the voltage buffer circuit341to generate an event driven output signal368in response to the output of the voltage buffer circuit341.

As shown in the example ofFIG. 3, the converter circuit340includes a first transistor372having a source coupled to the reset transistor324through hybrid bond364. First transistor372also includes a drain coupled to the voltage supply. A second transistor374has a gate coupled to the source of the first transistor372and the drain of the reset transistor through the hybrid bond364. The second transistor also has a source coupled to ground. A third transistor376has a drain coupled to a gate of the first transistor372and a first current source378. Third transistor376also has a source coupled to a drain of the second transistor374. In the depicted example, the voltage buffer circuit341includes a fourth transistor380having a gate coupled to the drain of the third transistor376and the gate of the first transistor372. The fourth transistor380also includes a source coupled to a second current source382.

In operation, the transfer transistors320-1,320-2,320-3,320-4and the reset transistor324are switched ON and row select transistor328is switched OFF during the event driven mode. As such, incident light362that is incident on photodiodes318-1,318-2,318-3,318-4photogenerates charge, which results in a photocurrent IEVENT336that flows through photodiodes318-1,318-2,318-3,318-4, transfer transistors320-1,320-2,320-3,320-4, reset transistor324, and event driven circuitry366during the event driven mode. If the external scene is static and thus there is no event occurring, the brightness of incident light362remains unchanged. As such, the photocurrent IEVENT336generated by photodiode318-1,318-2,318-3,318-4remains substantially constant. However, if an event occurs (e.g., movement, etc.) in the external scene, the event is indicated with an asynchronous change in the brightness of incident light362. As such, there is an asynchronous change or delta in the photocurrent IEVENT336generated by photodiodes318-1,318-2,318-3,318-4. Changes in the photocurrent IEVENT336are detected by the event driven circuit366as event data, which is indicated in the event driven output signal368generated by the comparator and handshake circuit343in accordance with the teachings of the present invention.

FIG. 4illustrates another example schematic of a stacked CIS system400with a pixel array including pixel cells408that can be read out with an image readout circuit416and an event driven circuit466to simultaneously capture images/video and detect events from the same column of the pixel array in accordance with the teachings of the present disclosure. It is appreciated the stacked CIS system400ofFIG. 4may be another example of the stacked CIS system300as shown inFIG. 3, or of stacked CIS system100as shown inFIG. 1, and that similarly named and numbered elements described above are coupled and function similarly below.

It is appreciated the stacked CIS system400ofFIG. 4shares many similarities with the stacked CIS system300as shown inFIG. 3. For instance, as shown in the example depicted inFIG. 4, a grouping of N pixel cells408<1>-408<N> of the pixel array that are included in top die402is illustrated. In one example, the grouping of N pixel cells408<1>-408<N> shown inFIG. 4may also correspond to the first portion (e.g., upper portion including 4 rows of photodiodes or 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208ofFIG. 2D, or the second portion (e.g., lower portion including 4 rows of photodiodes or 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208shown inFIG. 2Dabove. In the example, N=8 pixel cells408<1>-408<N> across two rows of 4C (2×2) pixel cells the pixel array208ofFIG. 2D. It is appreciated that in other examples, N may be equal to a different number.

As shown in the example depicted inFIG. 4, each pixel cell408<1>-408<N> is a shared pixel design that includes photodiodes418-1,418-2,418-3,418-4that are configured to photogenerate charge in response to incident light462. Thus, in one example, the four photodiodes418-1,418-2,418-3,418-4may be arranged as a 4C (2×2) arrangement of photodiodes in a pixel array as shown inFIGS. 2A-2D. In other examples, it is appreciated that a different number of photodiodes may be included.

In the example illustrated inFIG. 4, transfer transistors420-1,420-2,420-3,420-4are coupled to the photodiodes418-1,418-2,418-3,418-4, respectively. A floating diffusion422is coupled to the photodiodes418-1,418-2,418-3,418-4through transfer transistors420-1,420-2,420-3,420-4, respectively, to receive the photogenerated charge from photodiodes418-1,418-2,418-3,418-4. A gate of source follower transistor426is coupled to the floating diffusion422to generate an image data signal in response to the charge in the floating diffusion422that is photogenerated by the photodiodes418-1,418-2,418-3,418-4. In one example, the image data signal includes a current IIMAGE434that is coupled to be read out through a row select transistor428and received by image readout circuitry416in the bottom die406through a hybrid bond465from top die402to second die404, which is then routed through a through silicon via (TSV)410through the second die404to the bottom die406as shown.

In the example depicted inFIG. 4, a reset transistor424is coupled to floating diffusion422. One of the differences between the stacked CIS system400ofFIG. 4and the stacked CIS system300as shown inFIG. 3is that in the stacked CIS system400ofFIG. 4, the drain of reset transistor424is also coupled to a supply voltage in top die402to reset the photogenerated charge in pixel cell408in a normal image readout mode. For instance, in the example, reset transistor424is coupled to reset the charge in the floating diffusion422. In one example, reset transistor424is also coupled to photodiodes418-1,418-2,418-3,418-4through transfer transistors420-1,420-2,420-3,420-4to reset the charge in photodiodes418-1,418-2,418-3,418-4.

In one example, the three readout transistors including reset transistor424, source follower transistor426, and row select transistor428are isolated from the other devices included in the pixel cell408with an isolation structure470, which may include for example a full deep trench isolation (DTI) structure or the like. As such, the p doped regions of the photodiodes418-1,418-2,418-3,418-4as well as the transfer transistors420-1,420-2,420-3,420-4of the first die402are isolated from the reset transistor424, source follower transistor426, and row select transistor428. In addition, in various examples, each pixel cell408is isolated from each adjacent neighboring pixel cell408with isolation structures such as full deep trench isolation (DTI) structures or the like.

In various examples, the three readout transistors including reset transistor424, source follower transistor426, and row select transistor428may also be formed by devices that have native isolated body contact such as for example fin field effect transistors (FinFETs), gate-all-around (nanowire) FETs, silicon-on-insulator (SOI) FETs, transistors integrated into the back-end-of-line (BEOL) transistors such as for example Indium-Gallium-Zinc-Oxide (IGZO) transistors, or transistors integrated on a separate wafer, by means of three dimensional 3D integration via hybrid bonds or through silicon via (TSV) technologies, or other suitable structures that provide suitable isolation to isolate the pixel cells408and associated currents from neighboring pixel cells.

Continuing with the example depicted inFIG. 4, event data may be read out from each pixel cell408<1>-408<N> by event driven circuit466in second die404through hybrid bonds464when configured to operate in an event driven sensing mode. This example is described with respect the second portion (e.g., the lower 4 rows of photodiodes or lower 2 rows of 4C (2×2) pixel cells) of the pixel array208inFIG. 2Dabove from which event data signals may be read out from the pixel array208with an event driven circuit in accordance with the teachings of the present invention.

In the example depicted inFIG. 4, the event driven circuit466includes a current amplifier484having an input coupled to an anode of each one of the plurality of photodiodes418-1,418-2,418-3,418-4to receive a photocurrent IEVENT436of the event data signal from the plurality of photodiodes418-1,418-2,418-3,418-4. In one example, the anode of each one of the plurality of photodiodes418-1,418-2,418-3,418-4is in a p doped region of the first die402in which the pixel cells408are disposed. A converter circuit440is coupled to the current amplifier484to convert an output of the current amplifier484to a voltage. A voltage buffer circuit441is coupled to the converter circuit440to buffer an output of the converter circuit440. A comparator and handshake circuit443is coupled to the voltage buffer circuit441to generate an event driven output signal in response to an output of the voltage buffer circuit441to generate an event driven output signal468in response to the output of the voltage buffer circuit441.

In the depicted example, the converter circuit440includes a first transistor472having a source coupled to the output of the current amplifier484. The first transistor472also includes a drain coupled to a voltage supply. A second transistor474includes a gate coupled to the source of the first transistor472and the output of the current amplifier484. The second transistor474also includes a source coupled to ground. A third transistor476includes a drain coupled to a gate of the first transistor472and a first current source478. Third transistor476also includes a source coupled to a drain of the second transistor474. The voltage buffer circuit441includes a fourth transistor480that includes a gate coupled to the drain of the third transistor476and the gate of the first transistor472. The fourth transistor480also includes a source that is coupled to a second current source482.

Continuing with the example depicted inFIG. 4, the current amplifier484includes a fifth transistor488having a drain that is coupled to the anode of each one of the plurality of photodiodes418-1,418-2,418-3,418-4through hybrid bonds464to receive the photocurrent IEVENT436of the event data signal from the plurality of photodiodes418-1,418-2,418-3,418-4. Fifth transistor488also includes a source coupled to a VSS voltage reference. In various examples, the VSS voltage reference may be a suitable voltage reference value such as for example ground (e.g., zero volts), a positive voltage, or a negative voltage. A sixth transistor490includes a gate that is coupled to a gate of the fifth transistor488. The sixth transistor490also includes a source coupled to the VSS voltage reference. The sixth transistor490further includes a drain that is coupled to the output of the current amplifier484.

A first opamp486includes a first input that is coupled to a reference voltage. In one example, the reference voltage is substantially equal to ground. The first opamp486also includes a second input that is coupled to the drain of the fifth transistor488and the anode of each one of the plurality of photodiodes418-1,418-2,418-3,418-4through hybrid bonds464. The first opamp486further includes an output that is coupled to gate of the fifth transistor and the gate of the sixth transistor. During operation, the voltage difference between the first and second inputs of opamp486is small. As such, when the reference voltage coupled to the first input of opamp486is substantially equal to ground, the second input of opamp486is substantially equal to ground such that the anodes of the plurality of photodiodes418-1,418-2,418-3,418-4are, in effect, coupled to a virtual ground provided by the second input of opamp486.

In operation, the incident light462that is incident on photodiodes418-1,418-2,418-3,418-4photogenerates charge, which results in a photocurrent IEVENT436that flows through photodiodes418-1,418-2,418-3,418-4and event driven circuitry466during the event driven mode. In the depicted example, photocurrent IEVENT436is a hole current that is generated at the anodes of photodiodes418-1,418-2,418-3,418-4in response to the incident light462from the external scene. If the external scene is static and thus there is no event occurring, the brightness of incident light462remains unchanged. As such, the photocurrent IEVENT436generated by photodiodes418-1,418-2,418-3,418-4remains constant. However, if an event occurs (e.g., movement, etc.) in the external scene, the event is indicated with an asynchronous change in the brightness of incident light462. As such, there is an asynchronous change or delta in the photocurrent IEVENT436generated by photodiodes418-1,418-2,418-3,418-4. Changes in the photocurrent IEVENT436are detected by the event driven circuit466as event data, which is indicated in the event driven output signal468generated by the comparator and handshake circuit443in accordance with the teachings of the present invention.

FIG. 5illustrates yet another example schematic of a stacked CIS system500with a pixel array508that can be read out with an image readout circuit516and an event driven circuit566to simultaneously capture images/video and detect events from the same column in accordance with the teachings of the present disclosure. It is appreciated the stacked CIS system500ofFIG. 5may be another example of the stacked CIS system400as shown inFIG. 4, or of stacked CIS system300as shown inFIG. 3, or of stacked CIS system100as shown inFIG. 1, and that similarly named and numbered elements described above are coupled and function similarly below.

It is appreciated the stacked CIS system500ofFIG. 5shares many similarities with the stacked CIS system400as shown inFIG. 4. For instance, as shown in the example depicted inFIG. 5, a grouping of N pixel cells508<1>-508<N> of the pixel array that are included in top die502is illustrated. In one example, the grouping of N pixel cells508<1>-508<N> shown inFIG. 5may also correspond to the first portion (e.g., upper portion including 4 rows of photodiodes or 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208ofFIG. 2D, or the second portion (e.g., lower portion including 4 rows of photodiodes or 2 rows of 4C (2×2) groupings of photodiodes) of the pixel array208shown inFIG. 2Dabove. In the example, N=8 pixel cells508<1>-508<N> across two rows of 4C (2×2) pixel cells the pixel array208ofFIG. 2D. It is appreciated that in other examples, N may be equal to a different number.

As shown in the example depicted inFIG. 5, each pixel cell508<1>-508<N> is a shared pixel design that includes photodiodes518-1,518-2,518-3,518-4that are configured to photogenerate charge in response to incident light562. Thus, in one example, the four photodiodes518-1,518-2,518-3,518-4may be arranged as a 4C (2×2) arrangement of photodiodes in a pixel array as shown inFIGS. 2A-2D. In other examples, it is appreciated that a different number of photodiodes may be included.

In the example illustrated inFIG. 5, transfer transistors520-1,520-2,520-3,520-4are coupled to the photodiodes518-1,518-2,518-3,518-4, respectively. A floating diffusion522is coupled to the photodiodes518-1,518-2,518-3,518-4through transfer transistors520-1,520-2,520-3,520-4, respectively, to receive the photogenerated charge from photodiodes518-1,518-2,518-3,518-4. A gate of source follower transistor526is coupled to the floating diffusion522to generate an image data signal in response to the charge in the floating diffusion522that is photogenerated by the photodiodes518-1,518-2,518-3,518-4. In one example, the image data signal includes a current IIMAGE534that is coupled to be read out through a row select transistor528and received by image readout circuitry516in the bottom die506through a hybrid bond565from top die502to second die504, which is then routed through a through silicon via (TSV)510through the second die504to the bottom die506as shown.

In the example depicted inFIG. 5, a reset transistor524is coupled to floating diffusion522and also to a supply voltage in top die502to reset the photogenerated charge in pixel cell508in a normal image readout mode. For instance, in the example, reset transistor524is coupled to reset the charge in the floating diffusion522. In one example, reset transistor524is also coupled to photodiodes518-1,518-2,518-3,518-4through transfer transistors520-1,520-2,520-3,520-4to reset the charge in photodiodes518-1,518-2,518-3,518-4.

In one example, the three readout transistors including reset transistor524, source follower transistor526, and row select transistor528are isolated from the other devices included in the pixel cell508with an isolation structure570, which may include for example a full deep trench isolation (DTI) structure or the like. As such, the p doped regions of the photodiodes518-1,518-2,518-3,518-4as well as the transfer transistors520-1,520-2,520-3,520-4of the first die502are isolated from the reset transistor524, source follower transistor526, and row select transistor528. In addition, in various examples, each pixel cell508is isolated from each adjacent neighboring pixel cell508with isolation structures such as full deep trench isolation (DTI) structures or the like.

In various examples, the three readout transistors including reset transistor524, source follower transistor526, and row select transistor528may also be formed by devices that have native isolated body contact such as for example fin field effect transistors (FinFETs), gate-all-around (nanowire) FETs, silicon-on-insulator (SOI) FETs, transistors integrated into the back-end-of-line (BEOL) transistors such as for example Indium-Gallium-Zinc-Oxide (IGZO) transistors, or transistors integrated on a separate wafer, by means of three dimensional 3D integration via hybrid bonds or through silicon via (TSV) technologies, or other suitable structures that provide suitable isolation to isolate the pixel cells508and associated currents from neighboring pixel cells.

Continuing with the example depicted inFIG. 5, event data may be read out from each pixel cell508<1>-508<N> by event driven circuit566in second die504through hybrid bonds564when configured to operate in an event driven sensing mode. This example is described with respect the second portion (e.g., the lower 4 rows of photodiodes or lower 2 rows of 4C (2×2) pixel cells) of the pixel array208inFIG. 2Dabove from which event data signals may be read out from the pixel array208with an event driven circuit in accordance with the teachings of the present invention.

In the example depicted inFIG. 5, the event driven circuit566includes a current amplifier584having an input coupled to an anode of each one of the plurality of photodiodes518-1,518-2,518-3,518-4through hybrid bonds564to receive a photocurrent IEVENT536of the event data signal from the plurality of photodiodes518-1,518-2,518-3,518-4in response to the incident light562from the incident scene. In one example, the anode of each one of the plurality of photodiodes518-1,518-2,518-3,518-4is p doped region of the first die502in which pixel cells508are disposed. An integrator542is coupled to the current amplifier584to integrate an output of the current amplifier584. It is appreciated that the example integrator542illustrated inFIG. 5is a design example provided for explanation purposes, and may perform additional suitable functions such as for example but not limited to filtering, amplification, etc. A comparator and handshake circuit543, which includes the integrator542in the example depicted inFIG. 5, is configured to generate an event driven output signal568, which in the depicted example includes an On events signal568A and an Off events signal568B, in response to an output of the integrator542.

In the example depicted inFIG. 5, an optional mode select switch530is also included in second die504as shown. In the example, the mode select switch is coupled to the anode of each one of the plurality of photodiodes518-1,518-2,518-3,518-4through hybrid bonds564, the input of the current amplifier584through a node “E”, and ground through a node “I.” In the example illustrated inFIG. 5, the “E” label represents an event driven sensing mode and the “I” label represents a normal imaging mode for stacked CIS system500. In operation, the optional mode select switch530is configured to couple of the anode of each one the plurality of photodiodes518-1,518-2,518-3,518-4either to ground or to the input of the current amplifier584. As will be discussed, in an example in which optional mode select switch530is not included, the plurality of photodiodes518-1,518-2,518-3,518-4are coupled to a virtual ground provided by a second input of opamp586of current amplifier584.

As shown in the example ofFIG. 5, the current amplifier584includes a fifth transistor588that includes a source to be coupled to the anode of each one of the plurality of photodiodes518-1,518-2,518-3,518-4to receive the photocurrent IEVENT536of the event data signal from the plurality of photodiodes518-1,518-2,518-3,518-4generated in response to the incident light562from the external scene. The fifth transistor588also includes a drain coupled to a voltage supply NVDD. A sixth transistor590includes a gate that is coupled to a gate of the fifth transistor588. The sixth transistor590also includes a drain coupled to the voltage supply NVDD. The sixth transistor further includes a source that is coupled to a third current source592and the output of the current amplifier584. In the depicted example, the fifth transistor588and the sixth transistor590are PMOS transistors.

As shown in the depicted example, a first opamp586is also included, which includes a first input that is coupled to a reference voltage. In one example, the reference voltage is substantially equal to ground. The first opamp586also includes a second input that is coupled to the drain of the fifth transistor588and the anode of each one of the plurality of photodiodes518-1,518-2,518-3,518-4. The first opamp586further includes an output that is coupled to the gate of the fifth transistor588and the gate of the sixth transistor590. During operation, the voltage difference between the first and second inputs of opamp586is small. As such, when the reference voltage coupled to the first input of opamp586is substantially equal to ground, the second input of opamp586is substantially equal to ground such that the anodes of the plurality of photodiodes518-1,518-2,518-3,518-4are, in effect, coupled to a virtual ground provided by the second input of opamp586.

In the example illustrated inFIG. 5, the integrator542includes a second opamp548including an input that is capacitively coupled through a capacitor C1to the output of the current amplifier584. The integrator542also includes a capacitor C2that is coupled between the input of second opamp548and an output of second opamp548. A reset switch550is also coupled between the input of second opamp548and an output of second opamp548.

As shown in the depicted example, the comparator and handshake circuit543also includes a first threshold detection circuit552that is coupled to the output of the second opamp548to generate a first threshold detection output signal568A in response to the output of the second opamp548. The first threshold detection output signal568A is labeled “On events” inFIG. 5. The comparator and handshake circuit543also includes a second threshold detection circuit554that is coupled to the output of the second opamp548to generate a second threshold detection output signal568B in response to the output of the second opamp548. The second threshold detection output signal568B is labeled “Off events” inFIG. 5. In the illustrated example, it is appreciated that the first and second threshold detection output signals568A and568B provide an event driven output signal of the comparator and handshake circuit543.

In the depicted example, the comparator and handshake circuit543also includes a handshake protocol circuit594that is coupled to the first threshold detection circuit552and the second threshold detection circuit554to control switching of the reset switch550in response to the first threshold detection output signal568A and the second threshold detection output signal568B. In the depicted example, the handshake protocol circuit594is further configured to receive an acknowledge signal (Ack) and generate a request signal (Req) as shown.

In operation, the incident light562that is incident on photodiodes518-1,518-2,518-3,518-4photogenerates charge, which results in a photocurrent IEVENT536that flows through photodiodes518-1,518-2,518-3,518-4and event driven circuitry566during the event driven mode. In the depicted example, photocurrent IEVENT536is a hole current that is generated at the anodes of photodiodes518-1,518-2,518-3,518-4in response to incident light562from the external scene. If the external scene is static and thus there is no event occurring, the brightness of incident light562remains unchanged. As such, the photocurrent IEVENT536generated by photodiodes518-1,518-2,518-3,518-4remains constant. However, if an event occurs (e.g., movement, etc.) in the external scene, the event is indicated with an asynchronous change in the brightness of incident light562. As such, there is an asynchronous change or delta in the photocurrent IEVENT536generated by photodiodes518-1,518-2,518-3,518-4. Changes in the photocurrent IEVENT536are detected by the event driven circuit566as event data, which is indicated in the first threshold detection output signal568A and the second threshold detection output signal568B generated by the comparator and handshake circuit543in accordance with the teachings of the present invention.