Patent Publication Number: US-2017366772-A1

Title: Methods and apparatus for a multiple storage pixel imaging system

Description:
BACKGROUND OF THE TECHNOLOGY 
     Multi-storage node pixels may be beneficial in high dynamic range (HDR) imaging, which may include overlapped-exposure imaging techniques. In overlapped-exposure HDR imaging, multiple images are captured with an image sensor over the same time period but with different exposure periods, and the images are later combined into a high dynamic range image. 
     Images captured at different times for the purpose of HDR imaging may result in distorted images for moving objects. Capturing images in an overlapped manner reduces the distortion as multiple image captures receive the same motion at the same time. 
     Rolling shutter architectures also result in image distortion for moving objects since images are captured in a sequential row-by-row fashion. As such, a global shutter design may be preferred to minimize the motion distortion associated with rolling shutter circuits, since all pixels in a global shutter imager integrate light simultaneously. 
     Conventional global shutter pixels capable of correlated doubled sampling have at least one photo sensitive element, a dedicated in-pixel storage region, and a floating diffusion region. The pixels operate by transferring charge to the storage region after integration, resetting the floating diffusion region prior to readout, and then reading out the charge from the storage region to the floating diffusion region for sampling. As such, the floating diffusion region is utilized only during the readout period. 
     Creating a HDR image utilizing a global shutter design with pixels formed with multiple storage regions results in larger pixels, or for a given pixel size, reduces the area for the photodiode, thus decreasing sensitivity. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWING FIGURES 
       A more complete understanding of the present technology may be derived by referring to the detailed description when considered in connection with the following illustrative figures. In the following figures, like reference numbers refer to similar elements and steps throughout the figures. 
         FIG. 1  representatively illustrates a multi-storage pixel in accordance with an exemplary embodiment of the present technology; 
         FIG. 2  representatively illustrates a timing diagram in accordance with an exemplary embodiment of the present technology; 
         FIG. 3  representatively illustrates an image sensor in accordance with an exemplary embodiment of the present technology; and 
         FIG. 4  representatively illustrates an imaging system in accordance with an exemplary embodiment of the present technology. 
     
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
     The present technology may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of components configured to perform the specified functions and achieve the various results. For example, the present technology may employ various semiconductor devices, such as switches, capacitors, photosensors, and the like, which may carry out a variety of functions. In addition, the present technology may be practiced in conjunction with any number of imaging systems and the apparatus and method embodiments described are merely exemplary applications for the technology. Further, the present technology may employ any number of conventional techniques for capturing image data, sampling image data, readout of image data, and the like. 
     Methods and apparatus for a multi-storage node pixel  100  ( FIG. 1 ) imaging system according to various aspects of the present technology may operate in conjunction with any suitable imaging system, such as a camera system, video system, machine vision, vehicle navigation, surveillance system, motion detection system, image stabilization system, and the like. For example, referring to  FIG. 4 , the imaging system may comprise an electronic device, such as a digital camera  400 . The system may comprise a central processing unit (CPU)  405  that communicates with various devices over a bus  410 . Some of the devices connected to the bus  410  may provide communication into and out of the system, for example an input/output (I/O) device  415 . Other devices connected to the bus  410  provide memory, for example, a random access memory (RAM)  420 , hard drive, and one or more peripheral memory devices  425 , such as a floppy disk drive, compact disk (CD) drive, USB drives, memory cards, and SD cards. While the bus  410  is illustrated as a single bus, any number of busses may be used to provide communication paths to interconnect the devices. 
     The imaging system may further comprise an imaging sensor for capturing image data. For example, light may enter the camera through a lens  430  and strike the image sensor. The image sensor may detect and convey the information that constitutes an image, for example by converting the variable attenuation of waves (as they pass through or reflect off objects) into electronic signals. The image sensor may be implemented in conjunction with any appropriate technology, such as using semiconductor charge-coupled devices (CCD), active pixel sensors in complementary metal-oxide-semiconductors (CMOS) or N-type metal-oxide-semiconductors (NMOS), analog sensors, and/or flat panel detectors. 
     Referring to  FIG. 3 , an exemplary image sensor  300  may comprise a pixel array  305  comprising a plurality of multi-storage pixels  100  ( FIG. 1 ) arranged in rows and columns. The pixels  100  may be electrically connected via metal wirings or other suitable connections. The image sensor  300  may be formed in silicon using any suitable complementary metal-oxide semiconductor (CMOS) technology and/or fabrication process. 
     In various embodiments, the image sensor  300  may further comprise a processing circuit  310 . The processing circuit  310  may be coupled to the pixel array  305  via a communication bus  315 , wherein the communication bus  315  may transmit and receive signals, such as control and data signals, between the pixel array  305  and the processing circuit  310 . The processing circuit  310  may comprise a control unit (not shown) for transmitting control signals. The processing circuit  310  may also comprise a clocking mechanism to synchronize transmission of control signals. The processing circuit  310  may read the pixel signals through control lines accessed in a per-row manner. In various embodiments, the processing circuit  310  may be formed on the same chip as the pixel array  305 . 
     Each multi-storage pixel  100  of the image sensor  300  samples a portion of the image and generates a corresponding signal. Referring to  FIG. 1 , in an exemplary embodiment of the present technology, the multi-storage pixel  100  may comprise a photosensitive element  105 , a storage node  110 , and a floating diffusion node  115 . In the present embodiment, the photosensitive element  105  operates by converting light into an electric charge and may comprise a photodiode, a photogate, or other semiconductor device responsive to light. In various embodiments, the photosensitive element  105  may comprise a pinned photodiode capable of being fully depleted at a depletion voltage. 
     The floating diffusion node  115  (also referred to as a “floating diffusion region” or “FD node”) selectively stores charge. In the present embodiment, the floating diffusion node  115  may act as a sensing node and may be formed with any device or structure suitable for storing electric charge, such as a diode or capacitor. In various embodiments, the floating diffusion region  115  may be shielded from incoming light. 
     The storage node  110  (also referred to as a “storage region”) selectively stores charge. In the present embodiment, the storage node  110  may comprise a doped region into which electric charge is transferred and stored. The storage node  110  may be formed with any device or structure suitable for storing electric charge, for example, a pinned diode or a storage gate. In various embodiments, the storage region  110  may be shielded from incoming light. 
     The photosensitive element  105  may be selectively coupled to the floating diffusion node  115  via a first transfer gate  120 . In an exemplary embodiment, the first transfer gate  120  may comprise a switch, such as a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal, and the source and drain terminals are used to carry current or transfer charge. For example, the gate terminal of the first transfer gate  120  may receive a first control signal TX 1  from the processing circuit  310  ( FIG. 3 ). In other embodiments, the first transfer gate  120  may comprise any other suitable device for providing current flow or charge transfer. 
     The floating diffusion node  115  may also be selectively coupled to a voltage source V AA , for example via a switch, such as a reset gate  135 . In the present embodiment, the reset gate  135  may comprise a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal, and the source and drain terminals carry current when the reset gate  135  is activated. For example, the gate terminal of the reset gate  135  may receive a control signal RST from the processing circuit  310  ( FIG. 3 ). 
     The photosensitive element  105  may be selectively coupled to the storage node  110  via another switch, such as a second transfer gate  125 . In the present embodiment, the second transfer gate  125  may comprise a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal and the source and drain terminals carry current or transfer charge. For example, the gate terminal of the second transfer gate  125  may receive a second control signal TX 2  from the processing circuit  310  ( FIG. 3 ). In other embodiments, the second transfer gate  125  may comprise any other suitable device for controlling current flow or charge transfer. 
     The storage node  110  may be selectively coupled to the floating diffusion node  115  via a switch such as third transfer gate  130 . In the present embodiment, the third transfer gate  130  may comprise a transistor having a gate terminal, a drain terminal, and a source terminal, wherein the gate terminal may operate as a control terminal and the source and drain terminals carry current or transfer charge. For example, the gate terminal of the third transfer gate  130  may receive a third control signal TX 3  from the processing circuit  310  ( FIG. 3 ). In other embodiments, the third transfer gate  130  may comprise any other suitable device for providing current flow or charge transfer. 
     The pixel  100  may further comprise an amplifier  140  and a row select gate  150 . The amplifier  140  amplifies the signal generated by the photosensitive element  105 . For example, in the present embodiment, the amplifier  140  may comprise a source follower circuit comprising a transistor having a gate terminal, a drain terminal, and a source terminal. The gate terminal may receive signals corresponding to the charge generated by the photosensitive element and provide a corresponding amplified signal. 
     The row select gate  150  selectively connects the pixel  100  to the output. Each line in the imager sensor  300  may be selected and then read out, for example using a column select signal. The row select gate  150  for each pixel in a row may be activated to make that row active. In the present embodiment, the row select gate may comprise a transistor having a gate terminal, a drain terminal, and a source terminal. The gate terminal may be used as a control terminal and the source and drain terminals carry current. The gate terminal of the row select gate  150  may receive a control signal RS from the processing circuit  310  ( FIG. 3 ). In an exemplary embodiment, the source terminal of the amplifier  140  may couple to the drain terminal of the row select gate  150 . 
     Referring now to  FIGS. 1 and 2 , in operation  200 , the image sensor  300  ( FIG. 3 ) may operate in conjunction with a global reset period  205 , an integration period  210 , and a readout period  215 . In an exemplary embodiment, the image sensor  300  may capture multiple signals with different exposure periods. 
     In an exemplary embodiment, during the global reset period  205 , the first, second, and third transfer gates  120 ,  125 ,  130 , as well as the reset gate  135 , are simultaneously active. For example, control signals TX 1 , TX 2 , TX 3 , and RST are switched to a high voltage value simultaneously. Transmitting a control signal with a high voltage value to the first, second, and third transfer gates  120 ,  125 ,  130 , and reset gate  135  resets the photosensitive element  105 . After a predetermined period of time, the first, second, and third transfer gates  120 ,  125 ,  130 , and reset gate  135  may be deactivated, for example via low voltage control signals. In an exemplary embodiment, all pixels  100  in the pixel array  305  ( FIG. 3 ) receive the same control signals and are reset simultaneously. 
     After the reset period  205  has been completed, the global integration period  210  begins. During integration, the photosensitive element  105  absorbs light and converts the light into an electric charge. In an exemplary embodiment, the electric charge produced in the photosensitive element  105  is transferred to both the storage node  110  and the floating diffusion node  115 . Charge transfer between the storage node  110  and the floating diffusion node  115  may be controlled by selectively activating control signals TX 1  and TX 2  by pulsing the signals high at different times, wherein the pulses for control signals TX 1  and TX 2  do not overlap. For example, control signals TX 1  and TX 2  do not activate the first and second transfer gates  120 ,  125  at the same time. Instead, only one of the first and second transfer gates  120 ,  125  is activated at any given time. In an exemplary embodiment, the integration period may comprise one or more exposures. 
     After a first exposure period T 1 , typically measured in micro seconds (μs), the first transfer gate  120  may be activated, for example via a high voltage value control signal TX 1 , allowing the charge accumulated in the photosensitive element  105  to transfer to the floating diffusion node  115 . The first transfer gate  120  may be deactivated, such as via a low voltage value control signal TX 1 , to stop charge transfer to the floating diffusion node  115 . After a second exposure period T 2 , measured in microseconds (μs), a high voltage value control signal TX 2  activates the second transfer gate  125 , allowing the charge accumulated in the photosensitive region  105  to transfer to the storage node  110 . A low voltage value control signal TX 2  stops charge transfer to the storage node  110 . Charge transfer to the floating diffusion node  115  and the storage node  110  may begin in any order. For example, at the start of the integration period  210 , charge may be transferred and stored in either one of the floating diffusion node  115  or the storage node  110 . As such, the second exposure period T 2  may occur earlier in time than the first exposure period T 1 . 
     In an exemplary embodiment, during the integration period  210 , control signals TX 1  and TX 2  may be asserted multiple times such that the charges transferred the floating diffusion node  115  are summed Likewise, the charges transferred to the storage region  110  are summed (binned). Operation of the control signals TX 1  and TX 2  may be controlled by the processing circuit  310  ( FIG. 3 ) and may operate cyclically, such that charge is transferred from the photosensitive element  105  to the floating diffusion node  115  and then, immediately after another exposure period, charge is transferred from the photosensitive element  105  to the storage region  110 . In other embodiments, charge may not be transferred in a cyclical manner, but may be transferred in any suitable pattern. For example, charge may be transferred consecutively to the storage region  110  for any number suitable of times; likewise, charge may be transferred consecutively to the floating diffusion node  115  for any suitable number of times. 
     In an exemplary embodiment, where the integration period comprises multiple exposure periods T 1   i , T 2   i , the charge portions transferred to the storage region  110  may accumulate and are summed. Likewise, the charge portions transferred to the floating diffusion node  115  may accumulate and are summed. The transfer of multiple charge portions with corresponding exposure periods T 1   i , T 2   i  to each of the storage region  110  and the floating diffusion node  115  may result in the accumulated charges having different total exposure times T 1   total , T 2   total . 
     In an exemplary embodiment, the first exposure period T 1  may be shorter than the second exposure period T 2 . A long exposure period may be used to capture low-light portion of a scene, while a short exposure period may be used to capture bright portions of the scene. For example, the charge transferred to the floating diffusion node  115  after the first exposure period T 1  captures bright scenes (i.e. high light signals), and the charge transferred to the storage node  110  after the second exposure period T 2 , where T 2  is greater than T 1 , captures low-light scenes (i.e. low light signals). As such, the total exposure time for charge accumulated in the storage node  110  may be longer than that of the total exposure time for charge accumulated in the floating diffusion node  115 . The total exposure times may be described by the following equations, where n and m are the upper limits of the number of exposures and i is the index number for each exposure. 
       T 1   total =Σ i=1   n T 1   i  
 
       T 2   total =Σ i=1   m T 2   i  
 
     In various embodiments, n may be equal to m, n may be greater than m, or n may be less than m. The upper limits of the number of exposures n, m may be selected to suit a particular application. Additionally, the length of the exposure periods T 1   i , T 2   i  may vary during any one integration period  210 , for example, T 1   1  may be 50 λs (microseconds), while T 1   10  may be 400 μs. 
     Dynamic range of the image sensor may be defined as the ratio of the sum of second exposure periods T 2  to the sum of the first exposure periods T 1  (i.e. T 2   total /T 1   total ). The dynamic range may be adjusted during operation by increasing and/or decreasing the length of any of the exposures times T 1   i , T 2   i . The total exposure times T total , T 2   total  may be tracked in real time such that a running total may be computed during the integration period  210  and transmitted to the control unit, where the control unit may adjust the duration of any subsequent exposure periods T 1   i , T 2   i . 
     Pulsing of the control signals TX 1 , TX 2  may be represented as a duty cycle, where the duty cycle is defined as a length of time where the signal is pulsed high T h , to activate the transfer gates  120 ,  125  divided by the total amount of time in a cycle T c . In various embodiments, exposure periods T 1   i , T 2   i  may be predetermined, while in other embodiments, the exposure periods T 1   i , T 2   i  may be varied, resulting in a varied duty cycle. For example, the duty cycle and frequency of a voltage value applied to the control signals TX 1  and TX 2  may be synchronized to a pulsing light source for time of flight applications. The duty cycle and frequency of a voltage value applied to the control signals TX 1  and TX 2  may also be used to mitigate amplitude modulated light sources, such as LEDs (light emitting diodes). 
     Once the integration period  210  ends, the readout period  215  begins. In an exemplary embodiment, the pixel signals are read out on a row-by-row basis. A high voltage value is applied to the row select control signal RS, and in an exemplary embodiment, the row select control signal RS maintains a high voltage value for the duration of the readout period  215  for each row. Applying a high voltage value to the row select control signal RS allows the charge that has accumulated in the floating diffusion node  115  to be read out of the pixel  100  via a communication line  155  as a voltage which is proportional to the amount of accumulated charge. The readout process produces a first pixel signal comprising a corresponding voltage. The first pixel signal may be unable to undergo correlated double sampling since the floating diffusion reset voltage cannot be measured prior to readout. However, the first pixel signal can be double sampled, once before the reset gate  135  is activated  220  and once using the voltage level post reset  225 . 
     While the row select control signal RS maintains a high voltage value, the reset control signal RST activates the reset gate  135  and resets the floating diffusion node  115 . At this time  225 , the floating diffusion reset level can be sampled and used for double sampling of the first pixel signal, and used for correlated double sampling of the second pixel signal. 
     Once the floating diffusion node  115  has been reset and the reset level sampled, the charge that has accumulated in the storage node  110  can be read out. For example, activating the third transfer gate  130  via control signal TX 3  allows the charge to transfer from the storage node  110  through the floating diffusion node  115 , and the resulting voltage of pixel  100  is accessible via the communication line  155 . The readout produces a second pixel signal comprising a corresponding voltage. In various embodiments, as described above, the second pixel signal read out from the storage node  110  may undergo correlated double sampling to reduce noise since the voltage on the floating diffusion node  115  may be measured twice, once in a reset condition  225  (prior to activating the third transfer gate  130 ), and once after the charge from the storage node  110  has been transferred to the floating diffusion node  115 ,  230 . 
     After both pixel signals have been read out of the floating diffusion node  115  and the storage node  110 , the row select control signal applies a low voltage value to end the readout period  215 . A new global reset period  205  may begin again. 
     In an exemplary embodiment, after completion of the readout period  215 , the pixel signals (i.e. one capture using a long exposure and one capture using a short exposure) may be combined to produce a high dynamic range image utilizing an image signal processor (not shown). In various embodiments, the image signal processor may also perform color processing and tone mapping operations, such as demosaicing, white-balance, and color correction. 
     The embodiments of  FIGS. 2 and 3  in which the pixel array  305  is described as operating in global shutter mode is merely illustrative. If desired, the pixel array  305  having the processing circuit  310  for operating in global shutter mode may be operated in rolling shutter mode. The particular mode of operation depends on the timing and operation of the control signals control signals TX 1 , TX 2 , TX 3 , RST, and RS. For example, in rolling shutter mode, the signals controlling the integration  210  readout  215 , and reset  205  of each pixel  100  would be operated sequentially in a row-by-row fashion. 
     Additionally, the terms “first” and “second” do not connote a particular order in time, but rather distinguish between distinct elements, events, conditions, and the like. For example, the second exposure period T 2  may occur earlier in time than the first exposure period T 1 . 
     In the foregoing description, the technology has been described with reference to specific exemplary embodiments. The particular implementations shown and described are illustrative of the technology and its best mode and are not intended to otherwise limit the scope of the present technology in any way. Indeed, for the sake of brevity, conventional manufacturing, connection, preparation, and other functional aspects of the method and system may not be described in detail. Furthermore, the connecting lines shown in the various figures are intended to represent exemplary functional relationships and/or steps between the various elements. Many alternative or additional functional relationships or physical connections may be present in a practical system. 
     While the technology has been described with reference to specific exemplary embodiments, various modifications and changes may be made without departing from the scope of the present technology. The description and figures are to be regarded in an illustrative manner, rather than a restrictive one and all such modifications are intended to be included within the scope of the present technology. Accordingly, the scope of the technology should be determined by the generic embodiments described and their legal equivalents rather than by merely the specific examples described above. For example, the steps recited in any method or process embodiment may be executed in any order, unless otherwise expressly specified, and are not limited to the explicit order presented in the specific examples. Additionally, the components and/or elements recited in any apparatus embodiment may be assembled or otherwise operationally configured in a variety of permutations to produce substantially the same result as the present technology and are accordingly not limited to the specific configuration recited in the specific examples. 
     Benefits, other advantages and solutions to problems have been described above with regard to particular embodiments. Any benefit, advantage, solution to problems or any element that may cause any particular benefit, advantage or solution to occur or to become more pronounced, however, is not to be construed as a critical, required or essential feature or component. 
     The terms “comprises”, “comprising”, or any variation thereof, are intended to reference a non-exclusive inclusion, such that a process, method, article, composition or apparatus that comprises a list of elements does not include only those elements recited, but may also include other elements not expressly listed or inherent to such process, method, article, composition or apparatus. Other combinations and/or modifications of the above-described structures, arrangements, applications, proportions, elements, materials or components used in the practice of the present technology, in addition to those not specifically recited, may be varied or otherwise particularly adapted to specific environments, manufacturing specifications, design parameters or other operating requirements without departing from the general principles of the same. 
     The present technology has been described above with reference to an exemplary embodiment. However, changes and modifications may be made to the exemplary embodiment without departing from the scope of the present technology. 
     These and other changes or modifications are intended to be included within the scope of the present technology, as expressed in the following claims.