Abstract:
A method for reducing residual signal from a first image from appearing in a second image, the method includes the steps of capturing the first image on an image sensor having a known exposure that includes exposure time and a peak light intensity; estimating a flush time for the second image based on the known exposure in capturing the first image; and capturing the second image using the estimated flush time for substantially reducing the residual signal.

Description:
FIELD OF THE INVENTION 
     This invention relates to the field of digital imaging and, more particularly, to an apparatus and method for reducing the residual signal in digital images captured by a digital camera using an image sensor. 
     BACKGROUND OF THE INVENTION 
     A traditional camera system includes camera body, lens, electronics to control the functionality of the camera and most importantly the image-capturing media—film. In contrast, instead of using film, a digital camera system usually employs an image sensor made on semiconductor substrate. Typically the image sensor is either a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor. There are several types of CCD sensors, such as frame transfer CCD, full frame CCD, interline CCD and linear CCD sensors. For purposes of an illustrative embodiment, the image sensor of a full frame CCD is discussed for illustrating the effects of a residual signal. 
     A full frame CCD consists of a two-dimensional array of photosensitive elements of X columns by Y rows. The sensor captures light during exposure and converts photons to electrical charge, namely electrons. At the end of exposure, it starts the readout process. First it transfers all the electrons vertically row by row to a serial of horizontal registers. After each row transfer, the charge in the horizontal registers is transferred out to a sensing node, namely a floating diffusion, and converted to a voltage signal. An analog-to-digital converter converts this voltage to a digital count. A single image capture is completed when all the rows of signals are transferred out and converted. Finally the onboard electronics renders all the digital counts into a digital image. At this time, the camera is ready for the next capture. 
     A digital camera with a full frame CCD captures a single still image in the following manner. When a mechanical shutter is pressed, a flush cycle with a fixed amount of time is executed to clear out the existing charge in the CCD (see Japanese Patent No: JP62152281 for further discussion). This is because the existing charge, namely the dark current, is accumulated during either the camera power off period or camera standby period. This dark current needs to be cleared out. Then the exposure process begins and charge starts to accumulate in the CCD. The close of the shutter ends the exposure and starts the readout process in which the charge is transferred out of the CCD and converted to a digital image. 
     The digital camera with a full frame CCD usually only captures still images, not videos. The reason for this is that for the high frame rate required by the video mode, a mechanical shutter is not practical to use. As a result, photon-induced charge continues to accumulate even after the exposure period when the accumulated charge is being transferred out of CCD. This additional charge results in an artifact, namely smear, to appear in the images. Therefore, video capture usually is not an option for a digital camera using a full frame CCD sensor. There are ways to reduce smear without a mechanical shutter (see US Patent Application Publication No: US2002/0186308A1). 
     However, many high-end digital cameras using a full frame CCD sensor have a so-called continuous capture mode, which allows users to continuously capture several images in a row. In order to get high quality images for all images, several obstacles need to be overcome and they are smear, dark current and the residual signal. The smear problem can be solved by using a mechanical shutter if the frame rate is not very fast. For example, the frame rate for an 8 million pixel CCD operating at 30 Mhz pixel rate is about 0.3 seconds, which is long enough for a mechanical shutter to operate smoothly. Removal of dark current can be achieved by subtracting a dark image with the same exposure time captured at the end of continuous capture mode with the shutter closed. 
     The residual signal, however, cannot be solved by either a mechanical shutter or a dark current subtraction. It is defined that charge related to the previous image is superimposed on the next image when the camera operates in its continuous capture mode. Referring to  FIG. 1A  and  FIG. 1B , there are shown two scene settings for testing residual signal in a full frame CCD. In both cases, there is a candle  1  on a table. In  FIG. 1A , the candle  1  is lit and has a flare  2  while in  FIG. 1B  the candle  1  is put off and the flare is gone. Referring to  FIG. 1C  and  FIG. 1D , two images continuously captured from these two scenes using a camera in its two-image-capture mode are shown. In the  FIG. 1D  a residual image  3  of the flare  2  is still visible in the image although in the real scene ( FIG. 1B ) there is no such flare  2 . 
     When light strikes onto a full frame CCD during the first image exposure, some photon-generated charges are trapped in some pixels due to the crystal defects in the silicon. After exposure, these traps start to release trapped charge gradually. If during the second image exposure and readout, these traps still release charge, the total charge in the second image will be the summation of the charge due to second exposure and the charge released from the traps generated during the first exposure. Therefore, the second image will have a residual image of the first image in it. This phenomenon will become especially worse when the first image has a strong illumination scene such as  FIG. 1A . 
     The residual charge is due to the thermal release of charge from trapping sites in a CCD. The number of traps is exponentially proportional to light intensity and exposure time. Referring to  FIG. 2 , there is shown the relationship between the residual signal generated at the end of the first exposure and the exposure time for different light intensities. Under light intensity of 0.5 Esat, the residual signal increases from 0 to about 45 electrons for a five-second exposure. Esat is a relative light intensity unit. If an image sensor&#39;s output equals to its saturation level when it is illuminated under a light source for a fixed amount of exposure time, that light intensity is defined as 1 Esat. At the same exposure time, the residual signal also increases with the light intensity. At 20 Esat, it reaches about 115 electrons at 5 seconds. 
     After the first exposure, the traps will release the trapped charge and the process will slow down and eventually diminish with time. Theoretically, by adding infinite flush time between the end of the first image readout and the start of second image exposure, the residual signal will be eliminated in the second image because all the charge in the traps will be released and flushed out during this infinite time before the next exposure begins. However, in the practical use, the flush time cannot be set for too long. Otherwise, a camera user may feel a delay between each continuous capture and that delay may result in missing important scenes. 
     On the other hand, by setting a predefined maximum-allowable flush time sometimes can cause unnecessary slowdown of the continuous capture process if the illumination of a scene is not quite as strong as to generate a substantial residual signal beyond the camera&#39;s tolerance. 
     Therefore, there is a need for a method to automatically adjust the flush time based on the exposure information of a previous image capture and substantially reduce the residual signal in the next image in a continuous capture mode by a digital camera. 
     SUMMARY OF THE INVENTION 
     The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, the invention resides in a method for reducing residual signal from a first image from appearing in a second image, the method comprising the steps of capturing the first image on an image sensor having a known exposure that includes exposure time and a peak light intensity; estimating a flush time for the second image based on the known exposure in capturing the first image; capturing the second image using the estimated flush time for substantially reducing the residual signal. 
     It is therefore an object of the present invention to automatically adjust the flush time based on the exposure information of a previous image capture and substantially reduce the residual signal in the next image in a continuous capture mode by a digital camera using an image sensor. 
     This object is achieved by a method to determine an optimal flush time based on the relationship between the residual signal and the flush time. The residual signal in the next image is estimated based on the previous exposure which is the multiplication of the light intensity measured by a light meter and the exposure time. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIGS. 1A-1D  are illustrations of a residual signal; 
         FIG. 2  is a chart showing the relationship between the residual signal generated at the end of the exposure and the exposure time for different light intensities; 
         FIG. 3  is a block diagram for a digital camera system in accordance with the invention; 
         FIG. 4  is a block diagram of the image sensor timing to reduce the residual signal in the second image for a two-image-capture operation; 
         FIG. 5  is a chart showing the relationship between the residual signal and the flush time in the second image; and 
         FIG. 6  is a flow chart of a method of automatically reducing the residual signal for a two-image-capture operation. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     Referring to  FIG. 3 , there is shown a representative diagram showing a digital camera system  8  in accordance with the present invention. An image sensor  14  (preferably a CCD) is positioned in front of a lens with a mechanical shutter (the combination of the lens and the shutter is illustrated as  10 ). A light meter  12  measures the light intensity of the scene of interest passing through the lens-shutter fixture  10 . This value is saved in an onboard memory  32  for the future use. A microcontrol processor  22  controls the lens fixture  10  and it also controls a flush timing control  28 . The image sensor  14  is driven by clock drivers  20  and a clock-timing generator  30  which is also controlled by the microcontrol processor  22 . The analog-to-digital chain includes an analog gain and correlated double sampling (CDS) unit  16  and an analog-to-digital converter (ADC)  18 . The digital counts related to the scene of interest are processed by a digital signal processor (DSP) circuit  24  and finally transferred to either a viewing device  26  like a display or a memory card like a flash memory  34 . The onboard memory  32  saves exposure information of the last image such as exposure time and light intensity. The microcontrol processor  22  uses the information to calculate the flush time for the next image capture if the camera system is operated in a continuous image capture mode. 
     Referring to  FIG. 4 , there is shown a diagram showing the timing sequence in accordance with the present invention, assuming the digital camera system continuously captures two images in a row. The 1 st  flush  102  is a flush cycle which is used to primarily remove the dark current in the image sensor pixels during either power off or standby period. Then the 1 st  exposure  104  is initiated. After 1 st  exposure  104 , 1 st  readout  106  is executed and the first image is outputted. Immediately, based on the 1 st  exposure  104  including the exposure time (derived from the camera) and the light intensity, the microcontrol processor  22  calculates the optimal flush time for the 2 nd  flush  108  through either the lookup tables or the predefined fit-curves and starts the second image capture process including 2 nd  exposure  110  and 2 nd  readout  112 . The 2 nd  flush  108  will substantially remove the residual signal while in the meantime it keeps the delay between two images captured at minimum. 
     The process of determination of the optimal flush time is based on the relationship between the residual signal and the flush time for the second capture and this relationship is shown in  FIG. 5 . A threshold of the residual signal is defined as the maximum residual signal which a digital camera system can tolerate without sacrificing the image quality it captures. Suppose the estimated residual signal which will appear in the second image if, without any flush, is 90 electrons and the threshold is only is 50 electrons, the necessary flush time will be t 1 -t 0 , i.e., 0.75 seconds to remove the residual signal in the second image to the level which the camera can tolerate. 
     The onboard memory  32  in the digital camera system can store a plurality of lookup tables of the residual signal versus exposure time for different light intensities ( FIG. 2 ) and another lookup table of the residual signal versus flush time ( FIG. 5 ). Therefore, once the previous light intensity and exposure time are known, the system can immediately calculate the optimal flush time for the next capture by searching these lookup tables. 
     Another method is to store the parameters of a plurality of fit-curves of the residual signal versus exposure time for different light intensities ( FIG. 2 ) and the parameters of another fit-curve of the residual signal versus flush time ( FIG. 5 ). For example, in a linear curve, the only parameters of the fit-curve that are stored are the slope and the intercept. The flush time will then be interpolated and calculated based on these parameters of the fit-curves. 
     Referring to  FIG. 6 , there is shown a flowchart of a 2-image-capture mode in accordance to the present invention. When this mode  202  starts, a user presses a shutter button halfway to measure light intensity  204 . After the button is fully pressed down  206 , the 1 st  fixed flush  208  is executed to remove the background dark current. Then the mechanical shutter is opened to start 1 st  exposure  210 . The light intensities and exposure time are saved in the onboard memory  32  for later retrieval. After that, the mechanical shutter is closed to start 1 st  readout  212 . Once finishing the 1 st  readout  212 , the microcontrol processor  22  first retrieves the exposure information of the measured light intensity and the exposure time from the onboard memory  32  and then estimates the residual signal which will appear in the second image if without any flush based on the relationship between the residual signal and exposure ( FIG. 2 ). Finally it calculates the optimal flush time  214  based on a predefined threshold of residual signal level ( FIG. 5 ) and the relationship between the residual signal and the flush time. A comparison  216  decides if an optimal flush time  218  or a maximum allowable flush time  220  is used. A maximum flush time  220  is the maximum time the camera system allows before a user will not notice a significant amount of delay between two continuously captured images. This only occurs when the scene has an extreme bright illumination which generates a residual signal so large that a maximum flush time is needed to reduce the residual signal. After the flush, the mechanical shutter is opened again and starts the 2 nd  exposure  222 , and then the shutter is closed and the 2 nd  readout  224  is executed. The 2-image-capture mode  226  ends with two images either stored in memory card  34  or outputted to a viewing device  26 . 
     This invention has been described in detail with a particular configuration of a two-image continuous capture mode, but it will be understood that it can also apply to N-image continuous capture mode. 
     PARTS LIST 
     
         
           1  candle 
           2  flare 
           3  residual image 
           8  digital camera system 
           10  lens-shutter fixture 
           12  light meter 
           14  CCD image sensor 
           16  analog gain/CDS unit 
           18  A/D converter 
           20  clock drivers 
           22  microcontrol processor 
           24  DSP circuit 
           26  viewing device 
           28  flush timing control 
           30  clock timing generator 
           32  onboard memory 
           34  memory card 
           102  1 st  flush 
           104  1 st  exposure 
           106  1 st  readout 
           108  2 nd  flush 
           110  2 nd  exposure 
           112  2 nd  readout 
           202  start 
           204  meter exposure/light intensity 
           206  start capture 
           208  1 st  fixed flush 
           210  1 st  exposure 
           212  1 st  readout 
           214  calculate flush time 
           216  comparison 
           218  2 nd  optimal flush 
           220  maximum flush 
           222  2 nd  exposure 
           224  2 nd  readout 
           226  2-frame-capture end