Abstract:
Various embodiments of the present technology may comprise a method and apparatus for reducing spatial flicker artifacts in high dynamic range images produced from multiple image captures with differing integration times. The method and apparatus may comprise, in the image captures, measuring pixel intensity selecting pixels with a predetermined intensity thresholds, calculating a ratio based on a channel selection, normalizing the calculated ratio to an ideal ratio, and applying the normalized ratio to corresponding image pixels in a second image capture to produce a flicker-free image.

Description:
BACKGROUND OF THE TECHNOLOGY 
       [0001]    In multiple exposure high dynamic range (ME-HDR) imaging, an image sensor captures multiple image frames with different integration times, where the integration times have fixed ratios between them. The frames are then combined together to produce a high dynamic range (HDR) image. The integration times in most image sensors are controlled by an electronic rolling shutter (ERS). Capturing images with an ERS-based image sensor may produce a spatial flicker when a scene is illuminated by alternating current (AC) sources. The spatial flicker typically appears as intensity variations or ‘bands’ across the image (i.e. spatial artifacts). 
         [0002]    The number of bands may be affected by the AC source frequency and the exposure time. The amplitude of the intensity variation is dependent on the waveform driving the scene illumination. The flicker can also vary across different color channels depending on the type of illumination. In ME-HDR, all of these factors result in frames exhibiting spatial flicker with phase differences and amplitude variations between multiple frames. When the frames are combined together, color artifacts may be observed in the HDR image in addition to the intensity bands. 
         [0003]    A conventional method for avoiding flicker is to adjust the integration time to be an integer multiple of half the flicker period. This technique can eliminate flicker for the longest-exposure image (T 1 ). Adjusting the integration time for the shorter-exposure images (T 2 , T 3  . . . TN) while simultaneously maintaining the necessary integration ratios between them, however, does not produce flicker-free images, since the shorter integration times are almost always less than one flicker period. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWING FIGURES 
         [0004]    A more complete understanding of the present technology may be derived by referring to the detailed description when considered in connection with the following illustrative figures. In the following figures, like reference numbers refer to similar elements and steps throughout the figures. 
           [0005]      FIG. 1  representatively illustrates a block diagram of an imaging device in accordance with an exemplary embodiment of the present technology; 
           [0006]      FIG. 2  representatively illustrates a process for reducing spatial artifacts in accordance with an exemplary embodiment of the present technology; 
           [0007]      FIG. 3  representatively illustrates a diagram of part of a process for reducing spatial artifacts in accordance with an exemplary embodiment of the present technology; 
           [0008]      FIG. 4  representatively illustrates an imaging system in accordance with an exemplary embodiment of the present technology; 
           [0009]      FIG. 5  representatively illustrates a process in accordance with an exemplary embodiment of the present technology; 
           [0010]      FIGS. 6A-D  representatively illustrate alternative arrangements of a color filter array in accordance with an exemplary embodiment of the present technology; 
           [0011]      FIGS. 7A-B  representatively illustrate neighborhood searches in accordance with exemplary embodiments of the present technology; 
           [0012]      FIG. 8  representatively illustrates a graph in accordance with an exemplary embodiment of the present technology; and 
           [0013]      FIGS. 9A-B  representatively illustrate waveforms and estimated waveforms in accordance with exemplary embodiments of the present technology. 
       
    
    
     DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 
       [0014]    The present technology may be described in terms of functional block components and various processing steps. Such functional blocks may be realized by any number of components configured to perform the specified functions and achieve the various results. For example, the present technology may employ various semiconductor devices, circuitry, sensors, image signal processors, and the like, which may carry out a variety of functions. In addition, the present technology may be practiced in conjunction with any number of imaging systems, and the system described is merely one exemplary application for the technology. 
         [0015]    Methods and apparatus for reducing spatial flicker artifacts according to various aspects of the present technology may operate in conjunction with any suitable imaging system, such as a computer system, camera system, machine vision system, vehicle navigation system, video telephone, surveillance system, auto focus system, motion detection system, image stabilization system, smartphone, and the like. 
         [0016]    Referring to  FIGS. 1 and 4 , an exemplary imaging system may comprise an electronic device, in this case a digital camera  405 . The system may comprise a central processing unit (CPU)  410  that communicates with various devices over a bus  415 . 
         [0017]    An input/output (I/O) device  420  may be connected to the bus  415  and may, in some embodiments, provide communication into and out of the digital camera  405  or other system. For example, the I/O device  420  may comprise a port to facilitate communication with an external device, such as a computer display, memory card, or some other external unit. The I/O device  420  port may transfer digital image data, such as video data, image data, frame data, and/or gain information, for example from an image signal processor  145 . In other embodiments, the I/O device  420  may comprise an internal device, such as a display screen or a memory component, for directly viewing and/or storing the digital image data. 
         [0018]    Other devices connected to the bus  415  may provide memory, for example, a random access memory (RAM)  425 , hard drive, and/or one or more peripheral memory devices  430 , such as a floppy disk drive, compact disk (CD) drive, USB drive, memory card, and SD card. While the bus  415  is illustrated as a single bus, any number of busses or other connections may be used to provide communication paths to interconnect the elements. 
         [0019]    In various embodiments, the electronic device may further comprise a lens  435  configured to focus an image on a pixel array  105 . For example, the lens  435  may include a fixed and/or adjustable lens and may include micro lenses formed on an imaging surface of an imaging device  100 . 
         [0020]    The imaging device  100  (also referred to as an “image sensor”) captures and generates image data. The imaging device  100  may be combined with a processor, such as the CPU  410 , a digital signal processor, or a microprocessor, in a single integrated circuit, or may be on a separate chip. The imaging device  100  may also receive control or other signals from the system, such as a shutter release. In the present embodiment, the imaging device  100  may comprise a CMOS imager. 
         [0021]    Referring to  FIG. 1 , in an exemplary embodiment of the present technology, the imaging device  100  may comprise a plurality of pixels  110  arranged in rows and columns to form a pixel array  105 . In an exemplary embodiment, each pixel  110  comprises a photodetector, such as a photodiode, for capturing light and converting light into an electric signal. 
         [0022]    A color filter system, such as a color filter array (CFA)  155  comprising a pattern of color filters, may be situated on the pixel array  105  to capture color information. In the exemplary embodiment, each pixel  110  is covered with one color of the CFA  155 . For example, a Bayer color filter array comprising a pattern of red R, blue B, and green G filters may be provided. In other embodiments, the CFA  155  may be formed using other color filters, such as a CYYM filter (one cyan, two yellow, and one magenta), a CYGM filter (one cyan, one yellow, one green, and one magenta), a CRGB filter (one cyan, one red, one green, and one blue), and any other suitable color pattern. In various embodiments, the CFA  155  may comprise “clear” or transparent filter elements. The CFA  155  may form a 2×2 color pattern, a 4×4 color pattern, a 2×4 color pattern, or any other suitable pattern size. In various embodiments, the CFA  155  may repeat to cover the entire pixel array  105 . 
         [0023]    The imaging device  100  may capture and generate image data corresponding to one or more rows in the pixel array  105 . The image data may comprise image pixel data corresponding to one or more pixels  110  in the pixel array  105 . Once a scene, or part of a scene, is captured by the imaging device  100 , the resulting capture may be represented as an image capture  315  ( FIG. 3 ). Each image capture  315  may comprise image pixels  310  which correspond to the pixels  110  in the pixel array  105 . 
         [0024]    In an exemplary embodiment, the image pixels  310  may comprise image pixel data. The image pixel data may comprise information representing a measured voltage, current, or other quantity as a result of light absorbed by the pixel  110  (i.e., an intensity value), such that each image pixel  310  comprises an individual pixel intensity value. The intensity value may be a digital value. The image pixel data may also comprise row and column information corresponding to each pixel  110  in the pixel array  105 . 
         [0025]    The image data may be captured and generated with differing integration (i.e., exposure) times. In an M-image capture system, where M is the total number of image captures, each capture may have a different integration time, and each image capture may be generally referred to as TN, where N=1, 2, 3, . . . M. For example, a 4-image capture system has a first capture T 1  with a first integration time, a second capture T 2  with a second integration time, a third capture T 3  with a third integration time, and a fourth capture T 4  with a fourth integration time T 4 . Likewise, a 2-image capture system has a first capture T 1  with a first integration time and a second capture T 2  with a second integration time. In an exemplary embodiment, the image capture with the first integration time T 1  may be the longest, with the second, third, . . . M-captures having an integration time which is some fraction of the first integration time. In various embodiments, the image captures T 1 :TM may be obtained consecutively and within a short time span, wherein each image capture T 1 :TM may produce a single frame. 
         [0026]    In various embodiments, the imaging device  100  may further comprise row circuitry  115 , column circuitry  120 , and a timing and control unit  125 , for selectively activating sequential rows of pixels to be read out to a sample-and-hold circuitry  130 . The pixel signal may be then be transmitted to an amplifier  135  to amplify the signal prior to being converted to a digital signal by an analog to digital converter  140 . The digital pixel data may then be transmitted to and stored in the image signal processor  145  for further processing. 
         [0027]    In an exemplary embodiment, the pixel data may be stored utilizing line buffers (not shown) to temporarily store a portion of the pixel data, such as 3 lines of pixel data. In an alternative embodiment, the pixel data may be stored utilizing a frame buffer (not shown) to store an entire frame of pixel data. 
         [0028]    In an exemplary embodiment, the imaging device  100  may comprise a switch (not shown) configured to selectively enable one of a first circuit to perform a first ratio calculation (e.g., same-color channel ratio) or a second circuit to perform a second ratio calculation (e.g., cross-color channel ratio), described below. The switch may be manually or automatically operated. For example, the switch may be implemented using any suitable device, such as a transistor, or an external toggle switch for operation by a user. 
         [0029]    The imaging device  100  may further comprise the image signal processor  145  to perform demosaicing and/or other functions, such as autofocus, exposure, noise reduction, and white balance. The image signal processor  145  may comprise any number of semiconductor devices, such as transistors, capacitors, and the like, for performing calculations, transmitting and receiving image pixel data, and a storage unit for storing image pixel data. In various embodiments, the image signal processor  145  may be implemented with a programmable logic device, such as a field programmable gate array (FPGA) or any other device with reconfigurable digital circuits. In other embodiments, the image signal processor  145  may be implemented in hardware using non-programmable devices. In alternative embodiments, the image signal processor  145  may be formed partially or entirely within an integrated circuit in silicon using any suitable complementary metal-oxide semiconductor (CMOS) techniques or fabrication processes, in an ASIC (application-specific integrated circuit), using a processor and memory system, or using another suitable implementation. 
         [0030]    In an exemplary embodiment, the image signal processor  145  may store image pixel data from one or more image captures  315  having different integration times. For example, in a 2-image capture system, image pixel data from one or more rows of the first capture may be stored along with one or more corresponding (i.e., same spatial location) rows of the second capture. In other embodiments, for example in a 4-image capture system, the image signal processor  145  may receive image pixel data from more than two image captures  315 , such that corresponding portions of the image pixel data from four captures T 1  through T 4 , each capture with differing integration times, may be processed at any given time. Image pixel data may be defined as corresponding if the image pixel data from one image capture  315  is derived from the same spatially located pixel  110  as the image pixel data from a different image capture  315  taken at a different time, for example image pixel data from rows 1:5 of the first capture and image pixel data from rows 1:5 of the second capture, wherein the second capture is obtained later in time than the first capture. In alternative embodiments, image pixel data from an entire pixel array  105  may be received and stored at a given time. 
         [0031]    Referring now to  FIGS. 2 and 3 , in an exemplary embodiment, the 2-image capture system may comprise a first capture T 1  having a first integration time, and a second capture T 2  having a second integration time. In an exemplary embodiment, the first integration time is longer than the second integration time. In various embodiments, each image capture T 1 , T 2  may be realized by utilizing an electronic rolling shutter (not shown) wherein reset, integration and readout operate in a row-by-row manner starting with a first row of the pixel array  105 . 
         [0032]    In the 2-image capture system, the image signal processor  145  ( FIG. 1 ) may perform a process  200  for reducing spatial artifacts, including receiving pixel data from at least two captures ( 205 ), wherein the first capture T 1  has a first integration time, and the second capture T 2  has a second integration time, identifying valid and invalid image pixels ( 210 ), for example by determining the intensity value for each image pixel  310 , and then determining whether the intensity value falls above or below predetermined thresholds ( 215 ), ( 220 ). 
         [0033]    Referring now to  FIG. 8 , each image pixel  310  may be described using a transfer function  800 . The transfer function  800  illustrates an analog pixel signal (measuring the brightness level in lux) versus a digital output of the ADC (i.e., intensity value). The analog pixel signal may be converted to a digital value using various quantization levels. 
         [0034]    The image pixels  310  may be defined as either “valid”  815  or “invalid”  810 , for example based on whether or not the intensity value falls above or below predetermined thresholds. The valid and invalid pixels may be defined according to any suitable criteria. In the present exemplary embodiment, valid pixels correspond to actual brightness conditions, while invalid pixels do not. For example, image pixel data may not reflect actual brightness conditions when the sensor pixel is saturated or when the brightness is below the sensor pixel&#39;s capability to reliably measure the brightness. The valid/invalid criteria may also or alternatively be based on the resolution of the ADC  140  or other suitable criteria. 
         [0035]    In one embodiment, “valid”  815  image pixels may be defined as those image pixels with intensity values between a lower threshold TH L  and an upper threshold TH U . “Invalid”  810  image pixels  310  may be defined as those image pixels  310  with intensity values less than or equal to the lower threshold TH L , and those image pixels with intensity values greater than or equal to the upper threshold TH U . The upper and lower threshold value TH L , TH U  may be predetermined based on the resolution of the ADC  140  ( FIG. 1 ) being utilized. 
         [0036]    For example, a 12-bit ADC may have a lower threshold value set at 32 and an upper threshold value set at 3500. In various embodiments, as the resolution of the ADC  140  increases, the upper threshold TH U  value may also increase. In various embodiments, the lower threshold TH L  may be tuned within a small percentage of the full intensity range, based on the noise characteristics of the sensor. In various embodiments, the upper threshold TH U  value may be less than a saturation value SAT and may be selected such that the value does not fall within the non-linear region  805  of the pixel signal. A pixel with an intensity value which exceeds the maximum resolution of the ADC  140  may be defined as “saturated”  820 . For example, in a 12-bit ADC system, the saturation point is 4095. In various embodiments, saturated pixels are “invalid.” 
         [0037]    Referring back to  FIGS. 2 and 3 , in the 2-image capture system, the imaging device  100  produces the first capture T 1  and the second capture T 2 , wherein each capture T 1 , T 2  comprises an array  325 ( 1 ),  325 ( 2 ) of image pixels  310 . As described above, each image pixel  310  comprises image pixel data corresponding to the intensity value of the exposed pixel  110  and thus the corresponding image pixel  310 . As such, the image signal processor  145  may define each image pixel  310  as valid or invalid according to its intensity value. 
         [0038]    For example, the image pixel  310 ( 1 ) from the first capture T 1  with an intensity value above a first predetermined threshold (e.g., the upper threshold TH U ) may be defined as an invalid image pixel  330 ( 1 ), and the image pixel with an intensity value at or below the first predetermined threshold may be defined as a valid image pixel  335 ( 1 ) (S 215 ). Similarly, the image pixel  310  from the second capture T 2  with an intensity value below a second predetermined threshold (e.g., the lower threshold TH L ) may be defined as an invalid image pixel  330 ( 2 ), (S 220 ) and the image pixel  310 ( 2 ) with an intensity value at or above the second predetermined threshold may be defined as a valid image pixel  335 ( 2 ). In other embodiments, such as those having more than two image captures  315 , the image signal processor  145  may identify and define each of the resulting image pixels  310  formed from the additional image captures  315  according to their intensity value as described above. Referring to  FIG. 3 , the valid image pixels  335  are illustrated as white image pixels and the invalid image pixels  330  are illustrated as black image pixels. 
         [0039]    In various embodiments, if the image signal processor  145  identifies invalid image pixels  330 , the image signal processor  145  may modify, by removing and replacing, those invalid image pixels  330 . In the current embodiment, if the image captures T 1 , T 2  comprise invalid image pixels  330 , then a neighborhood search  225  for the nearest valid image pixels  335  may be performed. 
         [0040]    The image signal processor  145  may perform the neighborhood search  225  comprising detecting valid image pixels  335  within a neighborhood of the invalid image pixel  330 . The neighborhood of the invalid image pixel  330  may be defined in any suitable manner, such as image pixels in the same row or column or within a particular range of the invalid image pixel. In the present embodiment, neighborhood image pixels are defined as those pixels sharing the same row as the invalid image pixel  330 . For example, the image signal processor  145  may detect the valid image pixels  335  to the left and to the right of the invalid image pixel  330 . 
         [0041]    The neighborhood of the invalid pixel  330  may also be defined as the image pixels  310  in a predetermined number of rows above and/or below the invalid pixel  330 . In various embodiments, the total number of rows defining the neighborhood may be selected such that averaging the intensity ratios for those selected rows produces an accurate representation of a flicker waveform. For example, the image signal processor  145  may detect the valid pixels  335  in two (2) rows above and two (2) rows below the row with the invalid image pixel  330 , for a total of five (5) rows. In other embodiments, a total of three (3) rows may define the neighborhood (one row above and one row below the row of interest). 
         [0042]    In various embodiments, the image signal processor  145  may compute a maximum number of total rows to define the neighborhood. The following equations produce an estimated flicker waveform as a function of the row number, and may be used to determine the maximum number of rows that may be used to define the neighborhood. 
         [0000]        r   1 ( i )= mt   s1   +A   1  cos(ω i+θ   1 )  (equation 1)
 
         [0000]        r   2 ( i )= mt   s2   +A   2  cos(ω i+θ   2 )  (equation 2)
 
         [0043]    Where, i=row number; a=flicker amplitude; m=flicker mean; t flicker =flicker period: t r =row time; t t =frame time; n=frame number; t e1 =T 1  integration time; t e2 =T 2  integration time; r 1 (i)=T 1  flicker at row i; r 2 (i)=T 2  flicker at row i; 
         [0000]    
       
         
           
             
               
                 A 
                 1 
               
               = 
               
                 
                   
                     at 
                     flicker 
                   
                   π 
                 
                  
                 sin 
                  
                 
                   
                     
                       t 
                       
                         s 
                          
                         
                             
                         
                          
                         1 
                       
                     
                      
                     π 
                   
                   
                     t 
                     flicker 
                   
                 
               
             
             ; 
           
         
       
       
         
           
             
               
                 A 
                 2 
               
               = 
               
                 
                   
                     at 
                     flicker 
                   
                   π 
                 
                  
                 sin 
                  
                 
                   
                     
                       t 
                       
                         s 
                          
                         
                             
                         
                          
                         2 
                       
                     
                      
                     π 
                   
                   
                     t 
                     flicker 
                   
                 
               
             
             ; 
           
         
       
       
         
           
             
               ω 
               = 
               
                 
                   2 
                    
                   
                       
                   
                    
                   π 
                    
                   
                       
                   
                    
                   
                     t 
                     r 
                   
                 
                 
                   t 
                   flicker 
                 
               
             
             ; 
           
         
       
       
         
           
             
               
                 θ 
                 1 
               
               = 
               
                 
                   
                     2 
                      
                     n 
                      
                     
                         
                     
                      
                     
                       t 
                       frame 
                     
                      
                     π 
                   
                   
                     t 
                     flicker 
                   
                 
                 + 
                 
                   
                     π 
                      
                     
                         
                     
                      
                     
                       t 
                       
                         s 
                          
                         
                             
                         
                          
                         1 
                       
                     
                   
                   
                     t 
                     flicker 
                   
                 
               
             
             ; 
             and 
           
         
       
       
         
           
             
               θ 
               2 
             
             = 
             
               
                 
                   2 
                    
                   n 
                    
                   
                       
                   
                    
                   
                     t 
                     frame 
                   
                    
                   π 
                 
                 
                   t 
                   flicker 
                 
               
               + 
               
                 
                   π 
                    
                   
                       
                   
                    
                   
                     t 
                     
                       s 
                        
                       
                           
                       
                        
                       1 
                     
                   
                 
                 
                   t 
                   flicker 
                 
               
               + 
               
                 
                   
                     π 
                      
                     
                         
                     
                      
                     
                       t 
                       
                         s 
                          
                         
                             
                         
                          
                         2 
                       
                     
                   
                   
                     t 
                     flicker 
                   
                 
                 . 
               
             
           
         
       
     
         [0044]    In various embodiments, the flicker period t flicker  may calculated based on the frequency of the alternating current supply. For example, in some countries, the frequency is 50 Hz, while in other countries the frequency is 60 Hz. 
         [0045]    In general, the maximum number of rows that may be used to define a neighborhood equals the number of rows corresponding to one flicker period according to the equations 1 and 2 above. For example, if there are 10 rows per flicker period, then the maximum number of rows that may be used to define a neighborhood will be 10 rows, although the maximum number may not be an ideal number due to hardware constraints, such as an increase in the number of line buffers. 
         [0046]    The graphs in  FIGS. 9A-B  illustrate that as the total number of rows used to define the neighborhood increases, the less accurate the estimated flicker waveform will be.  FIG. 9A  illustrates a 3-row neighborhood, where the estimated flicker waveform is substantially similar to and representative of the actual flicker waveform. Conversely,  FIG. 9B  illustrates an 8 row-neighborhood, where the estimated flicker waveform does not accurately represent the actual flicker waveform. 
         [0047]    The image signal processor  145  may, during the neighborhood search  225 , compute a single representative value based on the neighboring valid image pixels  335 . For example, the representative value may be a computed average (i.e., arithmetic mean) of the neighboring valid image pixels  335 . In other embodiments, the representative value may be a median, a mode, a geometric mean, or any other computation suitable for obtaining a single value. In an exemplary embodiment, the representative value may be used to modify and/or replace the invalid image pixel  330 . 
         [0048]    Referring now to  FIGS. 7A-B , the image signal processor  145  may perform the neighborhood search any number of ways.  FIG. 7A  illustrates one exemplary embodiment of the neighborhood search in a 2-image capture case, wherein the intensity value for each of the image pixels from the first and second captures T 1 , T 2  is identified as valid or invalid. An intensity ratio may be calculated, for example by dividing the intensity value for each image pixel from the first capture T 1  with the intensity value of the same spatially located image pixel from second capture T 2 . The intensity ratio is then identified as valid or invalid using the same threshold criteria described above. For each invalid image pixel, the neighborhood of the invalid image pixel is searched to find valid intensity ratios. Once valid intensity ratios are identified, the representative value may be computed. The representative value may then be used to modify and/or replace the invalid image pixels to form a new first pixel map for capture  1 . 
         [0049]      FIG. 7B  illustrates another exemplary embodiment of the neighborhood search in a 2-image capture case, wherein the intensity value for each of the image pixels from the first and second captures is identified as valid or invalid. For each capture, the neighborhood search is performed to locate the image pixels with valid intensity values for each invalid intensity value. Once the valid image pixels are identified, the representative value may be computed. The representative value may be used to modify and/or replace the invalid image pixels  330  to form a corrected first and second pixel maps for captures  1  and  2  respectively. A new pixel map is then formed by dividing the image pixel intensity values of each of the corrected first pixel map by the same spatially located image pixels of the corrected second pixel map. The new pixel map may then be used to determine the predicted values described below. 
         [0050]    In various embodiments, then the image signal processor  145  may perform a ratio calculation  235  to compute a “predicted” value. A priori knowledge of the fixed integration time ratios between the integration times may be utilized to compute the “predicted” value for a particular color channel. For example, if there is no motion in the scene, the integration time ratios remain unchanged between the image captures  315  and a flicker-free value can be obtained by calculating the gain necessary to nullify the effect of flicker. As such, the “predicted” value will be flicker-free, and since the invalid image pixels  330  are modified, for example removed and replaced, with valid image pixels  335 , the “predicted” value will also be valid. 
         [0051]    In various embodiments, the image signal processor  145  may perform the ratio calculation  235  utilizing valid image pixel values, where the valid image pixel values may be the computed values for the new pixel map generated during the neighborhood search ( 225 ) and/or the original image pixel values. The ratio calculation  235  may be between two image captures  315  for the same color channel ( 240 ) (i.e., same-color channel). For example, in the 2-image capture system, since the first capture T 1  and the second capture T 2  have the same spatially located image pixels  310 , they also have corresponding color channels. Therefore, the ratio calculation  235  may be performed by dividing the pixel intensity value of the color channel of first capture T 1  with the corresponding pixel intensity value of the same color channel of the second capture T 2 . For example, referring to  FIG. 5 , the CFA  155  with the Bayer color pattern may form the following same-color channel ratios: R 1 /R 2 , B 1 /B 2 , G 11 /G 12  and G 21 /G 22 . In other embodiments, utilizing a CFA  155  other than a Bayer color pattern, same-color channel ratios may be formed based on the colors of the CFA  155  being used. 
         [0052]    Alternatively, the image signal processor  145  may perform the ratio calculation ( 235 ) between differing color channels within the same image ( 245 ) (i.e. cross-color channel). For example, the image signal processor  145  may perform ratio calculation ( 235 ) between differing color channels within the first capture T 1  (e.g. B 1 /G 21  as shown in  FIG. 5  using a Bayer color pattern), or between differing color channels within the second capture T 2  (e.g. B 2 /G 22  as shown in  FIG. 5  using a Bayer color pattern). 
         [0053]    In various embodiments, where invalid image pixels are identified and the neighborhood search is performed to identify the valid image pixels to create the new corrected image pixel map, the new corrected image pixel map is used by the image signal processor  145 , in place of the original first image capture T 1 , to perform either the same-channel ratio calculation or the cross-channel ratio calculation. 
         [0054]    Referring now to  FIGS. 6A-D , the image signal processor  145  may perform ratio calculation ( 235 ) on alternative CFA  155  configurations in the same manner described above. While the CFA  155  shown in  FIGS. 6A-D  illustrate various configurations using the Bayer colors (i.e. red, green, and blue), it should be understood that any CFA  155  may be manipulated to form various color patterns. 
         [0055]    In various embodiments, manual or automatic selection, via the switch, between the same-color channel ratio calculation ( 240 ) and the cross-color channel ratio calculation ( 245 ) may be based on the presence of a particular light source. Some light sources, such as florescent light, may produce variations between different color channels, such that the flicker amplitude and mean for one color channel may differ from that of another color channel. In this case, the same-color channel ratio calculation ( 240 ) may be selected and performed because a calculated gain in one color channel may not be the correct value for minimizing the flicker in another color channel. 
         [0056]    In various embodiments, the image signal processor  145  may perform filtering on the gain ratios ( 250 ) to remove unwanted noise in the signals. Filtering ( 250 ) may comprise utilizing any suitable filter to achieve a desired result. For example, filtering ( 250 ) may comprise a median filter, averaging filter, infinite impulse response filters, or any combination thereof. In various embodiments, the number of rows in a kernel may be selected by design so that the spatial extent of a single flicker period is not exceeded. In an exemplary embodiment, the gain ratios are filtered, rather than the actual pixel data. 
         [0057]    The image signal processor  145  may normalize the calculated ratios utilizing a known ideal ratio R ideal  ( 255 ). In various embodiments, the image signal processor  145  may normalize the calculated ratio ( 255 ) for both same-color channel ratios and cross-color channel ratios. 
         [0058]    A general expression for cross-channel normalization and prediction may be summarized in the following equations: 
         [0000]    
       
         
           
             
               
                 V 
                  
                 
                     
                 
                  
                 
                   1 
                   icc 
                 
               
               
                 V 
                  
                 
                     
                 
                  
                 
                   1 
                   cc 
                   ′ 
                 
               
             
             = 
             
               
                 V 
                  
                 
                     
                 
                  
                 
                   2 
                   icc 
                 
               
               
                 V 
                  
                 
                     
                 
                  
                 
                   2 
                   cc 
                 
               
             
           
         
       
     
         [0059]    Therefore, 
         [0000]    
       
         
           
             
               V 
                
               
                   
               
                
               
                 1 
                 cc 
                 ′ 
               
             
             = 
             
               
                 
                   V 
                    
                   
                       
                   
                    
                   
                     1 
                     icc 
                   
                 
                 
                   V 
                    
                   
                       
                   
                    
                   
                     2 
                     icc 
                   
                 
               
               × 
               V 
                
               
                   
               
                
               
                 2 
                 cc 
               
             
           
         
       
     
         [0060]    Now, 
         [0000]    
       
         
           
             
               V 
                
               
                   
               
                
               
                 2 
                 cc 
                 ′ 
               
             
             = 
             
               
                 V 
                  
                 
                     
                 
                  
                 
                   1 
                   cc 
                   ′ 
                 
               
               
                 R 
                 ideal 
               
             
           
         
       
     
         [0061]    Hence, 
         [0000]    
       
         
           
             
               V 
                
               
                   
               
                
               
                 2 
                 cc 
                 ′ 
               
             
             = 
             
               V 
                
               
                   
               
                
               
                 2 
                 cc 
               
               × 
               
                 
                   
                     V 
                      
                     
                         
                     
                      
                     
                       1 
                       icc 
                     
                   
                   
                     V 
                      
                     
                         
                     
                      
                     
                       2 
                       icc 
                     
                   
                 
                 
                   R 
                   ideal 
                 
               
             
           
         
       
     
         [0062]    Where, V 1 ′ cc =The predicted intensity value of the color channel that is under consideration in the first capture T 1 ; 
         [0063]    V 2 ′ cc =The predicted intensity value of the color channel that is under consideration in the second capture T 2 ; 
         [0064]    V 2   cc =The original intensity value of the color channel that is under consideration in the second capture T 2 ; 
         [0065]    V 1   icc =The original intensity value of the color channel that is being used as the basis for the prediction in the first capture T 1 ; 
         [0066]    V 2   icc =The original intensity value of the color channel that is being used as the basis for the prediction in the second capture T 2 ; and 
         [0067]    R ideal =The ideal known ratio between the integration times for the first and second captures T 1 , T 2 . 
         [0068]    Consider the following example, utilizing a CFA  155  with a Bayer color pattern, as illustrated in  FIG. 5 . 
         [0069]    Let V 1 ′ cc =G 21 ′ i.e. the predicted intensity value of the Green channel on the Green-Blue row in the first capture T 1 ; 
         [0070]    V 2 ′ cc =G 22 ′ i.e. the predicted intensity value of the Green channel on the Green-Blue row in the second capture T 2 ; 
         [0071]    V 2   cc =G 22  i.e. the original intensity value of the Green channel on the Green-Blue row in the second capture T 2 ; 
         [0072]    V 1   icc =B 1 , i.e. the original intensity value of the Blue channel that is being used as the basis for the prediction in the first capture T 1 ; 
         [0073]    V 2   icc =B 2 , i.e. the original intensity value of the Blue channel that is being used as the basis for the prediction in the second capture T 2 ; and 
         [0074]    R ideal =The ideal known ratio between the integration times for the first and second capture T 2 . 
         [0000]    
       
         
           
             
               
                 B 
                  
                 
                     
                 
                  
                 1 
               
               
                 G 
                  
                 
                     
                 
                  
                 
                   21 
                   ′ 
                 
               
             
             = 
             
               
                 B 
                  
                 
                     
                 
                  
                 2 
               
               
                 G 
                  
                 
                     
                 
                  
                 22 
               
             
           
         
       
       
         
           
             
               G 
                
               
                   
               
                
               
                 21 
                 ′ 
               
             
             = 
             
               
                 
                   B 
                    
                   
                       
                   
                    
                   1 
                 
                 
                   B 
                    
                   
                       
                   
                    
                   2 
                 
               
               × 
               
                   
               
                
               G 
                
               
                   
               
                
               22 
             
           
         
       
       
         
           
             
               G 
                
               
                   
               
                
               
                 22 
                 ′ 
               
             
             = 
             
               
                 
                   
                     G 
                      
                     
                         
                     
                      
                     
                       21 
                       ′ 
                     
                   
                   
                     R 
                     ideal 
                   
                 
                  
                 
                   
 
                 
                  
                 G 
                  
                 
                     
                 
                  
                 
                   22 
                   ′ 
                 
               
               = 
               
                 G 
                  
                 
                     
                 
                  
                 22 
                  
                 
                     
                 
                 × 
                 
                   
                     
                       B 
                        
                       
                           
                       
                        
                       1 
                     
                     
                       B 
                        
                       
                           
                       
                        
                       2 
                     
                   
                   
                     R 
                     ideal 
                   
                 
               
             
           
         
       
     
         [0075]    Predicted values for the remaining channels may be obtained in the same way. 
         [0076]    A general expression for same-channel normalization and prediction is described below: 
         [0000]    
       
         
           
             
               V 
                
               
                   
               
                
               
                 2 
                 sc 
                 ′ 
               
             
             = 
             
               V 
                
               
                   
               
                
               
                 2 
                 sc 
               
               × 
               
                 
                   
                     V 
                      
                     
                         
                     
                      
                     
                       1 
                       sc 
                     
                   
                   
                     V 
                      
                     
                         
                     
                      
                     
                       2 
                       sc 
                     
                   
                 
                 
                   R 
                   ideal 
                 
               
             
           
         
       
     
         [0077]    Where, V 2 ′ sc =The predicted intensity value of the color channel that is under consideration in the second capture T 2 ; 
         [0078]    V 2   sc =The original intensity value of the color channel that is under consideration in the second capture T 2 ; 
         [0079]    V 1   sc =The original intensity value of the same color channel that under consideration in the first capture T 1 ; and 
         [0080]    R ideal =The ideal known ratio between the integration times for the first and second image captures T 1 , T 2 . 
         [0081]    Consider the following example utilizing a CFA  155  with a Bayer color pattern, as illustrated in  FIG. 5 . 
         [0082]    Let V 2 ′ sc =G 22 ′ i.e., the predicted intensity value of the Green channel on the Green-Blue row in the second capture T 2 ; 
         [0083]    V 2   sc =G 22  i.e. the original intensity value of the Green channel on the Green-Blue row in the second capture T 2 ; 
         [0084]    V 1   sc =G 21  i.e. the original intensity value of the Green channel on the Green-Blue row in the first capture T 1 ; and 
         [0085]    R ideal =The ideal known ratio between the integration times for the first and second image captures T 1 , T 2 . 
         [0086]    Therefore, 
         [0000]    
       
         
           
             
               G 
                
               
                   
               
                
               
                 22 
                 ′ 
               
             
             = 
             
               G 
                
               
                   
               
                
               22 
               × 
               
                 
                   
                     G 
                      
                     
                         
                     
                      
                     21 
                   
                   
                     G 
                      
                     
                         
                     
                      
                     22 
                   
                 
                 
                   R 
                   ideal 
                 
               
             
           
         
       
     
         [0087]    Predicted values for the remaining channels may be obtained in the same way. 
         [0088]    In various embodiments, the intensity values of the image pixels  310  selected from each of the first and second captures T 1 , T 2  may share the same spatially located pixel  110 . 
         [0089]    In various embodiments, the predicted intensity values may be used to modify and/or replace the intensity values of the corresponding image pixels  310  in the second capture T 2  ( 260 ) to form a new flicker-free image. 
         [0090]    In various embodiments where more than two image captures  315  are desired, the same process  200  described above may be applied to subsequent image captures  315 . For example, in a case comprising four image captures T 1 , T 2 , T 3 , T 4 , each with differing integration times, the first two image captures T 1 , T 2  be processed according to the process described above. The subsequent image captures (e.g. image captures three and four T 3 , T 4 ) may be cascaded with the first and second image captures T 1 , T 2 . For example, in a cross-color channel case, the ratio calculations formed to obtain a new flicker-free third image, may be made between the first capture and third captures T 1 , T 3 , or between the second and third captures T 2 , T 3 , using the same ratio calculations and normalization described above, or between the second and third captures T 2 , T 3  using the predicted values from the second capture T 2 , and between the third and fourth captures T 3 , T 4  using the predicted values from the third capture T 3 . 
         [0091]    In the foregoing description, the technology has been described with reference to specific exemplary embodiments. The particular implementations shown and described are illustrative of the technology and its best mode and are not intended to otherwise limit the scope of the present technology in any way. Indeed, for the sake of brevity, conventional manufacturing, connection, preparation, and other functional aspects of the method and system may not be described in detail. Furthermore, the connecting lines shown in the various figures are intended to represent exemplary functional relationships and/or steps between the various elements. Many alternative or additional functional relationships or physical connections may be present in a practical system. 
         [0092]    While the technology has been described with reference to specific exemplary embodiments, various modifications and changes may be made without departing from the scope of the present technology. The description and figures are to be regarded in an illustrative manner, rather than a restrictive one and all such modifications are intended to be included within the scope of the present technology. Accordingly, the scope of the technology should be determined by the generic embodiments described and their legal equivalents rather than by merely the specific examples described above. For example, the steps recited in any method or process embodiment may be executed in any order, unless otherwise expressly specified, and are not limited to the explicit order presented in the specific examples. Additionally, the components and/or elements recited in any apparatus embodiment may be assembled or otherwise operationally configured in a variety of permutations to produce substantially the same result as the present technology and are accordingly not limited to the specific configuration recited in the specific examples. 
         [0093]    Benefits, other advantages and solutions to problems have been described above with regard to particular embodiments. Any benefit, advantage, solution to problems or any element that may cause any particular benefit, advantage or solution to occur or to become more pronounced, however, is not to be construed as a critical, required or essential feature or component. 
         [0094]    The terms “comprises”, “comprising”, or any variation thereof, are intended to reference a non-exclusive inclusion, such that a process, method, article, composition or apparatus that comprises a list of elements does not include only those elements recited, but may also include other elements not expressly listed or inherent to such process, method, article, composition or apparatus. Other combinations and/or modifications of the above-described structures, arrangements, applications, proportions, elements, materials or components used in the practice of the present technology, in addition to those not specifically recited, may be varied or otherwise particularly adapted to specific environments, manufacturing specifications, design parameters or other operating requirements without departing from the general principles of the same. 
         [0095]    The present technology has been described above with reference to an exemplary embodiment. However, changes and modifications may be made to the exemplary embodiment without departing from the scope of the present technology. These and other changes or modifications are intended to be included within the scope of the present technology, as expressed in the following claims.