Abstract:
Even when the light intensity of the light source varies, the flicker correction can thus be made flexibly. The present invention provides a flicker correction method comprising the steps of predicting, from an image of a present flicker-corrected frame, a flicker of an image of a next frame to generate two types of flicker images having flickers different in level from each other added thereto, detecting a flicker component through comparison between the generated two types of flicker images and an image of an input next frame, generating a flicker correction value on the basis of the detected flicker component, and making flicker correction by adding the generated flicker correction value to an input image frame by frame.

Description:
CROSS REFERENCES TO RELATED APPLICATIONS  
       [0001]     The present invention contains subject matter related to Japanese Patent Application JP 2005-141614 filed in the Japanese Patent Office on May 13, 2005, the entire content of which is incorporated herein by reference.  
       BACKGROUND OF THE INVENTION  
       [0002]     1. Field of the Invention  
         [0003]     The present invention relates to a flicker correction method and device, and an image pickup device, in which a flicker is corrected by subtracting a flicker correction signal from an image signal.  
         [0004]     2. Description of the Related Art  
         [0005]     In the conventional image sensor, the timing of charge storage differs depending upon whether the charge storage is made per plane or per line. Generally, timing of the charge storage per plane is called “global shutter system” while timing of the charge storage per line is called “rolling shutter system”. Most of the CCDs have adopted an image sensor of the global shutter type in the past. Recently, however, increasingly more attention has been paid to the CMOS image sensors that consume less power than the CCDs and can be produced more inexpensively than the CCDs because of their smaller number of parts. Many of the CMOS image sensors adopt the rolling shutter system for their structural problem. With one of the two shutter systems, when imaging is made under the light of a light source repeating turning on and off, light and dark fringes will appear in an entire image plane (plane flicker), while with the other shutter system, light and dark fringes will appear per line (in-plane flicker).  
         [0006]      FIG. 1  shows a difference in amount of charge storage in an image sensor adopting the global shutter system, and  FIG. 2  shows an example of image incurring a plane flicker when the image sensor is of the global shutter type.  FIG. 3  shows a difference in amount of charge storage in an image sensor adopting the rolling shutter system, and  FIG. 4  shows an example of image incurring an in-plane flicker. Also, a flicker component included in an image captured under the light of a light source cyclically turning on and off can be approximated to a sinusoidal wave, and there has been prevalent the method of forming a corrected image by removing the flicker on the basis of the nature of the sinusoidal wave.  
         [0007]     For the flicker correction, there was proposed a method of controlling the gain on the basis of a flicker component detected in an input image (as in the Japanese Patent Application Laid Open No. 2004-222228.  
       SUMMARY OF THE INVENTION  
       [0008]     When an object is imaged with a digital camera under the light of a light source that repeats turning on and off cyclically, such as a fluorescent lamp, cyclic light and dark fringes will appear in a captured image of the object, resulting in that they will seemingly run in the image. Otherwise, there will cyclically take place a difference in brightness between frames over an image. This is called “flicker”. The flicker is a problem unavoidably taking place when an object is imaged with a digital camera using an image sensor to image the object under the light of a flickering light source with the timing of charge storage being shifted.  
         [0009]     For flicker correction, the feature that a flicker can be approximated to a sinusoidal wave is utilized to detect a flicker component in an input signal. Similarly, a correction amount is calculated from the characteristic of the sinusoidal wave and detected flicker component, and the correction amount is added to the input signal or the latter is multiplied by the correction amount. For approximation of the flicker component to the sinusoidal wave, three features “cycle”, “phase” and “amplitude” have to be detected.  
         [0010]     A cycle can be detected based on a power supply frequency and frame rate.  
         [0011]     On this account, the Applicant of the present invention proposed a flicker correction method including the steps of acquiring a flicker correction signal corresponding to a flicker component included in each of specific periods of a video signal formed from a succession of the specific periods and containing the flicker component in response to a correction error signal for each specific period and calculating the flicker correction signal and each specific period to generate a corrected video signal for one specific period, whose flicker component has been corrected, detecting a correction error of the flicker component in the corrected video signal for one specific period before each of the specific periods and each specific period to acquire the correction error signal as one corresponding to the detected correction error, and acquiring the flicker correction signal as one which reduces the correction error correspondingly to the correction error signal (as in the Japanese Patent Application Laid Open No. 2004-330299).  
         [0012]     The above flicker correction method is performed with no flicker amplitude being detected but with a fixed value of flicker. In case the light from the light source does not vary, the method can be performed without any problem even with a fixed value of flicker amplitude. However, in case the light from the light source varies, the flicker amplitude has to be changed appropriately. This conventional method is thus not versatile.  
         [0013]     It is therefore desirable to overcome the above-mentioned drawbacks of the related art by providing a flicker correction method and apparatus and an image pickup device, in which flicker can flexibly be corrected even when the light intensity of a light source varies.  
         [0014]     According to the present invention, flicker correction is made by detecting a flicker component in an input image signal, calculating a correction value based on the detected flicker component and adding the correction value to the input image signal.  
         [0015]     According to the present invention, there is provided a flicker correction method including the steps of:  
         [0016]     predicting, from an image of a present flicker-corrected frame, a flicker of an image of a next frame to generate two types of flicker images having flickers different in level from each other added thereto;  
         [0017]     detecting a flicker component through comparison between the generated two types of flicker images and an image of an input next frame;  
         [0018]     generating a flicker correction value on the basis of the detected flicker component; and  
         [0019]     making flicker correction by adding the generated flicker correction value to an input image frame by frame.  
         [0020]     According to the present invention, there is also provided a flicker correction device including:  
         [0021]     a flicker correcting means for making flicker correction by adding a flicker correction signal to an input image signal; and  
         [0022]     a flicker correction signal generating means for predicting a flicker of an image of a next frame from the image signal having been flicker-corrected by the flicker correcting means and an image signal not yet flicker-corrected to generate two types of flicker images to which flickers different in level from each other, detecting a flicker component through comparison between the generated two types of flicker images and an image of an input next frame and generating a flicker correction value on the basis of the detected flicker component,  
         [0023]     the flicker correcting means adding, to the input image signal, the flicker correction signal generated frame by frame by the flicker correction signal generating means to make flicker correction.  
         [0024]     According to the present invention, there is also provided an image pickup device including a flicker correction device to make flicker correction by adding a flicker correction signal to an image signal captured by the image pickup device, the flicker correction device including:  
         [0025]     a flicker correcting means for making flicker correction by adding a flicker correction signal to an input image signal; and  
         [0026]     a flicker correction signal generating means for predicting a flicker of an image of a next frame from the image signal having been flicker-corrected by the flicker correcting means and an image signal not yet flicker-corrected to generate two types of flicker images to which flickers different in level from each other, detecting a flicker component through comparison between the generated two types of flicker images and an image of an input next frame and generating a flicker correction value on the basis of the detected flicker component,  
         [0027]     the flicker correcting means adding, to the input image signal, the flicker correction signal generated frame by frame by the flicker correction signal generating means to make flicker correction.  
         [0028]     According to the present invention, the amplitude of flickers can sequentially be predicted through comparison between an amplitude-predicted flicker image and an image captured by an image sensing device (will also be referred to as “imaging element” herein). Even when the light intensity of the light source varies, the flicker amplitude can flexibly be adjusted in an automatic manner to correct a flicker at any time. Also, even when imaging is made while relocating from a place with a flicker source to a flicker-free place, the flicker amplitude can automatically be varied and the flicker correction be ceased. Further, even when imaging is made while relocating a flicker-free place to a place with a flicker source, the flicker amplitude can automatically be varied and the flicker correction be made.  
         [0029]     Therefore, according to the present invention, even when the light intensity of the light source varies, the flicker correction can thus be made flexibly. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0030]      FIG. 1  schematically illustrates a difference in amount of charge storage in an image sensor of the global shutter type;  
         [0031]      FIG. 2  schematically illustrates an example of plane flicker image appearing when the global shutter system is adopted;  
         [0032]      FIG. 3  schematically illustrates a difference in amount of charge storage in an image sensor of the rolling shutter type;  
         [0033]      FIG. 4  schematically illustrates an example of in-plane flicker image;  
         [0034]      FIG. 5  is a schematic block diagram of an image pickup device as one embodiment of the present invention;  
         [0035]      FIG. 6  is also a schematic block diagram of a flicker correction circuit included in the image pickup device shown in  FIG. 5 ;  
         [0036]      FIG. 7  is a schematic block diagram of a correction value calculator in the flicker correction circuit;  
         [0037]      FIG. 8  schematically illustrates an algorithm for correction error detection in the image pickup device;  
         [0038]      FIG. 9  is a schematic block diagram of a correction phase error detector included in the image pickup device;  
         [0039]      FIG. 10  is a schematic block diagram of a correction amplitude error detector included in the image pickup device;  
         [0040]      FIG. 11  is a schematic block diagram of a flicker amplitude adjuster included in the image pickup device. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0041]     The present invention will be described in detail below concerning the embodiments thereon with reference to the accompanying drawings. It should be noted that the present invention is not limited to the embodiments which will be described herebelow but it may be can be modified in various manners, constructed alternatively or embodied in various other forms without departing from the scope and spirit thereof.  
         [0042]     The present invention is applicable to an image pickup device constructed as shown in  FIG. 5 . The image pickup device is generally indicated with a reference numeral  100 .  
         [0043]     The image pickup device  100  includes a red color image sensing device (image element)  10 R, green color image sensing device (imaging element)  10 G, blue color image sensing device (imaging element)  10 B, A-D converters  20 R,  20 G and  20 B to digitize image signals SI_R, SI_G and SI_B of color images captured by the image sensing devices  10 R,  10 G and  10 B, respectively, flicker correction circuits  30 R,  30 G and  30 B, correction phase error detectors  40 R,  40 G and  40 B and correction amplitude error detectors  50 R,  50 G and  50 B, supplied with the image signals DV_R, DV-G and DV_B digitized by the A-D converters  20 R,  20 G and  20 B, respectively, flicker amplitude adjusters  60 R,  60 G and  60 B supplied with correction amplitude error signals C_R, C_G and C_B detected by the correction amplitude error detectors  50 R,  50 G and  50 B, respectively, camera signal processing circuit  70  supplied with image signals CV_R, CV_G and CV_B flicker-corrected by the flicker correction circuits  30 R,  30 G and  30 B, respectively, etc.  
         [0044]     Supplied with the image signals CV_R, CV_G and CV_B flicker-corrected by the flicker correction circuits  30 R,  30 G and  30 B, respectively, and with the flicker amplitude signals A_R, A_G and A_B adjusted by the flicker amplitude adjusters  60 R,  60 G and  60 B, respectively, the correction phase error detectors  40 R,  40 G and  40 B detect correction phase errors of the image signals CV_R, CV_G and CV_B in the digitized image signals DV_R, DV_G and DV_, flicker-corrected image signals CV_R, CV_G and CV-B and flicker amplitude signals A_R, A_G and A_B to generate correction phase error signals E_R, E_G and E_B, and supply the generated correction amplitude error signals E_R, E_G and E_B to the flicker correction circuits  30 R,  30 G and  30 B, and correction amplitude error detectors  50 R,  50 G and  50 B, respectively.  
         [0045]     Supplied with the image signals CV_R, CV_G and CV_B flicker-corrected by the flicker correction circuits  30 R,  30 G and  30 B, respectively, and with the flicker amplitude signals A_R, A_G and A_B adjusted by the flicker amplitude adjusters  60 R,  60 G and  60 B, respectively, the correction amplitude error detectors  50 R,  50 G and  50 B detect correction phase errors of the image signals CV_R, CV_G and CV_B in the digitized image signals DV_R, DV_G and DV_B, flicker-corrected image signals CV_R, CV_G and CV_B and flicker amplitude signals A_R, A_G and A_B and correction phase error signals E_R, E_G and E_B to generate correction amplitude error signals C_R, C_G and C_B, and supply the generated correction amplitude error signals E_R, _G and E_B to the flicker amplitude adjusters  60 R,  60 G and  60 B, respectively.  
         [0046]     The flicker amplitude adjusters  60 R,  60 G and  60 B generate flicker amplitude signals A_R, A_G and A_B from the correction amplitude error signals E_R, E_G and E_B, respectively, and supplies the generated flicker amplitude signals A_R, A_G and A_B to the flicker correction circuits  30 R,  30 G and  30 B, correction phase error detectors  40 R,  40 G and  40 B and correction amplitude error detectors  50 R,  50 G and  50 B, respectively.  
         [0047]     In the image pickup device  100 , each of the flicker correction circuits  30 R,  30 G and  30 B uses a flicker correction circuit  30 * constructed as shown in  FIG. 6 . It should be noted here that the asterisk (*) stands for “R (red)”, “G (green)” and “B (blue)”.  
         [0048]     The flicker correction circuit  30 * includes an address calculator  31 * supplied with a correction error signal E_* from the correction error detector  40 *, correction value calculator  32 * supplied with an address AD calculated by the address calculator  31 *, multiplier  33 * supplied with flicker correction data FD calculated by the correction value calculator  32 * and flicker amplitude signal A_* generated by the flicker amplitude adjuster  60 *, level adjuster  34 * supplied with flicker correction data FDA resulted from multiplication of the flicker correction data FD by the flicker amplitude signal A_* in the multiplier  33 , and a low-pass filter (LPF)  35 * and operational circuit  36 *, supplied with an image signal DV_* digitized by the A-D converter  20 *. The image signal DV_* digitized by the A-D converter  20 * is supplied, via the low-pass filter (LPF)  35 *, to the level adjuster  34 * that will then generate a flicker correction value CFD which is to be supplied to the operational circuit  36 *.  
         [0049]     In the flicker correction circuit  30 * constructed as above, the address calculator  31 * calculates an address AD in ROMs (flicker memories  321  and  322  which will further be described in detail later) included in the correction value calculator  32 * on the basis of the correction error signal E_* supplied from the correction error detector  40 *.  
         [0050]     The address calculator  31 * calculates the address of a present line by calculating the address of a first line in a frame of interest from a power supply frequency and frame rate, and calculating an address increment at each advance by one line toward the address. More specifically, in case the power supply frequency is 50 Hz, frame rate is 30 Hz and the number of vertical clocks of the image sensing device  10 * is 1125 clk (these power supply frequency, frame rate and number of clocks of the image sensing device  10 * remain unchanged through the following description), the period T between light and dark fringes of a flicker will contain 337.5 lines as given below by an equation 1: 
 
 T =30 Hz×1125 clk/(50 Hz×2)=37.5 (clk)  (1) 
 
 Also, the ROM in the system holds flicker data resulted from division of one period by 512. At each advance by one line, the address in the ROM will be incremented by about 1.51703 as given below by an equation 2: 
 
512/337.5=1.51703  (2) 
 
 That is, on the assumption that the correction wave address on the first line is zero (0), the address on the 100th line counted from the first line will be 152 as given below by an equation 3: 
 
0+1.51703×100≈152  (3) 
 
         [0051]     As shown in  FIG. 7 , the correction value calculator  32 * includes flicker memories  321  and  322 , multipliers  323  and  324  to multiply two types of flicker data FD 1  and FD 2  read from the flicker memories  321  and  322  by coefficients α and α−1, respectively, and an adder  325  supplied with flicker data FD 1 _A and FD 2 _A multiplied by the coefficients α and α−1, respectively, by the multipliers  323  and  324 , respectively. The two types of flicker data FD 1  and FD 2  will be read from the flicker memories  321  and  322 , respectively, according to the address AD calculated by the address calculator  31 *.  
         [0052]     The correction value calculator  32 * calculates one flicker correction data FD by reading the two types of flicker data FD 1  and FD 2  from the flicker memories  321  and  322 , respectively, on the basis of the address AD calculated by the address calculator  31 *, multiplying the flicker data FD 1  and FD 2  by the coefficients α and α−1, respectively, by the multipliers  323  and  324 , respectively, correspondingly to a frame rate and shutter speed, and adding the results together by the adder  325 .  
         [0053]     Note that the periodicity of the flicker data is utilized, the correction value calculator  32 * is to hold a part of waveforms of the flicker data FD 1  and FD 2 . Also, flicker data can appropriately be calculated even with any other memory than the ROM. In this embodiment, one flicker correction data FD corresponding to a waveform is synthesized by combining the two flicker data FD 1  and FD 2  together. However, three or more flicker data can be combined together to synthesize various flicker correction data FD. The flicker correction data FD is updated once by a value depending upon each line per line.  
         [0054]     Since the flicker varies in level correspondingly to the value of each pixel, the level is adjusted for each by the use of the input image signal DV_*. However, a noise component in the image signal DV_* will influence the level adjustment.  
         [0055]     On this account, in the flicker correction circuit  30 * of the image pickup device  100 , the input image signal DV_* is passed through the low-pass filter (LPF)  35 * to remove the noise component from the image signal DV_* and the noise-free image signal DV_*′ is supplied to the level adjuster  34 *. The level adjuster  34 * can calculate a correction value CFD for each pixel not influenced by the noise from the noise-free image data DV_*′ and flicker correction data FD calculated by the correction value calculator  32 *.  
         [0056]     Note that this embodiment is adapted so that the correction value monotonously increases correspondingly to a pixel value for there has been observed a tendency that the flicker level also increases linearly correspondingly to a pixel value. Also, since no flicker is observed when the pixel value is extremely small or large, the embodiment is adapted to make a calculation taking this feature in account. However, the present invention is not limited to this embodiment.  
         [0057]     In the flicker correction circuit  30 *, the operational unit  35 * adds the correction value CFD for each pixel to the image signal DV_* to provide a corrected image signal CV_*.  
         [0058]     The algorithm for detection of a correction error in the image pickup device  100  will be described below with reference to  FIG. 8 .  
         [0059]     It is assumed here that a flicker of a certain frame image (image of the n-th frame) could have been corrected to a correct level. Since the flicker is continuous from one frame to another, a flicker of a next frame can be predicted. With the predicted next-frame flicker being kept at the same level as the flicker of an n-th frame, the flicker of a (n+ 1  )th frame is added to the corrected image. A flicker image thus produced is taken as “image A”. At the same time, with the level being made higher than that of the n-th frame, the flicker of the (n+1)th frame is added to the flicker-corrected image. A flicker image thus produced is taken as “image B”. The image A is an image resulted from addition of the flicker component of the (n+1)th frame to an object image of the n-th frame, while the image B is resulted from addition of the flicker component of the (n+1)th frame whose flicker level has been made higher than that of the image A to the object image of the frame n-th frame. When differences are calculated between the image of the (n+1)th frame actually supplied and images A and B, respectively, only a movement component of the object will appear as the differences because the next-frame flicker is faithfully reproduced in the image A. However, since the image B is made higher in level than the next-frame flicker, it contains two components, namely, the object movement and flicker component as the differences, that is, the predicted flicker component more approximate to the next-frame flicker will take a small value as the difference. Further, there are generated two images, namely, an image having added thereto a flicker component of the (n+1)th frame, higher in level than the flicker of the n-th frame and an image having added thereto a flicker component of the (n+1)th frame, lower in level than the flicker of the n-th frame. When differences are calculated between the two images and image of the (n+1)th frame, respectively, the difference more approximate to the flicker level of the (n+1)th frame will take a small value. Therefore, it is possible to adjust the initial level of an arbitrary flicker to an appropriate flicker level automatically as the time elapses by making a comparison between the differences for each frame.  
         [0060]     Each of the correction phase error detectors  40 R,  40 G and  40 B and correction amplitude error detectors  50 R,  50 G and  50 B is designed according to the above-mentioned algorithm.  
         [0061]     In this image pickup device  100 , each of the correction phase error detectors  40 R,  40 G and  40 B uses a correction phase error detector  40 * constructed as shown in  FIG. 9  according to the aforementioned algorithm. It should be noted here that the asterisk (*) stands for “R (red)”, “G (green)” and “B (blue)”.  
         [0062]     The correction phase error detector  40 * includes flicker-added signal generators  41 A and  41 B supplied with an image signal CV_* flicker-corrected by the flicker correction circuit  30 * and flicker amplitude signal A_* generated by the flicker amplitude adjuster  60 *, line integrators  42 A and  42 B supplied with flicker-added signals FDV 1  and FDV 2  generated by the flicker-added signal generators  41 A and  41 B, respectively, memories  43 A and  43 B supplied with line data LD 11  and LD 21  integrated by the line integrators  42 A and  42 B, respectively, difference detectors  44 A and  44 B supplied with line data LD 12  and LD 22  read from the memories  43 A and  43 B, respectively, line integrator  45  supplied with an image signal DV_* digitized by the A-D converter  20 *, integrators  46 A and  46 B supplied with difference data DD 1  and DD 2  detected by the difference detectors  44 A and  44 B, respectively, comparator  47  supplied with integrated data ID 1  and ID 2  provided by the integrators  46 A and  46 B, respectively, etc. Line data LD 3  provided by the line integrator  45  will be supplied to each of the difference detectors  44 A and  44 B, and a correction error signal E_* provided as a comparison output from the comparator  47  be supplied to each of the flicker-added signal generators  41 A and  41 B.  
         [0063]     Each of the flicker-added signal generators  41 A and  41 B includes address calculators  411 A and  411 B supplied with the correction error signal E_* supplied as a comparison output from the comparator  47 , address converters  412 A and  412 B supplied with addresses AD 11  and AD 21  calculated by the address calculators  411 A and  411 B, respectively, correction value calculators  413 A and  413 B supplied with addresses AD 12  and AD 22  calculated by the address converters  412 A and  412 B, respectively, multipliers  414 A and  414 B supplied with a flicker amplitude signal A_* generated by the flicker amplitude adjuster  60 *, level adjusters  415 A and  415 B supplied with flicker data FD 1  and FD 2  multiplied by the flicker amplitude signal A_* in the multipliers  414 A and  414 B, and low-pass filters (LPF)  416 A and  416 B and operational units  417 A and  417 B, supplied with an image signal DV_* digitized by the A-D converter  20 *. The image signal DV_* digitized by the A-D converter  20 * will be supplied, via the low-pass filters (LPF)  416 A and  416 B, respectively, to the level adjusters  415 A and  415 B, and correction values CFD 1  and CFD 2  generated by the level adjusters  415 A and  415 B, respectively, are supplied to the operational units  417 A and  417 B, respectively.  
         [0064]     In the correction error detector  40 * constructed as above, the address calculators  411 A and  411 B calculate addresses AD 11  and AD 21  in the ROM on the basis of the correction error signal E_*. The addresses to be thus calculated are resulted from shifting the top address of a flicker of a next frame in the positive- or negative-going direction. These addresses are calculated as in the address calculator  31 * in the flicker correction circuit  30 *. Also, the ROM included in the correction error detector  40 * is identical to that included in the flicker correction circuit  30 *.  
         [0065]     The address converters  412 A and  412 B convert the addresses AD 11  and AD 21  calculated by the address calculators  411 A and  411 B, respectively, into addresses AD 12  and AD 22 , respectively, from which flickers of a next frame can be reproduced. That is, they convert the addresses AD 1  and AD 2  into addresses having opposite phases. The addresses AD 12  and AD 22  converted by the address converters  412 A and  412 B, respectively, are resulted from prediction of flickers of the next frame, but not intended for correction of the flickers.  
         [0066]     The correction value calculators  413 A and  413 B calculate flicker data FD 1  and FD 2  on the basis of the addresses AD 12  and AD 22 , respectively, converted by the address converters  412 A and  412 B, respectively. The flicker data FD 1  and FD 2  are also determined per line as in the flicker correction circuit  30 *. The correction value calculators  413 A and  413 B are similarly constructed to the correction value calculator  32 * included in the flicker correction circuit  30 *.  
         [0067]     As in the flicker correction circuit  30 *, in the flicker-added signal generators  41 A and  41 B of the correction phase error detector  40 *, the image signal DV_* is passed through the low-pass filters (LPF)  416 A and  416 B to remove noises from the image signal DV_*, the noise-free image signal DV_*′ is supplied to the level adjusters  415 A and  415 B. The level adjusters  415 A and  415 B calculate correction values CFD 1  and CFD 2  for each pixel not influenced by the noises on the basis of the noise-free image signal DV_*′ and flicker data FD 1  and FD 2  calculated by the multipliers  414 A and  414 B, respectively.  
         [0068]     The level adjusters  415 A and  415 B are constructed like the level adjuster  34 * included in the flicker correction circuit  30 *.  
         [0069]     The operational units  417 A and  417 B add the correction values CFD 1  and CFD 2  for each pixel to the flicker-corrected image signal CV_* to generate flicker-added signals FDV 1  and FDV 2  for a next frame. The operational units  417 A and  417 B are also constructed like the operational unit  36  included in the flicker correction circuit  30 *.  
         [0070]     The line integrators  42 A and  42 B calculate line data LD 1  and LD 21  by integrating certain segments of the flicker-added signals FDV 1  and FDV 2  of the next frame, respectively. The “segment” may be of an arbitrary value as a horizontal size so far as it is within an image acquired horizontally. With a larger segment, a correction error can be detected with a higher accuracy. The vertical size of the segment may be an integral multiple of the cycle of the light and dark fringes of a flicker within one screen. More specifically, the segment may be given a size of 1000 horizontal pixels by 675 vertical pixels (=337.5×2), namely, of 1000×675 pixels.  
         [0071]     The line data LD 11  and LD 21  calculated by the line integrators  42 A and  42 B are stored in the memories  43 A and  43 B, respectively, until the image signal DV_* of a next frame is supplied. When the image signal DV_* of the next frame is supplied, the line integrator  45  makes line integration of the same segments as those of the flicker-added signals FDV 1  and FDV 2  which have been integrated to calculate the line data LD 3 .  
         [0072]     The line data LD 12  and LD 22  stored in the memories  43 A and  43 B, respectively, and line data LD 3  of the next-frame image signal DV_* corresponding to the lines of the line data LD 12  and LD 22  are supplied to the difference detectors  44 A and  44 B to provide difference data DD 1  and DD 2 .  
         [0073]     The integrators  46 A and  46 B provide integrated data ID 1  and ID 2 , respectively, by integrating the two difference data DD 1  and DD 2 , respectively.  
         [0074]     Then, the comparator  47  judges, by making a comparison in size between these integrated data ID 1  and ID 2 , in which direction an address is shifted for prediction of a correct flicker, positive- or negative-going. For example, in case the integrated data ID* obtained with the address shifted in the positive-going direction is smaller than the integrated data ID* obtained with the address shifted in the negative-going direction, a correction phase error signal E_* is outputted to shift the address in the positive-going direction.  
         [0075]     The correction error is minimized by supplying the correction phase error signal E_* to the address calculator  31  of the flicker correction circuit  30 * and address calculators  511 A and  511 B of the correction amplitude error detector  50 * to shift the address in a correct direction toward a flicker.  
         [0076]     Also in this image pickup device  100 , each of the correction amplitude error detectors  50 R,  50 G and  50 B uses the correction amplitude error detector  60 * constructed as shown in  FIG. 10  according to the aforementioned algorithm. It should be noted here that the asterisk (*) stands for “R (red)”, “G (green)” and “B (blue)”.  
         [0077]     The correction amplitude error detector  50 * includes flicker-added signal generators  51 A and  51 B supplied with an image signal CV_* flicker-corrected by the flicker correction circuit  30 * and flicker amplitude signal A_* generated by the flicker amplitude adjuster  60 *, line integrators  52 A and  52 B supplied with flicker-added signals FDV 13  and FDV 32  generated by the flicker-added signal generators  51 A and  51 B, respectively, memories  53 A and  53 B supplied with line data LD 31  and LD 32  integrated by the line integrators  52 A and  52 B, respectively, difference detectors  54 A and  54 B supplied with line data LD 31  and LD 32  read from the memories  53 A and  53 B, respectively, line integrator  55  supplied with an image signal DV_* digitized by the A-D converter  20 *, integrators  56 A and  56 B supplied with difference data DD 31  and DD 32  detected by the difference detectors  54 A and  54 B, respectively, comparator  57  supplied with integrated data ID 31  and ID 32  provided by the integrators  56 A and  56 B, respectively, etc. Line data LD 3  provided by the line integrator  55  will be supplied to each of the difference detectors  54 A and  54 B.  
         [0078]     The flicker-added signal generators  51 A and  51 B includes address calculators  511 A and  511 B, respectively, supplied with the correction phase error signal E_* provided as a comparison output from the comparator  47  in the correction phase error detector  40 *, address converters  512 A and  512 B supplied with an address AD 31  calculated by the address calculators  511 A and  511 B, respectively, correction value calculators  513 A and  513 B supplied with an address AD 32  calculated by the address converters  512 A and  512 B, respectively, amplitude amplifier  514 A and amplitude attenuator  514 B, supplied with the flicker amplitude signal A_* generated by the flicker amplitude adjuster  60 *, multipliers  515 A and  515 B supplied with flicker data FD 31  and CFD 31  calculated by the correction value calculators  513 A and  513 B, respectively, level adjusters  516 A and  516 B supplied with flicker data CFD 1  and CFD 2  multiplied by the flicker amplitude signal A_* in the multipliers  515 A and  515 B, and low-pass filters (LPF)  517 A and  517 B and operational units  518 A and  518 B, supplied with an image signal DV_* digitized by the A-D converter  20 *. Amplitude signals AM 31  and AM 32  resulted from controlling of the flicker amplitude signal A_* in the amplitude amplifier  515 A and amplitude attenuator  514 B are supplied to the multipliers  515 A and  515 B, respectively, the image signal DV_* digitized by the A-D converter  20 * will be supplied, via the low-pass filters (LPF)  517 A and  517 B, respectively, to the level adjusters  516 A and  516 B, and correction values CFD 31  and CFD 32  generated by the level adjusters  516 A and  516 B, respectively, are supplied to the operational units  518 A and  518 B, respectively.  
         [0079]     In the flicker-added signal generators  51 A and  51 B of the correction amplitude error detector  50 * constructed as above, the address calculators  511 A and  511 B calculate the address AD 31  in the ROM on the basis of the correction error signal E_*. The address to be thus calculated is the top address of a flicker component of a next frame. This address is calculated as in the address calculator  31 * in the flicker correction circuit  30 *. Also, the ROM included in the correction amplitude error detector  50 * is identical to that included in the flicker correction circuit  30 *.  
         [0080]     The address converters  512 A and  512 B convert the address AD 31  calculated by the address calculators  511 A and  511 B into an address from which a flicker of a next frame can be reproduced. More specifically, they convert the phase of the address AD 31  into an address AD 32  opposite in phase to the address AD 31 . The addresses AD 32  converted by the address converters  512 A and  512 B is resulted from prediction of flickers of the next frame, but not intended for correction of the flickers.  
         [0081]     The correction value calculators  513 A and  513 B calculate flicker data FD 31  on the basis of the address AD 32  converted by the address converters  512 A and  512 B. The flicker data FD 31  is also determined per line as in the flicker correction circuit  30 *. The correction value calculators  513 A and  513 B are similarly constructed to the correction value calculator  32 * included in the flicker correction circuit  30 *.  
         [0082]     Next, the amplitude amplifier  514 A calculates an amplitude signal AM 31  resulted from amplification of the amplitude signal A_*. At the same time, the amplitude attenuator  514 B attenuates the supplied amplitude signal A_* to calculate an amplitude signal AM 32 . The multipliers  515 A and  515 B multiply the calculated amplitude signals AM 31  and AM 32  by the flicker data FD 31  to provide flicker correction data FDA 31  having a flicker component thereof amplified and flicker correction data FDA 32  having a flicker component thereof attenuated.  
         [0083]     Also in the flicker-added signal generators  51 A and  51 B of the correction amplitude error detector  50 *, the image signal DV_* is passed through the low-pass filters (LPF)  517 A and  517 B as in the flicker correction circuit  30 * to remove noises from the image signal DV_*, and the noise-free image signal DV_*′ is supplied to the level adjusters  516 A and  516 B. The level adjusters  516 A and  516 B calculate correction values CFD 31  and CFD 32  for each pixel from the noise-free image signal DV_*′ and flicker data FDA 31  and FDA 32  calculated by the correction value calculators  515 A and  515 B, respectively.  
         [0084]     The level adjusters  516 A and  516 B are constructed like the level adjuster  34 * included in the flicker correction circuit  30 *.  
         [0085]     The operational units  518 A and  518 B add the correction values CFD 31  and CFD 32  for each pixel to the flicker-corrected image signal CV_* to generate flicker-added signals FDV 31  and FDV 32  of a next frame. The operational units  518 A and  518 B are similarly constructed to the operational unit  36  included in the flicker correction circuit  30 *.  
         [0086]     Then, the flicker-added signals FDV 31  and FDV 32  of a next frame, supplied from the operational units  518 A and  518 B, respectively, are processed as in the correction phase error detector  40 *.  
         [0087]     That is, the line integrators  52 A and  52 B integrate segments of the next frame where the flicker-added signals FDV 31  and FDV 32  exist to calculate line data LD 31  and LD 32 .  
         [0088]     The line data LD 31  and LD 32  calculated by the line integrators  52 A and  52 B are stored in the memories  53 A and  53 B, respectively, until the image signal DV_* of a next frame is supplied. When the image signal DV_* of the next frame is supplied, the line integrator  55  makes line integration of the same segments as those of the flicker-added signals FDV 31  and FDV 32  which have been integrated to calculate the line data LD 3 .  
         [0089]     The line data LD 31  and LD 32  stored in the memories  53 A and  53 B, respectively, and line data LD 3  of the next-frame image signal DV_* corresponding to the lines of the line data LD 31  and LD 32  are supplied to the difference detectors  54 A and  54 B to provide difference data DD 31  and DD 32 .  
         [0090]     The integrators  56 A and  56 B provide integrated data ID 31  and ID 32 , respectively, by integrating the two difference data DD 31  and DD 32 , respectively.  
         [0091]     Then, the comparator  57  makes a comparison in size between these integrated data ID 31  and ID 32  to provide a comparison signal C* indicative of whether the amplitude of the next-frame flicker should be amplified or attenuated.  
         [0092]     The comparison signal C_* provided in the correction amplitude error detector  50 * is supplied to the flicker amplitude adjuster  60 *. As shown in  FIG. 11 , the flicker amplitude adjuster  60 * includes a comparator  61  and amplitude increasing/decreasing unit  62  and always varies the flicker amplitude on the basis of the comparison signal C_*. The flicker amplitude adjuster  60 * functions to vary the flicker amplitude in a direction in which the comparison signal C_* supplied from the correction amplitude error detector  50 * will be smaller. It should be noted here that a smaller comparison signal C_* means that a prediction-based flicker image is more approximate to an actual image formed by the image sensing device  10 *. Namely, examining, by the comparator  61 , the relation in size of the comparison signal C_* between the frames permits to judge whether the flicker level has been predicted accurately. For example, in case the comparison signal C_* is larger than that of a preceding frame when the flicker amplitude has been increased, it can be determined that the flicker amplitude has been predicted inaccurately. In this case, the flicker amplitude is to be decreased by the amplitude increasing/decreasing unit  62 . On the contrary, in case the comparison signal C_* is smaller than that of the preceding frame, it can be determined that the flicker amplitude has been predicted accurately. In this case, the flicker amplitude is to be increased by the amplitude increasing/decreasing unit  62 . With these operations, the amplitude signal A_* is provided as an output.  
         [0093]     Thus, by varying, according to the aforementioned correction error detection algorithm, the level in a direction in which the integrated value of the two difference data DD 31  and DD 32  will be smaller, it is possible to vary the correction level to an optimum one as the time elapses.  
         [0094]     It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and alterations may occur depending on design requirements and other factors insofar as they are within the scope of the appended claims or the equivalents thereof.