Flicker band automated detection system and method

A flicker band automated detection system and method are presented. In one embodiment an incidental motion mitigation exposure setting method includes receiving image input information; performing a motion mitigating flicker band automatic detection process; and implementing exposure settings based upon results of the motion mitigating flicker band automatic detection process. The auto flicker band detection process includes performing a motion mitigating process on an illumination intensity indication. Content impacts on an the motion mitigated illumination intensity indication are minimized. The motion mitigated illumination intensity indication is binarized. A correlation of the motion mitigated illumination intensity and a reference illumination intensity frequency is established.

FIELD OF THE INVENTION

The present invention relates to the field of digital cameras. More particularly the present invention relates to a flicker band automated detection system and method.

BACKGROUND OF THE INVENTION

Electronic systems and circuits have made a significant contribution towards the advancement of modern society and are utilized in a number of applications to achieve advantageous results. Numerous electronic technologies such as digital computers, calculators, audio devices, video equipment, and telephone systems have facilitated increased productivity and reduced costs in analyzing and communicating data, ideas and trends in most areas of business, science, education and entertainment. For example, digital cameras have had a significant impact on the field of photography and usually offer a number of features that enhance image quality. Nevertheless, there are several phenomena such as flicker or running bands in an image frame that can adversely impact results. Setting a rolling shutter time to a multiple of a corresponding illumination variation (e.g., 1/100 seconds or 1/120 seconds) often mitigates adverse impacts associated with the flicker bands. However, determining the illumination variance frequency is usually very difficult and often susceptible to a number of inaccuracies.

A rolling shutter approach is often utilized by a variety of devices (e.g., a CMOS imagers) to control the optical integration time. Rolling shutter approaches typically enable an equal optical integration time to be achieved for pixels in an image frame however this optical integration does not typically happen for all pixels simultaneously. The actual interval used for integration is usually dependent on the vertical position of the pixel in an image frame. Rolling shutter approaches typically utilize at least two pointers, a reset pointer and a read pointer, to define shutter width. Shutter width is the number of lines (“distance”) between the two pointers. These pointers continuously move through the pixel array image frame from top to bottom, jumping from line to line at timed intervals. The reset pointer typically starts the integration for pixels in a line, and subsequently, the read pointer reaches the same line and initiates signal readout. Shutter width multiplied by the line time gives the duration of the optical integration time. If the illumination intensity of the light source does not remain constant over time, rolling shutter methods of integration time control can lead to the possibility of flicker or “running bands” in the image frame.

Flicker band issues are typically caused by scene illumination sources with varying light intensity. For example, illumination intensity usually fluctuates in light sources driven by alternating current (AC) supply power. The light intensity of AC powered light sources (e.g., fluorescent lamps) usually is enveloped by a 100 Hz or 120 Hz wave if the power source is operating at 50 Hz or 60 Hz correspondingly. If the video rate is 15 frames per second, there could be six (50 Hz power) or eight (60 Hz power) flicker dark bands overlying the preview image.

Determining the illumination variance frequency is traditionally resource intensive and difficult. A straightforward frequency transform (e.g., FFT) of the image frame consumes significant amounts of computation resources and the results can be relatively easily corrupted by the image content. Some traditional attempts at detecting mismatches between the frequency of an illumination source and the duration of optical integration time for an imager with rolling shutter involve detecting a sine image fluctuation in the difference of the two consecutive image frames. However, these approaches are often limited by averages from a single line or row of an image. In addition, a slight vertical shift in the camera such as the shaking hand of a photographer, could greatly deteriorate the results. Relatively low confidence levels associated with these traditional approaches are exacerbated by the utilization of zero crossing point estimation of flicker wave periods.

SUMMARY

A flicker band automated detection system and method are presented. In one embodiment an incidental motion mitigation exposure setting method includes receiving image input information; performing a motion mitigating flicker band automatic detection process; and implementing exposure settings based upon results of the motion mitigating flicker band automatic detection process. The motion mitigating flicker band automatic detection process includes performing a motion mitigating process on an illumination intensity indication. Content impacts on the motion mitigated illumination intensity indication are minimized. The motion mitigated illumination intensity indication is binarized. A correlation of the motion mitigated illumination intensity and a reference illumination intensity frequency is established.

DETAILED DESCRIPTION

Some portions of the detailed descriptions, which follow, are presented in terms of procedures, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means generally used by those skilled in data processing arts to effectively convey the substance of their work to others skilled in the art. A procedure, logic block, process, etc., is here, and generally, conceived to be a self-consistent sequence of steps or instructions leading to a desired result. The steps include physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, optical, or quantum signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussions, it is appreciated that throughout the present application, discussions utilizing terms such as “processing”, “computing”, “calculating”, “determining”, “displaying” or the like, refer to the action and processes of a computer system, or similar processing device (e.g., an electrical, optical, or quantum, computing device), that manipulates and transforms data represented as physical (e.g., electronic) quantities. The terms refer to actions and processes of the processing devices that manipulate or transform physical quantities within a computer system's component (e.g., registers, memories, other such information storage, transmission or display devices, etc.) into other data similarly represented as physical quantities within other components.

FIG. 1is a flow chart of incidental motion mitigation exposure setting method100in accordance with one embodiment of the present invention.

In step100, test illumination input information is received. The test illumination information can be initial illumination information sensed by a digital imager. The information can be received from a variety of digital imagers. In one embodiment the image input information is received from a CMOS imager. The auto exposure (AE) is set to a predetermined reference illumination frequency for capturing the initial image test input information. In one exemplary implementation, initial sensor timing for the received initial test input information is set so that frame height is not a multiple of the inverse of twice the reference illumination frequency. For example, if an auto exposure (AE) reference illumination frequency is initially set at the reference illumination frequency of 50 Hz the sensor timing frame is set up so that the frame height is not a multiple of 1/100 seconds. This ensures that the location of flicker bands will move from frame to frame if the flicker bands exist.

In step120, a motion mitigating flicker band automatic detection process is performed. The motion mitigating flicker band automatic detection process automatically identifies an illumination frequency. In one embodiment, the motion mitigating flicker band automatic detection process provides a confidence level by utilizing correlation scores to a know reference frequency. Down-sampled averages are utilized in the determination of the correlation score to minimize adverse impacts of incidental motion or jitters in hand movements while capturing the image. Additional explanation and implementations of motion mitigating flicker band automatic detection processes is presented below.

In step130, exposure settings are implemented. The exposure settings are implemented based upon said identified illumination frequency. In one embodiment of the present invention, a digital camera exposure reset from an initial exposure setting to an exposure based upon the identified illumination frequency is executed. In one exemplary implementation, auto exposure controls are utilized to switch between exposure tables (e.g., a 50 Hz exposure table or a 60 Hz exposure table).

FIG. 2is a block diagram of automated flicker band compensation imaging system200in accordance with one embodiment of the present invention. Automated flicker band compensation imaging system200includes lens210, shutter220, light sensor230, processor240and memory250. Lens210is coupled to shutter220which is coupled to light sensor230which in turn is coupled to processor240. Processor240is coupled to shutter220and memory250which includes flicker band automatic detection instructions255.

The components of automated flicker band compensation imaging system200cooperatively operate to automatically detect and minimize flicker band effects while capturing digital images. Lens210focuses light reflected from an object onto light sensor230. Light sensor230senses light from lens210and converts the light into electrical digital image information. In one embodiment of the present invention, light sensor230is a complementary metal oxide silicon (CMOS) light sensor. Shutter210controls the length of time light is permitted to pass from lens210to light sensor230. Light sensor230forwards the digital image information to processor240for processing. Processor240performs digital image information processing including automated flicker band detection and mitigation. Memory250stores digital image information and instructions for directing processor240, including flicker band detection instructions255for performing automated flicker band detection and mitigation. In one embodiment, a subtraction buffer included in the processor240stores content neutralized illumination intensity values utilized by the processor. In an alternate embodiment, the subtraction buffer is included in memory250.

FIG. 3is a flow chart of auto flicker band detection process300in accordance with one embodiment of the present invention.

In step310, a motion mitigating process is performed on an illumination intensity indication. In one embodiment, the motion mitigation process includes performing image down-sampling and averaging illumination intensity values from a plurality of pixel rows of the resulting sub-sampled image. In one exemplary implementation, the number of downsampled pixel rows is maintained above a minimum that is greater than the video frame rate. In one embodiment reliable down-sampled pixels are selected for the averaging, wherein the reliable down-sampled pixels include pixels associated with a smooth well lit surface.

In step320, content impacts are minimized on the motion mitigated illumination intensity indication. In one embodiment, minimizing content impacts on the illumination intensity indication includes subtracting illumination intensity values of a first frame from illumination intensity values of a second frame. In one exemplary implementation, the current frame column buffer is subtracted from the previous frame column buffer. The subtraction results are stored in a new buffer referred to as the subtraction buffer. In one exemplary implementation, a periodic waveform is included in the entries of the subtraction buffer if there is flicker band problem.

In step330, the motion mitigated illumination intensity indication is binarized. In one embodiment, binarizing the illumination intensity indication includes comparing each entry value in a subtraction buffer with a moving average value and if a row in the subtraction buffer has a value larger than the mean, the values changes to +1, otherwise it's change to −1. For example if the j-th row in the subtraction buffer has a value larger than the mean, the value is changed to +1, otherwise it's changed to −1.

In step340, a correlation of the motion mitigated illumination intensity to a reference illumination intensity frequency is determined. In one embodiment a correlation score is utilized to establish the correlation of the motion mitigated illumination intensity to a reference illumination intensity frequency. For example, the correlation score is assigned based upon the following manipulation:

score=∑0⁢<_⁢k⁢T2⁢i=1X⁢⁢S⁡[i]⁢R⁡[i+k]
Where S[i] is the brightness indication sequence, R[i+k] is the reference sequence, X is the number of samples, k is the offset and the parameter T is the distance between two flicker bands in the sub sampled image. The unit for the T parameter is the number of rows. Additional confidence is provided by the correlation of the motion mitigated illumination intensity to a reference illumination intensity frequency. The additional confidence level provided by the present invention enables a conscious decision (e.g., a decision that has very low false positive error probability) on the current flickering frequency when the reference frequency is known to be relatively stable. For example the reference frequency is selected to correspond with the power source frequency since the power frequency does not change very often. The present invention correlation score also provides enhanced noise immunity, since one or two earliest entries in the column vector can still produce high correlation results.

The downsampling reduces the influence of incidental movement or camera jitter on determining the illumination intensity frequency. The larger of the down scaling ratio, the more translational invariant that frame subtraction performed for minimizing content impacts in step320. In one exemplary implementation, the number of downsampled pixel rows is maintained above the minimum. For example, if the video frame rate is 15 frames per second and the luminance frequency is 60 hertz, there will be eight horizontal bands in the image frame. In order to capture the variations of those bands, the number of downsample rows is maintained above

It is appreciated that the generation of the column vector from the down sampled image can be implemented in a variety of different ways. In one exemplary implementation, the average of the downsample pixels is taken from multiple entire rows. If the surface is smooth the frame subtraction has a better chance of successfully removing the same content of camera jitter. Therefore the downsample pixels are chosen from a location corresponding to relatively smooth and well lit surface to generate the column vector. In another exemplary implementation simple tracking of the down sampled pixels is performed before the row averaging, so that the influence of the horizontal translation can be reduced.

Thus, the present invention facilitates automated selection of exposure settings with a high degree of confidence and minimal impacts due to jitter.