Methods and systems for anti shading correction in image sensors

Embodiments of the current invention provide for systems and methods for correcting shading effects in image sensors. More specifically, but not by way of limitation, embodiments of the current invention provide methods and systems for dynamically correcting shading effects for digitally converted outputs from individual pixels on a pixel array in the image sensor, wherein the shading correction may be calculated according to a function of an elliptical-type equation from the radial location of the pixel on the pixel array. In embodiments of the present invention, the correction is performed at the Bayer domain before demosaicing processing to provide for accuracy of shading correction and low power consumption.

BACKGROUND OF THE INVENTION

This disclosure relates in general to correcting shading effects in an image sensor. More specifically, but not by way of limitation, this disclosure relates to dynamically correcting digitally converted outputs from pixels in the image sensor for shading effects at a Bayer domain using correction functions of elliptical-type or circular-type equations and pixel position on a pixel array in the image sensor to ascertain a correction factor for shading effects before the output from the image sensor is interpolated and/or converted to a standard video format.

In an image sensor light entering the image sensor through a lens is never truly collimated. As a consequence image sensors, including CMOS image sensors, suffer from a phenomenon known as “shading.” Shading originates from the fact that light detected by photo-sensitive elements in the image sensor that are remote from the center of the image sensor array is obliquely incident upon the remote photo-sensitive elements, whereas the light detected by photo-sensitive elements at the center of the image sensor array is substantially perpendicularly incident upon the central photo-sensitive elements. As a consequence, the light incident on the remote photo-sensitive elements may not completely fall on the remote photosensitive elements and/or may be shaded from the remote photosensitive elements by structures in the image sensor. As such, to obtain a true or at least a more accurate representation of an object from the image sensor it is necessary to compensate the outputs of the photo-sensitive elements in different locations on the image sensor array for the variable shading effect. However, shading corrections made after color interpolation, demosaicing and/or video standardization may be complicated because raw output data from pixels may be lost and more image data may now be associated with the differently shaded pixels on the image sensor. Further, shading approximation measurements found from testing of the image sensor are often not practicable to apply to pixel outputs because of the large memory necessary to store such data and the associated power requirements necessary for operating such a large memory. It is, therefore, desirable to have a shading correction method and/or system that can accurately correct shading for pixels that does/do not require large amounts of memory

In the appended figures, similar components and/or features may have the same reference label.

DETAILED DESCRIPTION OF THE INVENTION

The current invention provides for systems and methods for correcting shading effects in image sensors. More specifically, but not by way of limitation, embodiments of the current invention may provide methods and systems for dynamically correcting shading effects for digitally converted outputs from individual pixels on a pixel array in the image sensor, wherein the shading correction is calculated from functions of elliptical-type or circular-type equations using the radial location of the pixel on the pixel array. In embodiments of the present invention, the correction may be performed at the Bayer domain before demosaicing processing. Performing shading correction at the Bayer domain may provide for accuracy of shading correction and reduced power consumption.

References suggest shading correction techniques involving measuring shading effects for pixels on a pixel array and storing correction factors in look up tables or other forms of memory. However, the storage of the correction factors requires large amounts of power and silicon area and adding to the power consumption. Further, if only representative correction values for each group of neighboring pixels are stored, less memory is needed, but the accuracy is degraded, causing poor image quality. Moreover, obtaining additional responses from pixels—such as obtaining a response from the pixels to a common illumination: (a) in a process of device calibration, (b) shortly before obtaining the imaging response; or (c) shortly after obtaining an imaging response from the pixels—adds an additional step to the imaging process complicating the process and increasing the cost of the image sensor device.

Further references suggest correcting shading effects in images from image sensors after the images have been demosaiced and converted to a standard video format, such as YUv. In demosaicing outputs from pixels at the Bayer domain are converted to an RGB-per-pixel representation, where R=Red, G=Green and B=Blue. Missing color components for each pixel are calculated from the outputs of neighboring pixels. The shading effect is wavelength dependant. Consequently, after demosaicing, each R, G, B component for each pixel will be determined from a mix of pixel outputs at the Bayer domain. The mix of pixel outputs determining the R, G, B components for a single pixel are from pixels that may be responsive, because of color filters associated with the pixels, to different illumination wavelengths and, thus, the different pixels will suffer from different shading effects. As such, precise correction of shading effects after demosaicing is not possible and approximations must be taken. Even if demosaicing is performed linearly so that each Y, U, V is calculated as a linear combination of R, G, B color components, the shading effect may be calculated but, because each component has a different shading degradation, correction at this stage is much more complex.

From observation and study of the shading phenomena, it may be seen that shading effects in image sensors may be somewhat circular or elliptical in nature such that for pixels lying on an ellipse that is centered on a center of the pixel array the shading effect is substantially similar. From this observation, it may be shown that, using two-dimensional cartesian coordinates, where (X0, Y0) are the coordinates of a center of the pixel array, the amount of light energy for pixel at coordinates (X, Y) equals:

In such an elliptical equation, an ellipse is defined that has two radial components about the center point (X0, Y0)—an x radius and a y radius—and, as such, the function F of the ellipse type equation varies radially about the center point of the ellipse (X0, Y0) according to the ellipse equation.

It may also be possible in some embodiments of the present invention to identify shading effects using a function of a circle equation, i.e., (x-h)2+(y-k)2=r2. A function of a circle type equation will vary radially like an ellipse type equation, however, with a function of a circle type equation the radial variance is the same for the x and the y components.

According to Equation (1), the light energy at different locations on the pixel array is a function F of the elliptical-type equation and varies with radial pixel location relative to the center of the pixel array where the radius defines an ellipse with different vertical and horizontal attributes. In Equation (1), for a pixel array in an image sensor, S and T are constants that may depend upon the magnitude of the physical dimensions of the pixel array. Further, function F is wavelength specific and different variations of function F must be used to ascertain shading for pixels associated with different incident wavelengths. In an image sensor utilizing red, green and blue color filters with the pixels in the image sensor, three different variations of function F must be used to determine shading factors for pixels associated with the red filters, shading factors for the pixels associated with the green filters and shading factors for the pixels associated with the blue filters. While this may be complex the correction factors produced will be more accurate than correction factors derived from correction methods that do not take wavelength effects into account.

From Equation (1) it can be shown that after the shading effect received by a pixel at location (X, Y) is Exy, the shading compensated energy E′xyis:

In embodiments of the current invention, outputs from each of the pixels in an image sensor may be corrected for shading using either equations (6) or (7), or by using a close approximation of either equations (6) or (7) such as a circle-type equation, and applying the radial location of a pixel to either of the equations to process an output of the equations for the pixel. As may be seen from the equations, the shading effect varies radially across the pixel array where the radial variation is elliptical in nature. In certain embodiments of the present invention the radial location of the pixel is determined relative to a center of the pixel array and described in cartesian coordinates and applied to the equations (6) and (7) to determine a shading correction factor for the pixel. In such embodiments and in alternative embodiments where other methods of solving the equations for individual pixels may be used, shading correction factors may be directly evaluated for a pixel on the pixel array using the pixel's radial location relative to a center of the pixel array. In other embodiments of the present invention, shading correction factors may be determined iteratively by calculating the differences of R or R2between neighboring pixels. In either of these embodiments, power is saved because the correction factors may be determined dynamically without requiring the use of large look up tables or large memory devices.

FIG. 1is a block diagram depicting typical image processing in a CMOS image sensor. In a CMOS image sensor100, a sensor array10comprising a plurality of pixels converts an image that is incident upon the CMOS image processor100into a plurality of analog voltages, where each pixel on the sensor array10outputs an analog voltage in response to the light incident on the pixel from the image. An analog pre-processor20, among other analog processing functions, may correct the offset and gain variations for each of the pixel outputs. The pre-processed analog outputs may then be converted by an analog to digital converter30to a digital form. As such, after analog to digital conversion, a digital representation of the image incident upon the CMOS image sensor100may be produced. This digital representation of the image is in the Bayer domain since it is in raw color data form and has not been processed to include missing color components. While the term Bayer domain is used to describe the raw color format of the digital image, the current invention may be used with image sensors using color patterns other than Bayer patterns, such as color patterns using yellow color filters, magenta color filters, and/or the like.

In Bayer domain processing40, the digital outputs from the pixels in the sensor array10may be used along with the understanding of the color pattern of the color filters used with the sensor array10, which patterns may vary between the different image sensors, to reconstruct and/or interpolate the missing color components for each pixel. Because each of the pixels on the sensor array10is associated with a color filter only a proportion of the red, green and blue light falling on the sensor array10is captured. Using demosaicing algorithms, in a process called demosaicing, the missing color components may be determined from the outputs of neighboring pixels on the sensor array10and an accurate color image from the sensor array10may be obtained. However, after demosaicing has occurred, the actual outputs from the pixels of the sensor array10are modified and accurate anti-shading correction is more complicated and is likely to be less accurate. Other processing of the raw image from the sensor array10may also be performed at this stage.

After processing at the Bayer domain40, a Bayer to YUV converter50may translate the image from the Bayer domain, which may be specific to the image sensor array, to a universal video standard. In the illustrated example the universal video standard is defined as YUV video standard, however, the conversion may be to another standard video signal. Further image processing may be done in the YUV domain60, including sharpening, white balance, filtering and similar functions. Some references also suggest processing for shading in the YUV or standardized video domain.

In a final step, the processed YU image may be converted to the desired output format by output format converter70. Each of the components in the CMOS image processor100may be controlled by a control bus80that may program the processing parameters of the various units and govern the timing of the selection and processing of the inputs and outputs of the components. For example, the control bus80may control the scanning of outputs from the pixels on the sensor array that may in turn be processed to form the output image from the CMOS image processor100. Control bus80may comprise one or more conductors or optical fibers that serve as a common connection for the components of the CMOS image processor100and the related image processing components. Control bus80may be controlled by an external control unit90. The external control unit may be a processor or processors, a processor component or software capable of being executed by a processor, and/or the like and may be incorporated on chip with the sensor array10, etc, or it may located off chip as an independent device.

In an embodiment of the present invention, shading correction may be performed as part of Bayer domain processing40. In the embodiment, shading correction may be performed on the digitally converted outputs of the pixels in the sensor array10at the Bayer domain, prior to demosaicing. The correction may be made dynamically according to essentially real-time calculations of shading correction factors without the need to resort to look up tables or memory storage of correction factors and without having to measure additional responses from the pixels in order to determine correction factors. The methods of the current invention comprise BAYER domain digital correction for the shading errors.

As discussed above, the correction to be applied to pixel energy at location (X, Y) is a function F1of R, where:
R=√{square root over ((X−X0)2+A*(Y−Y0)2)}{square root over ((X−X0)2+A*(Y−Y0)2)}  (8)
Or, alternatively, the correction is a function F2 of R2, where:
R2=(X−X0)2+A*(Y−Y0)2(9)
F1and F2are also functions of the color of the current pixel.

In an embodiment of the present invention, the calculation of R and/or R2may be performed iteratively based on the following equations:
((X+1)−X0)2 =(X−X0)2+2*(X−X0)+1   (10)
((Y+1)−Y0)2 =(Y−Y0)2+2*(Y−Y0)+1   (11)

In equations (10) and (11) the squaring operation of equations (8) and (9) is replaced by three additions. Moreover, according to the equations, the value of 2*(X−X0) is incremented by 2 whenever X is incremented by 1. As such, in some embodiments of the present invention, a register may be allocated to hold the value of (2*(X−X0)+1) and the value of the register may be incremented by two for every pixel on the sensor array10that is scanned logically across the x-axis of the sensor array10so that the only operations needed to obtain a correction factor for the newly scanned pixel are one increment of two and one addition.

Some image sensors provide an option to generate mirror images by reversing the scanning direction of pixels on the sensor array10in the X and/or the Y dimensions. For such image sensors, the above equations become:
((X+1)−X0)2=(X−X0)2±2*(X−X0)+1  (12)
((Y+1)−Y0)2=(Y−Y0)2±2*(Y−Y0)+1  (13)
Where the+or the−operations may be selected according to the scanning direction.

FIG. 2Aillustrates basic features of a pixel in an image sensor. Pixel210includes a microlens220that focuses the photons230incident upon the pixel210onto the photodiode240. The photodiode is fabricated into a silicon substrate245. To reach the photodiode the photons230must pass through a color filter250. In general, each pixel in a pixel array is associated with either a green, blue or red filter.

FIG. 2Billustrates distribution of color filters across a pixel array in an image sensor. In an image sensor the color filters associated with each pixel on a pixel array260may be arranged in a mosaic pattern. In the pixel array260illustrated inFIG. 2Athe mosaic pattern of colored filters comprises four lines of color filters, lines270,280,290and295. In the first line of color filters270, red filters272alternate with green filters275. In the second line of color filters, line280green filters275are alternated with blue filters282. The resulting mosaic pattern of filters is called a Bayer pattern and includes twice as many green filtered pixels as red or blue filtered pixels because the human eye is more sensitive to green. Many other types of color filter patters exist, such as CYGM using cyan, yellow, green, and magenta filters in equal numbers.

FIG. 3illustrates a simplified flow type diagram of shading correction in accordance with embodiments of the present invention. In an embodiment of the present invention, in step300a pixel in the image sensor is scanned and produces an output corresponding to the luminance falling upon the pixel. The output is raw and may contain shading effects that will vary depending upon the pixel location on the pixel array. In step305, information concerning the location of the pixel on the pixel array is retrieved. The pixel array may be scanned logically so that pixel location information may involve information regarding the scanning line and the pixel location on the scanning line. In certain embodiments, pixel location information is Cartesian in format having an X and a Y value.

In certain embodiments, in step310, X and Y values for the pixel location may be processed according to an elliptical-type equation to produce a value of R2. In other embodiments, in step315, the X and Y values may be processed according to an elliptical-type equation and then square rooted to produce a value of R. Processing may be performed by a processor either on or off the image sensor chip, by a combination of circuit components —such as multiplexers, registers, multipliers, adders, subtractors or the like, and/or similar devices. Processing may be iterative and/or use methods to approximate the value of the elliptical-type equation. In either of the embodiments, in step320the values of either R2or R may be applied to the relevant function—F1or F2—to obtain the relevant correction factor for the pixel's output. In step320the selected form of function F1or F2corresponds to the wavelength transmissibility of the color lens associated with the pixel. The value of F1or F2corresponding to the value of R2or R may be found in a look up table or other memory means or may be interpolated from an abbreviated look up table or the like. Since pixels with the same radial location on an ellipse may be affected by a substantially similar shading effect, the size of a look up table is less than if the look up table stored correction factors for each of the pixels in the image sensor. Further, the use of the radial-elliptical nature of the relationship between pixels regarding shading effects provides an accurate and effective way to limit look up table size. In step330, the shading correction factor for the scanned pixel deduced from either function F1or F2may be applied to the pixel output.

FIG. 4is a simplified flow type diagram of an embodiment for calculating shading correction calculation for pixels in an image sensor array according to pixel location.FIG. 4illustrates a process for scanning pixels and calculating correction factors. In some embodiments of the present invention, an image scanner may logically scan across the pixels in the image sensor array. Logical scanning may involve scanning across the image sensor array in lines so that when the image scanner reaches the end of the line it may move to a next line and scan across that line. The image sensor array may be analyzed in two dimensional Cartesian coordinates. As such each scanned line may be considered to have the same y coordinate and the scanner may scan a line with the same y coordinate along an x-axis. When the image scanner reaches an edge of the image sensor array it may move to a new y coordinate and repeat the previous process. Further, by identifying a reference point on the image array sensor the image scanner may identify relative x and y coordinates for each pixel it scans relative to the reference point. In certain aspects of the present invention the reference point may be chosen to coincide with an optical center of the image sensor array where the optical center is determined from the properties of the lens of the image sensor. In other aspects, the reference point may be the center of the image sensor array.

In the illustrated embodiment, in decision box405a decision is made depending upon whether an image scanner or the like has moved to a new pixel or a new line. In decision box405nothing will occur if neither the line-count nor the pixel-count changes, i.e., the image scanner does not scan an output from a new pixel. In this situation, the top decision junction loops through the NO terminal407. In decision box405when either the line-count Y or the pixel-count X changes, i.e., a new line or new pixel is being scanned, the decision junction passes through the YES terminal409.

Once it is determined that a new pixel or line is being scanned, in decision box410a determination is made concerning whether it is a new line or a new pixel that is being scanned. In aspects when a new line is scanned the YES terminal412of the second decision junction is selected. In the illustrated embodiment, when a new line is scanned the Y location of the pixels being scanned on the image sensor array changes relative to the reference point and as such the Y part of the correction factor may be updated. To update the shading correction equation the new value of Y is obtained in step420. In the illustrated embodiment, Y0, the Y coordinate of the reference location may be subtracted from Y in step422. In step425the difference between Y and Y0calculated in step422may be squared. In step427the result from step425may be multiplied by A, where A is as defined above in equation (3). In certain aspects, the calculated value of A*(Y−Y0)2may be stored in step429and used for further calculation of shading correction factors for pixels located on a line with the Y coordinate. Once the Y calculation has been performed the algorithm loops back to the top decision junction to wait for a change in X or another change in Y.

In aspects where a new pixel is found in decision box410the second decision junction for decision box410exits through the NO terminal415. When a new pixel is found the X location of the pixel is obtained in step430. Once the X location of the pixel is determined, the X coordinate of the reference location, X0, may be subtracted from X in step432to provide the X position of the pixel relative to the reference location. In step435the output from step432may be squared. In step437R2may be calculated by retrieving A*(Y−Y0)2from store429and adding it to the result of (X−X0)2calculated in step435. In step439R is calculated from the square root of the output of step437. The square root may be calculated by a processor or the like or may be determined using a table.

The output of439provides a calculation of equation (9) for the pixel according to its X and Y location relative to a reference location that may be used to provide a shading correction factor for the pixel. Once a calculation has been dynamically performed for a scanned pixel the correction process may loop back to decision box405to wait for the next change in X or Y as the image sensor array is scanned. In alternative embodiments, step439is not performed and the correction factor for the pixel may be determined from the value for R2from step437. In such embodiments, after R2is determined the correction process may loop back to decision box405to wait for the next change in X or Y as the image sensor array is scanned.

FIG. 5is a simplified flow type diagram of an embodiment for iteratively calculating shading correction calculation for pixels in an image sensor array according to pixel location. As illustrated, the iterative calculation process is initialized in step500. In step510, initial values from working registers MX, MY, R2may be loaded to reference registers MX0, MY0, R20from a control unit512. In the illustrated embodiment, calculations of equations (8) and (9) may be calculated for the reference location (X0, Y0) and loaded to reference registers MX0, MY0, R20from control unit512when the process of scanning the image sensor array is first initiated. In certain aspects, R20may be set to the value at the end of a first scanned line of pixels rather than the beginning of the line so that the correction for retracing from the last to the first pixel of each row, that may be performed when each new line begins, will work right for the first line as well.

In the illustrated embodiment, whenever the scanning of a new frame starts, in step517the working registers MX, MY and R2may be reloaded from the reference registers MX0, MY0, R20. In step515if it is determined that a new frame is not being scanned the process bypasses the reloading of reference registers of step517. In step519when a New Line is detected the process moves to step520where R2and MY are updated according to equations (10) and (11), and MX is reloaded from the reference register MX0. Further, in step520, R2is updated for the retrace of X from the rightmost to the leftmost pixel by subtracting Xretrace=(Xright-X0)2−(Xleft-X0)2.

After processing for the scanning of a new line or if no new line is being scanned, in step522it is determined whether a new pixel is being scanned. If a new pixel is not detected the process returns to decision box515to determine if a new frame is being scanned. If a new pixel is detected in decision box522then registers R2and MX are updated in step525. In step525an iterative calculation of equation (9) may be executed and a correction factor for the scanned pixel ascertained. In certain embodiments, at this point the process may return to decision box515to iteratively calculate a correction calculation for another scanned pixel or to wait for the scanning of a new frame. In other embodiments, the square root of R2is determined in step527and this value may be used to calculate the correction factor for the scanned pixel.

In certain embodiments of the present invention, instead of calculating the values of F1or F for pixel locations on the pixel array the values may be stored in a ROM or a RAM based table. However, in such embodiments, the use of large amounts of memory may be necessary that may provide for increased power consumption by the image sensor.

FIG. 6illustrates a method of an embodiment for combining calculation processes and memory storage tables to calculate shading correction factors with reduced memory requirements. In the embodiment illustrated inFIG. 6, F2may be approximated from a given R2. In the process an updated value for R2may be obtained in step610. In the embodiment R2is 10 bit wide. In step620the value of R2may be obtained from a32entry table for each color that may be addressed by bits [9:5] of R2. The table may be accessed twice—first, for the entry below the actual R2value, C1, and second for the next entry, C2. In step630a value C3, which is the difference between the two entries, C2-C1, may be calculated. C3may then be used for the interpolation of the correction factor. In step640, C3is multiplied by the residue of R2—the part that was truncated in the entry to the table. In step650the correction value is calculated by adding the C1value that was read from the table.

FIG. 7Aillustrates a simplified block diagram of a circuit to correct shading for a pixel on a pixel array according to an embodiment of the present invention. In the illustrated embodiment, a complete processing circuit700may be divided into two separate circuits, a first circuit710that calculates R2and a second circuit720that approximates the correction function F2and then multiplies the incoming pixels by the correction factor. In certain embodiments, there may be two clock cycles for each pixel. However, different clock level frequencies may be used in different embodiments of the present invention. In the illustrated embodiment, propagation of an output to be processed through the logic and arithmetic elements may be fast relative to the clock cycle to provide for accurate correction. To prevent a loss of synchronization in the processing of pixel outputs, a set of registers, registers735,739,750,770, and790, and a delay-line793may be clocked and may add a delay of at least one clock cycle. The delay of the multipliers may be more than the delay of the other elements in the circuit.

As shown inFIG. 7A, hardware units may be shared for the calculation of the X and the Y portions of R2. For example, a subtractor712, a multiplier730, and the like may be shared in the processing of X and Y. Merely by way of illustration, the subtractor712may be used during the horizontal-blank period of the scanning of the pixel array to calculate the value of Y−Y0and may also be used during the line scan period of the pixel array to calculate the value of X−X0, where X, Y, X0and Y0are selected by multiplexers702and702that are controlled a control CNT1707. An output from the subtractor712may enter an absolute value unit715that may calculate |X−X0|and |Y−Y0|, depending upon the scanning of the pixel array and, consequently, the output of a pair of multiplexers, multiplexers702and705. In theory, the absolute value unit715is mathematically redundant because the difference of the location components is squared in the next operation, but practically it may be used to allow the implementation of a simpler unsigned multiplier. In alternative embodiments, signed multipliers may be used without the absolute value unit715.

In the illustrated circuit, a multiplier730may be used to square an output of X−X0, an output of Y−Y0and the multiplication of the output of Y−Y0by A. During the X blank period of the scanning, a CNT2717may be set to high to force the multiplexers722and725to select a pair of 2-inputs727and present the output of |Y−Y0|to both inputs of the multiplier730to provide for the squaring of the output of |Y−Y0|. The output from the multiplexer730, (Y−Y0)2may be stored in a register735. During subsequent clock cycles of the X-blank period, the CNT2717may go low. When the CNT2717goes low, the multiplexer722may select a 1-input728and present the value of (Y−Y0)2obtained from the register735to one port of the multiplier730and the multiplexer725may also select its 1-input729and may retrieve a pre-programmed value of A from a memory device, not shown, and may present the pre-programmed A to the other port of the multiplier730. From these inputs, the multiplier730may produces=an output equivalent to A*(Y−Y0)2that may be stored in a register739. The register739may be controlled by control signal CNT3741. Since the image sensor array may be scanned in lines along the x-axis of the array, the register739may store the output A*(Y−Y0)2until a new line of the image sensor array is scanned. During the time that a line of the image sensor array is being scanned the control signal CNT2717may be held high to control inputs to the multiplier730to provide that the inputs both receive an input of |X−X0|. In this way, the multiplier730may output a square of the value of |X−X0|. The output from the multiplier730, |X−X0|2may be stored in a register735. The register735may be controlled by a regular frequency clock signal CLK736to provide that the stored value of the register735is output at a correct clock period to provide for synchronization with the circuit720. The register735may present the stored value |X−X0|2to a first input of an adder740and the register739may, in synchronicity, present the stored value A*(Y−Y0)2to the second input of the adder740. After receiving the inputs, the adder740may produce a value for the elliptical-type equation R2=(X−X0)2+A*(Y−Y0)2that may in turn be stored in the register750for application to the circuit720. A control signal CNT4749may be provided to operate the register750and control the output of bits [9:5]751and bits [4:0]752from the register750for application to the circuit720.

In the circuit720function F2may be processed. In the circuit720high order address bits in a RAM765may be driven by a color indicator bus767that identifies the color of the filter associated with the scanned pixel and selects the form of function F2required by the filter color—remembering that different function components are used in F2for different colors. In effect, three different look up tables are used in the RAM765, one per color component. The RAM765may be accessed twice per pixel—a first time to obtain C1, using the bits [9:5]751of the calculated R2and a second time to obtain C2, which corresponds to the next address. To provide for this double access, a multiplexer760controlled by a control signal CNT5757may first present the calculated R2[9:5] from the register750to the RAM765and then in the next clock cycle, select an output from the incrementor755to provide that the RAM755is presented with the subsequent address.

The C1result obtained from the RAM765may be stored in a register770that may be controlled by a control signal CNT6772. In the next clock cycle, the RAM765may output C2to a subtractor775. Synchronously, the register770may present C1to a subtractor775to provide for the calculation of C3wherein C3is equivalent to C2-C1. The difference of C2-C1may be output from the subtractor775to a multiplier780that may be controlled by a control signal MLP2-en782. The multiplier780may synchronously receive the difference of C2-C1on one input and the output of R2[4:0] from the register750on the other input. Having received the two inputs, the multiplier780may multiply the two inputs and present the product to an adder785. The other input of the adder785may receive a value of C1and the adder785may add the product of R2[4:0] and C2-C1to C1. The output of the adder785is the shading correction factor for the scanned pixel and it may be stored in a register790that is controlled by a clock signal clk736. Due to an inherent delay in calculating the correction factor for the scanned pixel, a delay unit793may be used to delay a digital output from the scanned pixel until the calculation factor can be processed. In an aspect of the present invention, the delay unit793may be a pair of registers connected together to form a two clock delay.

In an alternative embodiment of the present invention, the multipliers780and795may be replaced by a single multiplier unit with two multiplexers at its inputs. In such an embodiment, a control device and/or signal may provide that the multiplexers provide inputs to the single multiplier to provide that during odd clock cycles the single multiplier receives an output from the register750of the R2[4:0]752and an output from the adder775. Having received these inputs, the single multiplier multiplies the inputs together and presents the output of the multiplication to the register790. Then at even clock signals the multiplexers present the output from the register790to the single multiplier and the single multiplier multiplies the input with the delayed-pixel output from the delay unit793.

FIG. 7Billustrates timing waveforms for operation of the shading correction circuit according to an embodiment of the present invention shown inFIG. 7A.FIG. 7Billustrates the timing waveforms for the signals described inFIG. 7Ain accordance with an embodiment of the present invention.

FIG. 8Aillustrates a circuit for iteratively calculating R2and correcting a pixel output for shading effects in accordance with an embodiment of the present invention. In the embodiment, an iterative calculation circuit800may be used to iteratively calculate an elliptical-type equation for a pixel scanned on the image sensor array of an image sensor. The iterative calculation circuit may be combined with the correction function F2circuit720described inFIG. 7Ato calculate a shading correction factor for the scanned pixel.

In an embodiment of the present invention, an initial value for MX may be determined for the image sensor and loaded from an external CPU801onto a MX0register802. The CPU801may be located on the same integrated circuit in some embodiments and/or in another integrated circuit in other embodiments. The CPU801may be a processor, processors and/or a software program executed on a processor. An initial value for MY may be determined for the image sensor and loaded from the external CPU801onto a MY0register805. Further, an initial value for R2may be determined for the image sensor and loaded from the external CPU801onto a R20register880. The outputs from the external CPU801may be loaded to the registers802,805and880prior to the scanning of images on the image sensor.

In the illustrated embodiment, a timing and control unit807may control the operation of various circuits in the iterative calculation circuit800. The timing and control unit807may be a processor, a software program, and/or a device controlled by a processor, such as CPU801. As illustrated, the timing and control unit is shown providing control signals C1through C7, but may produce additional or less control signals. For illustrative purposes, connection of the control signals to the devices controlled by the control signals is not pictured, but will be appreciated by persons of skill in the art. In the illustrated embodiment, a multiplexer810may be controlled by a control signal C1817provided by the timing and control unit807. A second multiplexer815may also be controlled by the control signal C1817. When the control signal C1817is high the a multiplexer810may route the value of the MX0register802into an MX register820. Concurrently, when the control signal C1817is high the multiplexer815may route the value of the MY0register805into an MY register825. When the C1817is low the multiplexers810and815may route an output from an adder875to the MX register820and the MY register825. A control signal C2822controls when the MX register820receives an input. Similarly, a control signal C3827controls when the MY register825receives an input

In the illustrated embodiment, a three-way multiplexer830may be controlled by a control signal C4833to route outputs from the MX register820and the MY register825or a pre-programmed value of a minus-Xretrace835, where the Xretrace835may be equal to (Xleft-X0)2−(Xright-X0)2, to the adder875. A four-way multiplexer840may be controlled by a four-state control signal C5850. The four way multiplexer840may be used to provide inputs to the adder875. The four-way multiplexer840may be used to present a constant value of 2, a programmable value 2*A, a constant value of 0, or the current value of R2stored in a register890, to the top input of the adder875, depending upon the state of the control signal C5850.

In the illustrated embodiment, the adder875may provide outputs to the multiplexer810, the multiplexer815and a multiplexer860. The multiplexer860may be controlled by a control signal C6870to select an output from either the adder875or the R20register880for presentation to the input of a R2register890. The output of the R2register890may be controlled by a control signal C7892and the output may be divided to a low order part895and a high order part897. The output from the R2register890may be fed to the correction function F2circuit720and applied to function F2in the manner described with regard toFIG. 7Ato obtain a shading correction factor for the scanned pixel. The control signals C1, C2, C3, C4, CS, C6and C7may be generated by the timing and control unit807.

In embodiments of the present invention, the following operations may be done during the horizontal blank period (X-Blank):R2(x,y+1)=R2(X,Y)+MY(Y)R2(Xright,Y)=R2(Xleft,Y)−XretraceMY(Y+1)=MY(Y)+2*AMX=MX0
Further, in the first clock cycle, when X-Blank goes high, the control signal C1810may go high, routing an output from the MX0register802to the MX register820, and also routing an output from the MY0register805to the MY register825. At the end of the first clock cycle C2may go high to provide that an output from the MX0register802may be presented to the MX register820. When the control signal C3827is low the value of the MY register820will remain the same. Further, during the first clock cycle, the control signal C4833may be at state Xretrace to provide that the preprogrammed value of Xretrace, where Xretrace=(Xright-X0)2−(Xleft-X0)2, may be routed to the adder875. Synchronously to this input to the adder875, the control signal C5850may be at state R2to provide that the adder875may generate an x-retrace corrected value of R2that may be subsequently loaded into the R2register890at the next clock cycle, when the control signal C7892goes high.

During the second clock cycle, the control signal C4833may be set to an MY state and provide for the routing of the output from the MY register825through the multiplexer830to the bottom input of the adder875. The control signal C5850may remain at the R2state for the second clock cycle and the adder875may generate a sum of R2+MY. This output may then be routed through the multiplexer860to the R2register890. During the next clock cycle the control signal C7892may remain high to provide for the storing of R2+MY on the R2register890.

During the third clock cycle, the control signal C5850may be set to the 2*A state and provide that the multiplexer840routes the pre-programmed value of 2*A to the upper input of the adder875. During the third clock cycle, the control signal C4833may remain at the MY state to provide that the MY value is presented to the bottom input of the adder875. By presenting the MY to the bottom input of the adder875during the third clock signal it may be provided that the adder875may generate MY+2*A. Consequently, the value MY+2*A may be routed from the adder875through the multiplexer815to the MY register825. During the next clock cycles the control signal C3827may go high to provide that the value MY+2*A may be loaded to the MY register825.

In some embodiments, from the fourth clock cycle and until X-blank ends, in preparation of the scanning of first pixels of a new line on the image sensor array, the control signal C4833may stay in the MX state and the control signal C5850may stay in the R2state to provide that the adder875may output a value of MX+R2. Under the same conditions, after the first pixel of a new line is scanned the adder875may output a value of R2.

In some embodiments, the following operations may take place when new pixels on a line are scanned:R2(x+1,y)=R2(x,y)+MX(x)MX(x+1)=MX(x)+2
When a new pixel is scanned, the control signal C4833may be at state MX until the end of the line is scanned. As such, the control signal C4833may provide that the value of MX is presented to the bottom input of the adder875. As new pixels are scanned, the control signal C5850may alternate between the value of R2and the integer value 2 and provide that the adder875, in turn, alternately generates MX+2 and R2+MX. By keeping the control signal C1817low at this time the value of MX+2 generated by the adder875may be routed through the multiplexer810to the MX register820and may be latched there by the control signal C2822. For the succeeding pixels, the R2+MX values generated by the adder875that are equivalent to the R2values may be latched in the R2register890by the control signal C7892.

FIG. 8Billustrates timing waveforms for scanning an image sensor and controlling the circuit for iteratively calculating R2and correcting a pixel output for shading effects in accordance with an embodiment of the present invention.FIG. 8Billustrates the timing waveforms discussed above for operating the circuits illustrated inFIG. 8Ain accordance with an embodiment of the present invention.

In some embodiments of the present invention, the systems and methods described above for calculating either directly or iteratively, radial functions for an image sensor, that may be functions of circular-type or elliptical-type equations, may be used to define radial regions on the image sensor in the Bayer domain. In embodiments of the present invention, limitations may be applied to the circular-type and/or elliptical-type equations to provide for the identification of radial regions on the image sensor. Merely by way of example, a function of a circular-type equation may be provided with functional limitation factors to provide a zero (0) value when radial component, R or R2, is below a certain value, wherein R and/or R2may be calculated for the function using the processes described above either directly, using look up tables, iteratively, and/or the like. The zero value output from the function may be fed to an image processing system, color processing system, and/or the like to provide for image processing variations according to radial location on the image sensor in the Bayer domain. Such image processing in the Bayer domain may provide for creating a wide-variation of special effects, image processing and/or the like that may be directly derived from pixel outputs from the image sensor. Merely by way of example, color processing of pixel outputs may be turned off when the output from the function is zero or below a threshold value. In this way, selected regions of an image produced by the image sensor may be gray or colorless. In certain aspects, the function may be used to provide for the output of a gray circle at the center of the image that may indicate where auto-focusing of a camera, video camera or the like incorporating the image sensor is focusing. With different limitations on the functions of circular and/or elliptical type equations different regions on the image sensor may be identified for processing of pixel outputs in these regions prior to demosaicing.

The invention has now been described in detail for purposes of clarity of understanding. However, it would be appreciated that certain changes and modifications may be practiced within the scope of the appended claims.