Method, apparatus for correcting image signal from image sensor, and imaging system with apparatus

A method of correcting an image signal generated by a charge coupled device (CCD) image sensor in an imaging system is provided. The imaging system stores a number of gamma correction curves, each of which includes a respective correction factor for increasing contrast in a dark portion of the image signal. The method includes: measuring gray scale value of each pixel of the CCD image sensor; estimating a contrast level of an object scene to be imaged using the measured gray scale values; and correcting the image signal using a corresponding gamma correction curve depending on the estimated contrast level of the object scene.

BACKGROUND

1. Technical Field

Aspects of the present invention relate to image processing technology and, more specifically, relate to a method and an apparatus for correcting an image signal generated by an image sensor, such as, a charge coupled device (CCD) of an imaging system, and the imaging system having such a charge coupled device (CCD) image sensor.

2. Description of Related Art

In general, image sensors are used in imaging systems, such as, for example, digital cameras, optical scanners and video cameras as light sensing devices to convert a visual image into an electrical signal. An image sensor is typically an array of photoelectric cells, also known as picture elements (i.e., pixels), arranged in a matrix form so as to convert the light energy into electrical signal charge and subsequently output as an image signal, when a visual image is projected thereon. Each of the pixels, which form the visual image, is an optical sensor that is adapted to sense an incident light beam and photo-electrically convert the light beam into electrical signal charge corresponding to the amount of the received light beam. Such an image sensor can be any type of image sensor, such as a charge coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) device. CCD image sensors can be implemented in several different architectures, including, for example, full-frame, frame-transfer and interline techniques depending upon usage and applications.

One of the challenges of utilizing such an image sensor in an imaging system is to reduce image distortion and to minimize any loss of image details in a dark region or a bright region of an image signal. For example,FIG. 13illustrates a gray scale histogram of an image signal describing a visual image, such as, an object scene with a brightness range beyond a dynamic/effective range of the imaging CCD. As shown inFIG. 13, the gray scale histogram of an image signal is represented by a pixel amount as a function of a gray scale using Cartesian coordinates. Specifically, on the gray scale axis (i.e., X-axis) as a function of the pixel amount axis (i.e., Y-axis), “0” denotes pure black, and “D” denotes pure white. The range 0˜D is the dynamic range of the CCD. An area that is located in the lower part of the range 0˜D represents a dark portion of the image signal (i.e., shadows). Conversely, an area that is located in the higher part of the range 0˜D represents a bright portion of the image signal (i.e., highlights). The histogram contour curve10, as shown inFIG. 13, indicates: (1) any portion of the image signal brighter than “D” will be displayed as pure white by the pixels corresponding to that portion losing any detail that would be revealed with different degrees of whiteness displayed; and (2) any portion of the image signal blacker than “0” will be displayed as pure black by the pixels corresponding to that portion losing any detail that would be revealed with different degrees of blackness displayed. In other words, the CCD image sensor can not describe the bright portion of an object scene in detail. Additionally, the pixels corresponding to the brightest portion(s) of an object scene may bloom, inducing a smear effect in the CCD image sensor. As a result, characteristics of an image signal may distort. Likewise, the CCD image sensor can not describe the dark portion of the object scene in detail as well.

In order to avoid inducing the smear effect in an image, a CCD image sensor is typically exposed to a smaller amount of light intensity. As result, less image details are lost in the bright portion of an image signal. However, a loss of image details in the dark portion of the image signal is much more severe. Referring toFIG. 14, when a CDD image sensor is exposed to a smaller amount of light, less details are lost in the bright portion of the image signal, but a loss of details in the dark portion of the image signal is severe, as indicated by the histogram contour curve20.

Therefore, it is desirable to provide a method and an apparatus for correcting an image signal generated from an image sensor, such as a CCD image sensor, and an imaging system incorporating such a CCD image sensor so as to reduce image distortion and to minimize any loss of image details in a dark region or a bright region of an image signal.

SUMMARY

Several aspects and example embodiments of the present invention relate to an apparatus and a method of correcting an image signal generated by a CCD image sensor to reduce image distortion and to minimize any loss of image details in a dark region or the bright region of an image signal.

In accordance with an example embodiment of the present invention, an apparatus for correcting an image signal generated by a charge coupled device (CCD) image sensor in an imaging system is provided with the CCD image sensor having a plurality of effective pixels and a plurality of optical black (OB) pixels to determine a dark reference of the image signal, and the imaging system including a plurality of gamma correction curves, each of which comprises a respective correction factor for increasing contrast in a dark portion of the image signal. The image correction apparatus comprises a measuring unit to measure a plurality of gray scale values of the plurality of effective pixels and the plurality of optical black (OB) pixels; an estimating unit to estimate a contrast level of an object scene to be imaged using the plurality of measured gray scale values; and a correcting unit to correct the image signal using a corresponding gamma correction curve, depending on the estimated contrast level of the object scene.

In accordance with another example embodiment of the present invention, an imaging system is provided with a charge coupled device (CCD) image sensor comprising a plurality of effective pixels for generating an image signal, and a plurality of optical black (OB) pixels for determining a dark reference of the image signal; a memory to store a plurality of gamma correction curves, each of which comprises a respective correction factor for increasing contrast in a dark portion of the image signal; and an image correction apparatus arranged to correct the image signal generated from the CCD image sensor, wherein the image correction apparatus comprises a measuring unit to measure a plurality of gray scale values of the plurality of effective pixels and the plurality of optical black (OB) pixels; an estimating unit to estimate a contrast level of an object scene to be imaged using the plurality of measured gray scale values; and a correcting unit to correct the image signal using a corresponding gamma correction curve, depending on the estimated contrast level of the object scene.

In accordance with yet another example embodiment of the present invention, a method of correcting an image signal generated by a charge coupled device (CCD) image sensor in an imaging system is provided with the charge coupled device (CCD) image sensor having a plurality of effective pixels for producing the image signal and a plurality of optical black (OB) pixels to determine a dark reference of the image signal, and the imaging system having a memory for storing a plurality of gamma correction curves, each of which has a respective correction factor to increase contrast in a dark portion of the image signal. Such a method comprises measuring a plurality of gray scale values of the plurality of effective pixels and the plurality of optical black (OB) pixels in the CCD image sensor; estimating a contrast level of an object scene to be imaged using the plurality of measured gray scale values; and correcting the image signal using a corresponding gamma correction curve depending on the estimated contrast level of the object scene.

In addition to the example embodiments and aspects as described above, further aspects and embodiments will be apparent by reference to the drawings and by study of the following descriptions.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1is a block diagram of an imaging system including a pickup lens, a trigger mechanism and a charge coupled device (CCD) according to an example embodiment of the present invention. Referring toFIG. 1, an imaging system100includes an exposure section110, an image sensor, such as a charge coupled device (CCD) image sensor120, an analog-to-digital converter (ADC)130, a trigger mechanism140, a memory150and an apparatus200for correcting an image signal obtained from the CCD image sensor120(herein referred to as “an image correction apparatus” for the sake of brevity).

The exposure section110includes optics, such as a pickup lens112for directing image light from an object scene toward the CCD image sensor120, and a shutter114for regulating exposure time. The pickup lens112is arranged to expose the CCD image sensor120to image light for a predetermined period of time dependent upon exposure requirements, for example, a time period between 1/1000 to several seconds (s). The photo-induced image charges are then swept from the pixels in the CCD image sensor120so as to form an image signal of an object scene in an analog form. The ADC130is configured to digitize the photo-induced charge generated by each pixel of the CCD image sensor120into a respective gray scale value of that pixel, and generate a digital image signal from the analog input signal for each pixel. The image correction apparatus200is then utilized to correct the image signal.

The trigger mechanism140is arranged to trigger a shutter114of the exposure section110and start operation of the image correction apparatus200. The memory150is used to store a number of gamma correction curves, each of which includes a respective correction factor for increasing contrast between various dark portions of an image signal, and is used (e.g., selected and calculated) by the image correction apparatus200for correcting the image signal.

Turning now toFIG. 2, detailed circuit diagram of the CDD image sensor120and the image correction apparatus200are shown. Referring toFIG. 2, the CCD image sensor120includes an effective pixel region121, an optical black (OB) region122, a number of shift registers123, and a read out register124. The effective pixel region121comprises a number of effective pixels125configured to produce an image signal. The optical black (OB) region122comprises a number of optical black (OB) pixels126, each of which is configured to determine a respective dark reference (dark current) of a respective effective pixel125. Each shift register993is configured to store and buffer the photo-induced charge of each pixel of a respective line (vertical line). The read out register124is configured to transfer, on a row by row basis, the photo-induced charge of each pixel buffered in the shift registers123to the ADC130. Additional output amplifiers (not shown) may be used to output an image signal commensurate with the photo-induced charges transferred from the CCD image sensor120.

FIG. 3shows a photoelectric conversion characteristic of a CCD image sensor120, shown inFIG. 2. The curves30and32respectively indicate a photoelectric conversion characteristic of the optical black (OB) pixels126and the effective pixels125in the CCD image sensor120, and T and D respectively denote the highest gray scale value the optical black pixels126and the effective pixels125can generate.

Referring back toFIG. 2, the image correction apparatus200includes a measuring unit210, an estimating unit220, and a correcting unit230. The measuring unit210is arranged to measure a gray scale value of each pixel of the CCD image sensor120. The estimating unit220is configured to estimate a contrast level of an object scene using the measured gray scale values. The correcting unit230is configured to correct the image signal using a corresponding gamma curve selected, depending on the estimated contrast level of the object scene, from the memory150.

Specifically, the estimating unit220includes a first counter222, a judging sub-unit224, a second counter226, and an estimator228. The first counter222is arranged to count a first amount of optical black (OB) pixels126in the CCD image sensor120with a gray scale value in a first predetermined gray scale range. The judging sub-unit224is configured to determine whether a smear effect induced in the CCD image sensor120is acceptable (e.g., no smear effect induced or the level of the smear effect is below a predetermined threshold) based the first counted amount from the first counter222. The second counter226is arranged to count a second amount of effective pixels125in the CCD image sensor120with a gray scale value in a second predetermined gray scale range in the acceptable smear effect case. The estimator228is configured to estimate the contrast level of an object scene using the second amount in the acceptable smear effect case, or according to the level of the smear effect in the unacceptable smear effect case (e.g., the level of the smear effect exceeds the predetermined threshold).

Turning now toFIG. 4, a method of correcting an image signal generated from a charge coupled device (CDD) image sensor according to an example embodiment of the present invention is shown. Such a method can be performed by the image correction apparatus200, shown inFIG. 2. Referring toFIG. 4, a method of correcting an image signal generated from the CCD image sensor120includes the following operations:

Operation410: Measuring the gray scale value of each pixel of the CCD image sensor120from the ADC130. Specifically, before measuring, previewing by the imaging system100(e.g., zooming and focusing) is carried out to obtain a high quality image signal. Then, the trigger mechanism140is activated to trigger the shutter114of the exposure section110and start the image correction apparatus200, as shown inFIG. 1. Thus, the image signal with the corresponding dark reference is generated, and is digitized by the ADC130into (D+1) scales.

Operation420: Estimating the contrast level of an object scene using the measured gray scale values.

Operation430: Correcting the image signal using a corresponding gamma correction curve depending on the estimated contrast level of the object scene.

As for operation420(estimating operation), it can be inferred that the higher contrast level of an object scene is, the more the detail loss of the dark portion of an image signal is, and the higher level the smear effect is. Therefore, the contrast level of the object scene can be estimated by either the level of lost image details of the dark portion of the image signal, or alternatively, the level of the smear effect. Whereas, if the smear effect is acceptable, the contrast level of the object scene can be advantageously estimated according to the amount of detail lost in the dark portion of the image signal, because, in this case, the level of the smear effect may be too low to discern the contrast levels of the object scene. Considering the detail loss of the dark portion of the image signal is related to the amount of the effective pixels125in the dark portion of the image signal (hereafter referred as “DP amount”, the larger the DP amount is, the more the detail loss of the dark portion of the image signal is), the contrast level of the object scene can be estimated using the DP amount in the acceptable smear effect situation.

On the other hand, if the smear effect is not acceptable, photo-induced charges of the blooming effective pixels125in the CCD image sensor120will overflow to other pixels in the same line along the respective shift register123, resulting in: (1) some effective pixels125in the same line corresponding to the dark portion of an object scene will exhibit an untrue high gray scale value (hereafter referred as “abnormal effective pixel”), the DP amount can not be accurately counted in this case; (2) some optical black (OB) pixels126in the same line exhibit an untrue gray scale value higher than the T (hereafter referred as “abnormal optical black (OB) pixel”), these abnormal optical black (OB) pixels are countable because their gray scale values exceed T; and (3) the greater the level of the smear effect, the greater number of the abnormal effective pixels and the abnormal optical black (OB) pixels there are, the level of the smear effect is related to the amount of abnormal optical black pixel (hereafter referred as “OB amount”).

Therefore, the contrast level of an object scene can be advantageously estimated according to the level of the smear effect in the unacceptable smear effect situation, in view of the abovementioned (1) some effective pixels125in the same line corresponding to the dark portion of an object scene will exhibit an untrue high gray scale value (hereafter referred as “abnormal effective pixel”), and (3) the greater the level of the smear effect, the greater number of the abnormal effective pixels and the abnormal optical black (OB) pixels there are, the level of the smear effect is related to the amount of abnormal optical black pixel (the DP amount cannot be accurately counted in a high level smear effect situation).

FIG. 5is a sub-flow chart of the contrast level estimating operation420, shown inFIG. 4. Referring toFIG. 5, the contrast level estimating operation420includes the following operations:

Operation510: Determining whether the smear effect induced in the CCD image sensor120is acceptable;

Operation520: Counting the DP amount if the smear effect induced in the CCD image sensor120is determined to be acceptable;

Operation530: Estimating the contrast level of the object scene using the DP amount if the smear effect induced in the CCD image sensor120is determined to be acceptable; and

Operation540: Estimating the contrast level of the object scene according to the level of the smear effect if the smear effect induced in the CCD image sensor120is determined not to be acceptable.

FIG. 6is a sub-flow chart of the smear effect determining operation510, shown inFIG. 5. Referring toFIG. 6, the smear effect determining operation510includes the following operations:

Operation610: Counting the OB amount, in detail, that is, the first counter222counts the first amount of the optical black (OB) pixels126with a gray scale value in the first predetermined gray scale range of T˜D, and the first amount is the OB amount. Since the total amount of the optical black (OB) pixels126is invariable, the OB amount can be determined by the first counter222in counting the first amount of the optical black (OB) pixels126with a gray scale value in a first predetermined gray scale range of 0˜T, and deducting the first amount from the total amount of the optical black (OB) pixels126in the CCD image sensor120.

Operation620: Determining whether the OB amount is below a predetermined threshold, where the predetermined threshold is settable, and is determined depending on a total amount of the optical black (OB) pixels126and/or the quality requirement of the image signal. As a part of operation620, if the OB amount is determined to be below the predetermined threshold, the smear effect is acceptable at operation630. Otherwise, the smear effect is not acceptable at operation640. Particularly, the judging sub-unit224within the estimating unit220, shown inFIG. 2, compares the OB amount with the predetermined threshold to determine whether the OB amount is below the predetermined threshold.

Referring back toFIG. 5, in operation520(DP amount counting step), the second counter226within the estimating unit220, shown inFIG. 2, counts the second amount of the effective pixels125with a gray scale value in the second predetermined gray scale range of 0˜31D/256 (the gray scale range of the dark portion of an image signal in this example embodiment, but not limited thereto), the second amount is the DP amount. Similar to counting OB amount, an alternative technique for the second counter226to determine the DP amount is to count the second amount of the effective pixels125in the CCD image sensor120with a gray scale in the second predetermined gray scale range of 31D/256˜D, and deduct the second amount from the total amount of the effective pixels125in the CCD image sensor120.

In operation530and operation540, the estimator228within the estimating unit220, shown inFIG. 2, divides a number range of 0˜N into a plurality of first number sub-ranges, and records the first number sub-ranges therein, where N is the total amount of the effective pixels125or the optical black (OB) pixels126in the CCD image sensor120. Each first number sub-range is related to one respective contrast level of the object scene. The estimator228compares the DP amount with each first number sub-range to find out the contrast level of the object scene the DP amount is related to, if the smear effect is judged to be acceptable. On the other hand, the estimator228further divides a second number range of P˜N into a plurality of second number sub-ranges, and records the second number sub-ranges therein, where P is the predetermined number. The estimator228compares the OB amount with each second number sub-range and determines the contrast level of the object scene the OB amount is related to, if the smear effect is determined not to be acceptable.

In this example embodiment of the present invention, the estimator228within the estimating unit220, shown inFIG. 2, divides and records three first number sub-ranges and three second number sub-ranges, each first number sub-range is related to a respective second number sub-range, and is related to a respective contrast level of the object scene. In other words, the estimator228can discern three contrast levels of the object scene, and the memory150stores three gamma correction curves therein.

Referring back toFIG. 4, in operation430(correcting operation), the correcting unit230, shown inFIG. 2, selects a gamma correction curve according to the estimated contrast level of the object scene, and corrects the image signal. Two examples of such an image correction operation430are described in connection withFIG. 7andFIG. 10herein below.

For example,FIG. 7illustrates a gray scale histogram of an image signal to be corrected in accordance with a first example. As shown inFIG. 7, an area70designated represents a dark portion of the image signal. Referring toFIG. 8, the curve80denotes a non-linear input-output characteristic curve of a display device for displaying the image signal; the curve82denotes a typical gamma curve for pre-correcting the non-linear input-output characteristics of the display device; the curve84is the corresponding gamma curve selected from the memory150for correcting the image signal; and the area86(defined by the curve84,82and the ordinate) is the respective correction factor for increasing contrast of the dark portion of the image signal.

FIG. 9shows a gray scale histogram of a corrected image signal in accordance with the first example. As shown inFIG. 9, after correcting, the detail loss of the dark portion of the image signal is improved significantly.

Turning now toFIG. 10, a gray scale histogram of an image signal to be corrected in accordance with a second example is shown. As shown inFIG. 10, an area70arepresents a dark portion of the image signal.FIG. 11shows another gamma correction curve84aincluding a corresponding correction factor86a, the correction factor86ais capable of increasing contrast of the dark portion of the image signal.FIG. 12illustrates a gray scale histogram of a corrected image signal in accordance with the second example.

It is to be understood that all the predetermined terms in the example embodiment of the present invention, e.g., the first predetermined gray scale range in operation610, shown inFIG. 6, and the second predetermined gray scale range in operation520, shown inFIG. 5, are settable, and can be preset depending on the quality requirement of the image signal, the total amount of the gray scales, and/or the total amount of the pixels of the CCD image sensor120. Additionally, the correction factor introduced in each gamma curve is predetermined according to what degree of contrast is desired in the dark regions of an image signal and applied to that portion of the gamma curve affecting the dark regions of an image signal.

As described in the foregoing, the imaging system100, the image correction apparatus200, and the method as shown inFIG. 4,FIG. 5andFIG. 6, are capable of improving the detail loss of the dark portion of the image signal, thus the quality of the image signal is improved.

Various components of the image correction apparatus200, as shown inFIG. 1andFIG. 2, such as the measuring unit210, the estimating unit220and the correcting unit230can be integrated into a single control unit, or alternatively, can be implemented in software algorithm or hardware, such as, for example, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). In addition, method steps ofFIG. 4,FIG. 5andFIG. 6, may be performed by the same control unit or a processor executing instructions organized into a program module or a custom designed state machine. As such, it is intended that the processes described herein be broadly interpreted as being equivalently performed by software, hardware, or a combination thereof. As previously discussed, software modules can be written, via a variety of software languages, including C, C++, Java, Visual Basic, and many others. These software modules may include data and instructions which can also be stored on one or more machine-readable storage media, such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; and optical media such as compact discs (CDs) or digital video discs (DVDs). Instructions of the software routines or modules may also be loaded or transported into the wireless cards or any computing devices on the wireless network in one of many different ways. For example, code segments including instructions stored on floppy discs, CD or DVD media, a hard disk, or transported through a network interface card, modem, or other interface device may be loaded into the system and executed as corresponding software routines or modules. In the loading or transport process, data signals that are embodied as carrier waves (transmitted over telephone lines, network lines, wireless links, cables, and the like) may communicate the code segments, including instructions, to the network node or element. Such carrier waves may be in the form of electrical, optical, acoustical, electromagnetic, or other types of signals.

While there have been illustrated and described what are considered to be example embodiments of the present invention, it will be understood by those skilled in the art and as technology develops that various changes and modifications, may be made, and equivalents may be substituted for elements thereof without departing from the true scope of the present invention. Many modifications, permutations, additions and sub-combinations may be made to adapt the teachings of the present invention to a particular situation without departing from the scope thereof. Accordingly, it is intended, therefore, that the present invention not be limited to the various example embodiments disclosed, but that the present invention includes all embodiments falling within the scope of the appended claims.