Apparatus for compensating for color shading on a picture picked up by a solid-state image sensor over a broad dynamic range

In a solid-state image pickup apparatus, a preliminary pickup circuit performs, during preliminary pickup, divided photometry on a photometry signal outputted from the primary cells of photo-sensors, determines an exposure condition in which photometry data in all divided blocks do not exceed the saturation maximum value of the primary cells, and calculates the individual photometry data tints under the above exposure condition. During actual pickup, a shading corrector included in an image processing circuit divides subsidiary image data in the same manner as during the divided photometry, calculates shading correction gains in accordance with the photometry data tints and those of the subsidiary image data, and interpolates the shading correction gains in accordance with the pixel for thereby executing color shading correction on the subsidiary image data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a solid-state image pickup apparatus for compensating for color shading on a picture picked up by a solid-state image sensor having photo-sensors arranged, each of which has a set of photosensitive cells different in sensitivity from each other, to produce image information over a broad dynamic range, and a method of compensating for color shading for the same.

2. Description of the Background Art

It is a common practice with a solid-state image pickup apparatus capable of picking up a picture over a broad dynamic range by a solid-state image sensor, which has photo-sensors arranged, each having a primary and a secondary photosensitive cell different in sensitivity from each other. The primary cell, higher in sensitivity to incident light than the secondary cell, forms an image with a well modulation. The secondary cell accurately captures image, or contrast, information even on an imaged region that would cause a white blur in a usual exposure condition or would cause the primary cell to produce a saturated signal. U.S. patent publication No. US 2003/0141564 A1 of Kondo et al, for example, proposes a solid-state image pickup device capable of picking up a picture over a broad dynamic range by processing signals outputted from the main and subsidiary photosensitive fields.

Generally, in a solid-state image pickup device, image signals outputted from a solid-state image sensor may involve shading due to, e.g. unevenness in quantity of light incident to the respective photo-sensors. For example, in the case of a digital camera including a solid-state image sensor which has microlenses formed, the quantity of incident light to the respective photo-sensors noticeably varies in dependence upon the direction of the incidence particular to the microlenses. More specifically, to a photosensitive cell located in the vicinity of the edge of the imaging frame of the image sensor, light is incident often inclined, so that the incident light is poorer in quantity than one incident to a photosensitive cell located near the center of the frame. Consequently, the luminance of the signals produced in the vicinity of the edge of the frame is poorer, thus resulting in shading.

Japanese patent laid-open publication No. 79773/1996 discloses a shading correcting device for correcting shading by multiplying image signals outputted from an image sensor by a shading correction coefficient, which is calculated by approximating by a quadric curved surface function. The shading correcting device with this configuration can accomplish a shading correction feasible for mass-production of image sensors and therefore free the image sensors from irregularity.

Further, U.S. patent publication No. US 2002/0008760 A1 of Nakamura teaches a digital camera for correcting shading on the basis a pixel-by-pixel correction value. The digital camera divides an imaging area into a preselected number of blocks and stores light amount correction data particularly assigned to the blocks beforehand. When correcting shading, the digital camera calculates the correction value of each pixel by weighting the light amount correction data in accordance with the positions of target pixels to thereby generate correction values for respective pixels.

The prior art documents described above have the following problems left unsolved. In Kondo et al, the solid-state image pickup device picks up an image over a broad dynamic range by processing signals outputted from the main and subsidiary photosensitive fields each constituting a particular pixel. However, the problem with Kondo et al is that all subsidiary fields in an imaging frame are arranged at one side with respect to the main photosensitive field without regard to the pixel position in the imaging frame, thus involving shading on a picture picked up which depends upon, e.g. the exit pupil position of a lens or an iris value of the camera.

The shading correcting device disclosed in Japanese publication No. 79773/1996 needs to pick up, during adjustment, a subject radiated with uniformly light, and, on the basis of the result from adjustment, the shading correction gain is determined. Then, in the adjustment procedure, correction data have to be obtained in accordance with, e.g. a zoom lens position or an iris value, and therefore, a long period of time is necessary for calculation. Although the shading correction of such a shading correcting device may be desirable for manufacturing solid-state image sensors, it is not feasible for adjusting or calibrating digital cameras.

Nakamura has a problem that because a memory in the digital camera stores correction data in one-to-one correspondence to various zoom lens positions and various iris values, the memory is required to have its storage capacity increased. Should the number of zoom lens positional sections and that of iris value sections be reduced in order to reduce the data amount required to be stored in the memory, the digital camera would involve image pickup conditions in which a complete correction is unable.

Apart from the problems stated above, secondary image data outputted from the secondary cell are used to reproduce high-luminance information and therefore often remain contained even in a completed picture in the form of high-luminance information. On a picture containing such high-luminance information, luminance shading, common to R (red), G (green) and B (blue) pixels, is not conspicuous. RGB or coloristic shading is dependent upon the wavelength of the colors R, G and B. Consequently, the coloristic shading is different in the degree of deviation between colors so as to be deviate much more toward the longer wavelength. This causes a picture to be viewed worse and critically degraded in image quality.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a solid-state image pickup apparatus capable of compensating for shading on an image picked up by a primary cell and a secondary cell positioned at each pixel over a broad dynamic range to thereby obviate critical degradation of image quality while reducing a period of time necessary for an adjustment step and the amount of data stored in a memory, and a method of compensating for shading for the same.

A solid-state image pickup apparatus of the present invention includes a solid-state image sensor including first photosensitive cells for photoelectrically converting incident light and second photosensitive cells for photoelectrically converting the incident light with lower sensitivity than the first photosensitive cells. The first photosensitive cells each form a particular pixel together with corresponding one of the second photosensitive cells. A signal processor processes a first and a second image signal outputted from the first and second photosensitive cells, respectively. The signal processor includes an actual pickup circuit for processing an actual pickup signal representative of a subject field actually picked up, and a photometry circuit for processing a photometry signal representative of the quantity of light incident from the field to perform photometry before actual pickup. The actual pickup circuit includes a shading corrector for executing shading correction on the actual pickup signal. The photometry circuit divides an image represented by the photometry signal into a preselected number of blocks, measures the quantity of incident light block by block to thereby produce the result of photometry block by block, and generates first color shading correction information in accordance with the result of photometry. The shading corrector includes a color shading corrector for generating shading correction gains in accordance with the first color shading correction information and executing, based on the shading correction gains, color shading correction on the actual pickup information.

A method of correcting shading for the above image pickup apparatus is also disclosed.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring toFIG. 1of the drawings, a solid-state image pickup apparatus embodying the present invention is generally designated by the reference numeral10. As shown, the image pickup apparatus10includes optics12to which light is incident from a subject field being picked up. When the operator of the image pickup apparatus10operates a control panel14, a system controller16and a timing generator18control other various-sections so that an image pickup section20picks up a subject field. Image data representing an image thus captured is outputted from the image pickup section20and sequentially processed by a preprocessor22and a signal processor22to be then inputted to a compressor28connected to a bus26as an image signal. The image signal is compressed by the compressor28to be delivered to a monitor30and a recorder32. It is to be noted that parts and elements not directly relevant to the understanding of the present invention are not shown, and a detailed description thereof will not be made in order to avoid redundancy.

The optics12includes a lens, a diaphragm control mechanism, a shutter mechanism, a zoom mechanism and an AF (Automatic Focus) control mechanism, although not shown specifically. The lens is positioned substantially at the front of a barrel, not shown, and may be configured as a combination of lenses. The diaphragm control mechanism varies the radius of an optical aperture to control the quantity of light to be incident to the image pickup section20. The shutter mechanism includes a mechanical shutter configured to close the above aperture to cut off the incident light on its optical path. Further, the zoom mechanism zooms in or out by moving the lens or lenses while the AF control mechanism focuses the image pickup apparatus10on a desired subject in accordance with the distance therebetween.

In the optics12, the diaphragm control mechanism, shutter mechanism, zoom mechanism and AF control mechanism are driven in response to control signals, generally106, to pick up the image of a desired field and focus it on a photosensitive array of the image pickup section20. In this sense, the optics12plays the role of a light incidence mechanism. Considering the fact that an optical range capable of being measured by photometry at a time is limited, the diaphragm control mechanism and an electronic shutter, for example, may be adapted to be repeatedly driven to output several segments of photometric data, if desired. It is to be noted that signals are hereinafter designated by reference numerals attached to connections on which they appear.

The control panel14includes, e.g. a shutter release button, not shown, that can be operated by the user at the time of a shot. The control panel14sends out a signal104representative of, e.g. the operation of the shutter release button to a system controller16.

In the illustrative embodiment, the shutter release button is adapted to be depressed into its two consecutive strokes. More specifically, when the shutter release button, held in its initial or non-depressed position, is depressed into its first stroke, or half-stroke position, the control panel14sends out a stroke signal104indicative of preliminary pickup step for detecting pickup conditions suitable for a desired subject field to the system controller16. When the shutter release button is further depressed into its second stroke, or a full-stroke position, the stroke signal104, sent out from the control panel14to the system controller16, indicates an actual pickup step for shooting and recording the scene in accordance with the pickup conditions detected in the preliminary pickup step. In the illustrative embodiment, in the preliminary pickup step, an image of the field is picked up to be inputted to the image pickup section20so as to obtain information necessary for AF control and AE (Automatic Exposure) control form the image.

The system controller16controls the entire or general operation of the image pickup apparatus10in response to the stoke signal104outputted from the control panel12. For example, in the illustrative embodiment, the system controller16delivers the control signal106, a control signal108and a control signal110to the optics12, a timing generator18and the bus26, respectively, in response to the stroke signal104. If desired, the system controller16may include a CPU (Central Processing Unit) not shown. Further, as shown inFIG. 2, the system controller16controls a pickup controller90on the basis of AE control information and AF control information fed from an AE controller42and an AF controller44, respectively.

Referring again toFIG. 1, the timing generator18includes a clock oscillator, not shown, for generating a basic or system clock112on the basis of which the image pickup apparatus10operates. The timing generator18delivers the basic clock112to the system controller16and to the almost all of the components in the apparatus10as well, although not shown specifically inFIG. 1, in response to the control signal108fed from the system controller16.

Further, in the illustrative embodiment, the timing generator16generates various timing signals by dividing the basic clock in response to the control signal108outputted from the system controller16. For example, the timing signal generator18feeds a timing signal114including a vertical and a horizontal sync signal and electronic shutter drive pulses to the image pickup section20. Also, the timing signal generator18feeds a timing signal116including sampling pulses for correlated double sampling and a conversion clock for analog-to-digital conversion to the preprocessor22.

The image pickup section20includes a photosensitive cell array300, corresponding to a single frame of picture picked up, and a horizontal transfer path not shown.FIG. 4shows part of a photosensitive cell array300including a plurality of photo-sensors302and vertical transfer paths308. Each of the photo-sensors302corresponds to a particular pixel. The image pickup section20photoelectrically transduces light incident to the photosensitive cell array300to a corresponding electric signal118, seeFIG. 1. The photosensitive cell array300corresponds to an image capturing field. In the illustrative embodiment, the image pickup section20may be implemented by any suitable image sensor, e.g. a CCD (Charge Coupled Device) or a MOS (Metal Oxide Semiconductor) image sensor. As shown inFIG. 1, the image pickup section20is controlled in response to the timing signal114so that light102incident from a subject field via the optics12is photoelectrically transduced to corresponding signal charges by the photo-sensors302. In the image pickup section20, the signal charges is then transformed to a corresponding analog electric signal118and outputted to the preprocessor22.

More specifically, the photo-sensors302may be arranged in a so-called honeycomb pattern, in which each photo-sensor302are shifted from nearby photo-sensors302by a distance half as long as the pitch of the photo-sensors in the row or column direction on the photosensitive array300. Alternatively, the photosensitive array300may be adapted such that the photo-sensors302are arranged in a square matrix pattern at a fixed pitch in the row and column directions, if desired. In the illustrative embodiment, each photo-sensor302is made up of a primary cell or high-sensitivity region304and a secondary cell or low-sensitivity region306. The primary cell304and secondary cell306, each photoelectrically transducing incident light to a corresponding electric signal representative of the quantity or intensity of incident light, may be implemented by photodiodes by way of example. With this configuration, the image pickup section20is capable of outputting the analog electric signal118containing both of main image data and subsidiary image data derived from the primary cell304and secondary cell306, respectively.

The image pickup section20may be configured to transform, in a preliminary pickup step, the quantity of incident light to a photometry signal while reducing, or thinning out, pixels for rapid read-out to output the photometry signal, whereas reading out, in an actual pickup step following the preliminary pickup step, image signals from the whole pixels of the photosensitive cell array to output the entire signals.

As shown inFIG. 1, the preprocessor22executes analog signal processing on the analog electric signal118representative of an image in response to the timing signal116for thereby outputting a resulting digital image signal120. The preprocessor22may include a CDS (Correlated Double Sampling) circuit, a GCA (Gain Controlled Amplifier), an AD (Analog-to-Digital) converter and so forth, although not shown specifically.

The signal processor24performs digital signal processing on the digital image signal120inputted from the preprocessor22for thereby outputting resulting digital image signals122and130. The digital image signals122and130are fed to the bus26and compressor28, respectively. More specifically, in the illustrative embodiment, the signal processor24receives the digital image signal120containing the main and subsidiary image data from the preprocessor22and effects digital signal processing in response to a control signal122, which is the control signal110received from the system controller16over the bus26.

FIG. 2shows a specific configuration of the signal processor24. As shown, the digital image signal120inputted to the signal processor24is separated into main image data202and subsidiary image data204. The main image data202and subsidiary image data204thus separated are temporarily stored in a main image memory52and a subsidiary image memory54, respectively, on a pixel-by-pixel basis. A preliminary pickup circuit40performs AF control, AE control and other preliminary pickup operations, in the preliminary pickup step, on the basis of main image data206outputted from the main image memory52. An image processing circuit50generates a digital image signal in the actual pickup step on the basis of the main and subsidiary image data stored in the image memories52and54, respectively.

The preliminary pickup circuit40includes an AE controller42and an AF controller44, in which AE and AF control information is determined based on the main image data206from the main image memory52. Particularly, the preliminary pickup circuit40includes an addition circuit46for generating shading correction information and stores it in a RAM (Random Access Memory)70of the image processing circuit50. The shading correction information may be generated in accordance with the AE control information by the system controller16.

More specifically, the preliminary pickup circuit40receives the main image data, i.e. a photometry signal206read out from the main image memory52in the preliminary pickup step. The preliminary pickup circuit40may be configured to separate, as shown inFIG. 7, the main image data206into256pieces of R image data602,256pieces of G image data604and256pieces of B image data606for processing. The preliminary pickup circuit40may deal with the thus separated photometry signal206even when actual pickup step is effected without the intermediary of preliminary pickup step. Further, when green image data G included in the photometry signal206are greater in the number of pixels than the image data of the other colors, e.g. two times as much as the image data of the other colors, the preliminary pickup circuit40may subdivide the image data G into green image data604and608. The preliminary pickup circuit40so adjusts the photometry signal206as to establish an exposure condition in which a white blur is not involved, i.e. a picture is not picked up beyond the dynamic range, thereby generating shading correction information under optimum pickup conditions.

In the preliminary pickup circuit40, an AE controller42divides an image represented by the photometry signal206into a preselected number of blocks and determines an exposure condition for accomplishing optimum automatic exposure. The preselected number mentioned above is one or above, but smaller than the number of pixels, and may be selectable. For example, as shown inFIG. 5, the entire image402represented by the photometry signal206may be divided into 8*8 blocks, i.e. sixty-four blocks, in which case the AE controller42effects divided photometry such that an integrated luminance value is produced in each of the sixty-four blocks. More specifically, the AE controller42adds data from pixels lying in each block to thereby calculate an addition block404, which includes R photometry data R_AE(i,j), G photometry data G_AE(i,j) and B photometry data B_AE(i,j). It is to be noted that letters i and j are indices representative of the x and y coordinates of the addition block404, respectively, and initially zero each.

To narrow down an exposure condition in accordance with the photometry data R_AE(i,j), G_AE(i,j) and B_AE(i,j), the AE controller42calculates block by block:

In the above expression, R_gain(fine), G_gain(fine and B_gain(fine) are gain values for fine days and assigned to red pixels, green pixels and blue pixels, respectively. To narrow down an exposure condition, the weighted mean of Y_AE(i,j) may be used as an evaluation function; when priority is given to the center of the frame by way of example, importance is attached to Y_AE(i,j) around the center of the frame. The AE controller42feeds the system controller16with AE control information derived from the exposure condition thus determined.

An AF controller44also included in the preliminary pickup circuit40produces AF control information controlling a diaphragm, a shutter speed and so on, in response to the photometry signal206and exposure information derived from the AE control circuit42. The AF control information is also outputted to the system controller16. An addition circuit46determines shading correction information in accordance with the photometry data R_AE(i,j), G_AE(i,j) and B_AE(i,j) derived from the AE controller42block by block and writes in the information into a RAM (Random Access Memory)70, which is included in the image processing circuit50. In the illustrative embodiment, the addition circuit46calculates tints R_AE(i,j)/G_AE(i,j) and B_AE(i,j)/G_AE(i,j) and then stores the tints in the RAM70, e.g. as shading correction information.

As shown inFIG. 3, the image processing circuit50includes an offset corrector56, an LMTX (Linear Matrix) corrector60and a WB (White Balance) corrector64cooperating to correct the main image data and subsidiary image data stored in the main image memory52and subsidiary image memory54, respectively. In the illustrative embodiment, the offset corrector56, LMTX correct or60and WB corrector64are connected to EEPROMs (Electrically Erasable Programmable Read-Only Memory)58,62and66, respectively.

An SHD (Shading) corrector72also included in the image processing circuit50compensates for color shading, particularly color shading of the subsidiary image data in the illustrative embodiment in accordance with the shading correction information stored in the RAM70. This color shading should preferably be compensated for at least after the subsidiary image data have been subjected to offset correction. The SHD corrector72may additionally compensate for luminance shading of the main image data and subsidiary image data stored in the main image memory52and subsidiary image memory54, respectively.

The SHD corrector72will compensate for color shading as described more specifically hereinafter. For color shading correction, the subsidiary image data stored in the subsidiary image memory54are divided in matching relation to the specific addition blocks shown inFIG. 5. The SHD corrector72adds subsidiary image data block by block to thereby produce R subsidiary image data r(i,j), G subsidiary image data g(i,j) and B subsidiary image data b(i,j) and then calculates subsidiary image data tints r(i,j)/g(i,j) and b(i,j)/g(i,j) block by block. Further, by obtaining photometry data tints R_AE(i,j)/G_AE(i,j) and B_AE(i,j)/G_AE(i,j) from the RAM70, the SHD corrector72calculates comparative gains r_gain(i,j) and b_gain(i,j) block by block in accordance with the subsidiary image data and photometry data tints by using the following expressions (1) and (2):
r_gain(i,j)=(R—AE(i,j)/G—AE(i,j) /(r(i, j)/g(i,j)   (1)
b_gain(i,j)=(B—AE(i,j)/G—AE(i,j) /(b(i,j)/g(i,j))   (2)

The comparative gains r_gain(i,j) and b_gain(i,j) are necessary for matching the average subsidiary image data tint to the average main image data tint corresponding thereto.

As shown inFIG. 6, in the illustrative embodiment, the SHD corrector72matches comparative gains504represented by r_gain(i,j) and b_gain(i,j) to the addition blocks to thereby produce a comparative gain image502, which corresponds to the image402ofFIG. 5. The SHD corrector72smoothly, bidimensionally interpolates the comparative gains504, i.e. r_gain(i,j) and b_(i,j) by expanding them on a pixel basis by, e.g. spline interpolation while applying the gains pixel by pixel.

Assume that the SHD corrector72interpolates the comparative gains in the vertical and horizontal directions on an image by tridimensional spline interpolation by way of example. Then, if an interpolation image has 2,400*1,600 pixels, the comparative gains r_gain(i,j) and b_gain(i,j) are produced by:
r_gain(i,j)=r_gain_hokan(150+300i,100+200i)   (3)
b_gain(i,j)=b_gain_hokan(150+300i,100+200i)   (4)

In the above expressions (3) and (4), r_gain_hokan and b_gain_hokan are representative of shading correction gains to be determined for each pixel. Assuming that a given pixel is indicated by x and y coordinates, then shading correction gains r_gain_hokan(x,y) and b_gain_hokan(x,y) assigned to the pixel are expressed as:

The functions r_gain_hokanX(x,j) and b_gain_hokanX(x,j) included in the expressions (5) and (6), respectively, are expressed as:

Further, the functions M(k), N(i), P(j) and Q(j) included in the above expressions (7) and (8) satisfy the following expressions:

The SHD corrector72compensates for, based on the shading correction gains r_gain_hokan(x,y) and b_gain_hokan(x,y), the shading of the subsidiary image data r(x,y) and b(x,y) stored in the subsidiary image memory54pixel by pixel by using the following expressions (13) and (14):
r′(x,y)=r_gain_hokan(x,y)*r(x,y)   (13)
b′(x,y)=b_gain_hokan(x,y)*b(x,y)   (14)

In the image processing circuit50, a γ (gamma) corrector/combiner74executes gamma correction with the main and subsidiary image data subjected to shading correction described above and combines the resulting image data. A synchronizer78synchronizes a digital image signal, after combined, outputted from the y corrector/combiner74while a CMTX (Color Matrix) corrector82adjusts the colors of the digital image signal thus synchronized. In the illustrative embodiment, the γ corrector/combiner74, synchronizer78and CMTX corrector82are connected to EEPROMs76,80and84, respectively.

The image processing circuit50delivers the digital image signal122generated by the procedure described above to the compressor28, monitor30and recorder32over the bus26. In the illustrative embodiment, the compressor28, monitor30and recorder32are controlled by the control signal110fed from the system controller16via the bus26. If desired, the image processing circuit50may be connected to the compressor28without the intermediary of the bus26and feed a digital image signal130to the compressor28.

The compressor28compresses the digital image signal inputted thereto according to, e.g. the JPEG (Joint Photographic Experts Group) standard based upon orthogonal transform in response to the control signal110outputted from the system controller16. The resulting compressed data are delivered to, e.g. the recorder32.

The monitor30visualizes on its display screen, not shown, an image represented by the digital image signal122fed from the signal processor14and may be implemented by an LCD (Liquid Crystal Display) panel by way of example.

The recorder32records the digital image signal input thereto. As shown inFIG. 2, in the illustrative embodiment, the recorder32includes a card interface (I/F)92and a recording medium94and may write in the compressed image signal outputted from the compressor28into the recording medium94via the card interface92. For the recording medium94, use may be made of a memory card loaded with a semiconductor memory device or a drive package containing, e.g. a magneto-optical disk or similar disk-like recording body. Such a recording medium may be removably mounted to the recorder32, if desired.

Reference will be made toFIGS. 8,9and10for describing a specific operation of the solid-state image pickup apparatus10. As shown inFIG. 8, assume that the user of the apparatus10depresses the shutter release button of the control panel14when the apparatus10is held in its stand-by state (step702). Then, whether or not the shutter release button is depressed to its half-stroke position is determined (step704). If the answer of the step S704is positive (YES), there is executed the tint analysis of the addition blocks (subroutine SUB1). At the same time, the signal104, commanding photometry, is fed from the control panel14to the system controller16. On the other hand, if the answer of the step704is negative (NO), the procedure returns to the step702.

As shown inFIG. 9, the subroutine SUB1begins with exposure control (step722). At this stage, exposure may be controlled in the initial exposure condition.

After the step722, photometry data are obtained (step724), so that AE and AF control are executed by the divided photometry. At this instant, the system controller16generates the control signals106and108, including a photometry command, in accordance with an exposure state determined by AE control of the step722and delivers the signals106and108to the optics12and timing generator18, respectively. In response, the timing generator18generates the timing signals112,114and116including a photometry command, in response to the control signal108, and feeds the signals112,114and116to the system controller16, image pickup section20and preprocessor22, respectively.

Light102coming from a subject field is incident to the image pickup section20via the optics12, so that the image of the field is focused on the photosensitive array300,FIG. 4, of the image pickup section20. In the image pickup section20, signal charges for photometry on the photosensitive array300are read out in response to the timing signal114. At this instant, signal charges may be read out only from the primary cell304of the individual photo-sensors302or reduced, or thinned out, for rapid read-out. The resulting analog electric signal, or photometry signal,118is fed from the image pickup section20to the preprocessor22.

The preprocessor22executes CDS, GCA and AD conversion and other preprocessing on the analog electric signal118in response to the timing signal116to thereby generate a digital image signal120and delivers the signal120to the signal processor24. In the signal processor24, main image data202and subsidiary image data204separated from the image signal120are written into the main image memory52and subsidiary image memory54, respectively.

In the illustrative embodiment, the photometry signal206is fed from the main image memory52to the preliminary image pickup circuit40in the form of R, G and B photometry data. In the preliminary image pickup circuit40, the photometry signal206is inputted to the AE controller42and used to effect AE control in the preliminary pickup mode. The AE controller42, in turn, delivers the resulting AE control information to the system controller14and AF controller44. The AF controller44performs AF control in accordance with the AE control information and photometry signal206.

In the divided photometry executed by the AE controller42, the image402represented by the photometry signal206,FIG. 5, is divided into sixty-four addition blocks404by way of example. In this condition, pixel data lying in each addition block404are added so as to produce photometry data. Subsequently, AE control information derived from the photometry data thus generated block by block are delivered to the system controller14.

After the step S724, whether or not the photometry data lie in the dynamic range is determined block by block (step726). More specifically, in the illustrative embodiment, R, G and B photometry data R_AE(i,j), G_AE(i,j) and B_AE(i,j) are compared with a saturation maximum value EVMAX assigned to the primary cells304of the photo-sensors302. In the illustrative embodiment, if the three kinds of photometry data all are smaller than the saturation maximum value EVMAX (NO, step726), it is determined that the exposure condition is optimum. This is followed by a step728for the calculation and storage of tints. If the answer of the step726is YES, the procedure returns to the step S722so as to repeat exposure control.

In the second and successive times of exposure control (step722), the AE controller42produces a new exposure condition controlled such that any one of the current photometry data greater than the saturation maximum value EVMAX becomes smaller than the maximum value EVMAX. The AE controller42then feeds AE control information based on the new exposure condition to the system controller16. In this manner, the exposure control (step722) and the calculation of photometry data (step724) are repeated until an exposure condition that prevents main image data from exceeding the dynamic range has been determined.

In the step728, the R, G and B photometry data R_AE(i,j), G_AE(i,j) and B_AE(i,j) are input to the addition circuit46of the preliminary pickup circuit40. In response, the preliminary pickup circuit40calculates the above photometry data tints R_AE(i,j)/G_AE(i,j) and B_AE(i,j)/G_AE(i,j) and then stores the tints in the RAM70as shading correction data.

After the step728, the AE controller42generates AE control information for actual pickup on the basis of the current exposure condition in which all photometry data are smaller than the saturation maximum value EVMAX (step730). Subsequently, the system controller determines an exposure condition for actual pickup in accordance with the above AE control information inputted thereto (step732). This is the end of the subroutine SUB1.

Referring again toFIG. 8, after the subroutine SUB1, whether or not the shutter release button is depressed to its full-stroke position is determined (step708). If the answer of the step708is YES, actual pickup is executed (step708), or otherwise the procedure returns to the step704.

In the step708, the system controller16generates control signals106and108, including an actual pickup command, in accordance with the exposure condition determined in the step732,FIG. 9, and delivers the control signals106and108to the optics12and timing generator18, respectively. Timing signal generator18generates timing signals112,114and116, including a photometry command, in response to the control signal108and feeds the timing signals112,114and116to the system controller16, image pickup section20and preprocessor22, respectively.

Light102is incident from the subject field to the image pickup section20via the optics12, so that the image of the field is focused on the photosensitive array300. Subsequently, an analog electric signal118, derived from signal charges read out from the photo-sensors302in response to the timing signal114, is fed from the image pickup section20to the preprocessor22. The preprocessor22executes CDS, GCA and AD conversion and other preprocessing on the analog electric signal118for thereby outputting a digital image signal120. The digital image signal120is fed to the signal processor24.

The signal processor24separates the input digital image signal120into main image data202and subsidiary image data204and writes in the image data202and204into the main image memory52and subsidiary image memory54, respectively. The main image data202are then read out from the main image memory52and fed to the offset corrector56, LMTX corrector60and WB corrector64. Particularly, in the illustrative embodiment, the subsidiary image data204read out form the subsidiary image memory54are compensated for color shading (subroutine SUB2).

As shown inFIG. 10, in the subroutine SUB2, the subsidiary image data204are fed from the subsidiary image memory54to the offset corrector56, LMTX corrector60and WB corrector64and then subjected to correction thereby (step742). Subsequently, the SHD corrector72divides the subsidiary image data corrected by the circuits56,60and64into the preselected number of addition blocks in the same manner as during the divided photometry effected in the preliminary pickup mode and then generates added subsidiary image data block by block (step744). In the illustrative embodiment, the added subsidiary image data are made up of R, G and B subsidiary image data r(i,j), g(i,j) and b(i,j).

Further, the SHD corrector72calculates the subsidiary image data tints r(i,j)/g(i,j) and b(i,j)/g(i,j) on the basis of the subsidiary image data r(i,j), g(i,j) and b(i,j) (step746). In addition, the SHD corrector72calculates comparative gains r_gain(i,j) and b_gain(i,j) by using the above expressions (1) and (2) (step748).

After the step748, the SHD corrector72performs spline interpolation, i.e. calculates shading correction gains r_gain_hokan(x,y) and b_gain_hokan(x,y) by using the above expressions (3) and (4) (step750). Finally, the SHD corrector72compensates for color shading of the subsidiary image data r(x,y) and b(x,y) (step752), which are generated in the step742, on the basis of the shading correction gains r_gain_hokan(x,y) and b_gain_hokan(x,y). In the illustrative embodiment, for such color shading correction, use is made of the expressions (13) and (14). This is the end of the subroutine SUB2.

As shown inFIG. 8, the subroutine SUB2is followed by a step710of executing image processing and recording. More specifically, in the step710, the y corrector/combiner74executes gamma correction on the main image data202and subsidiary image data204stored in the main and subsidiary image memories52and54, respectively, combines the image data202and204thus corrected with each other, and then writes in the resulting composite image data to the main image memory52.

Subsequently, the composite image data are synchronized by the synchronizer78and then subject to CMTX correction by the CMTX corrector82. In this manner, the signal processor24of the illustrative embodiment processes the digital image signal120and writes in the resulting image data into the main image memory52.

When the system controller14controls the signal processor24to command image recording and display, the digital image signal130processed by the procedure described above is read out from the main image memory52and fed to the compressor28. In response, the compressor28performs compression and other processing on the digital image signal130and causes the recorder32to store the image signal130thus compressed in the recording medium94while causing the monitor30to display it.

An alternative embodiment of the image pickup apparatus in accordance with the present invention will be described with reference also made toFIGS. 2 and 3. As shown, in the alternative embodiment, the image processing circuit50of the signal processor24additionally includes an addition circuit68for producing shading correction information from the main image data subjected to WB correction. In the alternative embodiment, the addition circuit68compares the main image data subjected to WB correction with, e.g. the maximum charge level in order to see if the former is smaller than the latter. Subsequently, the SHD corrector72performs, based on the result of the above comparison, shading correction on either one of the shading correction information210derived from the photometry data and shading correction information218derived from the data subjected to WB correction.

More specifically, the addition circuit68reads out the main image data216subjected to WB correction from the WB corrector64and then divides the main image data216into the preselected number of blocks in the same manner as during divided photometry effected by the AE controller42. The addition circuit68then adds image data block by block for thereby calculating WB corrected data, which consist of R WB corrected data R_WB(i,j), G WB corrected data G_WB(i,j) and B WB corrected data B_WB(i,j).

Further, the addition circuit68uses the WB corrected data R_WB(i,j), G_WB(i,j) and B_WB(i,j) to calculate the WB corrected data tints R_WB(i,j)/G_WB(i,j) and B_WB(i,j)/G_WB(i,j), and then writes in the tints thus calculated into the RAM70. At this instant, the addition circuit68compares the WB corrected data G_WB(I,j), G_WB(I,j) and B_WB(I,j) with the maximum charge level QLMAX. If the WB corrected data R_WB(I,j), G_WB(I,j) and B_WB(I,j) all are smaller than the maximum charge level QLMAX, the addition circuit68then commands the SHD corrector72to select the shading correction information210based on the WB corrected data. Otherwise, the addition circuit68commands the SHD corrector72to select the shading correction information218based on the photometry data. The addition circuit68may store a flag indicative of the above command in the RAM70so that the SHD corrector72ascertains this flag, or may not store the shading correction information218in the RAM70when the above condition is not satisfied so that the SHD corrector72ascertains whether the shading correction information218has been stored in the RAM70. The SHD corrector72may feed such WB corrected data to the system controller24so as to cause it to generate shading correction data.

The SHD corrector72of the alternative embodiment calculates the comparative gains r_gain(i,j) and b_gain(i,j) on the basis of the subsidiary image data tints r(i,j)/g(i,j) and b(i,j)/g(i,j). At this instant, the SHD corrector72selectively uses the shading correction information210based on the photometry data or the shading correction information218based on the WB corrected data. For example, if the main image data saturate to cause a white blur, i.e. if at least one of the WB corrected data R_WB(i,j), G_WB(i,j) and B_WB(i,j) is greater than the maximum charge level, the SHD corrector72uses the shading correction information210and expressions (1) and (2) to calculate the comparative gains. On the other hand, if the WB correction data R_WB(i,j), G_WB(i,j) and B_WB(i,j) all are smaller than the maximum charge level, the SHD corrector72uses the shading correction information218based on the WB corrected data and the following expressions (15) and (16) to calculate the comparative gains:
r_gain(i,j)=(R—WB(i,j)/G—WB(i,j)) /(r(i,j)/g(i,j))   (15)
b_gain(i,j)=(B—WB(i,j)/G—WB(i,j)) /(b(i,j)/g(i,j))   (16)

In the alternative embodiment, the SHD corrector72calculates, based on the comparative gains r_gain(i,j) and b_gain(i,j) thus calculated, shading correction gains r_gain_hokan(x,y) and b_gain_hokan(x,y) by using the expressions (5) and (6) and executes color shading on the subsidiary image data stored in the subsidiary image memory54by using the expressions (13) and (14).

Reference will be made toFIGS. 11 through 14showing specific operational flows controlling the alternative embodiment. Steps702,704,706,708and710, and subroutine SUB1shown inFIG. 11are identical with the steps and subroutine shown inFIG. 8and will not be described specifically in order to avoid redundancy. A subroutine SUB3shown inFIG. 11characterizes the specific operation of the alternative embodiment and will be described with reference toFIG. 12in detail. As shown inFIG. 12, the subroutine SUB3begins with a subroutine SUB4for a determination of WB gain. The subroutine SUB4shown inFIG. 12will be described with reference toFIG. 13in detail.

As shown inFIG. 13, the subroutine SUB4begins with a step802in which the main image data202are read out from the main image memory52and then processed by the offset corrector56, LMTX corrector60and WB corrector64(step802). The addition circuit68divides the WB corrected data fed from the WB corrector64into the plurality of blocks and adds image data block by block for thereby calculating R, G and B WB correction data G_WB(i,j), G_WB(i,j) and B_WB(i,j). The addition circuit68then compares each of the WB corrected data R_WB(i,j), G_WB(i,j) and B_WB(i,j) with the maximum charge level QLMAX (step806).

If the WB corrected data R_WB(i,j), G_WB(i,j) and B_WB (i, j) all are smaller than the maximum charge level QLMAX, (YES, step806), a step808will be executed. On the other hand, if at least one of the WB corrected data R_WB(i,j), G_WB(i,j) and B_WB(i,j) is greater than the maximum charge level QLMAX (NO, step806), a step810will be executed.

In the step808, tints R_WB(i,j)/G_WB(i,j) and B_WB(i,j)/G_WB(i,j) are calculated on the basis of the WB corrected data R_WB(i,j), G_WB(i,j) and B_WB(i,j) and then written into the RAM70. In the step S810, a WB position is automatically determined on the basis of the WB corrected data R_WB(i,j) through B_WB(i,j) in accordance with, e.g. the color temperature of the scene captured. In a step812following the step810, a WB gain is calculated from the WB corrected data R_WB(i,j) through B_WB(i,j) at the above WB position. This is the end of the subroutine SUB4.

As shown inFIG. 12, the subroutine SUB4is followed by a subroutine SUBS for a determination of subsidiary image data tints. The subroutine SUBS shown inFIG. 12will be described with reference toFIG. 14in detail. As shown inFIG. 14, steps742,744and746following the step812are identical with the steps742,744and746ofFIG. 8in that the SHD corrector72calculates the subsidiary image data tints r(i,j)/g(i,j) and b(i,j)/g(i,j) from the subsidiary image data204stored in the subsidiary image memory54. This is the end of the subroutine SUB4.

As shown inFIG. 12, the subroutine SUB5is followed by a step814. Particularly, in the alternative embodiment, the SHD corrector72determines whether or not the WB corrected data tints R_WB(i,j)/G_WB(i,j) and B_WB(i,j)/G_WB(i,j) are stored in the RAM70(step814). If the answer of the step814is YES, a step816is executed. Otherwise, a step748is executed. In the step816, the SHD corrector72uses the expressions (15) and (16) to calculate comparative gains r_gain(i,j) and b_gain(i,j). On the other hand, in the step748, the SHD corrector72uses the expressions (1) and (2) to calculate comparative gains r_gain(i,j) and b_gain(i,j) as in the step748of the previous embodiment,FIG. 8.

Steps750and752following the step748or816are identical with the steps750and752ofFIG. 10in that the SHD corrector72executes spline interpolation. More specifically, the SHD corrector72calculates shading correction gains r_gain_hokan(x,y) and b_gain_hokan(x,y) by using the expressions (5) and (6) and then executes shading correction on the subsidiary image data r(x,y) and b(x,y) corrected in the step742by using the above gains r_gain_hokan(x,y) and b_gain_hokan(x,y) and expressions (13) and (14). This is the end of the subroutine SUB3.

As shown inFIG. 11, the subroutine SUB3described above is also followed by a step710identical with the step710of the previous embodiment,FIG. 8.

In summary, in accordance with the present invention, an image pickup apparatus determines an optimum exposure condition by divided photometry, generates shading correction gains in accordance with the photometry data tints block-by-block, and executes color shading correction on an image signal outputted by actual pickup in accordance with the shading correction gains. It is therefore possible to reduce a period of time necessary for generating shading correction data at a pickup adjustment stage. Also, in the optimum exposure condition thus determined, the color shading of subsidiary image data generated by actual pickup can be desirably corrected.

Further, in accordance with the present invention, main image data used to calculate a white balance gain include data corresponding to all pixels. By calculating color shading correction gains on the basis of the all pixel data tints, it is possible to generate color shading correction gains more accurately than by calculating them on the basis of pixel data reduced or thinned out.

The entire disclosure of Japanese patent application No. 2004-80190 filed on Mar. 19, 2004, including the specification, claims, accompanying drawings and abstract of the disclosure is incorporated herein by reference in its entirety.

While the present invention has been described with reference to the particular illustrative embodiments, it is not to be restricted by the embodiments. It is to be appreciated that those skilled in the art can change or modify the embodiments without departing from the scope and spirit of the present invention.