Calibration methods and apparatus for digital cameras

Digital cameras are calibrated so that detailed highlights and shadows in scenes being photographed are reproduced in digital images of those scenes by calibrating holographic data obtained from target images at those scenes. In one embodiment the target image is obtained from an actual target having black and white, and optionally gray, target areas; and in another embodiment, the target image is obtained from the scene being photographed. In both embodiments, histographic data is manipulated so that shadow and highlight data spikes are automatically placed in boundary areas by automatically adjusting lens aperture and/or shutter speed prior to taking the desired photograph of the scene. The resulting photograph is adjusted for highlight and shadow details.

FIELD OF THE INVENTION

The present invention is directed to calibration methods and apparatus for digital cameras. More particularly, the present invention is directed to calibration methods and apparatus for selecting exposure settings and achieving white balance in digital cameras.

BACKGROUND OF THE INVENTION

In contrast with the digital photography, film photography traditionally has an exposure tolerance of two to four stops. An original scene exposed within this tolerance range will retain a good quality tonal curve due to the film, which is the recording medium, having a light sensitivity range which exceeds the tonal extremes in an average scene of two to four stops. Exposure for digital photography has a very narrow range, so that when there is over exposure or under exposure, part of the information is lost. Consequently, unless you initially achieve the desired exposure you make an inferior image file, which will not be in full detail either in highlight areas or shadow areas. There is no way to retrieve these lost details for the image file. Consequently, in digital photography techniques have evolved to fix images after they have been taken. This is a time consuming and relatively expensive undertaking in which it is still very difficult to compensate for information missing from an image.

Inside digital cameras there is a sensor on which a latent light impression of an original scene is made. Based on latent information from the sensor, a central processing unit in the camera processes the information into a proper color spectrum and into a proper color curve that retains the detail from highlight to shadow. The quality of image produced using the latent light impression frequently has diminished quality due to exposure error.

All sensors inside cameras have an optimal sensitivity setting (ISO), whether it is due to the sensitivity of film or to the sensitivity of image sensors in digital cameras. Sensitivity has an optimal range where it produces the absolute best image file in terms of color fidelity with the least image defects. In producing an optimal image file, the image file will be given a rated ISO, e.g., 100. Digital cameras have the capability of shooting other ISOs, but as one deviates from the optimum image quality suffers. When deviating from the optimal ISO, noise is introduced into the image files from the CPU and arbitrary abnormalities known as artifacts become visually apparent and the quality and color degrade. Thus, producing optimal digital image files is difficult for the professional photographer and extremely difficult for the consuming public.

Hand held light meters do not adequately compensate for inaccuracies in exposure because tolerances are typically plus or minus a half stop of exposure. Typically light meters select a middle tone, the placement of which varies from one manufacturer to another. Since light meters peg the middle of the tonal curve, light meters select gray rather than the black and white extremes. Pegging the middle of the tonal curve can result in the photographer loosing information at one or the other extreme so that light metering does not work effectively. This forces camera manufactures to develop methods to fix latent information. In order to compensate for inadequate latent information, camera manufacturers provide you with software solutions for manipulating improperly exposed and color-balanced images. But, these “back end fixes” almost invariably produce inferior image files with which before you can even start to produce a print, require very labor intensive efforts having three times the amount of work to process an image file. Accordingly, there is a need for a technique to correctly set exposure and color balance on the front end, i.e. prior to recording an image.

Obtaining correct exposure is part of the problem, the other part being correcting white balance. Most digital cameras provide different options for white balance correction which may be automatic or set by the photographer. Such settings are ballpark settings based on daylight, flash, overcast sky, tungsten filament lighting or fluorescent lighting. Daylight varies depending on the time of day with the color temperature being different at morning, midday and afternoon. There are also differences in white balance due to brightness of tungsten light bulbs because brightness determines color temperature. Since current white balance settings for digital cameras are quite inaccurate, there is a need for improvement.

In view of the aforementioned considerations, in digital photography there is need for improvement in the ability to select correct or desired exposure settings, as well as a need for improving the ability to correct or select desired white balance settings.

SUMMARY OF THE INVENTION

The present invention is related to calibration methods and apparatus for digital cameras performed prior to exposing an image of an original scene to an image sensor within the camera by exposing the image sensor to calibration information provided at or by the original scene.

In one embodiment of the calibration, use of a digital calibration target involves adjusting the exposure of digital cameras to retain maximum detail by referencing the brightest important highlight detail (white area of the target) and the darkest important shadow detail (black area of the target) instead of the traditional metering method which references 18% gray. Additionally by ensuring the digital calibration target panels are absolutely neutral (without color bias), the target allows the adjustment of the camera's white point to be set accurately in virtually all lighting situations.

In another embodiment of the calibration, the target is referenced at the location of an original scene and the camera automatically adjusts the exposure and white balance based on the information recorded on the camera's sensor without the photographer's intervention.

In still another embodiment of the calibration, the incorporated information provided by the target is in the scene itself, which information provides important highlight and shadow detail and adjusts automatically so that the need to record an external target is eliminated.

In another embodiment of the invention a calibration target having a substantially white target area that reflects substantially all wavelengths of visible light and a substantially black target area that absorbs substantially all wavelengths of visible light when focused upon, achieves an exposure setting for the digital camera. The exposure setting is then adjusted for the target by a controller in the camera until an adjusted exposure setting for black and white detection is substantially balanced within the camera for a selected intensity and distribution of light at the original scene. The adjusted exposure setting is then applied to at least one image photographed at the scene.

In a further aspect of the invention, at least one substantially gray target area is provided and exposed simultaneously with exposure to the black and white target areas.

In still a further aspect of the invention, the adjusted exposure setting for an exposure balanced image is exposed to a substantially neutral, white or gray target area without color bias that reflects equally substantially all wavelengths of light to produce settings for a color balanced, calibrated image having a selected white balance for the original scene.

In still another aspect of the invention, adjusting the exposure is performed by viewing a histogram display within the digital camera and automatically adjusting the exposure settings until spikes representing black and white detection are not offset laterally with respect to a histogram function.

In still another aspect of the invention, an initial exposure setting is obtained of the location prior to exposing the image sensor to the calibration target. The initial exposure setting is then automatically adjusted without the photographic intervention by subsequent exposures to the calibration target to achieve correct or desired exposure settings.

In a further aspect of the invention, the target further includes at least one middle gray target area, which reflects substantially all wavelengths of light without color bias.

In still a further aspect of the invention, the target includes a neutral white or gray target area without color bias, which neutral area when positioned at locations of scenes prior to recording images of the scenes, determines settings for desired color balances for the images.

In a further aspect of the invention, the target configuration is a panel having black and white target areas on a first side and the neutral target area on a second side of the panel.

The invention further relates to methods of digital photography utilizing the aforedescribed calibration methods and calibration targets.

DETAILED DESCRIPTION

Referring now toFIG. 1there is shown a location10at which an original scene11is disposed, which scene may include a subject12to be photographed and a background13. The location10may be any location and the scene11may be any scene with any particular object12, or no particular object. The background13may be any background. The scene11is illuminated by a light source16, which may be any light source, for example, a lamp, the sun, a photoflash or indirect light, which illuminates the location10of the scene11.

The scene11has shadows18and highlights20. The shadows18and highlights20may be definite and readily apparent, or may be subtle. For any subject12, the shadows and highlight are usually intermingled, varied and complex.

The subject12and background11also has colors and tones, which may vary widely and interact with highlights and shadows to produce a visual image of the original scene11. A digital camera24having a lens focuses the scene11on image sensors in the camera that record light electronically. Typically, in digital cameras the image sensors are charged coupled devices (CCD) or complimentary metal oxide semiconductor (CMOS) devices. A computer device in the form of a central processing unit (CPU) converts electrical charges from the image sensors into digital data, which is then stored as digitalized image information in the camera's memory.

As is seen inFIG. 2, a light meter22takes a reading proximate the subject12to determine initial exposure settings for the camera24.

As is seen inFIG. 3, a calibration target40, configured in accordance with the principles of the present invention, is inserted into the scene11at the approximate location of the subject12(if there is a subject). The calibration target40has a black target area42which absorbs substantially all wavelengths of visible light incident thereon from the light source16and a white target area44, which reflects substantially all wave lengths of visible light incident thereon from the light source16. The black and white target areas42and44cover substantially equal portions of the target40. In the illustrated example, the target areas42and44are approximately rectangular with curved sides and separate from one another at line45, however it is only necessary that the black and white target areas have the same amount of area. Accordingly, the black and white areas may have a checkerboard configuration or may be in the form of writing or a logo. For example, the white area44may form a background for the black area42, which is in the form of writing with perhaps a logo or other design also in black. It is emphasized that the black and white target areas42and44need only be equal in area, but may have any desired configuration. While the black and white target areas42and44are illustrated as being on a panel, these areas may be on separate cards which are juxtaposed with one another, may be on adjacent sides of a carton, on a poster, on a screen or on a folded insert packaged with a digital camera.

The black and white target areas42an44are neutral in that they contain no color cast, the black target area42being substantially pure black and the white target area44being substantially pure white. The target40is not limited to specific size constraints or materials provided that it's reflective qualities are pure black and pure white.

Referring now toFIGS. 4-7in conjunction withFIG. 3, the method of the present invention is practiced by focusing the camera24on the calibration target40and setting the camera to a “image and histogram” or “histogram only” display50(seeFIGS. 5 and 7) on the LCD display panel of the camera24. Care is taken to ensure the light falling on the calibration target40is of the same intensity as the light, which will be illuminating the subject12(FIG. 1). In otherwords, if the source16of light is the sun providing full illumination, then the illumination falling on the target should be full sunlight and not sunlight obscured by clouds or haze. If the light falling on subject12is from a cloudy sky, then the light on the calibration target40should be from the same cloudy sky. If the light source is a tungsten lamp, then the same lamp at the same position is used for the calibration target40as for the subject12.

Preferably in practicing the method of the invention, an initial exposure is made using the camera's internal light meter or a handheld light meter22to provide initial exposure settings (seeFIG. 2). As is seen inFIG. 5this results in a histogram display50with two distinctive spikes52and54representing an initial exposure settings. The spike52represents black and the outside spike54represents white. Typically, the first exposure using the cameras internal light meter or an external light meter22will result in a histogram similar toFIG. 5in which the black and white spikes52and54are shifted with respect to a left side boundary line56or a right side boundary line58. The exposure settings of the camera24are then adjusted to move the spikes52and54to be equally contained in the area60; equally balanced between the boundaries56and58, and not offset to either the right or left side.

A second exposure (FIG. 3) is made to see if the spikes52and54remain in the area60and equally balanced between the left and right boundaries56and58, respectively and not offset to either the right or left side. If the spikes52and54both remain in the area60and are equally balanced between the boundaries56and58with no lateral offset, the exposure settings are correct and the digital camera24is properly calibrated to photograph the subject12, once the subject is returned to the scene11(if there is a subject12) (seeFIG. 1). If the black and white spikes52and54are still not both in the area60and equally balanced between boundaries56and58with no lateral offset, then the exposure settings of the camera are again adjusted and another exposure made to see if the spikes52and54are equally balanced within the left and right boundaries56and58. The process of adjusting settings and then exposing image sensors in the camera to the calibration target40, with the adjusted settings, and then viewing the additionally adjusted histogram display is continued until the spikes52and54are in the area60and equally balanced between the left and right boundaries56and58with no lateral offset. This may take a single adjustment or several adjustments to achieve. In any event, the subject12may now be digitally photographed with exposure settings properly set so that highlights and shadows of the photographed image of the original scene11will correspond to the original scene.

While a histogram display50is used in accordance with one embodiment of the invention, the characteristics of the histogram display are sensed electronically in another embodiment of the invention (not illustrated) and centered automatically by the CPU of the camera24. In this alternative embodiment values assigned to the locations of black and white spikes52and54are compared to values assigned to the boundaries56and58. After exposing the image sensors in the camera24initially to the camera's internal light meter or a hand held light meter22to obtain values for initial exposure settings corresponding to the positions of values for spikes52and54, the camera is focused on the calibration target40and another exposure made. The value for the settings for this exposure are then compared to the values for the boundaries56and58. If the calibration target values fall between the values for the boundaries56and58, the exposure settings are calibrated. If not, an additional exposure is made or several additional exposures are made, until the values for the black and white spikes52and54do fall between the values for the boundaries56and58.

The additional exposures may be done manually or automatically using a sequence of exposures with the histogram display simply informing the photographer visually or audibly that the exposure settings are correct.

While an initial exposure based on light meter readings would appear desirable, when using an automatic sequence of exposures this step could be either dispensed with or incorporated into the sequence, relying preferably on the camera's internal light meter.

Referring now toFIG. 8-12, a second embodiment70of the calibration target is disclosed wherein the calibration target includes a black target area72, a white target area74and a middle gray target area76. The middle gray target area76is 50% gray, i.e. halfway between the black target area72and the white target74. As is seen inFIGS. 10 and 12, there is a middle target area spike53between the black target area spike52and the white target area54. The middle spike53represents the gray area and is used to indicate whether the spikes52and54are shifted left or right.

InFIG. 10the spikes52and54are shifted to the left with part of spike52being sheared off. This indicates that an under exposed image of the original scene with the subject12will occur if the setting obtained from the light meter22are used. The exposure settings of the camera24are then corrected to shift the spikes52,53and54to the right as is shown inFIG. 12. The entire spike52is then inside the boundary56with the spike remaining inside boundary58.

While not its primary purpose or design, because the calibration target40or70contains known values for highlight, mid tone and shadow, it can be useful for back end correction when included in an original scene. While this is a desirable feature, it is primarily a convenience so that a photographer does not have to carry multiple adjustment tools in case a mistake is made and backend corrections are necessary.

A single gray target area76is shown inFIG. 8, which is equal in area to the target areas72and74. In accordance with a further embodiment of the invention, there may be two or more gray target areas, each having a tone different from the gray area76. These additional gray areas will each have an area equal to the black and white target areas72and74, and the single target area76. By having multiple tones for the gray scale, finer level adjustments may be made on the backend after the image of the original scene has been recorded.

Referring now toFIG. 13, in conjunction with the calibration target40or70there is shown a neutral target area85that is either white or neutral gray, which neutral target area substantially reflects all wave lengths of light neutrally without color bias. Subsequent to obtaining exposure settings using the calibration target40or70, the neutral calibration target area85is focused upon and exposure made by the digital camera24in order to achieve white balance for the aforementioned corrected exposure settings. Most cameras24provide several options for white balance by providing settings on a menu for daylight, flash, overcast skies, tungsten lighting and fluorescent lighting. These settings are generally not accurate because daylight varies depending on the time of day with the color temperature of daylight being different in the morning, midday and late afternoon. The difference in tungsten depends on how bright the tungsten light bulb really is and what the color temperature will be. All other preset settings also have variances and therefore these other settings cannot be totally accurate either.

Prior to exposing the image of an original scene11, the menu on the digital camera24is accessed to select a custom white balance function illustrated inFIG. 14and the set button pressed using as the white point the neutral target area85. The camera24now has a clear white display86in the viewfinder59ofFIG. 15. A tone selector display87ofFIG. 16is then used to select from the previously recorded images, images86of the neutral calibration target85. The camera24has now been color corrected so that when the subject12is photographed, the colors of the image are identical thereto even though the color temperature is different. This allows one to customize the camera24so that the camera knows what the white point should be and by setting the white point you get perfect color images from the camera. Alternatively, white balance can be achieved prior to adjusting exposure by relying on handheld meter or the camera meter to achieve a base exposure, and then following the custom white balance procedures as outlined above.

As long as the settings for exposure are correct, as obtained by using the calibration targets40and70ofFIGS. 3 and 8, and as long as the white balance is correctly set, it does not matter what light source16is utilized. The photographer will get consistent color, and when the subject12is put in front of the camera, the subject will look exactly the same to the camera. Different custom white balances may be stored in the memory of the camera24so that each time situations or locations10are changed; the appropriate white balance settings can be clipped to the situations so that they can be set up in advance. For example, a wedding photographer might set white balance at the front of the church, white balance for the back of the church and white balance for outside the church. As the photographer moves to each of these locations, the photographer picks a correct custom white balance settings for that location.

Referring now more specifically toFIGS. 8 and 13, the specific calibration target70used in practicing the principles of the present invention is in the form of panels101having a first side102with the black target area72, the white target area74and the gray target area76providing an exposure setting calibration target. On a second side110of the panel101there is the neutral target area85which is gray or white providing white balance as described inFIGS. 13-16. The panel101is fabricated from four separate pieces of cloth fabric, with the black, white and gray target areas72,74and76of the first side102being stitched together to form the calibration target for determining exposure settings, and the neutral white or gray target area85being fabricated from a single sheet of fabric to form the second side110of the panel. The first and second sides102and110of the panel101are stitched to a cloth tube120, which contains a flat peripheral spring, which has flat sides facing radially.

As is seen inFIGS. 17 and 18, the panel101has relatively straight edges130-133joined by curved edge portions135-139so that the panel101is foldable into overlapping portions. The panel can then be inserted into a zippered pouch140for convenient carrying.

The histogram display50ofFIGS. 5 and 7andFIGS. 11 and 12are graphical displays of brightness values, showing important highlight details exemplified by highlight spikes52and important shadow details as exemplified by shadow spikes54. Highlight details are associated with the color white, such as the white areas44and74on the targets40and70, and shadow details are associated with the color black, such as the black areas42and72on the targets40and70. The brightness values used to provide images of the spikes52and54are derived from brightness data obtained by monitoring light entering the camera24.

InFIGS. 19-22brightness data is shown by data relating to important highlight detail illustrated by highlight data spike252and by data relating to important shadow detail illustrated by shadow data spike254. The data of spikes252and254are compared to fixed brightness data regions, i.e., highlight region256and shadow region258, respectively, which correspond to highlight and shadow data that is recordable by the digital camera25. Data that is too dark or less bright than an acceptable level for a photograph will not appear in the photographed image and data that is too bright will wash out and also will not appear in the digitally photographed image. It is therefore desirable to keep the data forming the highlight spike252in the highlight data region256and the data forming the shadow data spike254in the shadow data region258when adjusting exposure settings of the camera25.

This is accomplished by automating the camera25to use exposure control arrangements (already available in the art) but which are according to this invention driven by the histographic data ofFIGS. 19-22. This histographic data is preferably the same histographic data that drives the histogram display50ofFIGS. 5,7,10and12.

Referring now toFIG. 23, it is seen that a target, which may be the targets40or70(or in a subsequent embodiment may be portions of a scene, such as the scene10ofFIG. 1), is viewed through a lens300of the camera24. A light sensor304monitors light from the target40,70for brightness and a brightness data generator306generates a range of light brightness values that are arranged by a histographic data organizer308into a histogram of light brightness data.

This histographic data, which is configured as the spikes may or may not be transmitted to a histogram display, such as display50ofFIG. 5,7,10or12. If the target is the target40, the only histogram data sets are those for black and white, i.e., a shadow data spike256and a highlight data spike254which are separated for convenience of illustration by boxes310and312, respectively. The shadow and highlight data spikes252and254are fed to a brightness data comparator316which contains the data boundary regions256and258and initial data321from the histogram data organizer308.

The initial histogram data321of the scene prior to calibration for brightness is shown inFIGS. 19 and 21(FIG. 21includes gray target area). In these figures, the highlight data spike254is inboard of the boundary region258. This information is temporarily stored in the brightness data comparator316. Upon comparing the data ofFIG. 20(orFIG. 22) from the highlight data spikes310and312to the initial histogram data321with the brightness data comparator216, the brightness data comparator emits a difference signal to at least a lens adjustment driver320if the difference is significant. The camera adjustment driver320adjusts the aperture, shutter speed or both of the camera24until the detected histographic data is organized so that both data spikes are in the boundary regions252and258as is shown inFIGS. 20 and 22so that when an image of the scene is recorded, the brightness setting of the camera corresponds to that of the scene.

When and if this match occurs, optionally a signal is sent from the brightness data comparator316to a visual or audible indicator322indicating to the photographer that brightness detected by the camera25is properly calibrated for highlight and shadow.

Referring now toFIG. 24, there is shown another embodiment for brightness adjustment using data organized into a histogram, wherein the data utilized for practicing the invention is derived directly from the scene400to be, or being photographed. In this embodiment, a similar initial procedure for obtaining brightness data for a histogram of that data is followed as is used for obtaining data for the histogram display50ofFIGS. 5,6,10and12. The data ofFIG. 24is initially processed according to the procedure ofFIG. 23, however a brightness selector352input is provided inFIG. 24to select portions of the holographic brightness data, which are of interest with respect to the scene being photographed.

In the embodiment ofFIG. 24, histogram data in the histogram data organizer308is read by a data pattern recognition comparator356and matched with data patterns in a library360of histographic data sets from standardized images to select the closest data set. Associated with each histographic data set in the library360are uncalibrated shadow and highlight data spikes252and254, as shown inFIGS. 19 and 21, and calibrated shadow and highlight histographic data spikes252and254similar to those ofFIGS. 20 and 22. The calibrated data spikes252and254have associated therewith histographic data in which the data spikes252and254are located in the data boundaries256and258. The calibrated data spikes252and254are used to calibrate the camera24by adjusting lens aperture or shutter speed, or both, by comparing the calibrated data spikes to the uncalibrated data spikes in a brightness comparator316and applying the difference to the camera adjustment driver320which is connected to the camera lens controller300. Each time a new photograph is taken, there is the option of recording the histographic data thereof in the library360by using a library update262having an input of original histographic data from the organizer308and an input of shadow and highlight initial and calibrated data spikes252and254from the patter recognition comparator356.

The pattern recognition comparator356has outputs310and312for the data spikes.