Method and apparatus for adjusting image colors of image projector

Disclosed is a method for correcting color in a portable image projector in consideration of an effect of a background color when the portable image projector projects images. The method corrects the effect of the background color to be projected, using color constancy technique, and thereby represents high-quality image which can be projected on a white screen. The method includes the steps of determining if a background color exists on a projection screen using a portable camera, converting the size of a mask image in order to connect a spatial position of an original image and an image photographed by a camera, correcting brightness of the mask image in order to correct non-homogeneity of the brightness of the projected image due to the distribution of different brightness values of the background color, and correcting the background color in order to correct color distortion of the image projected in different colors on the background color. The background color correction step includes the sub-steps of initializing chromaticity of the white screen in advance, and correcting chromaticity using a technique of color constancy of human eyesight.

CLAIM OF PRIORITY

This application claims the benefit under 35 U.S.C. §119(a) of an application entitled “Method and Apparatus For Adjusting Image Colors Of Image Projector” filed in the Korean Intellectual Property Office on Oct. 13, 2006 and assigned Serial No. 2006-99871, the contents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an apparatus and method for adjusting colors of images projected via an image projector, i.e., a portable communication terminal.

2. Description of the Related Art

Generally, an apparatus having a function of projecting images can project the image at different directions designated by a user. The image projecting apparatus typically displays image data, which are stored in a memory, or image signals, which are input through an image input terminal, on a screen. A white wall or a flat surface with a certain color may be used as a screen.

Such an image projector generally has a distinguished characteristic from an internal display unit, such an LCD. In the internal display unit, a necessary setting can be easily performed, and colors of the image may be effectively displayed using set values. However, an external display unit such as an image projector is affected by the environment in which colorful images is projected, for example, brightness of the external light source, color of a screen, etc. The projection condition can affect the quality of projected images, especially in the image projector, such as a portable terminal, when used in a moving environment.

The portable terminal in today is recognized as a multi-entertainer, which provides users with a variety of information on life and culture, for example, Internet shopping, phone banking, photographing, watching broadcast, entertainment, biological diagnosis, etc. as well as a means for communication. In order to realize various contents, such as moving pictures, movies, three-dimensional games, etc., more multimedia devices such as a digital camera, a Digital Multimedia Broadcasting (DMB) system, a MP3 player, etc. are miniaturized and embedded in a portable terminal.

Now, there are efforts to install a function of a general beam projector to the portable terminal while overcoming the limitation of size limit in the portable terminal, so that a user to watch movie or to play a game. However, a hardware-technical problem including a lens of a general image projector, a low luminance according to the miniaturization of a backlight, and a distortion of projected image, and a distortion problem of projected image caused by effect of circumstance and color quality of the portable image projector have been a roadblock to the development of these products.

Unlike a general Liquid Crystal Display (LCD) or a Digital Light Processing (DLP) in which a light source and a screen are fixed, the portable image projector provides images, the quality of which is affected by the background color of an object on which the images are projected, or the surrounding light sources because of frequent movement of the portable terminal. To this end, a color correction technology has been developed, which considers the color of an object on a projection screen when a portable image projector is implemented.

As an example of the conventional color correction technology considering a background color, there is a background color correction technology using a color adaptability of human eyesight for a general projection equipment (for example, Masato Tsukada, “Projector Color Reproduction Adapted to the Colored Wall Projection” CGIV 2004). This method is a technology in which a color adaptable model of human eyesight is introduced into a color space of CIEXYZ in order to correct the background color. However, this method has many errors in the correction of the background color due to an error in eliciting RGB output value and CIEXYZ value of a color sensor. Particularly, it is difficult to realize the color correction technology in the CIEXYZ color space in real-time because of an amount of calculation in gamut mapping and device characterization procedures.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve the above-mentioned problems occurring in the prior art and provides additional advantages, by providing a color correction method which can determine if a background color exists, through a camera embedded in a portable terminal, and remove an effect of the background color by applying a color constancy technique of human eyesight to non-homogeneity of brightness of projected image and color distortion, caused by different brightness values of the background color and a distribution of color when the background color exists.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. For the purposes of clarity and simplicity, a detailed description of known functions and configurations incorporated herein will be omitted as it may make the subject matter of the present invention unclear.

FIG. 1is a block diagram illustrating the configuration of a portable communication terminal including an image projector according to the embodiment of the present invention. For illustrative purposes, a portable communication terminal will be described as an example in this specification. However, it should be noted the teachings of the present invention is applicable to that other devices equipped with an image projector

Referring toFIG. 1, the portable communication terminal according to the embodiment of the present invention includes a controller2, a memory unit4, a key input unit6, a voice processor6, a wireless unit10, a first display unit12, an image projector14, and a camera16.

The key input unit6is used to input phone numbers or characters by a user. It includes keys for inputting numeric and characters and function keys for setting various functions. The key input unit6outputs the input signals from the keys to the controller2. The first display unit12is an internal display unit of the portable communication terminal which may include a Liquid Crystal Display (LCD) and the like. The first display unit12carries out a function of outputting image output signals received from the controller2.

The wireless unit10converts user's voice data and control data into wireless signals and then transmits the wireless signals to a base station (not shown) of a mobile communication network. The wireless unit10receives wireless signals from the base station and then converts the wireless signals into voice data and control data. The voice processor8decodes the voice data received from the wireless unit10and then outputs the voice data as audible sound through a speaker. Further, the voice processor8converts users' voice signals received from a microphone into data and then outputs the data to the wireless unit10.

The camera16carries out a function of a general digital camera under control of the controller2, and converts visible image signal, input from an external object, into a digital image data in a suitable format. The image projector14is used for performing a general projector function in the portable communication terminal and may include a light source, a permeable display unit, a lens, etc.

The controller2controls the operation of respective units collectively, so as to perform a function of controlling the overall operation of the portable communication terminal. Specifically, the controller2performs an operation depending on signals relating to the number and menu selection input through the key input unit6, and also receives through a camera16image signals relating to a photographed object so as to operate an operation depending on the image signals. Furthermore, the controller2outputs image input signals, which are required to display images taken with the camera, using the first display12or an image projector14. The controller2reads or stores contents to be output from/into the memory4during operation. The memory4stored programs and data relating to the operation of the controller2stored therein, and is also used for storing information on images taken by the camera as well as necessary information in operating the portable communication terminal.

The portable communication terminal operates the general communication services, and conducts an image projecting function relating to the present invention through the image projector14. Technology relating to a portable communication terminal including such an image projector is disclosed in Korean Patent Application No. 2004-29693, filed on Apr. 28, 2004 by Sung-Goo KIM, entitled with “Portable Terminal having Image Projector and Method For Controlling the Same”, the content of which is incorporated by reference herein.

Meanwhile, according to the present invention, the controller2additionally performs the operation of adjusting image colors as well as the above-mentioned functions. In addition, the controller2stores a program relating to an image projecting operation or relating information to adjust colors in the memory4, and if necessary, the controller2outputs the information.

Now, the operation of adjusting the colors according to the present invention will be described in detail with reference to the accompanying drawings below.

FIG. 2is a flowchart illustrating the operation of adjusting image colors from the image projector according to the present invention. Referring toFIG. 2, in the operation of adjusting the image colors, first, white signals (255, 255, 255) of RGB is output on a projection screen in order to detect a background color of a projection screen, in step202. Then, it is determined whether there is another background color in addition to white color by taking a picture of the background color of the projection screen using a camera16embedded in the portable terminal and calculating chromaticity of average value of RGB images, in step204.

If another background color exists, the background color correction unit according to the present invention applies the color correction technology to the projecting image in step208and then outputs resultant images, in which RGB is corrected, in real-time in step210. Meanwhile, if the background color is white, the original images are projected on a portable image projection, or the color correction technology embedded in the portable image projector may be applied selectively to the images in step206.

FIG. 3is a flowchart illustrating steps of determining if the background color exists using a camera, as described above. Referring toFIG. 3, first, white color patch including RGB (255, 255, 255) signals is output through the image projector14before original images are projected, in step302. Then, it takes a picture of the background color by means of the camera16, in step305. Next, it is determined if the background color exists by calculating an average value of each channel of the pictured images (background color) and obtaining chromaticity value showing a ratio of RGB signals to the sum of RGB color signals instead of absolute magnitude of the RGB signal using the following equation (1), in step306.

In the equation (1), (Rave, Gave, Bave) value means an average value of the background color captured by the camera, and (r, g) represents chromaticity of the background color.

The obtained chromaticity is compared with a specified threshold (TH1, TH2) and (TH3, TH4), in step308. As a result of comparison, if the chromaticity is in a range of the specified threshold, the background color is regarded as white, in step310. If the chromaticity is out of a range of the specified threshold, it is determined that the background color exists, in step312. Then, in the next process, the background color correction unit performs an operation of correcting the background color, as explained hereinafter with reference toFIGS. 4-6.

FIG. 4is a flowchart illustrating the operation of the background color correction unit according to the embodiment of the present invention. Note that the background color correction unit is employed in an image processing module, such as the controller.

Referring toFIG. 4, the color correction unit according to the present invention carries out converting a size of a mask image in step402, correcting brightness of the mask image nonhomogeneously in step403, and correcting chromaticity using color homeostasis in step404.

InFIG. 4, the mask image represents the background image captured by the camera in order to determine if the background color exists in the projection screen. In step402of converting the size of the mask image, the size of the image captured by the camera is converted into the size of an original image. The size of the original image display on the portable screen is 240×320. However, the size of images captured by the camera for determination of the background color is variable depending on the performance of the camera and the picturing option. Therefore, converting the size of the pictured image into the size of the original images is necessary in order to obtain the corrected image resulting from the original images, in view of the effect of the background color in step402.

Converting the size of the mask image can be implemented by calculating the ratio of width to length of the pictured images and the original images in order for down-sampling using the following equation (2) or by using the conventional interpolation methods.

In equation (2), (Wc, Hc) represents the values of width and length of an image pictured by the portable terminal, and (WD, HD) represents the values of width and length of an original image. n1and n2represent the ratio of width to length in the pictured image and the original image. ‘MaskImage’ represents the pictured mask image, and (x, y) refers to a coordinate of a special position of a pixel. The pictured mask image is down-sampled by the ratio of n1and n2in order to convert the size of the image.

FIG. 5is a flowchart illustrating a step of correcting the brightness of the mask image (step404) inFIG. 4nonhomogeneously. In a case where the mask image does not have a single background color, specifically various kinds of textures or a background picture are inserted in the background color, distortion, in which the brightness of images projected with different reflective ratios is spatially non-homogeneous depending on the position of the background color, is corrected. Since the background color of the projection screen has different reflective ratios depending on the spatial position, the brightness of the projected image can be uniformly corrected by correcting the brightness of the original image using the brightness of the pictured mask image.

More specifically, referring toFIG. 5, according to the technology of correcting the brightness of the pictured mask image, first, the RGB value of the original image and the mask image is moved to a color spatial coordinate YCbCr in which a luminance signal and a color signal can be separated from the RGB value. Then, an average value YMaskImage-Aveof the luminance of the mask image is calculated in steps502,504and506. Then, the difference of the luminance value YMaskImage(x, y) corresponding to each pixel of the mask image is obtained on the basis of the average value YMaskImage-Aveof the luminance of the mask image in step508. If the luminance value is larger than the average value of the luminance of the mask image, the luminance value Y(x, y) of the original image is reduced by the difference between the luminance value and the average value. Otherwise, if the value of the luminance corresponding to each pixel of the mask image is smaller than the average value of the luminance of the mask image, the value of the luminance of the original image increases by the difference between the luminance of the mask image and the average luminance of the mask image, thereby correcting irregularity of the brightness of the background color. The step of correcting the irregularity of the brightness in such a mask image can be expressed by the following equation (3).
YMaskImage(x,y)=0.299RMaskImage(x,y)+0.587GMaskImage(x,y)+0.114BMaskImage(x,y)Y(x,y)=o.299ROriginalImage(x,y)+0.587GOrigianlImage(x,y)+0.114BOriginalImage(x,y)Ycorrect(x,y)=Y(x,y)+(YMaskImage-ave−YMaskImage(x,y))  (3)

In equation (3), (RMaskImage(x, y), GMaskImage(x, y), and BMaskImage(x, y)) are values obtained by normalizing each pixel value of the mask image to 0-1, and (ROriginalImage(x, y), GOriginalImage(x, y), and BOriginalImage(x, y)) are values obtained by normalizing each pixel value of the original image to 0-1. YMaskImage(x, y) and Y(x, y) are luminance values obtained by converting the values of the mask image and the original image into a color space coordinate of YCbCr.

In the YCbCr color space coordinate, a color signal component still remains kept, nonhomogeneity of the brightness due to different reflexibilities of the background color can be corrected by adding the difference between the luminance signal (YMaskImage(x, y)) of the mask image and the average value of the luminance (YMaskImage-ave) to the luminance value of the original image, in step510. The corrected luminance value (YCorrect(x, y)) of the original image is converted into RGB image through YCbCr inverse transformation, in step512.

FIG. 6is a flowchart illustrating a step of correcting chromaticity using constancy of color (step406) inFIG. 4. In such a step, when the original image is projected on the background color having the different colors, the image quality deteriorated by the background color is improved. Unlike the conventional image projector projecting image on the white projection surface of a fixed screen, the portable image projector of the present invention can project the image on the background color having the different colors because of characteristic of the portable terminal. Thus, there is required a color correction technology concerning the background color of the projection screen. A method for correcting the background color using the conventional color accommodation model has an error in characterization of a color sensor and a difficulty in a real-time realization. Accordingly, the present invention proposes a background color correction technology for an image projector to which a color constancy technique of human being is applied.

Human beings recognize natural color of an object by countervailing effect of surrounding light sources based on light energy reflected from the object. The human eyesight performs a specifically mechanical operation which can effectively countervail the change of the light source at an initial time when the human being sees the object. This refers to the color constancy of the eyesight of the human being. In the color constancy technique excluding the effect of the light source in the photographed image, the natural color of the object can be obtained by estimating the chromaticity of the light source. The conventional color constancy technique may be used for correcting the effect of the background color of the portable image projection.

Referring toFIG. 6, in step602of initializing the chromaticity of the white screen using a camera, the chromaticity of the object having different white background color can be obtained even when the white balance function of the camera is inactivated. Since the portable image projector can project image on different white background color, it is required to set a criterion chromaticity of the white screen. Hence, after the average value of each channel is calculated by taking a picture of an object having different white background color using a camera embedded in the portable terminal, the chromaticity of the criterion white screen is initialized by using equation (4).

In equation (4), (Raverage-while, Gaverage-white, and Baverage-white) is the average value of the white background color photographed by the camera, and (rwhite, gwhite) represents the chromaticity of the white screen.

In step604of estimating the average value and the chromaticity of the background color ofFIG. 6, the chromaticity of the background color to be projected is estimated. After the average value of the background color is calculated by using the camera, the chromaticity of the background color can be calculated by the following equation (5).

In equation (5), (Raverage-back, Gaverage-back, Baverage-back) is the average value of a white background color photographed by a portable camera, and (rbackground, gbackground, bbackground) is the chromaticity of the background color.

In step606of obtaining the corrected image in which the effect of the background color is removed inFIG. 6, the corrected image can be obtained by multiplying the original image by the ratio of the chromaticity of the background color and the chromaticity of the white screen, which is estimated in steps. In order to remove the effect of the projection screen having the background color, the pixel value of the original image applied to the portable image projection should be changed. For example, in order to obtain the image shown on the white screen from the image projected on the blue-based background color, gain of B-channel corresponding to the blue color should be reduced. Specifically, the corrected image can be obtained by a following equation (6), in which the effect of the background color is removed by multiplying the entire image by the ratio of the chromaticity of the white screen and the chromaticity of the background color.

In equation (6), (R, G, B) is the pixel value of the original image, and R′, G′, and B′ are the corrected images to which the color constancy is applied. Therefore, the corrected image is output through the image projection, thereby representing the resultant image on the white screen from which the effect of the background color is removed.

As is apparent from the foregoing, in the method of adjusting the image color of the image projector according to the present invention as described above, there is proposed a technique of determining if the background color exists, in which the range of chromaticity is estimated by using a color correction technology of a portable image projector concerning the background color of a screen after the chromaticity of the background color is extracted by using a portable camera embedded in the portable terminal. Moreover, a color constancy of human eyesight is applied to the color correction of the background color to obtain a resultant image similar to the image displayed on the white screen, thereby achieving the high-quality color representation. Furthermore, since the color correction technology includes simple processes and has an amount of calculation, it can be implemented in real-time. Of course, the color adjustment method of the present invention can be applied to general image projectors as well as portable image projectors.