Method and apparatus for image processing using a template image

A digital camera includes a memory card. The memory card is previously recorded with template image data that has been subjected to conversion of Y-data effective range and JPEG compressing. A CPU expands this template image data by a JPEG method, and then subjects only expanded data greater than a predetermined value to effective-range conversion that is reverse to the above. The CPU composites the template image data thus obtained with photographed image data, thereby creating composite image data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a method of compositing a template image data with a photographed image as well as a digital camera, and more particularly to a template/photographed image compositing method and digital camera arranged to read the template image data that has been compressed by a JPEG method and previously recorded in a memory card so that the template image data is read out of the memory card and expanded for compositing with photographed image data.

This invention also relates to a method and apparatus of recording a template image, and more particularly to a template image recording method and apparatus adapted to record template image data into a memory card through compression.

2. Description of the Related Art

In a conventional digital camera, template image data is previously recorded in a memory card. An operator can create a composite image of a template image with a photographed image by desirably selecting a template image.

However, the template image data is recorded in the memory card in a state of compression according to a JPEG method. There has been a problem in that the template image data after expansion contains noise resulting from errors induced between compression and expansion of the data, resulting in a problem that noise occurs in the composite image.

SUMMARY OF THE INVENTION

Therefore, it is a primary object of this invention to provide a method of compositing a template image with a photographed image as well as a digital camera which can improve the image quality of a composite image.

It is another object of this invention to provide a template image recording method and a template image recording apparatus which can prevent noise from occurring.

This invention is a method of compositing a template image with a photographed image, comprising the steps of: (a) preparing a recording medium for holding third template image data created by converting first template image data into second template image data by a first converting method and compressing the second template image data by a predetermined compressing method; (b) expanding the third template image data by a predetermined expanding method to create fourth template image data; (c) converting the fourth template image data having a data value in a first determined range into fifth template image data by a second converting method; and (d) compositing the fifth template image data with the photographed image data.

The third template image data held in the recording medium is data that is created by converting first template image data into second template image data by a first converting method and compressing the second template image data by a predetermined compressing method. The third template image data reproduced out of the memory medium is expanded by a predetermined expanding method to thereby create the forth template image data. The fourth template image data having a data value in a first determined range only is converted into fifth template image data by a second converting method. The fifth template image data is composited with the photographed image data.

In one embodiment of this invention, data having a data value in a second predetermined range is converted by a first converting method into data having a data value in a third predetermined range smaller than the second predetermined range, while data having a data value in the third predetermined range is converted by a second converting method into data having a data value in the second predetermined range.

In another embodiment of this invention, the second predetermined range includes the first predetermined range, and the first predetermined range including the third predetermined range.

In one aspect of this invention, the first template image data includes at least one image-quality-related data component, and the second template image being created by converting the image-quality-related data component by the first converting method. The step (c) includes a step of comparing the image-quality-related data component contained in the fourth template image data with a predetermined value, and a step of converting the fourth template image data including the image-quality-related data component greater than the predetermined value into the fifth template image data.

According to this invention, the fourth template image data included in the first predetermined range is converted by a second converting method into the fifth template image data. It is therefore possible to remove away noises due to errors induced between compression and expansion, thereby improving the image quality of a composite image.

This invention is a method of recording a template image, comprising the steps of: (a) converting first template image data by a predetermined converting method to create second template image data; (b) compressing the second template image data by a predetermined compressing method to create third template image data; and (c) recording the third template image data into a recording medium.

The second template image data is created by converting the first template image data by a predetermined converting method, while the third template image data is created by compressing the second template image data by a predetermined compressing method. The third template image data thus created is recorded in the memory medium such as a memory card.

According to this invention, the template image data, that has been converted by the predetermined converting method and compressed by a predetermined compressing method, is recorded in the memory medium. This makes possible to remove noises during reproduction.

This is a digital camera, comprises: a recording medium for holding third template image data created by converting first template image data into second template image data by a first converting method and compressing the second template image data by a predetermined compressing method; a creating means for creating fourth template image data by expanding the third template image data by a predetermined expanding method; a converting means for converting the fourth template image data having a data value in a first predetermined range into fifth template image data by a second converting method; and a compositing means for compositing the fifth template image data with photographed image data.

The third template image data held in the recording medium is data that is created by converting first template image data into second template image data by a first converting method and compressing the second template image data by a predetermined compressing method. The third template image data reproduced out of the memory medium is expanded by a predetermined expanding method to thereby create the forth template image data. The fourth template image data having a data value in a first determined range only is converted into fifth template image data by a second converting method. The fifth template image data is composited with the photographed image data.

According to this invention, the fourth template image data included in the first predetermined range is converted by a second converting method into the fifth template image data. It is therefore possible to remove away noises due to errors induced between compression and expansion, thereby improving the image quality of a composite image.

This invention is a template image recording apparatus, comprising: a creating means for creating second template image data by converting template image data by a predetermined converting method; a creating means for creating third template image data by compressing second template image data by a predetermined compressing method; and a recording means for recording the third template image data into a recording medium.

The second template image data is created by converting the first template image data by a predetermined converting method, while the third template image data is created by compressing the second template image data by a predetermined compressing method. The third template image data thus created is recorded in the memory medium such as a memory card.

According to this invention, the template image data, that has been converted by the predetermined converting method and compressed by the predetermined compressing method, is recorded in the recording medium. Noises can be removed away during reproduction.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring toFIG. 1, a digital camera10of this embodiment includes a lens12. An optical image incident upon this lens12is given to a CCD imager14through a color filter having Cy, Ye,Mgand G arranged in a mosaic form.

When outputting a motion picture through a monitor34, the CCD imager14performs so-called well-known pixel mixing readout to supply a resulting pixel signal to a CDS/AGC circuit16. The CDS/AGC circuit16performs well-known noise removal and level adjustment on the inputted pixel signal. The pixel signal processed by this CDS/AGC circuit16is then converted by an AID converter18into digital data, i.e. pixel data. A first signal processing circuit20receives the pixel data outputted from the AID converter18to calculate luminance data (Y data) and color-difference data (U data and V data). Upon creating Y data, the inputted pixel data is averaged according to Equation 1.

For a line (H3+H4)
Y={(G+Ye)+(Mg+Cy)}/2={(G+G+R)+(R+B+B+G)}/2=(2R+3G+2B)/2

Since the Y data is proportional to 2R+3G+2B, the component B is reproduced brighter than the y data (=0.3R+0.59G+0.11B) specified by the NTSC standard. This, however, raises no practical problem.

Meanwhile, when creating U data and V data, the first signal processing circuit20performs subtraction between adjacent pixels according to Equation 2.

For the line (H1+H2)
U={(Mg+Ye)−(G+Cy)}={(R+B+R+G)−(G+B+G)}=2R−G[Equation 2]

For the line (H3+H4)
V={(G+Ye)−(Mg+Cy)}={(G+G+R)+(R+B+B+G)}=G−2B

However, the color-difference data is available only every other line. Accordingly, the first signal processing circuit20supplements for deficient color-difference data on a current line by using the color difference data on the preceding line. That is, V data only is available on a line (H3+H4) so that the U data on a line (H1+H2) is utilized for the U data for the line (H3+H4).

The Y, U and V data thus created are written by a memory control circuit26into a memory area24aof a DRAM24, and are thereafter outputted to a second signal processing circuit30. The second signal processing circuit30, when outputting a motion picture, performs predetermined horizontal and vertical interpolations on the Y, U and V data (motion picture data) from the DRAM24so that these data become suited for a display-screen size of an LCD34. The motion picture data, supplied from the second signal processing circuit30, is converted by a D/A converter32into an analog signal. This analog signal is supplied to the LCD34, and also outputted through an output terminal36. As a result, a motion picture is outputted through the LCD34.

When an operator depresses the shutter button40, a system controller42controls the CCD imager14so as to perform so-called all-pixel readout. Due to this, the CCD imager14outputs pixel signals at every other line. Since the CCD imager14is mounted with a color filter in a mosaic form, Cyand Yeare alternately outputted at an odd line, while Mgand G are alternately outputted at an even line. The CDS/AGC circuit16performs noise removal and level adjustment on the pixel signal, similarly to the above. The A/D converter18converts the pixel signal from the CDS/AGC circuit16into digital data, i.e. pixel data. The CCD imager14is disabled after outputting 1 frame of pixel signals. The 1-frame pixel data outputted from the A/D converter18is directly delivered onto a bus22without processed by the first signal processing circuit20. The pixel data is written into the memory area24aby the memory control circuit26.

The CPU28converts the pixel data of the memory area24ainto Y (=Y.sub.2), U and V data, according to Equation 3 to Equation 5, with using a working area24b. The converted Y, U and V data, i.e. photographed image data, are compressed according to a JPEG format, and written into a memory card46.
Yh=Cy+Ye+Mg+G
Cb=(Cy+Mg)−(Ye+G)
Cr=(Ye+Mg)−(Cy+G)  [Equation 3]
R=k11×Yh+k12×Cb+k13×Cr
G=k21×Yh+k22×Cb+k23×Cr
B=k31×Yh+k32×Cb+k33×Cr[Equation 4]
YL=0.299×R+0.587×G+0.114×B
U=B−YL
V=R−YL[Equation 5]

Due to the color separation and YUV conversion as stated above, adjacent4(2 ×2) pixels of Cy, Mg, Yeand G data are used to create 1-pixel Y, U and V data of the 4 pixels, thereby providing Y, U and V data for all the pixels.

The memory control circuit26reads Ye,CyMgand G pixel data on a line-by-line basis out of the memory area24a, and supplies them to the second signal processing circuit30. The second signal processing circuit30, when outputting a photographed image, performs color separation and YUV-conversion on the pixel data according to Equation 3 to Equation 5, thereby outputting a photographed image (freeze image) on the LCD34.

If the operator depresses a reproduced button52, the CPU28reads desired compressed data out of a memory card46, and expands the compressed data with using a working area24b. The expanded image data (YUV data) is stored in the memory area24a, and read out later. The second signal processing circuit30, in a reproducing mode, performs horizontal and vertical interpolations on the image data. This allows a reproduced image to be displayed on the LCD34.

The memory card46may use, for example, a template card that has a template image previously prepared as shown inFIG. 2(A)and a template code written therewith. With such a card, the operator can set either one of a normal reproducing mode or a card-function reproducing mode by operating a reproducing mode setting button48.

In the normal reproducing mode, if the operator manipulates a + button54or − button56, any one of image data that has been recorded in the memory card46is read out. Thus, a reproduced image is displayed on the LCD34, similarly to the above. That is, if a template image shown inFIG. 2(A)and a photographed image shown inFIG. 2(B)are recorded in the memory card46, then either one of the images will be outputted through the LCD34.

Where the card-function reproducing mode is set, if the operator operates the + button54, the − button56and the selecting button50, an image composited by the template image and the photographed image can be created as shown inFIG. 2(C).

The template image data shown inFIG. 2(A)is subjected to effective-range conversion and JPEG compression as shown inFIG. 3, and then recorded in the template card. That is, 8 bits of the template image data are taken into an image processing apparatus (not shown), such as a personal computer, where the Y data constituting the template image data is converted in effective range from “0”–“255” to “128”–“255”, according to Equation 6. Incidentally, the range “0”–“255” is defined as a second predetermined range, while the range “128”–“255” as a third predetermined range.
d2=(d1÷2)+128  [Equation 6]

d1: the Y data before conversion on each pixel

d2: the Y data after conversion on each pixel

Consequently, a converted template image is displayed on a display of the personal computer. If the operator designates an unwanted portion (a portion other than the curtain), this unwanted portion is cut away. That is, this unnecessary portion has a data value turned to “0”. The personal computer compresses the template image data like this according to a JPEG format, and records the compressed data into the template card, depending upon a designation of recording by the operator.

Incidentally, the compressed data of the template image is accommodated in an image file with a file name “syn0000S.jpg” (S is an integer), while the compression data of the photographed image is to an image file having a file name “pic0000P.jpg” (P is an integer).

The CPU28performs the above-stated operation according to a program stored in a flash memory38. When a card-function reproducing mode is set, the operation depends upon the flowcharts shown inFIG. 4toFIG. 6. In the card-function reproducing mode, the CPU28first determines, at a step S1, whether a template code exists in the memory card46or not. If “NO”, the process returns to the normal reproducing mode. That is, when a desired memory card is not mounted, even if the operator operates the reproducing mode setting button48, the card-function reproducing mode cannot be established. If the determination is “YES” at the step S1, the CPU28resets, at a step S3, a count value S of the syn counter28a, and determines at a step S5whether the image file “syn0000S.jpg” exists or not. If “NO”, the process returns to the step S3, while if “YES”, the process proceeds to steps S7–S11to expand the compressed data in the image file to store it into the memory area24aof the DRAM24.

Explaining in further detail, the Y data contained in the compressed data is first expanded at the step S7, and stored in the memory area24a. Then, the U data contained in the compressed data is expanded at a step S9, and stored in the memory area24a. Subsequently, the V data contained in the compressed data is expanded at a step S11, and stored in the memory area24a. The CPU28then reversely converts, at a step S13, an effective range of the Y data stored in the memory area24afrom “128”–“255” into “0”–“255” according to Equation 7, and writes the converted Y data into the working area24b. That is, the CPU28performs on the Y data a JPEG expansion and reverse conversion in data value, as shown in FIG.3.
d3=(d2−128)×2  [Equation 7]

d3: the Y data after reverse conversion on each pixel

The CPU28subsequently writes, at a step S15, the Y data and the V data as they are into the working area24b, and reads out the Y, U and V data written in the working area24bat a step S17. This causes a template image having a desired luminance to be outputted through the LCD34.

The CPU28thereafter determines at a step S19whether a selecting button50is depressed or not. If “NO”, it is determined at a step S21whether the + button54or− button56is depressed or not. If neither of the + button54nor the − button56is depressed, the process returned to the step S19, whereby a same image continues to display on the LCD34. On the other hand, if the + button54or − button56is depressed, the CPU28increments or decrements the count value S of the syn counter28aaccording to the button operation at a step S23, and the process returns to the step S5. This changes over an image to be displayed on the LCD34. If the selecting button53is depressed by the operator, the CPU proceeds the process from the step S19to a step S25.

In this manner, the template image data merely expanded is held in the memory area24a, and the template image data having Y data reversely converted in effective range is held in the working area24b.

The CPU28resets at the step S25a count value P of a pic counter28b, and then determines at a step S27whether an image file “pic0000P.jpg” exists in the memory card46or not. If the determination here is “NO”, the process returns to the step S25, while if “YES”, the process proceeds to steps S29–S33to write the photographed image data in the image file over the working area24b. That is, the Y data, U data and V data constituting the photographed image data are expanded respectively at steps S29–S33, and stored in the working area24b. The CPU28subsequently reads the Y, U and V data out of the working area24bat a step S35. This causes the photographed image to be displayed on the LCD34.

The CPU28thereafter determines at a step S37whether or not the selecting button50is depressed. If “NO”, it is determined at a step S39whether the + button54or− button56is depressed or not. If the determination at the step S39is “NO”, the process returns to the step S37. However, if “YES”, the count value P is changed by incrementing or decrementing at a step S41, and the process returns to the step S27. This changes over the photographed image to be displayed on the LCD34.

If the operator depresses the selecting button50, the CPU determines as “YES” at the step S37, and reads, at a step S43, the Y data on any pixel from the memory area24a. It is determined at a step S45whether the Y data has a data value lying in a first predetermined range, i.e. “64”≦Y≦“255” or not. Here, the first predetermined range has its lower limit taken smaller than the third predetermined range, because there may be a case that a deviation occurs in the data value after expansion due to errors caused between expansion and compression. The lower limit of the first predetermined range is set at “64”, because the value “64” lies intermediate between a minimum value “128” of the third predetermined range and “0”.

If “YES”at the step S45, the CPU28at the step S45reversely converts the Y data according to Equation 7 as stated above, and writes the reversely-converted Y data into the working area24b. This causes the data of a relevant pixel of the photographed image data held in the working area24bto be updated by a reversely-converted Y data. Incidentally, the Y data value before the reverse conversion is less than “128”, the data value after conversion becomes smaller than “0”. In such a case, the CPU28forcibly renders the reversely-converted data value “0”.

The CPU28further writes, at steps S49and S51, the U data and the V data on a same pixel as they are to the working area24b. Consequently, the U data and the V data of a relevant pixel of the photographed image data had in the working area24bare updated by the template-image U data and V data. The CPU28thereafter advances the process to a step S53. If “NO” at the step S45, the CPU28proceeds the process directly to the step S53without performing any process. That is, if the value of the Y data is not included within the first predetermined range, it is considered that the pixel has no data existing thereon or, even if existing, it would be due to a noise, thus proceeding the process directly to the step S53.

The CPU28determines at the step S53whether all the pixels have completed of Y data readout or not. If “NO”, the process returns to the step S43. However, if “YES”, it is determined that the template image data and the photographed image data have completed of compositing. Then, at a step S55, composite image data (YUV data) is read out of the working area24b. This causes a composite image to be outputted on the LCD34as shown inFIG. 2(C). The CPU28also compresses, at a step S57, this composite image data according to the JPEG format, and records the compressed data into the memory card46. Then the process is ended.

According to this embodiment, a template card is prepared which is recorded with template image data that has Y data having data value converted in effective range and compressed according to a JPEG format. During reproducing the template image data from the template card, the Y data is reversely converted in effective range. Then, the template image data, on except for a pixel having a Y data value smaller than a predetermined value, is composited with photographed image data. Therefore, it is possible to prevent noises from occurring on a composite image and hence improve the image quality of a composite image.

Incidentally, the memory card may use various kinds of cards involving SSFDC (Solid-State Floppy Disc Card). Although this embodiment was explained using a complementary-colored filter having Ye, Cy, Mgand G arranged in a mosaic form, a primary-colored filter may be employed that has R, G and B arranged in a mosaic form. Further, although in this embodiment the Y data is converted in effective range, the U data or the V data may be converted in effective range. Where using a primary-colored filter, it is possible to convert the effective range of any one of the R, G and B data.