Abstract:
First image data and second image data different from the first image data are input in an image processing apparatus/method. At least one of the first image data and the second image data is encoded while being subjected to orthogonal transformation processing. The first image data and the second image data are transformed into first and second orthogonal transformation coefficient data, and the first and second orthogonal transformation coefficient data are synthesized. A recording medium capable of being read by a computer stores an image processing program for the image processing method.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The present invention relates to an image processing apparatus and method for efficiently synthesizing a plurality of image data while encoding the data, and a storage medium capable of being read by a computer in which an image processing program is stored. 
     2. Description of the Related Art 
     When editing images picked up by a video camera, as shown in FIG. 1, output image signals from two video cassette recorders (hereinafter abbreviated as “VCRs”)  110  and  112  are input to an image editor  114 . An image signal obtained by editing the image signals is output to another VCR  116  in order to be recorded on a recording medium, such as a magnetic tape or the like. 
     FIG. 2 is a schematic block diagram illustrating the configuration of a camcorder (a video camera incorporating a recording/reproducing apparatus). In FIG. 2, an image pickup device  120  converts an optical image of an object into an electric signal. An A/D (analog-to-digital) converter  122  converts an analog signal from the image pickup device  120  into a digital signal. A compression circuit  124  performs data compression of data output from the A/D converter  122 , and records the compressed image data on a recording medium  126 . The output of the compression circuit  124  is also supplied to a digital interface (I/F)  128 . The digital interface  128  conforms, for example, to IEEE 1394 standards. The image data is output to the outside in a format conforming to the standards. 
     Data reproduced from the recording medium  126  and the data input to the digital interface  128  are supplied to an expansion circuit  130 , which expands the received data from a compressed state. Output data from the expansion circuit  130  is converted into an analog signal by a D/A (digital-to-analog) converter  132 , and is supplied to a monitor  134  for image display. 
     Various image compression methods have been proposed and actually used. Many of these methods use both orthogonal transformation and variable-length encoding. 
     Conventionally, when synthesizing two images which are recorded in a compressed state as described above, image synthesizing processing is performed after completely expanding image data. FIG. 3 is a schematic block diagram illustrating a conventional image synthesis apparatus. 
     Compressed image data are input to input terminals  140   a  and  140   b . Variable-length decoding circuits  142   a  and  142   b  perform variable-length decoding of the data from the input terminals  140   a  and  140   b , respectively. Inverse orthogonal transformation circuits  144   a  and  144   b  perform inverse orthogonal transformation of outputs from the variable-length decoding circuits  142   a  and  142   b , respectively, and output restored image data. 
     A system control circuit  146  controls the entire apparatus in accordance with the user&#39;s operation through an operation unit  148 . Particularly, the system control circuit  146  controls a coefficient generation circuit  150  so as to generate coefficients to be used when synthesizing the two image data. The generated coefficients are supplied to multipliers  152   a  and  152   b . The multipliers  152   a  and  152   b  multiply the outputs of the inverse orthogonal transformation circuits  144   a  and  144   b  by the corresponding coefficients from the coefficient generation circuit  150 , respectively. An adder  154  adds the results of the multiplication by the muntipliers  152   a  and  152   b . That is, the adder  154  synthesizes the two images. 
     An orthogonal transformation circuit  156  performs orthogonal transformation of an output image from the adder  154 . A variable-length encoding circuit  158  performs variable-length encoding of the output of the orthogonal transformation circuit  156 . The output of the variable-length encoding circuit  158  is output to the outside from an output terminal  160 . 
     As described above, conventionally, the two inverse orthogonal transformation circuits  144   a  and  144   b  are required, and orthogonal transformation processing is required when again encoding data after image synthesis. Accordingly, extensive hardware or software is required, and a long time is required for calculation. 
     SUMMARY OF THE INVENTION 
     The present invention has been made in consideration of the above-described problems. 
     It is an object of the present invention to provide an image synthesis apparatus and method capable of providing a smaller circuit and fewer signal processing steps than previously used, and a storage medium that stores program software for realizing the image processing method. 
     According to one aspect of the present invention, an image processing apparatus includes an input unit for inputting first image data and second image data different from the first image data. At least one of the first image data and the second image data is encoded while being subjected to orthogonal transformation processing. The apparatus also includes a transformation unit for transforming the first image data and the second image data into orthogonal transformation coefficient data, and a synthesis unit for synthesizing the first image data and the second image data transformed by the transformation unit. 
     According to still another aspect of the present invention, a storage medium capable of being read by a computer stores an image processing program, the program including an input step of inputting first image data and second image data different from the first image data. At least one of the first image data and the second image data is encoded while being subjected to orthogonal transformation processing. The program also includes a transformation step of transforming the first image data and the second image data into orthogonal transformation coefficient data, and a synthesis step of synthesizing the first image data and the second image data transformed in the transformation step. 
     According to still another aspect of the present invention, in a storage medium, capable of being read by a computer storing an image processing program, the program includes an input step of inputting first image data and second image data different from the first image data. At least one of the first image data and the second image data is encoded while being subjected to orthogonal transformation processing. The program also includes a transformation step of transforming the first image data and the second image data into orthogonal transformation coefficient data, and a synthesis step of synthesizing the first image data and the second image data transformed in the transformation step. 
    
    
     The foregoing and other objects, advantages and features of the present invention will become more apparent from the following detailed description of the preferred embodiments taken in conjunction with the accompanying drawings. 
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a schematic block diagram illustrating the configuration of an editing system for editing two images; 
     FIG. 2 is a schematic block diagram illustrating the configuration of a VCR shown in FIG. 1; 
     FIG. 3 is a block diagram illustrating the configuration of a conventional image synthesis apparatus; 
     FIG. 4 is a block diagram illustrating the configuration of an image synthesis apparatus according to a first embodiment of the present invention; 
     FIG. 5 is a flowchart illustrating image synthesizing processing according to the first embodiment; 
     FIG. 6 is a block diagram illustrating the configuration of an image synthesis apparatus according to a second embodiment of the present invention; 
     FIG. 7 is a block diagram illustrating the configuration of an image synthesis apparatus according to a third embodiment of the present invention; 
     FIG. 8 is a diagram illustrating image switching in the image synthesis apparatus shown in FIG. 7; 
     FIG. 9 is a flowchart illustrating image synthesizing processing by the image processing apparatus shown in FIG. 7; 
     FIG. 10 is a block diagram illustrating the configuration of an image synthesis apparatus according to a modification of the first embodiment; 
     FIG. 11 is a block diagram illustrating the configuration of an image synthesis apparatus according to a modification of the second embodiment; and 
     FIG. 12 is a block diagram illustrating the configuration of an image synthesis apparatus according to a modification of the third embodiment. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     Preferred embodiments of the present invention will now be described in detail with reference to the drawings. 
     FIG. 4 is a schematic block diagram illustrating the configuration of an image synthesis apparatus according to a first embodiment of the present invention. In FIG. 4, image data compressed according to orthogonal transformation and variable-length encoding are input to input terminals  10   a  and  10   b . Variable-length decoding circuits  12   a  and  12   b  perform variable-length decoding of the compressed image data from the input terminals  10   a  and  10   b , respectively. A system control circuit  14  controls the entire apparatus. An operation unit  16  is used for inputting various instructions to the system control circuit  14 . A coefficient generation circuit  18  outputs multiplication coefficients to be used when synthesizing images input to the input terminals  10   a  and  10   b , under the control of the system control circuit  14 . Multipliers  20   a  and  20   b  multiply the outputs of the variable-length decoding circuits  12   a  and  12   b  (orthogonal transformation coefficient data) by the coefficients Ka and Kb from the coefficient generation circuit  18 , respectively. An adder  22  adds the outputs of the multipliers  20   a  and  20   b . A variable-length encoding circuit  24  performs variable-length encoding of the output of the adder  22 . The output of the variable-length encoding circuit  24  is output to the outside from an output terminal  26 . A recording circuit  28  records the output of the variable-length encoding circuit  24  on a recording medium, such as a video tape, a hard disk or the like. 
     FIG. 5 is a flowchart illustrating the operation of the first embodiment. The operation of the first embodiment will now be described with reference to FIG.  5 . 
     Image data A and B compressed according to orthogonal transformation and variable-length encoding are input to the input terminals  10   a  and  10   b , respectively (step S 1 ). The variable-length decoding circuits  12   a  and  12   b  perform variable-length decoding of the compressed image data A and B from the input terminals  10   a  and  10   b , respectively, and output orthogonal transformation coefficient data (step S 2 ). The system control circuit  14  causes the coefficient generation circuit  18  to generate synthesis coefficients Ka and Kb for images input to the input terminals  10   a  and  10   b , respectively (step S 3 ). The coefficients Ka and Kb generated by the coefficient generation circuit  18  are supplied to the multipliers  20   a  and  20   b , respectively. The multipliers  20   a  and  20   b  multiply the orthogonal transformation coefficient data from the variable-length decoding circuits  12   a  and  12   b  by the coefficients Ka and Kb, respectively (step S 4 ). The adder  22  adds the output of the multipliers  20   a  and  20   b  (step S 5 ). The variable-length encoding circuit  24  performs variable-length encoding of the output of the adder  22  (step S 6 ). The output of the variable-length encoding circuit  24  is output to the outside from the output terminal  26 , or recorded on a recording medium by the recording circuit  28  (step S 7 ). 
     In the first embodiment, since image synthesis is performed using desired weighting coefficients Ka and Kb in a state of orthogonal transformation coefficient data, it is unnecessary to provide an inverse orthogonal transformation circuit. By adjusting the coefficients Ka and Kb, special editing, such as wipe or the like, can be realized. 
     If image data input to the input terminals  10   a  and  10   b  is image data subjected to orthogonal transformation processing followed by quantization and variable-length encoding, then, as shown in FIG. 10, inverse quantization circuits  19   a  and  19   b  and a quantization circuit  23  may be inserted. 
     Although in the above-described first embodiment, both of the two input image data are subjected to compression encoding, the present invention may also be applied to a case in which only one of the two image data is subjected to compression encoding. 
     FIG. 6 is a schematic block diagram illustrating the configuration of an image synthesis apparatus according to a second embodiment of the present invention. In the second embodiment, one input data is image data itself, and another input data is image data subjected to orthogonal transformation and variable-length encoding. 
     In FIG. 6, image data which has been picked up by a video camera  29  but has not been subjected to compression encoding is input to an input terminal  30   a , and image data compressed according to orthogonal transformation and variable-length encoding is input to an input terminal  30   b . An orthogonal transformation circuit  32   a  performs orthogonal transformation of the image data from the input terminal  30   a . A variable-length decoding circuit  32   b  performs variable-length decoding of the compressed image data from the input terminal  30   b . A system control circuit  34  controls the entire apparatus. An operation unit  36  is used for inputting various instructions to the system control circuit  34 . A coefficient generation circuit  38  outputs multiplication coefficients to be used when synthesizing images input to the input terminals  30   a  and  30   b , under the control of the system control circuit  34 . Multipliers  40   a  and  40   b  multiply the outputs of the orthogonal transformation circuit  32   a  (orthogonal transformation coefficient data) and the output of the variable-length decoding circuit  32   b  (orthogonal transformation coefficient data) by the corresponding coefficients Ka and Kb from the coefficient generation circuit  38 , respectively. An adder  42  adds the outputs of the multipliers  40   a  and  40   b . A variable-length encoding circuit  44  performs variable-length encoding of the output of the adder  42 . The output of the variable-length encoding circuit  44  is output to the outside from an output terminal  46 . A recording circuit  48  records the output of the variable-length encoding circuit  44  on a recording medium, such as a video tape, a hard disk or the like. 
     The operation of the second embodiment will now be described. 
     Image data which has been picked up by the video camera  29  and has not been subjected to compression encoding is input to the input terminal  30   a , and image data compressed according to orthogonal transformation and variable-length encoding is input to the input terminal  30   b . The orthogonal transformation circuit  32   a  performs orthogonal transformation of the image data from the input terminal  30   a , and the variable-length decoding circuit  32   b  performs variable-length decoding of the compressed image data (image data subjected to orthogonal transformation and variable-length encoding) from the input terminal  30   b . Thus, both of the outputs from the orthogonal transformation circuit  32   a  and the variable-length decoding circuit  32   b  become orthogonal transformation coefficient data. The system control circuit  34  causes the coefficient generation circuit  38  to generate synthesis coefficients Ka and Kb for images input to the input terminals  30   a  and  30   b , respectively. The coefficients Ka and Kb generated by the coefficient generation circuit  38  are supplied to the multipliers  40   a  and  40   b , respectively. The multipliers  40   a  and  40   b  multiply the output from the orthogonal transformation circuit  32   a  and the orthogonal transformation coefficient data from the variable-length decoding circuit  32   b  by the coefficients Ka and Kb, respectively. The adder  42  adds the outputs of the multipliers  40   a  and  40   b . The variable-length encoding circuit  44  performs variable-length encoding of the output of the adder  42 . The output of the variable-length encoding circuit  44  is output to the outside from the output terminal  46 , or recorded on a recording medium by the recording circuit  48 . 
     If image data input to the input terminal  30   b  is image data subjected to orthogonal transformation processing followed by quantization and variable-length encoding, and the image data output from the adder  42  is quantized and subjected to variable-length encoding, then, as shown in FIG. 11, an inverse quantization circuit  39  and a quantization circuit  43  may be inserted. 
     The configuration shown in FIG. 6 may also be applied to a process of performing switching between two images while partially overlapping the images. FIG. 7 is a block diagram illustrating the configuration of an image synthesis apparatus when the configuration shown in FIG. 6 is used for image switching processing, according to a third embodiment of the present invention. In FIG. 7, the same components as those shown in FIG. 6 are indicated by the same reference numerals, and further description thereof will be omitted. 
     In FIG. 7, compressed image data reproduced from a recording medium  52  is input to an input terminal  30   b , and the compressed image data output from an output terminal  46  is recorded on a recording medium  50 . A coefficient Ka supplied from a coefficient generation circuit  38 ′ to a multiplier  40   a  equals 1−K, and a coefficient Kb supplied from the coefficient generation circuit  38 ′ to a multiplier  40   b  equals K. 
     By continuously changing the coefficient K from 1 to 0 every time image synthesis processing for one frame is completed, an output image (an image to be recorded on the recording medium  50 ) continuously changes, as shown in FIG. 8, from image data stream B (waveform  801 ) input to the input terminal  30   b  (the image reproduced from the recording medium  52 ) to input image data A (waveform  802 ) to an input terminal  30   a.    
     At that time, for example, by displaying the same picture frame or using only the DC component of the orthogonal transformation coefficient when the intensity of the image data stream B input to the input terminal  30   b  is halved (waveform  803 ), the amount of calculation necessary for image synthesis can be reduced. 
     FIG. 9 is a flowchart illustrating processing in which, when the synthesis coefficient of the image data stream B input to the input terminal  30   b  is equal to or less than ½, the same data is used for the image data stream B. In FIG. 9, “a” represents a positive coefficient less than 1. When the value “a” is large, image switching is slow, and when the value “a” is small, image switching is fast. 
     The multiplication coefficient K is set to 1 as an initial value (step S 11 ). In step S 12 , it is determined if the value K is larger than 0.5. If the result of the determination in step S 12  is affirmative, the data A and B are received from the input terminals  30   a  and  30   b , respectively (step S 13 ). Then, the image data stream B input to the input terminal  30   b  is subjected to variable-length decoding by a variable-length decoding circuit  32   b  (step S 14 ). Then, the input image data A input to the input terminal  30   a  is subjected to orthogonal transformation (for example, discrete cosine transform) by an orthogonal transformation circuit  32   a  (step S 15 ). A multiplier  40   a  multiplies the output of the orthogonal transformation circuit  32   a  by the coefficient 1−K, and a multiplier  40   b  multiplies the output of the variable-length decoding circuit  32   b  by the coefficient K (step S 16 ). Then, an adder  42  adds the outputs of the multipliers  40   a  and  40   b  (step S 17 ). A variable-length encoding circuit  44  performs variable-length encoding of the output of the adder  42 , serving as synthesized image data, and outputs the resultant data from the output terminal  46  to the recording medium  50  (step S 18 ). 
     In step S 19 , it is determined if the value K equals 0. If the result of the determination in step S 19  is negative, the process proceeds to step S 20 , where it is determined if processing of one frame has been completed. If the result of the determination in step S 20  is negative, the processing starting from step S 12  is repeated. If the result of the determination in step S 20  is affirmative, the multiplication coefficient K is updated to K×a (step S 21 ), and the processing starting from step S 12  is repeated. 
     If the result of the determination in step S 12  is negative, only the input image data A at the input terminal  30   a  is received (step S 22 ). Then, orthogonal transformation is performed (step S 15 ), and image synthesis is performed using the variable-length decoded data of the previously used image data stream B without modifying the data (steps S 16  and S 17 ). 
     If image data input to the input terminal  30   b  is image data subjected to orthogonal transformation processing followed by quantization and variable-length encoding, and the image data output from the multiplier  42  is quantized and subjected to variable-length encoding, a configuration as shown in FIG. 12 may be adopted. In FIG. 12, the same components as those shown in FIGS. 7 and 11 are indicated by the same reference numerals. 
     The present invention may be applied to a system comprising a plurality of apparatuses (such as a host computer, an interface apparatus, a reader, a printer and the like), or to an apparatus comprising a single unit (such as a copier, a facsimile apparatus or the like). 
     The present invention may also be realized by supplying a computer (a CPU (central processing unit) or an MPU (microprocessor unit)) within an apparatus or a system connected to various devices in order to operate the devices with program codes of software for realizing the functions of the above-described embodiments, and causing the computer to operate the various devices in accordance with a stored program. 
     In such a case, the program codes of the software realize the functions of the embodiments, so that the program codes themselves and means for supplying the computer with the program codes, such as a storage medium storing the program codes, constitutes the present invention. For example, a floppy disk, a hard disk, an optical disk, a magnetooptical disk, a CD(compact disk)-ROM (read-only memory), a CD-R (recordable), a magnetic tape, a nonvolatile memory card, a ROM or the like may be used as the storage medium for storing the program codes. 
     The present invention may, of course, be applied not only to a case in which the functions of the above-described embodiments are realized by execution of supplied program codes by a computer, but also to a case in which the functions of the above-described embodiments are realized by cooperation of the program codes with an OS (operating system) operating in the computer, other application software or the like. 
     The present invention may, of course, be applied to a case in which, after storing supplied program codes in a memory provided in a function expanding board of a computer or in a function expanding unit connected to the computer, a CPU or the like provided in the function expanding board or the function expanding unit performs a part or the entirety of actual processing, and the functions of above-described embodiments are realized by the processing. 
     The individual components designated by blocks in the drawings are all well known in the image processing apparatus and method arts and their specific construction and operation are not critical to the operation or the best mode for carrying out the invention. 
     While the present invention has been described with respect to what are presently considered to be the preferred embodiments, it is to be understood that the invention is not limited to the disclosed embodiments. To the cotrary, the present invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.