Image synthesizing apparatus and image synthesizing method

An image synthesizing apparatus according to the present invention comprises coefficient setting means for setting a blending coefficient α(0≦α≦1) at a specified value when a value of a specific picture element component included in picture element components A of a first image is a predetermined value; and arithmetic means for performing an operation on the picture element components A, picture element components B of a second image, and the blending coefficient α as follows: A*α+B*(1−α), and performing the operation on all the picture element components A and the picture element components B of a picture element that has the specific picture element component representing the predetermined value by using the blending coefficient α set by the coefficient setting means.

BACKGROUND OF THE INVENTION

The present invention relates to an image synthesizing apparatus for synthesizing two pieces of image data and outputting synthesized image data, and particularly to an image synthesizing apparatus for effecting image synthesis by using an α blending method.

Recently, functions of a television receiver have been diversified. Since conventional mechanical switches such for example as buttons provided on the body of the television receiver and a remote control are not sufficient for setting the functions of the television receiver, the functions are more often set by menu operation such as displaying a menu image on a monitor screen, selecting a setting item, and supplying an input on a displayed setting screen. In some of such cases, an image for menu operation is superimposed on a television image currently selected for viewing by a television tuner on the monitor screen.

FIG. 7shows an example of a display screen of a television receiver for such menu operation.

FIG. 7shows a menu screen71to be superimposed, a television screen72selected by a television tuner or the like, and a synthesized screen73in which the menu screen71is superimposed on the television screen72. Selection buttons71a,71b, and71cshown on the menu screen71for example represent a “sound selection” button for switching between a main-channel sound and a sub-channel sound, a “channel” button for setting the display number of a channel, and a “return” button for stopping the display of the menu screen, respectively.

The television screen72being viewed can be seen through the menu selection buttons71a,71b, and71con the synthesized screen73in which the menu screen71is superimposed on the television screen72. An image synthesizing method that makes one image seen through the other image in such a manner is referred to as α blending. This processing method is often used for menu operation on a television screen.

According to the α blending method, when an image to be superimposed is set to be A, a background image is set to be B, and a blending coefficient α is set to be α(0≦α≦1), an operation is performed on each element forming respective one of the images A and B so that an output image S satisfies the following equation, and then the result is outputted.
S=A*α+B*(1−α)
An arbitrary value in a range of 0≦α≦1 is selected for the blending coefficient α to thereby define transparency; when α is zero, the image A to be superimposed on the image B is completely transparent, while when α is unity, the image A to be superimposed on the image B is completely opaque.

When the menu screen is superimposed on the television screen by using the α blending method as described above, the blending coefficient α is generally determined for each menu screen using a table referred to as a color look-up table (hereinafter referred to as a CLUT).

FIG. 8shows an example of a CLUT. InFIG. 8, Y, Cb, and Cr are picture element components that form a picture element, and Y represents a luminance component while Cb and Cr represent color difference components. Coordinate values represent coordinates (x, y) of a picture element on a monitor screen obtained when an x-component is taken in a horizontal direction and a y-component is taken in a vertical direction. In order to simplify description, it is assumed that only four sets of coordinates are present inFIG. 8. The values of the luminance component Y, the color difference component Cb, the color difference component Cr, and α are converted into 8-bit data for convenience of operation.

In order to change transparency of a synthesized screen for each picture element, the α blending method using a CLUT requires information on an α value for each picture element, as shown inFIG. 8. Therefore, the finer the picture elements on a monitor screen, the larger the amount of information of the CLUT, thereby increasing memory size and the load of the above-mentioned operation processing, and thus decreasing image-generating rate. In order to deal with the problem, circuit scale, power consumption, and cost of the television receiver must be increased.

SUMMARY OF THE INVENTION

The present invention has been made in view of the above problems. It is an object of the present invention to provide an image synthesizing apparatus that reduces the amount of data handled in image synthesis processing by α blending and thereby reduces the load of image synthesis processing.

It is another object of the present invention to provide an image synthesizing method that reduces the amount of data used for α blending and thereby reduces the load of image synthesis processing.

In order to solve the above problems, according to an aspect of the present invention, there is provided an image synthesizing apparatus for synthesizing two images, including coefficient setting means for setting a blending coefficient α(0≦α≦1) at a specified value when a value of a specific picture element component included in picture element components A of a first image is a predetermined value; and arithmetic means for performing an operation on the picture element components A, picture element components B of a second image, and the blending coefficient α as follows:
A*α+B*(1−α)

and performing the operation on all the picture element components A and the picture element components B of a picture element that has the specific picture element component representing the predetermined value by using the blending coefficient α set by the coefficient setting means.

When a specific component of a picture element that forms the first image represents a predetermined value, the image synthesizing apparatus according to the present invention performs α blending operation by the arithmetic means on all the picture element components of the picture element by fixing the blending coefficient α at a specified value, thereby eliminating the need for setting the blending coefficient α for every picture element. Thus, the amount of information for the operation is reduced, and thereby it is possible to reduce required memory capacity and the load of image processing.

According to another aspect of the present invention, there is provided an image synthesizing method for synthesizing two images, wherein an operation is performed on picture element components A of a first image, picture element components B of a second image, and a blending coefficient α(0≦α≦1) as follows:
A*α+B*(1−α)

and when a specific picture element component included in the picture element components A represents a predetermined value, the blending coefficient α is set at a specified value in the operation on all the picture element components A and the picture element components B of a picture element that has the specific picture element component representing the predetermined value.

According to the image synthesizing method, when a specific component of a picture element that forms the first image represents a predetermined value, α blending operation is performed on all the picture element components of the picture element by fixing the blending coefficient α at a specified value, thereby eliminating the need for setting the blending coefficient α for every picture element. Thus, the amount of information for the operation is reduced, and thereby it is possible to reduce required memory capacity and the load of image processing.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIG. 1is a block diagram for illustrating principles of the present invention.

An image synthesizing apparatus1synthesizes a first image1aand a second image1bat arithmetic means1dby α blending to output data of a synthesized image to a display unit2or the like. The first image1aand the second image1bare each formed by a set of a plurality of adjacent picture elements, and each of the picture elements is formed by a plurality of picture element components. The synthesized image S is outputted after the following operation:
S=A*α+B*(1−α)(0≦α≦1)
where picture element components of the first image1aare A, picture element components of the second image1bare B, and α is a blending coefficient for defining transparency; when α=0, the first image1ais completely transparent while when α=1, the first image1ais completely opaque. The image synthesizing apparatus1performs such an operation on picture element components of all the picture elements that form each image.

Coefficient setting means1cin the image synthesizing apparatus1detects data of a specific picture element component from the picture element components A that form the first image1a. When the data represents a predetermined value, the coefficient setting means1csets the blending coefficient α used for the blending operation at a specified value. The arithmetic means1dperforms the above operation on all picture element components A and B. At this point, the arithmetic means1dperforms the above operation on all picture element components A and B of the picture elements whose specific picture element components represent the predetermined value by using the blending coefficient α set by the coefficient setting means1c. Synthesized image data outputted after the arithmetic means1dhas performed the operation is displayed on the display unit2as a synthesized image or stored in a memory not shown in the figure.

When the picture element components A and B are formed by two kinds of components, that is, a luminance component Y and a color difference component Cb or Cr, for example, the coefficient setting means1cdetects the value of the luminance component Y from all the picture element components A of the first image1a, and when the value of the luminance component Y is zero, the coefficient setting means1csets the blending coefficient α at zero. According to the setting by the coefficient setting means1c, the arithmetic means1dperforms the operation on all the picture element components, that is, the luminance component Y and the color difference component Cb or Cr of a picture element whose luminance component Y is zero by setting the blending coefficient α to be zero. As a result, when the picture elements of the first image1ahave a zero luminance component Y, the first image1abecomes completely transparent in the displayed synthesized image and therefore is not displayed on the screen, and only the second background image1bis displayed.

Such processing by the coefficient setting means1cand the arithmetic means1dis realized by a processor, a ROM (Read Only Memory) and a RAM (Random Access Memory) using semiconductors, or an integrated circuit designed specifically for that purpose, for example.

A specific example of processing by the image synthesizing apparatus1will next be described in detail.

In the present embodiment, image data in an ITU-601 format, which is referred to as a 4:2:2 format, is used as an example of data of an image to be displayed. Incidentally, ITU stands for the International Telecommunication Union.

FIG. 2is a schematic diagram of image data.

Picture element components of the image data in the 4:2:2 format are represented by luminance components Y and color difference components Cb and Cr, which are each formed by 8-bit data. The sampling frequency of the luminance component Y is 13.5 MHz, while the sampling frequency of the color difference components Cb and Cr is 6.75 MHz. Since of the color difference components Cb and Cr, one with a higher frequency component can be discarded, each picture element22in a display screen21is formed by a combination of two picture element components, that is, a luminance component Y and a color difference component Cb or a luminance component Y and a color difference component Cr, as shown inFIG. 2. Although the actual value of the luminance component Y ranges from 0 to 1 and the actual value of the color difference components Cb and Cr ranges from −0.5 to 0.5, data of these components is converted into 8-bit form in the 4:2:2 format, and it is specified that the luminance component Y, the color difference component Cb, and the color difference component Cr range from 16 to 235, from 16 to 240, and from 16 to 240, respectively.

According to the present invention, attention is directed to a specific component of the picture element components in setting the blending coefficient α used for the above-mentioned operation so that the blending coefficient α is set according to the value of the specific component. An example in which the luminance component Y is used as the specific component will be described in the following.

FIG. 3shows an example of screens to be synthesized.FIG. 4is an enlarged view of a menu screen31shown inFIG. 3.

In the example shown inFIG. 3, the menu screen31for setting various functions of the television receiver is superimposed on a television screen32selected for viewing by a TV tuner or the like. Specifically, the aforementioned first image1ainFIG. 1corresponds to the menu screen31to be superimposed and the second image1bcorresponds to the television screen32being viewed. The menu screen31and the television screen32are both in the 4:2:2 format, and thus are obtained from image data comprising the same number of picture elements, for example 720×480.

When the operation for synthesizing these images by a blending is performed, the blending coefficient α is set according to the value of the luminance component Y forming each picture element of the menu screen31to be superimposed. The luminance component Y of the menu screen31assumes values other than the specified values (16 to 235) of the luminance component Y to know the setting of the blending coefficient α.

For example, when the luminance component Y is zero, the blending coefficient α is set at zero, while the luminance component Y is other than zero, the blending coefficient α is set at unity. When the luminance component Y of a picture element is zero, operation on not only the luminance component Y but also the color difference components Cb and Cr of the picture element is performed by setting the blending coefficient α at zero. Similar operation is performed when the luminance component Y is other than zero. Such operation is performed on all picture elements.

InFIGS. 3 and 4, selection buttons31a,31b, and31cshown on the menu screen31for example represent a “sound selection” button for switching between a main-channel sound and a sub-channel sound, a “channel” button for setting the display number of a channel, and a “return” button for stopping the display of the menu screen, respectively.

Since monitor display for function setting is not necessary in a shaded portion31dofFIG. 4that does not include portions of the selection buttons31a,31b, and31c, the luminance components Y of picture elements in the shaded portion31dare set at zero. Thus, operation on all picture elements included in the shaded portion31dis performed by setting the blending coefficient α at zero, whereby only the background television screen32is displayed in a synthesized screen33shown inFIG. 3. Luminance components Y in the portions of the selection buttons31a,31b, and31chave numerical values set within a specified range of values that indicate normal image components. Thus, operation on picture elements in the portions of the selection buttons31a,31b, and31cis performed by setting the blending coefficient α at unity, whereby only the selection buttons31a,31b, and31care displayed on the synthesized screen33, as shown inFIG. 3.

It is to be noted that the blending coefficient α is not limited to zero and unity as set in the above example. For example, the blending coefficient α may be set at zero when the luminance component Y is zero, and the blending coefficient α may be set at 0.5 when the luminance component Y is other than zero. In this case, the portions of the selection buttons31a,31b, and31cwhen the luminance components Y are other than zero will be displayed on the synthesized screen33of the monitor as an image of the menu screen31being superimposed on the television screen32.

Thus, operation is performed by setting the blending coefficient α according to the value of the luminance component Y, thereby eliminating the need for a conventionally required CLUT in which a blending coefficient α is assigned to each picture element. Therefore, a memory for storing the CLUT having a large amount of data is not required, thereby reducing the scale and the power consumption of the circuit to be mounted in the apparatus. In addition, the stepwise setting of the blending coefficient α makes it possible to achieve a simpler configuration of a circuit for performing the operation and thereby increase processing rate in image synthesis.

Moreover, since the ITU-601 format employed in the example described above specifies that the value of the luminance component Y is 16 to 235, the value of the luminance component Y=0 is not included in a range of values that can be assumed by original image data. Therefore, the value of the luminance component Y=0 does not affect display of image data for an image having luminance components Y set at such values, that is, the menu screen31as shown inFIG. 3. Thus, the value of the luminance component Y for setting the blending coefficient α at zero may be any of 0 to 15.

Even in the case of image data in a different format from ITU-601, when a range of values assumed by the data is determined, the present invention can be carried out by assigning data having a value outside the range. According to the specifications of the ITU-601 format employed in the example described above, each picture element includes a luminance component Y without exception, and therefore a luminance component Y is used to know a setting value of the blending coefficient α. However, when image data in a different format includes a color difference component or a color component without exception, for example, the color difference component or the color component may be used for the above purpose.

Next,FIG. 5shows an example of the main configuration of an image synthesizing apparatus according to the present invention.

An image synthesizing apparatus1comprises a microcomputer11, an image generation control unit12, a memory13, a memory control unit14, an image generating unit15, an image synthesizing unit16, and an image output unit17. The image synthesizing apparatus1captures moving images generated by a moving image generating apparatus3, and synthesizes an image stored in the memory13with each of the moving images to output the result to a monitor2for display. The image synthesizing apparatus1is for example disposed inside a television receiver, in which case a television image receiving apparatus corresponds to the moving image generating apparatus3, and the image synthesizing apparatus1superimposes a menu image or the like generated by the microcomputer11upon a received television image to thereby display the result on the monitor2.

The microcomputer11is connected to the image generation control unit12and the memory control unit14, and controls the image generating unit15, the image synthesizing unit16, and the image output unit17via the image generation control unit12. The memory control unit14controls memory access by the microcomputer11and the image generating unit15, and also writes and reads various data to and from the memory13. Functions of the image generation control unit12, the memory control unit14, the image generating unit15, and the image synthesizing unit16are realized by control and processing by a processor, a ROM and a RAM that use semiconductors and the like, or an integrated apparatus designed specifically for that purpose. The memory13is a ROM and RAM using semiconductors and the like, or a hard disk formed by a magnetic disk or the like.

The microcomputer11generates encoded image data in which image data such as luminance components Y, color difference components Cb, and color difference components Cr is encoded, and then writes the encoded image data to the memory13via the memory control unit14. In generating the encoded image data, image data inputted from another image generating apparatus by an input unit not shown in the figure provided for the image synthesizing apparatus1is encoded into specified data form by control and processing by the microcomputer11to be stored in the memory13. The memory13also stores specified information such as a display position and a display size of image data to be displayed on the monitor2.

When processing for image synthesis begins, the encoded image data and information such as its display position and display size are supplied from the memory13to the image generating unit15via the memory control unit14. The image generating unit15decodes the encoded image data according to a control signal from the image generation control unit12to thereby generate decoded image data, and then outputs the decoded image data to the image synthesizing unit16as image data for display on the monitor2.

Under control of the image generation control unit12, the image synthesizing unit16reads the decoded image data from the image generating unit15and moving image data from the moving image generating apparatus3. The image synthesizing unit16sequentially reads picture element components that form picture elements from these pieces of data, and obtains the value of a luminance component Y from the picture element components of the decoded image data. The image synthesizing unit16sets a blending coefficient α corresponding to the value of the luminance component Y by using information on the relation between the luminance component Y and the blending coefficient α supplied from the image generation control unit12. Then the image synthesizing unit16performs α blending operation by using the blending coefficient α set for a picture element of the moving image data and the decoded image data that includes the luminance component Y, and thereafter outputs the result to the image output unit17. The information on the relation between the luminance component Y and the blending coefficient α may be preset in the image generation control unit12, or the relation between the luminance component Y and the blending coefficient α may be specified by the user and stored in the memory13or the like by an input unit not shown in the figure so that the relation between the luminance component Y and the blending coefficient α is read by the image synthesizing unit16.

The image output unit17is connected to the monitor2, and according to a control signal from the image generation control unit12, converts display image data of a synthesized image generated at the image synthesizing unit16into a data form for output that enables display of the data on a display screen of the monitor2.

FIG. 6is a flow chart of image synthesis processing by the image synthesizing unit16.

When image synthesis processing begins, encoded image data stored in the memory13is read to be decoded by the image generating unit15, and decoded image data for each picture element is supplied to the image synthesizing unit16(S61). At the same time, moving image data for each picture element at the same position, serving as a background of the image of the decoded image data, is supplied from the moving image generating apparatus3to the image synthesizing unit16(S62).

Next, the image generation control unit12supplies information on relation between the luminance component Y and the blending coefficient α of the decoded image data (S63). Under the control of the image generation control unit12, the luminance component Y is detected from picture element components included in picture elements of the read decoded image data, and is then subjected to comparison with the information on the relation between the luminance component Y and the blending coefficient α (S64). The value of the blending coefficient α corresponding to the detected luminance component Y is set according to the information (S65). Then, α blending operation is performed on each picture element component by using the set blending coefficient α (S66).

Synthesized image data obtained from the picture element components after the operation is outputted to the image output unit17(S67). When display of a synthesized image is to be continued (s68), a picture element at the next position is read from the decoded image data and the moving image data (S61), and thereafter the same operation is performed on picture element components of the picture element.

After operation on all the picture elements of one image is finished, the moving image generating apparatus3supplies the next piece of moving image data for operation on each picture element of the next image. Thus, a loop from S61to S67is repeated until display of a synthesized image is stopped and thereby image synthesis processing is finished.

It is to be noted that in the embodiment described above, the background image is a moving image and the image to be superimposed is a still image; however, these images may be either a moving image or a still image as long as the image to be superimposed has specific data written into the luminance components Y of picture elements that do not need to be displayed on the monitor. Thus, the present invention is applicable regardless of whether these images are moving images or still images.

As described above, when a specific component of a picture element that forms one image represents a predetermined value, the image synthesizing apparatus according to the present invention performs α blending operation on all the picture element components of the picture element by fixing the blending coefficient α at a specified value, thereby eliminating the need for setting the blending coefficient α for every picture element. Thus, the amount of information for the operation is reduced, and thereby it is possible to reduce required memory capacity and the load of image processing.

In addition, when a specific component of a picture element that forms one image represents a predetermined value, the image synthesizing apparatus according to the present invention performs α blending operation on all the picture element components of the picture element by fixing the blending coefficient α at a specified value, thereby eliminating the need for setting the blending coefficient α for every picture element. Thus, the amount of information for the operation is reduced, and thereby it is possible to reduce required memory capacity and the load of image processing.