Method and unit to process an image signal involving frame rate conversion to improve image quality

An image signal processing unit includes a frame rate conversion circuit performing double frame rate conversion on an input image signal from a first frame frequency to a second frame frequency. When performing frame rate conversion with the motion correction process, a motion vector is determined between a first frame image and a third frame image, and three interpolation frame images are formed through the motion correction process to the first frame image based on the motion vector, and are inserted between the first and third frame images so as to establish the second frame frequency. When performing frame rate conversion without the motion correction process, an interpolation frame image same as the first frame image is inserted between the first and second frame images, and an interpolation frame image same as the second frame image is inserted between the second and third frame images.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent Application No. JP 2008-203859 filed in the Japanese Patent Office on Aug. 7, 2008, the entire content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique to improve image quality in the case where, for example, a frame rate is converted twice in such a manner that a frame frequency is converted from 50 fps (frames per second) to 100 fps in a TV (television) receiver. In particular, the present invention relates to an image signal processing unit and a method of processing an image signal, which improve the image quality in the case where, for example, an image in which a normal signal (for example, a telop) with a frame rate of 50 fps is superimposed on a film signal with a frame rate converted from 25 fps to 50 fps is input.

2. Description of the Related Art

In the TV receiver, for example, when an image with a frame rate of 50 fps such as a PAL (phase alternating line) signal is displayed, there is a shortcoming that a whole screen flickers, that is, “plane-flicker disturbance” occurs in, for example, a CRT-based display apparatus. To solve this shortcoming, a frame rate conversion which solves the plane-flicker disturbance by setting the frame frequency high has been used. In this frame rate conversion, for example, the frame frequency is converted from 50 fps to 100 fps. Such a frame rate conversion is also used to improve blurring of a moving image in, for example, a display apparatus of a liquid crystal method.

On the other hand, for example, an image signal used for a movie shoot has the frame frequency lower than that of a typical TV signal, and is, for example, a film image signal with a frame rate of 25 fps. For this reason, a process called a television-cinema conversion (telecine conversion) in which a signal format of a film image signal is converted to that of the typical TV signal has been performed. As an example of the telecine conversion, for example, there is a 2-2 pull-down in which the film image signal with a frame rate of 25 fps is converted to the image signal of the PAL method with a frame rate of 50 fps.

FIG. 5indicates an example of a film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down.FIG. 5is an example of the film image10in which an object moves in a right direction in a screen. Although the TV signal has a frame rate of 50 fps, the image content is the a film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. Thus, each two successive frames in the TV signal have the same image content. Accordingly, in the case where the object moves in the screen, as indicated inFIG. 5, a first frame F1and a second frame F2have the same image content, and the object is in the same position. The position of the object moves in a third frame F3. The third frame F3and a fourth frame F4have the same image content, and the object is in the same position in the third frame F3and the fourth frame F4. The position of the object moves in a next fifth frame F5. In this manner, in the film image, since the image moves with a frame rate of 25 fps in the screen, there is a shortcoming that the motion of the image is not visually smooth. This unsmooth motion is called “judder”.

To solve this shortcoming, “a motion correction process” is added. In the motion correction process, a motion vector of an input image is obtained, and an image which is shifted in accordance with the amount of the obtained motion vector is inserted as an interpolation frame image, thereby realizing smooth motion. This process has been widely used.

Here, with reference toFIGS. 6A and 6B, operation to smooth the motion of the image will be described. In this case, the input image with a frame rate of 50 fps is the film image configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down, and the frame rate is converted from 50 fps to 100 fps by adding the motion correction process.FIG. 6Aindicates the input image before the frame rate conversion is performed, andFIG. 6Bindicates the output image after the frame rate conversion involving the motion correction process is performed. InFIGS. 6A and 6B, the vertical axis indicates time, and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 6Ais substantially the same as that of the film image10indicated inFIG. 5. Similarly toFIG. 5, in the image content, the object moves in the right direction. This input image is configured by converting the frame rate of the film image10from 25 fps to 50 fps through a 2-2 pull down. In the input image, each two successive frames have the same image content. Thus, when the object moves in the screen, as indicated inFIG. 6A, the image of the first frame F1and the image of the second frame F2are in the same position, and the object largely moves in the third frame F3. The fourth frame F4is displayed in the same position as that of the third frame F3. At this time, between the first frame F1and the third frame F3which is two frames after the first frame F1in the input image, the motion vector of the image is obtained by, for example, block matching method as a known method. The motion vector is obtained for each pixel unit, or for each block having a certain size. The obtained motion vector amount is regarded as A.

As indicated inFIG. 6B, in accordance with the obtained motion vector amount A, interpolation frame images are formed by shifting the image of the first frame F1in the input image by the amounts of A×¼, A× 2/4, and A×¾. These interpolation frame images obtained by shifting are displayed as a frame F1′, a frame F2, and a frame F2′, respectively, in the output image with a frame rate of 100 fps, and thereby smooth motion with a frame rate of 100 fps is realized.

FIG. 7indicates an example of the configuration of an image signal processing unit of the related art which performs the frame rate conversion as descried above. In the image signal processing unit, an input signal152is input from an input terminal151to a frame rate conversion circuit153. As indicated inFIGS. 5 and 6A, the input signal152is a film image signal configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down (as an image format, an image signal with a frame rate of 50 fps). The frame rate conversion circuit153includes a first frame memory154, a second frame memory155, and a third frame memory156. The frame memories154,155, and156are image memories each temporarily storing image data of one frame. The frame rate conversion circuit153also includes a film phase detection circuit160, a motion vector detection circuit161, a switch162, and a double rate conversion/image shift circuit166.

In the frame rate conversion circuit153, the image is delayed with the first frame memory154and the second frame memory155which are frame memories for two frames, and, by using the input signal152and a two-frame delayed signal158, a motion vector165between two frames is obtained with the motion vector detection circuit161by block matching method or the like. Moreover, by using the input signal152and a one-frame delayed signal157, a correction timing in accordance with a film phase of the input image is obtained with the film phase detection circuit160. The above-mentioned two-frame delayed signal158is further delayed by one frame with the third frame memory156, thereby obtaining a three-frame delayed signal159. The two-frame delayed signal158and the three-frame delayed signal159are supplied to the switch162. In the switch162, by inverting the switching operation for every one frame, the image of the odd-numbered frame in the input image is generally selected, and regarded as a selection signal163.

On the other hand, the obtained motion vector165and a correction timing signal164are input to the double rate conversion/image shift circuit166. By the operation of the switch162, the selection signal163in which the image of the odd-numbered frame in the input image is generally selected is input to the double rate conversion/image shift circuit166. In the double rate conversion/image shift circuit166, the frame rate of the image of the odd-numbered frame is converted so that the converted frame rate is twice that of the input signal, and then an image position is appropriately shifted in response to the motion vector165and the correction timing signal164. Thereby, the output signal167whose motion is improved to be smooth is obtained.

As described above, the image is appropriately moved when the motion vector is obtained and the frame rate is converted, and thereby the smooth motion is realized. Japanese Patent Publication Nos. 3596521 and 3855761 each discloses the invention regarding the image signal processing unit which performs such a frame rate conversion.

SUMMARY OF THE INVENTION

In TV images, there are some cases where an image such as a telop which is different from a source image is superimposed on the source image. In this case, even if the source image is a film image with a frame rate of 25 fps, the image which is superimposed on that source image is a normal signal with a frame rate of 50 fps.

FIG. 8indicates an example of an image in which a normal image11with a frame rate of 50 fps is superimposed on the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. In this example, a telop (normal image11) moving to the right with a frame rate of 50 fps is superimposed on a background screen (film image10) moving to the right with a frame rate of 25 fps in the screen. The background screen is the film image10which is the same as that in the example ofFIG. 5. In the background screen, the first frame F1and the second frame F2have the same image content, and the moving object is in the same position. After that, the image content changes in the third frame F3which is two frames after the first frame F1, and the object moves. On the other hand, the superimposed telop is displayed while the position of the telop moves in each frame with a frame rate of 50 fps.

Here, with reference toFIGS. 9 and 10, the description will be made for the process when the frame rate conversion involving the motion correction process is performed in the case where the image as inFIG. 8is input as an input signal.FIG. 9indicates the input image before the frame rate conversion is performed, andFIG. 10indicates the output image after the frame rate conversion involving the motion correction process is performed. InFIGS. 9 and 10, the vertical axis indicates time, and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 9is substantially the same as that of the image indicated inFIG. 8. Similarly toFIG. 8, the image content inFIG. 9is an image in which the normal image11with a frame rate of 50 fps is superimposed on the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. As indicated inFIG. 9, in the input image, between the first frame F1and the third frame F3which is two frames after the first frame F1, the motion vector of the image is obtained by, for example, block matching method as a known method. The motion vector of the image is obtained for each pixel unit, or for each block having a certain size. Here, the motion vector amount of the object in the film image portion with a frame rate of 25 fps is regarded as A. The motion vector amount of the superimposed telop portion with a frame rate of 50 fps is regarded as B.

As indicated inFIG. 10, in accordance with the obtained motion vector amount A, the film image portion in the image of the first frame F1in the input image is shifted by the amounts of A×¼, A× 2/4, and A×¾. In accordance with the obtained motion vector amount B, the telop portion in the image of the first frame F1in the input image is shifted by the amounts of B×¼, B× 2/4, and B×¾. The images in which the telop portions obtained by shifting are superimposed on the film image portions obtained by shifting, respectively, are displayed as the frame F1′, the frame F2, and the frame F2′ with a frame rate of 100 fps.

In this manner, even in the case where the image on which the normal image11with a frame rate of 50 fps is superimposed is input as the input image, the same process is performed as in the above-described case where the image configured with only the film image10is input as the input image (FIGS. 6A and 6B), and thereby it is realized that both of the film image portion and the normal image portion (telop) smoothly move with a frame rate of 100 fps.

Here, in the above-described frame rate conversion, to make the motion smooth (to improve judder), “the motion correction process” is necessary. Next, the case where “the motion correction process” is stopped will be described. For example, in a block matching circuit to obtain the motion vector, when the image moves fast to the degree that it is over a search range of the vector, the motion vector is not correctly obtained. Thus, there is a case where the motion correction process is incorrectly performed by using the incorrect vector. In this case, there is an issue that an output image is broken. To avoid this issue, in the case of the image moving fast, measures are taken in many cases such that the motion correction process is stopped, and the input image is output as it is with the original position and the original image content. Alternatively, there is a case where some moving-image modes are prepared in the TV receiver. In that case, the motion correction process is turned off when a mode which stops the motion correction process is selected according to user preference.

FIGS. 11A and 11Bindicate an example of the frame rate conversion in the case where the motion correction process is stopped.FIG. 11Aindicates the input image before the frame rate conversion is performed, andFIG. 11Bindicates the output image after the frame rate conversion is performed. InFIGS. 11A and 11B, the vertical axis indicates time and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 11Ais substantially the same as that of the film image10indicated inFIGS. 5 and 6A. The image content is the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. In the case where the film image10is input, since the mode is set so that the motion correction process is stopped, the output image after the frame rate conversion is performed is as indicated inFIG. 11B. That is, without shifting the image of the first frame F1, the interpolation frame images with the same position and the same image content as those of the first frame F1are formed as the frame F1′, the frame F2, and the frame F2′, and displayed with a frame rate of 100 fps. With such a process, although the judder of the image is not improved from a frame rate of 25 fps which is the same as that in the input image, it is possible to avoid the breaking of the output image which is caused by incorrectly performing the motion correction process through the use of the incorrect vector.

Next, with reference toFIGS. 12A and 12B, the description will be made for the case where the motion correction process is stopped and the frame rate conversion is performed when the image in which the normal image11with a frame rate of 50 fps is superimposed on the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down.FIG. 12Aindicates the input image before the frame rate conversion is performed, andFIG. 12Bindicates the output image after the frame rate conversion is performed. InFIGS. 12A and 12B, the vertical axis indicates time, and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 12Ais the same as that of the image indicated inFIGS. 8 and 9. In the case where the frame rate conversion with the mode which stops the motion correction process is performed to this input image, the image is as indicated inFIG. 12B. That is, without shifting the image of the first frame F1, the interpolation frame images in which the whole images (whole including the film image content portion and the telop portion) with the same position and the same image content as those of the first frame F1are formed as the frame F1′, the frame F2, and the frame F2′, and displayed with a frame rate of 100 fps. In this case, as indicated inFIG. 12B, the judder in the film image content portion still has a frame rate of 25 fps which is the same as that in the input image. On the other hand, in the telop portion, the input image has a frame rate of 50 fps, but the image in the output image has a frame rate corresponding to 25 fps which is the same as that in the film image10. This means that the judder is deteriorated in comparison with the original input image.

In this manner, in the relate art, in the case where the input image is the image in which the normal image and the film image are synthesized, there is an issue that, when the frame rate conversion is performed with the mode which stops the motion correction process, the judder in the normal image portion is deteriorated. In Japanese Patent Publication Nos. 3596521 and 3855761, such an issue is not considered.

In view of the foregoing, it is desirable to provide an image signal processing unit and a method of processing an image signal, which improve image quality after a frame rate conversion is performed in the case where an input signal is an image signal in which, for example, a normal image with a frame rate of 50 fps is superimposed on a film image configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down.

According to an embodiment of the present invention, there is provided an image signal processing unit including: a frame rate conversion circuit converting a frame rate of an input image signal from a first frame frequency to a second frame frequency which is twice the first frame frequency, and selecting whether a motion correction process is performed or not at the time of the frame rate conversion. In a case of performing frame rate conversion with the motion correction process, a motion vector is determined between a first frame image and a third frame image in the input image signal, followed by forming three interpolation frame images through performing the motion correction process to the first frame image based on the motion vector, and then the three interpolation frame images are inserted between the first frame image and the third frame image so as to establish the second frame frequency. In a case of performing frame rate conversion without the motion correction process, an interpolation frame image having the same content as the first frame image is formed and inserted between the first frame image and a second frame image in the input image signal, and an interpolation frame image having the same content as the second frame image is formed and inserted between the second frame image and the third frame image.

According to an embodiment of the present invention, there is provided a method of processing an image signal including: a step of converting a frame rate of an input image signal from a first frame frequency to a second frame frequency which is twice the first frame frequency; and selecting whether a motion correction process is performed or not at the time of the frame rate conversion, in the step of converting the frame rate. In a case of performing frame rate conversion with the motion correction process, a motion vector is determined between a first frame image and a third frame image in the input image signal, followed by forming three interpolation frame images through performing the motion correction process to the first frame image based on the motion vector, and then the three interpolation frame images are inserted between the first frame image and the third frame image so as to establish the second frame frequency. In a case of performing frame rate conversion without the motion correction process, an interpolation frame image having the same content as the first frame image is formed and inserted between the first frame image and a second frame image in the input image signal, and an interpolation frame image having the same content as the second frame image is formed and inserted between the second frame image and the third frame image.

In the image signal processing unit or the method of processing the image signal according to the embodiment of the present invention, a frame rate of an input image signal is converted from a first frame frequency to a second frame frequency which is twice the first frame frequency. At the time of the frame rate conversion, in a case of performing frame rate conversion with the motion correction process, a motion vector is determined between a first frame image and a third frame image in the input image signal, followed by forming three interpolation frame images through performing the motion correction process to the first frame image based on the motion vector, and then the three interpolation frame images are inserted between the first frame image and the third frame image so as to establish the second frame frequency. On the other hand, in a case of performing frame rate conversion without the motion correction process, an interpolation frame image having the same content as the first frame image is formed and inserted between the first frame image and a second frame image in the input image signal, and an interpolation frame image having the same content as the second frame image is formed and inserted between the second frame image and the third frame image.

That is, according to the embodiment of the present invention, in the case where the motion correction process is performed, the first frame to the fourth frame with the second frame frequency after the frame rate conversion is performed are the images based on the first frame image in the input image signal. On the other hand, in the case where the motion correction process is not performed, the first frame to the fourth frame with the second frame frequency are the images based on not only the first frame image, but also the second frame image in the input image signal. Thereby, in the case where the motion correction process is not performed, the image quality after the frame rate conversion is performed is improved in comparison with the case where, for example, the frame rate conversion is performed in such a manner that the three interpolation frame images having the same content as the first frame image in the input image signal are formed, and inserted between the first frame image and the third frame image in the input image signal with the second frame frequency. In particular, in the case where the input image is the image in which the normal image is superimposed on the film image, deterioration of the judder in the normal image portion is improved.

In the image signal processing unit or the method of processing the image signal according to the embodiment of the present invention, in the case where the frame rate conversion is performed, each signal is optimally processed according to whether the motion correction process is performed or not. Thus, it is possible to improve the image quality after the frame rate conversion is performed in the case where, for example, the input image signal is the image signal in which the normal image with a frame rate of 50 fps is superimposed on the film image with a frame rate of 25 fps. In particular, in the case where the motion correction process is not performed, deterioration of the judder in the normal image portion is improved.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1indicates an example of the configuration of an image signal processing unit according to an embodiment of the present invention. The image signal processing unit includes an input terminal51to which an input signal52is input, and a frame rate conversion circuit53which performs a frame rate conversion to the input signal52. The image signal processing unit also includes a switching control section70instructing whether the motion correction process is performed or not at the time of the frame rate conversion in the frame rate conversion circuit53.

The frame rate conversion circuit53performs the frame rate conversion so that, in the image signal as being the input signal52, a first frame frequency (for example, 50 fps) is converted to a second frame frequency (for example, 100 fps) which is twice the first frame frequency. Then, the frame rate conversion circuit53outputs an output signal67as a result of the frame rate conversion. The frame rate conversion circuit53selects whether the motion correction process is performed or not at the time of the frame rate conversion.

The input signal52is an image signal in which, for example, each two successive frames with the first frame frequency have the same image content. As an example of such an image signal,FIGS. 5 and 6Aindicate a film image configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down (as an image format, an image signal with a frame rate of 50 fps). The input signal52may be an image signal in which a first image portion and a second image portion are mixed. The first image portion is configured with an image in which each two successive frames have the same image content with the first frame frequency and the second image portion is configured with a moving image with the same frequency as the first frame frequency. As an example of such an image signal,FIGS. 8 and 9indicate an image signal in which a normal image11(second image portion) with a frame rate of 50 fps is superimposed on a film image10(first image portion) configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down.

The frame rate conversion circuit53includes a first frame memory54, a second frame memory55, and a third frame memory56. The frame rate conversion circuit53also includes a film phase detection circuit60, a motion vector detection circuit61, a first switch62, a second switch68, and a double rate conversion/image shift circuit66.

The first frame memory54, the second frame memory55, and the third frame memory56are connected in series, and each of the first frame memory54, the second frame memory55, and the third frame memory56is an image memory temporarily storing image data of one frame. The input signal52is input to the first frame memory54, and the first frame memory54outputs a one-frame delayed signal57which is one frame delayed from the input signal52. The one-frame delayed signal57output from the first frame memory54is input to the second frame memory55, and the second frame memory55outputs a two-frame delayed signal58which is two frames delayed from the input signal52. The two-frame delayed signal58output from the second frame memory55is input to the third frame memory56, and the third frame memory56outputs a three-frame delayed signal59which is three frames delayed from the input signal52.

The input signal52and the one-frame delayed signal57are input to the film phase detection circuit60. On the basis of the input signal52and the one-frame delayed signal57, the film phase detection circuit60forms a correction timing signal64which takes timing of the image correction at the time of performing the frame rate conversion, and outputs the correction timing signal64to the double rate conversion/image shift circuit66. The input signal52and the two-frame delayed signal58are input to the motion vector detection circuit61. On the basis of the input signal52and the two-frame delayed signal58, the motion vector detection circuit61obtains a motion vector65which is used at the time of performing the motion correction process, and outputs the motion vector65to the double rate conversion/image shift circuit66.

The two-frame delayed signal58and the three-frame delayed signal59are supplied to the first switch62. The first switch62generally selects an image of an odd-numbered frame in the input image signal by inverting a switching operation for every one frame, and outputs the image as a first selection signal63.

The two-frame delayed signal58and the first selection signal63from the first switch62are supplied to the second switch68. The second switch68is switchable according to whether the motion correction process is performed or not in the double rate conversion/image shift circuit66. In the case where the motion correction process is performed, the second switch68outputs the first selection signal63output from the first switch62as it is to the double rate conversion/image shift circuit66, as a second selection signal69. In the case where the motion correction process is stopped, the second switch68outputs the two-frame delayed signal58as the second selection signal69to the double rate conversion/image shift circuit66.

The double rate conversion/image shift circuit66performs the frame rate conversion to the input signal52on the basis of the correction timing signal64output from the film phase detection circuit60, the motion vector65output from the motion vector detection circuit61, and the second selection signal69output from the second switch68, and outputs an output signal67which has the frame frequency twice that of the input signal52. Like a specific example which will be described later, the double rate conversion/image shift circuit66performs an optimal frame rate conversion according to whether the motion correction process is performed or not.

The switching control section70performs a switching control of the second switch68according to whether the motion correction process is performed or not. The switching control section70also instructs to the double rate conversion/image shift circuit66whether the motion correction process is performed or not at the time of the frame rate conversion. According to user preference, the determination whether the motion correction process is performed or not at the time of performing the frame rate conversion may be selectable through, for example, an operation section not illustrated in the figure. In this case, the switching control section70determines whether the motion correction process is performed or not on the basis of the instruction from the user, and appropriately instructs to the second switch68and the double rate conversion/image shift circuit66. In the case of an image particularly moving fast, there are many cases that the motion correction process is incorrectly performed, and the output image is broken. Thus, such an image moving fast is automatically detected in the frame rate conversion circuit53, and the motion correction process may be automatically turned on or off. In this case, it is possible that the determination whether it is an image moving fast or not is performed, for example, on the basis of the motion vector detected in the motion vector detection circuit61.

In the embodiment, the frame rate conversion circuit53corresponds to a specific example of “a frame rate conversion circuit” in the present invention. The first frame memory54, the second frame memory55, and the third frame memory56each corresponds to a specific example of “an image memory” in the present invention. The first switch62corresponds to a specific example of “a first switch” in the present invention, and the second switch68corresponds to a specific example of “a second switch” in the present invention. The double rate conversion/image shift circuit66corresponds to a specific example of “a conversion circuit” in the present invention.

Next, operation of the image signal processing unit according to the embodiment will be described.

In the image signal processing unit, the input signal52is input from the input terminal51to the frame rate conversion circuit53. In the frame rate conversion circuit53, the image is delayed with the frame memories54and55which are frame memories for two frames, and, by using the input signal52and the two-frame delayed signal58, the motion vector65between two frames is obtained in the motion vector detection circuit61by block matching method or the like. By using the input signal52and the one-frame delayed signal57, a correction timing in accordance with a film phase of the input image is obtained in the film phase detection circuit60. Moreover, the above-mentioned two-frame delayed signal58is further delayed by one frame in the third frame memory56, thereby obtaining the three-frame delayed signal59. The two-frame delayed signal58and the three-frame delayed signal59are supplied to the first switch62. In the first switch62, a switching operation is inverted for every one frame. Thereby, the image of the odd-numbered frame in the input image (input signal52) is generally selected, and regarded as the first selection signal63. The two-frame delayed signal58and the first selection signal63from the first switch62is supplied to the second switch68. The second switch68is switchable according to whether the motion correction process is performed or not in the double rate conversion/image shift circuit66.

Here, in the case where the motion correction process is performed, the first selection signal63from the first switch62is output as it is. That is, the image signal of the odd-numbered frame in the input image is output as the second selection signal69to the double rate conversion/image shift circuit66. Moreover, the motion vector65and the correction timing signal64are input to the double rate conversion/image shift circuit66. In the double rate conversion/image shift circuit66, the frame rate of the image of the odd-numbered frame is converted so that the converted frame rate is twice that of the input signal52. Then, like a specific example which will be described later, the image position is suitably and appropriately shifted in response to the motion vector65and the correction timing signal64. Thereby, the output signal67whose motion is improved to be smooth is obtained.

In the case where the motion correction process is not performed, the two-frame delayed signal58is output as the second selection signal69to the double rate conversion/image shift circuit66. In the double rate conversion/image shift circuit66, like a specific example which will be described later, the frame rate of the two-frame delayed signal58is appropriately converted so that the converted frame rate is twice that of the input signal52. Thereby, the output signal67in which deterioration of the judder is prevented in comparison with the case of the related art is obtained.

Next, a specific example of the frame rate conversion in the case where the motion correction process is performed and in the case where the motion correction process is not performed will be described.

FIG. 8indicates an example of an image in which the normal image11with a frame rate of 50 fps is superimposed on the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. In this example, the telop (normal image11) moving to the right with a frame rate of 50 fps is superimposed on the background screen (film image10) moving to the right with a frame rate of 25 fps in the screen. The background screen is the film image10as in the example ofFIG. 5. The first frame F1and the second frame F2have the same image content, and the moving object is in the same position. After that, the image content changes in the third frame F3which is two frames after the first frame F1, and the object moves. On the other hand, the superimposed telop is displayed while the position of the telop moves in each frame with a frame rate of 50 fps.

With reference toFIGS. 9 and 10, the description will be made for the process when the frame rate conversion involving the motion correction process is performed in the case where the image as inFIG. 8is input as the input signal52.FIG. 9indicates the input image before the frame rate conversion is performed, andFIG. 10indicates the output image after the frame rate conversion involving the motion correction process is performed. InFIGS. 9 and 10, the vertical axis indicates time, and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 9is substantially the same as that of the image indicated inFIG. 8. Similarly toFIG. 8, the image content inFIG. 9is an image in which the normal image11with a frame rate of 50 fps is superimposed on the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. In the motion vector detection circuit61, as indicated inFIG. 9, in the input image, between the first frame F1and the third frame F3which is two frames after the first frame F1, the motion vector of the image is obtained by, for example, block matching method as a known method. The motion vector of the image is obtained for each pixel unit, or for each block having a certain size. Here, the motion vector amount of the object in the film image portion with a frame rate of 25 fps is regarded as A. The motion vector amount in the superimposed telop portion with a frame rate of 50 fps is regarded as B.

In the double rate conversion/image shift circuit66, as indicated inFIG. 10, in accordance with the obtained motion vector amount A, the film image portion in the image of the first frame F1in the input image is shifted by the amounts of A×¼, A× 2/4, and A×¾. In accordance with the obtained motion vector amount B, the telop portion in the image of the first frame F1in the input image is shifted by the amounts of B×¼, B× 2/4, and B×¾. The images in which the telop portions obtained by shifting are superimposed on the film image portions obtained by shifting, respectively, are inserted as the frame F1′, the frame F2, and the frame F2′ between the first frame F1and the third frame F3in the input image with a frame rate of 100 fps.

In this manner, even in the case where the image on which the normal image11with a frame rate of 50 fps is superimposed is input as the input image, the same process is performed as in the above-described case where the image configured with only the film image10is input as the input image (FIGS. 6A and 6B), and thereby it is realized that both of the film image portion and the normal image portion (telop) smoothly move with a frame rate of 100 fps.

Next, with reference toFIGS. 2A and 2B, the description will be made for the case where the frame rate conversion is performed while the motion correction process is stopped when the image in which the normal image11with a frame rate of 50 fps is superimposed on the 2-2 pull-down film image10with a frame rate of 25 fps is input as the input image52.FIG. 2Aindicates the input image before the frame rate conversion is performed, andFIG. 2Bindicates the output image after the frame rate conversion is performed. InFIGS. 2A and 2B, the vertical axis indicates time, and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 2Ais the same as that of the image indicated inFIGS. 8 and 9. In the case where the frame rate conversion with the mode which stops the motion correction process is performed to this input image, the image is as indicated inFIG. 2B. That is, in the double rate conversion/image shift circuit66, without shifting the image of the first frame F1, the interpolation frame image in which the whole image (whole including the film image content portion and the telop portion) with the same position and the same image content as those of the first frame F1is formed as the frame F1′, and inserted between the first frame F1and the second frame F2in the input image with a frame rate of 100 fps. Similarly, in the case of the second frame F2, the interpolation frame image in which the whole image (whole including the film image content portion and the telop portion) with the same position and the same image content as those of the second frame F2is formed as the frame F2′, and inserted between the second frame F2and the third frame F3in the input image with a frame rate of 100 fps. The same process is performed for the subsequent frames. With such a process, as indicated inFIG. 2B, the judder in the film image content portion still has a frame rate of 25 fps which is the same as that in the input image. The judder in the telop portion also still has a frame rate of 50 fps which is the same as that in the input image. Thereby, deterioration of the judder is prevented.

That is, in the embodiment, in the case where the motion correction process is performed, the first frame to the fourth frame F1, F1′, F2, and F2′ with the second frame frequency (frame rate of 100 fps) after the frame rate conversion is performed are the images based on the first frame F1in the input image signal. On the other hand, in the case where the motion correction process is not performed, the first frame to the fourth frame F1, F1′, F2, and F2′ with the second frame frequency are the images based on not only the first frame F1, but also the second frame F2in the input image signal. Thereby, in the case where the motion correction process is not performed, the image quality after the frame rate conversion is performed is improved in comparison with the case where, for example, the frame rate conversion is performed in such a manner that the interpolation frame images of three frames with the same content as that of the first frame F1in the input image signal are formed, and inserted between the first frame F1and the third frame F3in the input image signal with the second frame frequency (refer toFIGS. 12A and 12B). In particular, in the case where the input image is the image in which the normal image is superimposed on the film image, deterioration of the judder in the normal image portion is improved.

In the specific example described above, the description is made for the image signal as being the input signal52in which the normal image11with a frame rate of 50 fps is superimposed on the film image10configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. However, the same process is performed in the case where the input signal52is configured with only the film image10.

Here, as a comparative example to the specific example where the motion correction process is performed as indicated inFIGS. 9 and 10, the case where the process is performed as indicated inFIGS. 4A and 4Bis considered.FIG. 4Aindicates the input image before the frame rate conversion is performed, andFIG. 4Bindicates the output image after the frame rate conversion is performed. InFIGS. 4A and 4B, the vertical axis indicates time, and the horizontal axis indicates a position of the image in the horizontal direction.

The content of the input image indicated inFIG. 4Ais the same as that of the image indicated inFIGS. 8 and 9. When the motion correction process is performed as indicated inFIG. 4B, it is assumed that the frames F1and F1′ in the output image are formed on the basis of the first frame F1in the input image, and the frames F2and F2′ in the output image are formed on the basis of the second frame F2in the input image. More specifically, the film image portion in the image of the first frame F1in the input image is shifted by the amount of A×¼ in accordance with the motion vector amount A, and the telop portion is shifted by the amount of B×¼ in accordance with the motion vector amount B, thereby forming the frame F1′ in the output image. Moreover, the film image portion in the image of the second frame F2in the input image is shifted by the amounts of A× 2/4 and A×¾ in accordance with the motion vector amount A, and the telop portion is shifted by the amounts of B× 2/4 and B×¾ in accordance with the motion vector amount B, thereby forming the frames F2and F2′ in the output image.

In the case of such a method, as indicated inFIG. 4B, the smooth output image with a frame rate of 100 fps is obtained in the film image portion. However, the image with unsmooth motion is obtained in the normal image portion (telop). Accordingly, irrespective of whether the motion correction process is performed or not, when the frame F1′ in the output image is formed on the basis of the first frame F1in the input image, and the frames F2and F2′ in the output image are formed on the basis of the second frame F2in the input image, the image quality is generally deteriorated. As in the embodiment, according to whether the motion correction process is performed or not, it is necessary to appropriately switch the frame which is used as the basis when forming the interpolation images. That is, when the motion correction process is performed, all of the frames F1′, F2, and F2′ in the output image are formed on the basis of the first frame F1in the input image. When the motion correction process is not performed, the frame F1′ in the output image is formed on the basis of the first frame F1in the input image, and the frames F2and F2′ in the output image are formed on the basis of the second frame F2in the input image. In this manner, it is necessary to appropriately switch the frames used as the basis.

FIG. 3indicates a method of the phase detection to the film image signal configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. In the embodiment, when the motion correction process is performed, for example, as indicated inFIG. 9, it is necessary to obtain the motion vector between the first frame F1and the third frame F3which is two frames after the first frame F1in the input image. In this case, although the signal format of the input signal52is a frame rate of 50 fps, the image signal which is the base of the input signal52has a frame rate of 25 fps. Thus, it is necessary to know which frame in the input image with a frame rate of 50 fps corresponds to which phase of the film image with a frame rate of 25 fps. In the film phase detection circuit60, to perform this film phase detection, it is utilized that each two successive frames in the film image signal configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down have the same image content. That is, the first frame F1and the second frame F2have the same content, and the third frame F3and the fourth frame F4have the same content. Similarly, in the subsequent frames, it is utilized that each two successive frames have the same image content, and the same image content is displayed twice.

As inFIG. 3, when the difference between the two frames immediately adjacent to each other in the input image signal is obtained, a change in the film phase is detected in a two-frame cycle as follows,

the difference between the first frame F1and the second frame F2is zero

the difference between the second frame F2and the third frame F3is not zero

the difference between the third frame F3and the fourth frame F4is zero

the difference between the fourth frame F4and the fifth frame F5is not zero

By utilizing this, it is possible to detect the film phase in the two-frame cycle.

As described above, in the image signal processing unit according to the embodiment, in the case where the frame rate conversion is performed, each signal is optimally processed according to whether the motion correction process is performed or not. Thus, it is possible to improve the image quality after the frame rate conversion is performed in the case where, for example, the input image signal is the image signal in which the normal image with a frame rate of 50 fps is superimposed on the film image configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down. In particular, in the case where the motion correction process is not performed, deterioration of the judder in the normal image portion is improved.

The present invention is not limited to the embodiment, and various modifications may be made.

For example, the format of the input signal applied in the present invention is not limited to the film image signal configured by converting the frame rate from 25 fps to 50 fps through a 2-2 pull down, in the TV signal with a frame rate of 50 fps, which is described in the embodiment. For example, the present invention is also applicable to the case where the content of the image is a CG (computer graphics) image with a frame rate of 30 fps in the TV signal with a frame rate of 60 fps. Moreover, it is obvious that the present invention is applicable to not only the TV receiver, but also a signal converter or the like to which the TV receiver is connected.