METHOD FOR CONVERTING FRAME RATE AND VIDEO PROCESSING APPARATUS USING THE SAME

A method for converting the frame rate of a video signal includes a film footage detection step of detecting a M:N (M and N are integers) pulldown video signal from an input video signal; a frame rate conversion step of performing frame rate conversion for a series of frames of the input video signal; and a switching control step of, if the M:N pulldown video signal is detected based on a detection result obtained at the film footage detection step, switching, at the frame rate conversion step, the frame rate conversion for the series of frames to frame rate conversion for M:N pulldown, and controlling the switching at a frame in which there is no difference between time required for the frame rate conversion for the series of frames and time required for the frame rate conversion for M:N pulldown.

DETAILED DESCRIPTION

First Embodiment

FIG. 1is a block diagram illustrating a frame rate conversion method of a first embodiment of the present disclosure.

Referring toFIG. 1, a video signal5100input from an input unit100is input to a frame rate conversion unit101, and then is input to a storage102using a memory. Depending on output of a switching controller107, a memory controller103reads the followings: frame data S102A, S102B required for a motion vector detector104; frame data S102C, S102D required for an interpolation frame generator105; and frame data S102E, S102F required for a film footage detector106. A video signal S105having a frame rate converted by the interpolation frame generator (frame rate converter)105is output from an output unit108.

FIG. 9is a timing chart illustrating video processing in the case where a 4:4 pulldown video signal is input in the first embodiment.FIG. 10is a timing chart illustrating film footage processing in the case where a 4:4 pulldown video signal is input in the first embodiment.FIGS. 11A-11Dare diagrams of interpolation frames, and illustrate switching in the case where a 4:4 pulldown video signal is input in the first embodiment.

An operation of the first embodiment will be described with reference toFIGS. 1,9,10, and11A-11D. For example, a video signal5100changing every four frames in a pattern of A, A, A, A, B, B, B, B, . . . is input as illustrated inFIG. 9. A first read signal containing the frame data S102C and a second read signal containing the frame data S102D are read from the storage102. Comparison between the frame data S102C contained in the first read signal and the frame data S102D contained in the second read signal results as follows. Of first eight frames of the first read signal, six frames contain the same image signals as that contained in corresponding six frames of the second read signal. Thus, a motion vector is “0” during a six-frame period of an output video signal, and no interpolation frame is generated. On the other hand, an interpolation frame is generated by using the motion vector between two frames of the first and second read signals having different image signals. However, since frames having the same image signal are repeated, smooth movement of a video picture corresponding to first eight frames of the output video signal cannot be realized, resulting in juddering of the video picture.

For the foregoing reasons, in the present embodiment, the film footage detector106detects, based on the frame data S102E, S102F output from the storage102, the number of repeat frames and detects whether or not a M:N (“M” and “N” are integers) pulldown signal is input. Then, the film footage detector106outputs a detected video pattern signal S106B. If the M:N pulldown video signal is detected, the film footage detector106outputs a frame phase signal S106A corresponding to a frame phase per cycle of the pulldown signal. The present embodiment describes one example where a 4:4 pulldown video signal is input. Upon detection of 4:4 pulldown, a signal with a repetition of values 0, 1, 2, 3 is output as the frame phase signal S106A corresponding to the frame phase per cycle. In order to read, based on the frame phase signal S106A and the video pattern signal S106B output from the film footage detector106, frame data required for each video processing pattern in the switching controller107, the memory controller103is switched to switch frame data to be output to the motion vector detector104and the interpolation frame generator105based on an output pulldown signal S107A.

FIG. 10is a timing chart in the case where a 4:4 pulldown video signal is detected. A second read signal containing frame data S102D is, corresponding to pulldown processing, read such that the input video signal is delayed by four frames. Moreover, the switching controller107outputs a signal S107B indicating that a video pattern is switched to the motion vector detector104, and then switching and setting of the motion vector detector104are performed. The motion vector detector104outputs a motion vector signal S104indicating motion from the frame containing the image signal A to the frame containing the image signal B. Moreover, the switching controller107outputs a signal107C indicating the video pattern to the interpolation frame generator105. Upon generation of an interpolation frame(s), switching of the frame interpolation ratio of the motion vector signal S104and setting of the interpolation frame generator105are performed. Frame rate conversion corresponding to the detected video pattern is performed in the interpolation frame generator105, and a video signal having the converted frame rate is output from the output unit108.

Switching timing in an operation of the switching controller107will be described with reference toFIGS. 8A-8Fand11A-11D. In the case of switching at an N+1th frame (seeFIG. 8D), switching at an N+2th frame (seeFIG. 8E), and switching at an N+3th frame (seeFIG. 8F), reversing of video data occurs, resulting in unnatural movement of an object(s) at the switching point. Thus, there is a disadvantage that continuity of object movement is not maintained and reversing becomes noticeable. For such a reason, in the case where image signals indicating a series of movement are input at the Nth, N+1th, N+2th, and N+3th frames, even if the film footage detector106determines that a M:N pulldown signal is input at the N+1th, N+2th, and N+3th frames, the processing is not switched to processing for a pulldown video picture. Since the same processing as that at the Nth frame is performed at the frame subsequent to the N+3th frame, the processing is switched only at the Nth frame. Control is performed such that the processing is switched when the frame phase signal S106A having a predetermined value is output from the film footage detector106.FIG. 11Aillustrates a 4:4 pulldown signal changing every four frames is input.FIG. 11Billustrates an output video signal with consecutive frames for which frame rate conversion is performed.FIG. 11Cillustrates an output video signal for which 4:4 pulldown is performed and then frame rate conversion is performed.FIG. 11Dillustrates that a pulldown signal is detected and switching is performed fromFIG. 11BtoFIG. 11Cat an Nth frame. No switching is performed after detection of a pulldown signal until a frame phase signal having a value of “3” is generated. When a pulldown signal is detected and a frame phase signal having a predetermined value is generated, the processing switched to pulldown processing in frame rate conversion. In the case where the number of repeat frames is “K (an integer),” a value for frame phase signal changes in a pattern of 0, 1, . . . , (K−1), and the predetermined value is generated every K-multiple frames. The predetermined value is a value corresponding to a frame in which a difference in time of movement of an object(s) between frame rate conversion for a series of frames and frame rate conversion for pulldown processing is zero or small. It has been described that switching is performed at the frame phase corresponding to one of the K repeat frames. However, if little impact such as reversing of movement of an object(s) is on a video picture, the processing may be switched at plural points. In the case of a M:N pulldown signal, since the value for frame phase signal changes in a pattern of 0, 1, . . . , (M+N−1), a predetermined value is generated every M+N-multiple frames. The predetermined value is a value corresponding to a frame in which a difference in time of movement of an object(s) between frame rate conversion for a series of frames and frame rate conversion for pulldown processing is zero or small. It has been described that the processing is switched at the frame phase corresponding to one of the (M+N) repeat frames. However, if little impact such as reversing of movement of an object(s) is on a video picture, the processing may be switched at plural points, or a different value for frame phase signal may be, in the case of a M:N pulldown signal, assigned to each repetition of the frame phase values. In the switching controller107, switching is performed depending on the frame phase. Alternatively, the number of determinations as detecting a pulldown signal may be set. That is, switching may be performed based on a threshold value for the number of determinations which is set at, e.g., multiples of K or multiples of (M+N).

(First Configuration Example of Film Footage Detector)

Next, a first configuration example of the film footage detector106of the first embodiment will be described with reference toFIGS. 2 and 3.

FIG. 2is a block diagram illustrating the first configuration example of the film footage detector106in the frame rate conversion method of the first embodiment.

Referring toFIG. 2, frame data S102E, S102F read with a frame delay from the storage102using the memory is input to a frame difference extraction unit201, and frame difference values S201are obtained. The frame difference values S201are accumulated for one frame period in an integration unit202, and a cumulative integral value S202is obtained. Then, the integral value S202is compared with a threshold value preset in a comparison unit203. If the integral value S202is less than the threshold value, a frame correlation signal S203indicating that a frame of the frame data S102E contains the same data as that contained in a corresponding one of frames of the frame data S102F is output. If the integral value S202is greater than the threshold value, a frame correlation signal S203indicating that a frame of the frame data S102E contains different data from that contained in a corresponding one of frames of the frame data S102F is output. In a film footage detection unit204, the number of repeat frames is determined based on the frame correlation signal S203. Moreover, it is also determined, based on the frame correlation signal S203, whether or not a M:N pulldown signal is input. Meanwhile, in a phase extraction unit205, a frame phase signal S106A is output based on a determination result S204and the frame correlation signal S203. For example, if a video picture changing every four frames is input, a frame correlation signal S203has values of 0, 1, 1, 1 . . . where “0” represents that there is no correlation and “1” represents that there is a correlation. Since the number of repeat frames is determined by the comparison with the preset correlation pattern, video patterns for various pulldown signals and various numbers of repeat frames can be detected.

(Second Configuration Example of Film Footage Detector)

FIG. 3is a block diagram illustrating a second configuration example of the film footage detector106in the frame rate conversion method of the first embodiment.

In addition to frame data S102E, S102F read with frame delay from the storage102using the memory, frame data S102G is input as illustrated inFIG. 3. Each of these types of frame data S102E, S102F, S102G is input to both of frame difference extraction units301,304, and frame difference values S301, S304are obtained. The frame difference values S301are accumulated for one frame period in an integration unit302, and the frame difference values S304are accumulated for one frame period in an integration unit305. As a result, cumulative integral values S302, S305are obtained. Then, each of the integral values S302, S305is compared with a corresponding one of threshold values preset in comparison units303,306. If the integral value S302, S305is less than the threshold value, a frame correlation signal S303, S306indicating that frames of the frame data S102E, S102F, S102G corresponding to each other are the same as each other in data contained therein is output. If the integral value S302, S305is greater than the threshold value, a frame correlation signal S303, S306indicating that frames of the frame data S102E, S102F, S102G corresponding to each other are different from each other in data contained therein is output.

In a film footage detection unit307, the number of repeat frames is determined based on the frame correlation signals S303,306. Moreover, it is also determined, based on the frame correlation signals S303,306, whether or not a M:N pulldown signal is input. Meanwhile, in a phase extraction unit308, a frame phase signal S106A is output based on a determination result S307and the frame correlation signals S303, S306. Since the number of repeat frames is determined by the comparison with the preset correlation pattern, video patterns for various pulldown signals and various numbers of repeat frames can be detected.

InFIGS. 2 and 3, it has been described that two or three types of frame data are input. However, the present disclosure is not limited to such a configuration. Four or more types of frame data may be processed to detect video patterns for various pulldown signals and various numbers of repeat frames.

FIG. 12is a first flowchart for frame rate conversion of the present disclosure, and a process therefor will be described with reference toFIG. 12.

It is determined, by film footage detection, whether or not a video signal corresponding to an input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed.

On the other hand, if the video signal is the M:N pulldown video signal, a frame phase is determined, and it is determined whether or not the determined frame phase is a predetermined phase. If the frame phase is not the predetermined phase, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. If the frame phase is the predetermined phase, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for M:N pulldown is performed. During frame rate conversion for pulldown processing, film footage detection is performed for each frame. If pulldown video signals are processed in the same pulldown pattern, frame rate conversion for pulldown processing continues. If pulldown video signals are not processed in the same pulldown pattern, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. If frame rate conversion for a series of frames is performed, the process returns to “START” at a subsequent frame, and the process is repeated.

FIG. 13is a second flowchart for frame rate conversion of the present disclosure, and a process therefor will be described with reference toFIG. 13.

It is determined, by M:N pulldown determination, whether or not a video signal corresponding to an input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed.

On the other hand, if the video signal is the M:N pulldown video signal, it is determined, by scroll detection, whether or not the video picture is a scroll video picture in which a displayed object(s) moves in a uniform manner or in the same direction across the entirety of a screen. If the video picture is not the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. On the other hand, if the video picture is the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for M:N pulldown is performed.

FIG. 14is a third flowchart for frame rate conversion of the present disclosure, and a process therefor will be described with reference toFIG. 14.

It is determined whether or not an input video picture is a scroll video picture in which a displayed object(s) moves in a uniform manner or in the same direction across the entirety of a screen. If the video picture is not the scroll video picture, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. On the other hand, if the video picture is the scroll video picture, it is determined, by M:N pulldown determination, whether or not a video signal corresponding to the input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed. If the video signal is the M:N pulldown signal, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for M:N pulldown is performed.

In the foregoing frame rate conversion, the processes of the second and third flowcharts are different from each other in the order of performing M:N pulldown detection and scroll detection. However, M:N pulldown detection and scroll detection may be performed in parallel, and frame rate conversion for M:N pulldown may be performed based on determination results of M:N pulldown detection and scroll detection.

Second Embodiment

FIG. 4is a block diagram illustrating a video processing apparatus using a frame rate conversion method in a second embodiment of the present disclosure.

Referring toFIG. 4, a reference numeral “400” represents a storage unit such as DVD discs, Blu-ray discs, or hard discs, a reference numeral “401” represents an input processing unit configured to reproduce video signals recorded from broadcasting or video signals from recording media, a reference numeral “402” represents the frame rate conversion apparatus of the first embodiment, a reference numeral “403” represents an output processing unit configured to output a video signal to an external unit or a display unit, and a reference numeral “404” represents a display unit such as liquid crystal displays or plasma displays.

In the input processing unit401configured to process video data generated or recorded in the storage unit400, a video signal containing a M:N pulldown signal is generated. For example, such a pulldown video signal is converted from a 60 Hz video signal into a 120 Hz video signal by using the frame rate conversion method of the first embodiment. Since the video signal having the converted frame rate is output from the output processing unit403to the external unit, such a video signal is processed into such a signal format that the video signal can be output via connection through a cable or via wireless connection. For display of a video picture on the display unit404, a synchronization signal to be reference for display and a timing signal for controlling a display apparatus are, in addition to a video signal, generated and processed to display the video picture on a screen.

The display unit404is required for video processing apparatuses configured to display a video picture on a screen, such as television sets or displays. However, video processing apparatuses, such as DVDs, Blu-ray discs, or hard discs, configured to record and/or reproduction a video picture can be realized with a configuration which does not include the display unit404, and a video signal having a converted frame rate is output via an output terminal connected to the external unit.

Third Embodiment

FIG. 15is a flowchart for a frame rate conversion method of a third embodiment of the present disclosure.

Referring toFIG. 15, it is determined, by film footage detection, whether or not a video signal corresponding to an input video picture is a M:N pulldown video signal. If the video signal is not the M:N pulldown signal, settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to frame rate conversion for a series of frames, and then such frame rate conversion is performed.

On the other hand, if the video signal is the M:N pulldown video signal, it is determined, by scroll detection, whether or not the video picture is a scroll video picture in which a displayed object(s) moves in a uniform manner or in the same direction across the entirety of a screen. If the video picture is not the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to X:Y pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion for X:Y pulldown is performed. On the other hand, if the video picture is the scroll video picture, the settings of memory control, motion vector detection, and interpolation frame generation are switched to those corresponding to M:N pulldown to switch processing for memory control, motion vector detection, and interpolation frame generation. Then, frame rate conversion corresponding to M:N pulldown is performed.

Note that M:N and X:Y satisfy relationships of M>X and N>Y. Suppose that M:N pulldown is 8:7 pulldown. In such a case, X:Y pulldown is 3:2 pulldown, and frame rate conversion is switched between frame rate conversion for the lower number of repeat frames and frame rate conversion for the higher number of repeat frames depending on whether or not a displayed object(s) moves in a uniform manner or in the same direction on a screen.

As described above, the frame rate conversion apparatus and method of the present disclosure are useful as apparatus and method for performing frame rate conversion used for, e.g., display apparatuses, video processing apparatuses, or television sets and performing judder compensation for a pulldown video signal.