Video system with de-motion-blur processing

A video system that performs TV signal decoding, deinterlacing, and de-motion-blurring for progressive scan flat panel display is introduced. The system embeds a frame buffer and a scaler for conducting format and resolution conversions for display panels of different sizes. The output of the scaler is sent to a de-motion-blur processor for reducing the blurriness due to the motion of image objects and the slow response time of flat panel display devices. The de-motion-blur processor gets the motion and noise indication signal from scaler or the pre-frame-buffer video processor. Based on the motion and noise information and the information of temporal difference, the de-motion-blur processor performs over driving for the display panel interface and improves the rising and falling response time of the flat panel display. The decoding, deinterlacing, and de-motion-blur processing share the same frame buffer controller so the entire system can be optimized in cost and performance.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to the flat panel TV (especially the LCD TV) display system, and more specifically to a video system for improving the response time and reducing the motion-blur of the panel display.

2. Description of Prior Art

The development of flat panel displays has been accelerated. In the current display market, flat panel displays are gradually replacing the traditional CRT displays. However, some of the flat panel displays, for example, TFT-LCD panel displays, are generally slower in the response time compared to the CRT displays. This disadvantage hampers the development of the LCD TV, for which the display quality (especially motion blur) is highly concerned.

To improve the quality of motion picture on the flat panel (especially TFT-LCDs), various driving techniques have been introduced. Among them, the over driving method is well known as the effective scheme for improving the response time. The over driving method makes the response time fast by applying an over driving voltage to a motion pixel. In order to apply the over driving method to the flat panel, the motion pixels are detected by comparing the pixel values of current and previous frames, and determining the value to be added to the original value. The over driving level is decided through a decision model. Such a model heavily depends on the characteristics of the flat panel. For TFT-LCDs, the decision model is normally implemented by a look-up table built in a over drive circuit chip. The over drive decision requires one-frame delay mechanism for getting the pixel value of the previous frame. A compression and decompression mechanism is used for reducing the bandwidth demands of the frame buffer access for implementing the one-frame delay. The over driving method is proposed in some prior arts to combine with the timing controller (TCON) in the flat panel display for improving the video quality of motion picture.

To further improve the response time of a flat panel display, the present invention proposes an extra video processing for de-motion-blurring. A video system in accordance with the present invention combines the color TV decoding, deinterlacing, scaling, and de-motion-blur process and many other video enhancement features to optimize the overall system in cost.

SUMMARY OF THE INVENTION

An objective of the present invention is to provide a video system, which improves the response time of the flat panel display.

Another objective of the present invention is to provide a video system, which is optimized in cost and performance.

A further objective of the present invention is to provide a de-motion-blur processor, which conditionally disables compression/decompression and over-driving operations so as to provide better quality and rapid image processing.

According to an aspect of the present invention, a video system for flat panel display comprises a pre-frame-buffer processor receiving video signals for decoding and deinterlacing the video signals; a frame buffer controller connected with the pre-frame-buffer processor for providing one or a plurality of field delay to the video signals; a scaler receiving the video signal output from the pre-frame-buffer processor directly or via the frame buffer controller for converting the signals to a desired mode according to the specification of the flat panel display; and a de-motion-blur processor comparing current video signals from the scaler and previous video signals from the frame buffer controller to derive a temporal difference, and controlling over driving to the flat panel display according to the temporal difference.

According to another aspect of the present invention, in the video system, the pre-frame buffer processor provides motion information to the de-motion-blur processor.

According to a further aspect of the present invention, in a video system, a de-motion-blur processor comprises compression and decompression units for compressing and decompressing the video signals; a motion and noise detector receiving motion information, comparing the current and previous video signals, calculating temporal difference value of a certain pixel on the screen; and an over drive processing unit using the temporal difference value of the pixel to determine level of over driving.

DETAILED DESCRIPTION OF THE EMBODIMENT

The present invention proposes a video system1for a flat display device. The video system1includes a pre-frame-buffer processor10, a frame buffer controller20, a scaler30, and a de-motion-blur processor40, as shown inFIG. 1. The flat display device can be (but not limited to) one of TFT-LCD panel, PDP (plasma display panel), OLED (organic light emitting display), and LCOS (liquid crystal on silicon).

In the system, the de-motion-blur processor40implements a novel over driving mechanism which uses one-frame or many-frame delay generated through YC-DPCM (Differential Pulse Code Modulation) compression and decompression units41-43. The de-motion-blur processor40further comprises a motion and noise detector45, and an over drive processing unit44. The YC-DPCM compression unit performs de-correlation and entropy coding on Y and C (or luminance and chrominance) components separately.

The pre-frame-buffer processor10has the functions of color TV decoding and deinterlacing. As shown inFIG. 2, the pre-frame-buffer processor10comprises a color TV decoder11, a video source selector12, a pre-scaler video processing unit13, a deinterlacer14, H&V scaling-down circuits15and16, and many other video enhancement devices as required.

Signals from one or more video sources are input into the color TV decoder11. The color TV decoder11receives composite video broadcast signal (CVBS) or separated Y/C signals from a TV tuner or set-top-box/DVD player. It may also receive the component (Y/Pb/Pr) video inputs, for which no further color decoding is needed. The composite, separated YC, or component inputs are analog signals with peak-to-peak value of 1V in amplitude, for example. An ADC (Analog to Digital Converter; not shown) converts the analog signals to digital signals for further processing. The color TV decoder11slices the horizontal and vertical synchronization signals (hereinafter referred to HS and VS) from the input and uses a PLL (Phase Locked Loop; not shown) to lock the signals to the color carrier frequency. The chrominance signal, comprising U and V for PAL or I and Q for NTSC, is decoded using in-phase and quadrature mixers (not shown) and low-pass filters (not shown). Since the spectra of luminance and chrominance signals are interweaved at the neighboring frequencies of the color carrier, a 3D (three-dimensional) comb filter (not shown) of the color TV decoder11is used to separate the luminance and chrominance signals. The comb filter may be adapted for different TV standards (e.g. NTSC, PAL, etc.) A frame buffer50as shown inFIG. 1is used for one- and many-field delay. The color TV decoder11has an access to the frame buffer controller20controlling the frame buffer50so that the frame buffer50acts as the temporal delay mechanism for the operation of the 3D comb filter in the color TV decoder11. The 3D comb filter detects the motion pixels and decides the filter coefficients for separating the interweaved frequency components of the luminance and chrominance signals.

The video source selector12receives the video signals output from the color TV decoder11at an input port, and receives image data from other sources at a digitized image data port. The video sources selector12optionally uses a color space converter (not shown) for RGB to YUV color domain conversion. The video source selector12may also switch the input ports to main-picture or sub-picture image paths. Both main-picture and sub-picture are processed separately before being combined in a PIP (picture-in-picture) blending unit33of the scaler30, which will be described later.

The pre-scaler video processing unit13receives output from the video source selector12for video improvement and color adjustment. The pre-scaler video processing unit13may have functions of LTI (Luminance Transient Improvement), CTI (Color Transient Improvement), flesh tone adjustment, and 2D or 3D noise reduction. In the flesh tone adjustment, the colors near the human flesh tone are shifted to a more saturated region, and colors in other ranges are remained unchanged. In the case that 3D noise reduction is implemented, an access to the frame buffer50(through the frame buffer controller20) of the display is necessary for providing one-field or one-frame delay mechanism. This pre-scaler video processing unit13provides mainly the subjective color preference adjustment and restores the video quality degraded due to quantization error or channel noise of video compression and transmission. Compared with the adjustment by the video processing unit in the scaler30after the deinterlacing, this video adjustment made by the pre-scaler video processor13provides the processing for quality improvement of video sources. The video processing in the scaler30is more adaptable for quality improvement to panel display.

Receiving output from the pre-scaler video processor13, the deinterlacer14converts the interlacing TV or HDTV signals to a progressively scanned format.

The deinterlacer14has an access to the frame buffer controller20, so that the frame buffer50acts as the field delay memory unit for the operation of the deinterlacing. The deinterlacer14used a 3-dimensional interpolation filter (not shown), which uses data in the frame buffer50, to convert the interlaced image into a progressive scanned image.

The deinterlacer14checks the temporal and spatial neighboring pixel data of a certain pixel and determines to adopt the motion or still pixel mode operation. In the case of a still pixel mode, a direct pixel merge (even and odd field scan lines are combined to form the progressively scanned frame) is conducted. While in the case of a motion pixel mode, a sophisticated algorithm is used to find the possible motion vector and the intra-frame interpolation is performed accordingly. The deinterlacer14also takes into the consideration of the possible conversion from cinema film to TV signals. As known, in so-called 3-2 pull down conversion, it detects the motion pixels, counts the pixels and decides the 3-2-3-2 repetition sequence of the image fields. The 3-2-3-2 repetition sequence, in which 3 frame periods of high state are followed by 2 frame periods of low state and such an arrangement repeats, is introduced due to the 24 (cinema film) to 60 (TV signal) frame-per-second conversion. In film mode, the 3-2-3-2 repetition sequence reveals the facts that 3 consecutive identical pictures are followed by another 2 consecutive identical pictures, and this 3-2-3-2 picture sequence repeats again and again until the so called bad-editing clip is detected. This 3-2-3-2 picture sequence indication is also very important for the de-motion-blur processing. If the film mode is detected, the fact that 3 consecutive identical pictures followed by another 2 consecutive identical pictures means the de-motion-blur processing can be disabled if the two adjacent pictures are exactly the same. Therefore, the finding of this 3-2-3-2 repetition sequence saves the power consumption and reduces the side-effects of de-motion-blur processing.

The pre-frame-buffer video processor10may also include scaling down circuits15and16for horizontal and vertical sample rate down conversion. The horizontal and vertical scaling down circuits provide a decimation of image resolution and pixel data rate. It supports the flat panel display with resolution smaller than the input image or enables the sub-picture display of a PIP (Picture-In-Picture) feature.

The output of this pre-frame-buffer processing10is sent to the scaler30through the frame buffer controller20or sometimes bypassing the frame buffer controller20.

Although there is the scaler30connected with the frame buffer controller20(seeFIG. 1) in accordance with the present invention, the utilization of the scaling down circuits15and16in the pre-frame-buffer processing unit10has the benefit of lower bandwidth for the frame buffer controller access and easier implementation using input clock ICLK. After the reduction of image resolution, the data amount for each frame can be reduced significantly before the writing to the frame buffer. Besides, with one result per interpolation (per ICLK), this horizontal and vertical scaling down block is easier to be implemented. If a panel clock (PCLK) is used for scaling, there may be two results for one interpolation. Accordingly, it is advantageous to separate the scaling function into scaling down circuits15,16before the frame buffer controller20and the scaler30after the frame buffer controller20. The reason to locate the scaling down circuits after the deinterlacer14is explained as follows. The decimation of image resolution can seriously destroy the motion information of an interlaced image. The sophisticated motion vector calculation and film mode detection are no longer valid if the processing order of the deinterlacer14and the scaling down circuits15,16is reversed. A vertical scaling down circuit with better display quality can be implemented using some line buffers, but in fact a simple vertical scaling down can be easily realized by proportional line dropping (remove a scan line for every some lines) according to the scaling factor.

The frame buffer controller20, which can be a DRAM controller, is used to provide field or frame delay. As shown inFIG. 3, the frame buffer controller20includes a PLL21for generating memory access clock signal (MCLK), a DRAM access controller22, an access arbitrator23, and three or more access gates connected with the access arbitrator23. The access gates provide read or write paths so that data can be stored into the frame buffer50and restored later.

The frame buffer controller20offers the field delay or frame delay for video processing. The access gates include at least a pre-frame-buffer access gate24, a scaler access gate25, and a de-motion-blur access gate26. The pre-frame-buffer access gate24offers the field delay or frame delay for color TV decoder, deinterlacer, and in some cases the 3D noise reduction. The scaler access gate25fetches the progressive scanned image data and performs further format conversion for fitting the panel resolution. The de-motion-blur access gate26is connected to YC-DPCM compression and decompression units of the de-motion-blur processing unit40. Since the data amount may be dramatically increased after the deinterlacing and scaling up processing, these compression and decompression functions can help to reduce the bandwidth requirement for frame buffer storage. To improve the access efficiency, dual-port SRAM or FIFO (First-In-First-Out) memories (not shown) can be used between the access gates and the access arbitrator for temporary data storage before the burst-write and burst-read of DRAM. These dual-port SRAM or FIFO memories provide also the clock domain isolation, so the meta-stable issue of the interface of different clock frequencies can be avoided. Since at one time only one access gate can get the chance to write or read the frame buffer50, the access arbitrator23acts as a controller for coordination to all access gates. Furthermore, according to the present embodiment, once in some mini-seconds, the bits and bytes stored in the DRAM need at least a read or write operation to maintain the intact content. A refresh access gate (not shown) can be also included in the frame buffer controller but is not discussed in this invention.

The scaler30is used for converting input images to different formats and resolutions for LCD panels of different sizes. The scaler30has a main scaling mechanism32to perform the horizontal scaling up and vertical scaling up/down to covert resolution for different display panels. A main-picture passes through the main scaling mechanism32and then feeds into a PIP (picture-in-picture) blending unit33, while a sub-picture bypasses the main scaling mechanism32and directly fed into the PIP blending unit33. With reference toFIG. 4, the scaler30comprises a PLL35for panel clock generation, line buffers322and a line buffer controller/scaling parameter generator320, and the horizontal scaling units325followed by a vertical scaling unit326. The input image is input to one of the line buffer via a 2-input-2-output switch31and a 1-input-5-output switch321. Then, the image data are read from the line buffers322and sent to the horizontal scaling unit325. The line buffer controller and scaling parameter generator320handles the switching and selection of line buffers322for writing and reading image data by controlling the switch321. The line buffer controller320also generates address signal and the scaling parameters and sends the parameter setting to the horizontal and vertical scaling units325,326. For the case of nonlinear scaling, which is commonly used for 4:3 images to 16:9 wide screen displays, the horizontal scaling factor (and hence the scaling parameters) should be changed for different display zones along the scan line. For keystone correction, the horizontal scaling parameters are different from one to the next scan line. In the embodiment, five line buffers322are used and four horizontal scaling units321followed by a vertical scaling unit326.

A PIP (picture-in-picture) blending unit33receives the main-picture from the main scaling mechanism32and the sub-picture from the switch31to combine the pixel data of the main-picture and sub-picture. Since the frame rate of main-picture and sub-picture should be exactly the same, either the main-picture or sub-picture should pass through the frame buffer controller20to be adjusted, so that the frame rates of the main-picture and the sub-picture are synchronized. There is a video adjusting and peaking processing unit34after the PIP blending unit33for brightness, contract, hue, and saturation adjustment. The unit34also boosts the middle or high frequency components of the video signal. As mentioned in the previous description, this video processing is more for quality improvement of panel display and hence it heavily depends on the characteristics of display panels.

With reference toFIG. 5, the de-motion-blur processor40comprises compression and decompression units41,42, and43, an over drive processing unit44, and a motion and noise detector45. The operation of the over chive processing unit44is well known as the driving scheme of improving the response time of a flat panel display. The compression and decompression units41to43help to reduce the bandwidth requirement of frame buffer access since the amount of image data expanded significantly after the deinterlacing and scaling up. The over drive processing unit44uses temporal difference value of a pixel of current and previous images to determine the display data for over driving. This improves the response time but it also increases the noise level in some cases, hencc, a motion and noise detector45is uscd to avoid the adverse effect. The motion and noise detector45receives motion information from the pre-frame buffer processor and compares current and previous frames to calculate temporal difference value of a certain pixel using signals from YC-DPCM compression/decompression units41and42. It also calculates spatial difference between a pixel and its neighboring pixels and controls the over driving of the over drive processing unit44according to the motion infonnation and the spatial and temporal difference values. The motion infonnation from the pre-frame-buffer processor includes at least a motion/noise indication signal and a film mode indication signal, such as a still picture indication signal (e.g. 3-2-3-2 repetition signal for NTSC TV system or 2-2-2-2 repetition signal for PAL TV system). The over driving level of the over drive processing unit44is decennined for the display device according to the spatial and temporal difference value and the motion information of the pixel. A look-up table is used for mapping the difference values to the over driving level. In the over drive processing unit44, it is detennined to directly use the compressed-decompressed image or the image from the scaler30bypassing the compression and decompression units (in a word, the over drive processing unit44is disabled), if the motion information indicates that the picture is in the still mode.

As mentioned above, the motion and noise detector45calculates the difference of image values of the pixels on some consecutive frames. This motion and noise detector45also gets motion information, such as the motion and noise indication, from the color TV decoder11, deinterlacer14, and scaler30, In order to reduce the complexity of implementation, we use the YC-DPCM scheme for the image compression/decompression before and after the storage to the frame buffer50. An image is decomposed into the Y and C (or luminance and chrominance) components. The DPCM method then applies to Y and C compression and decompression units simultaneously. The aforementioned frame buffer controller20(seeFIG. 3) not only provides the field delay mechanism to the pre-frame-buffer processor10, it also offers one or a plurality of frames of temporal delay to the de-motion-blur processor40for generating the temporal difference in over driving calculation. The motion and noise detector45calculates difference from the current frame image YC(t), the one-frame delayed image YC(t−1), and up to the N-frame delayed image YC(t−N), so as to control the over drive processing unit44.

As mentioned above, for image compression, several characteristics are desirable in a method used. An ideal method completely de-correlates the image data before quantization and coding. Some commonly used image compression methods such as JPEG or MPEG, use DCT (Discrete Cosine Transform), are expensive in cost of implementation.

In the embodiment in accordance with the present invention, YC-DPCM method is used. The method uses a scheme in which the prediction is subtracted from the actual pixel value to obtain a differential image, which is much less correlated than the original image data. The differential image is then quantized and encoded.

FIG. 6schematically depicts the YC-DPCM compression unit43. The YC-DPCM compressor43comprises a quantizer431, an entropy coder433and a YC predictor432. The difference between the actual pixel value YC(t) and the most likely predicted valueYC(t) from the predictor432is calculated and referred to the differential signal or error signal E(t). That is,
E(t)=YC(t)−YC(t)  (1)
We also define E(t) as the quantized error signal output from the quantizer431. Using the entropy coder433, this quantized error signal E(t) is coded before being sent to the frame buffer50. The predictor432can be a liner predictor. In general, linear predictor is not the optimal choice as compared to the predictor maximizing the conditional probability using Markov model. However, in most cases, this small loss due to the use of the linear predictor in performance can be compensated for by the significant reduction in computation complexity.

FIG. 7illustrates the block diagram of the YC-DPCM de-compressor42, which comprises an entropy decoder421and a YC predictor422. The reconstructed pixel YĊ(t) is given by:
YĊ(t)=YC(t)+Ė(t)  (2)
Note that we observe that the prediction at this de-compressor block based on reconstructed values of the fetched data from frame buffer50. In order to prevent error generated at the de-compression side, the prediction in both compressor and de-compressor blocks must be identical. Therefore, the predictor in the compressor43also uses the reconstructed values of the previously encoded samples in the prediction instead of their actual values. For this, the compression simulates the decoder in the prediction loop, as shown in theFIG. 6.

FIG. 8shows an embodiment of the predictor422. For practical application, the Y and C predictors can be implemented with a simple circuit that calculates the linear combination of the neighboring pixels. In this drawing, the YC predictor comprises a luminance predictor4221and a chrominance predictor4222. A pixel is decomposed into luminance and chrominance components, denoted as Y and C respectively. Due to the different visual sensitivity to human eyes, the luminance and chrominance prediction are implemented separately. The luminance and chrominance predictors4221and4222are designed and characterized according to the response time of display devices and visual response of human eyes. In fact, since the response time of the color information is less critical than the gray-level information, we can use linear predictors of different orders for luminance and chrominance processing. For example, the second-order chrominance predictor:
XC=0.5*AC+0.5*CC,  (3)
And the third-order luminance predictor:
XY=0.75*AY−0.5*BY+0.75*CY,  (4)
Where X is the pixel to be predicted, and A, B, and C indicate the neighboring pixels on the current and previous scan lines used for prediction (seeFIG. 9). Subscripts Y and C respectively denote the luminance and chrominance components of a pixel data. A third-order luminance predictor is depicted inFIG. 10as an example. The circuit can be easily extended to a linear predictor of other orders.

Note that inFIG. 5, there are some input image data buses fed into the over drive processing unit44for the over driving processing. The input image buses are current frame data YC(t), restored (compressed and then decompressed) current frame data YĊ(t), and restored previous frame data YĊ(t−1), . . . , and YĊ(t−N). The output of the over driving processing is obtained by the following equation:
YCout(t)=YC(t)+ODTable(YĊ(t),YĊ(t−1), . . . ,YĊ(t−N)),  (5)
where the second term ODTable0 is the acceleration value. For the special case that N=1, the function ODTableis the look-up table that depends on the amplitude of YĊ(t) and the difference between YĊ(t) and YĊ(t−1). The acceleration value (also called over driving value) is zero if restored current and restored previous frame data are substantially equal. In a word, if the image is a still image, the output data YCout(t) will be exactly the current frame image YC(t). Hence, the distortion due to compression and decompression can be totally removed and the quality of still images will not be degraded. On the other hand, if a pixel is detected as the motion pixel, through the look-up table, an over driving value is determined and added to the current frame image. For practical application, the look-up table used in the present invention can be implemented by a table, which has 6-bit resolution for luminance and 4-bit resolution for chrominance. For each input and output voltage value, the content of the over-drive table should be derived from the response time characteristics for a particular display device.

The over driving process used to improve the response time could lead to noise amplification if the input image is noisy. In accordance with the present invention, the motion detection circuit45is used to adaptively adjust the threshold for determining the image state is motion or still. Noise information such as noise level can be collected from the image data during the vertical retrace period (supposed to be black pixels) and be derived from the 3D noise reduction function. During the pre-frame-buffer processing, the video decoder11and deinterlacer14need to determine the status of a pixel is motion or still. Based on this information, a motion and noise indication signal is generated and then forwarded to the motion and noise detector45.

As mentioned above, if the film mode is detected in the deinterlacer14, the 3-2-3-2 repetition sequence signal for NSTC system (or 2-2-2-2 for PAL) designates 3 identical pictures followed by another 2 consecutive identical pictures. The de-motion-blur processor40gets the indication at every frame and determines to disable the over driving processing of the over drive processing unit44if the two adjacent pictures are exactly the same (no motion pixels at all and hence the over driving values are always zero for the entire frame). With this (3-2-3-2 for NTSC and 2-2-2-2 for PAL) repetition sequence indication, the over driving processing saves (⅗ for NTSC and or ½ for PAL) the total power consumption and reduce the unfavorable side effect.

The present invention discloses a video system that performs color TV decoding, deinterlacing, format and resolution conversion, and de-motion-blur processing. In the system of the present invention, the motion pixel detection at earlier processing stages can be forward passed to the later stage. Furthermore, the frame buffer controller20provides the frame delay mechanism for over drive processing of the motion-blur processor40, and also for 3D comb filter in color TV decoder11and the deinterlacer14of the pre-frame-buffer processor10. This saves the computation, the storage space, and the frame buffer access bandwidth. By sharing the frame-buffer controller20and forward-passing the motion and noise information, this invention minimizes the overall system cost and improves the display quality.

While the embodiment of the present invention as illustrated and described, various modifications and alterations can be made by persons skilled in this art. The embodiment of the present invention is therefore described in an illustrative but not restrictive sense. It is intended that the present invention may not be limited to the particular forms as illustrated, and that all modifications and alterations which maintain the spirit and realm of the present invention are within the scope as defined in the appended claims.