Video system with blocking artifact two-dimensional cross filtering

A video system includes: analyzing video data, having a block; performing a transition change detection for determining a spatial intensity transition within the block; performing a block-wise similarity measurement on the block in the video data for identifying a blocking artifact; and filtering with a two dimensional cross filter every pixel in the block for removing the blocking artifact.

TECHNICAL FIELD

The present invention relates generally to video decompression for high definition applications, and more particularly to a system for addressing the picture degradation caused by picture compression algorithms.

BACKGROUND ART

With the transition to digital television mandated by the United States government, many manufacturers are actively preparing to deal with a broad range of quality in broadcast and downloaded programming. Many consumers are already confining their video applications to a single unit. In many cases, a computer monitor might support computer applications, down loaded video applications and broadcast television programs. Some families have chosen to utilize a central television for family viewing and computer applications.

A trend has started as more and more families replace their old CRT television with big screen LCD and Plasma televisions with high definition television (HDTV) capabilities. While the new technologies provided better experience with higher resolution and more details, they also reveal more obvious artifacts and noise if the received signals are not of a good enough quality. For instance, displaying YouTube™ video clip on the HDTV will show very ugly coding artifacts caused by the compression algorithm. Technology that can produce superior visual quality on the digital video and image products is highly desirable.

In the current digital video and image standard, block based transformation and quantization of transform coefficients are used to achieve high compression efficiency. Since quantization is a process subject to losing detail, the combination of block based transform and quantization may generate some perceptually annoying artifacts such as blocking artifacts and ringing artifacts.

A blocking artifact may be an artificially induced pattern or intensity change that may be generated when a block of compressed video data is decompressed for display purposes. The blocking artifact, if it is present, will show-up at the boundaries of the decompressed block of video data. As a single block represents only a small portion of a video picture, hundreds or thousands of these blocking artifacts may be present in a single frame of video data. Left unaddressed, these artifacts may ruin the picture completely.

These artifacts may show up on the viewing screen as sharp discontinuities or blurry areas in a textured pattern. Since coding artifacts reduction is fundamental to many image processing applications, it has been investigated for many years.

Many post-processing methods have been proposed. In general, most prior art methods either focus on blocking artifacts reduction or ringing artifacts reduction. To reduce the blocking artifacts, most prior arts methods only focus on the block boundary pixels, the general quantization noise in the middle of the block has not been handled. Also, the de-blocking process depends on the quantization parameter. Obviously, these approaches are not effective. Although some of prior art methods show very good results on the selected applications, they are not good enough for new digital HDTV. As the result, either the artifacts are still visible or the texture detail is blurred.

Many of the consumers have questioned why the original mandate was put in place. With the purchase of a new HDTV, they are not satisfied with the picture quality of many applications that they have enjoyed for some time. Consumers mistakenly believe that their newly purchased HDTV is faulty in some way. The actual culprit is the compression algorithm that was used to process the data for transmission.

Thus, a need still remains for a blocking artifact filter system that can provide a crisp picture without losing the texture detail from the picture. In view of the mandated transition to all digital television broadcast called for by the United States government, it is increasingly critical that answers be found to these problems. In view of the ever-increasing commercial competitive pressures, along with growing consumer expectations and the diminishing opportunities for meaningful product differentiation in the marketplace, it is critical that answers be found for these problems. Additionally, the need to save costs, improve efficiencies and performance, and meet competitive pressures, adds an even greater urgency to the critical necessity for finding answers to these problems.

DISCLOSURE OF THE INVENTION

The present invention provides a video system including: analyzing video data, having a block; performing a transition change detection for determining a spatial intensity transition within the block; performing a block-wise similarity measurement on the block in the video data for identifying a blocking artifact; and filtering with a two dimensional cross filter every pixel in the block for removing the blocking artifact.

BEST MODE FOR CARRYING OUT THE INVENTION

In the following description, numerous specific details are given to provide a thorough understanding of the invention. However, it will be apparent that the invention may be practiced without these specific details. In order to avoid obscuring the present invention, some well-known circuits, system configurations, and process steps are not disclosed in detail. Likewise, the drawings showing embodiments of the system are semi-diagrammatic and not to scale and, particularly, some of the dimensions are for the clarity of presentation and are shown greatly exaggerated in the drawing FIGS. Where multiple embodiments are disclosed and described, having some features in common, for clarity and ease of illustration, description, and comprehension thereof, similar and like features one to another will ordinarily be described with like reference numerals.

For expository purposes, the term “horizontal” as used herein is defined as a plane parallel to the plane or surface of the Earth at a point, regardless of its orientation. The term “vertical” refers to a direction perpendicular to the horizontal as just defined. Terms, such as “above”, “below”, “bottom”, “top”, “side” (as in “sidewall”), “higher”, “lower”, “upper”, “over”, and “under”, are defined with respect to the horizontal plane. The term “on” means there is direct contact among elements. The term “system” as used herein means and refers to the method and to the apparatus of the present invention in accordance with the context in which the term is used. The term “processing” as used herein includes calculating, manipulating, ordering, measuring, filtering, or displaying as required in forming a described structure.

Referring now toFIG. 1, therein is shown a functional block diagram of a video system with blocking artifact filtering100, in an embodiment of the present invention. The functional block diagram of the video system100depicts a circuit element102, such as an integrated circuit, a circuit board, or a flex circuit, having a spatial range extraction circuit104for determining the range of intensities in an uncompressed block of video data106. The spatial range extraction circuit104may be coupled to a transition change detector108.

The transition change detector108calculates an intensity transition or spatial variation for both the vertical and horizontal directions at the same time for detecting a natural transition change in the spatial variation. It has been discovered that blocking artifacts usually share common phenomena including the spatial variation that is very small relative to the rest of the block and there is an abrupt intensity change along the direction of the small spatial variation direction in the block boundary. The direction of the small spatial variation is known as the dominant direction and may be identified by a dominant direction detection block110.

A blocking artifact detector112may detect the presence of the blocking artifact by monitoring the intensity changes in the dominant direction. If no blocking artifact is detected in the uncompressed block of video data, the block is passed without filtering to a video output bus114. By not applying filtering to a block that does not have blocking artifacts, the process preserves the natural edge smoothness and sharpness for generating natural intensity transitions between the uncompressed blocks of the video data106.

If the blocking artifact detector112determines that blocking artifacts are present in the uncompressed block of video data, an artifact level estimation block116may process the data. The artifact level estimation block116differentiates the natural transition with texture and edge from the encoding artifacts caused changes. The data is then passed to a block-wise similarity checker118, for comparison with adjacent blocks of the uncompressed video data. This mechanism helps differentiate the natural transitions between blocks from the blocking artifacts induced by the compression algorithm.

A filter coefficients generator120may be used to determine the amount of filter correction that is appropriate for the current uncompressed block of the video data106. The filter coefficients are passed with the data to an adaptive filtering circuit122. The adaptive filtering circuit122may apply the filter coefficients to all of the pixels in the block of uncompressed video data. The output of the adaptive filtering circuit122passes the filtered data to the video output bus114for display.

Referring now toFIG. 2, therein is shown a diagram of a segment of a video picture200during an uncompress process. The diagram of the segment of the video picture200depicts an array of a pixel202, such as the smallest component of a video picture, not shown. The segment of the video picture200may be formed by a series of a block204, such as an uncompressed block of video data, which may be aligned to display the current picture, not shown. Each of the block204may have a horizontal boundary206and a vertical boundary208. An adjacent block210may be present on the horizontal boundary206or the vertical boundary208.

The number of the pixel202that may be aligned along the horizontal boundary206and the vertical boundary208may be dependent on the compression scheme used to prepare the data. In the embodiment example, the block204of the uncompressed video data is represented by 64 of the pixels202forming an 8×8 pattern. Each of the blocks204may have a first column212, a last column214, a first row216, and a last row218.

Referring now toFIG. 3, therein is shown a functional block diagram of a spatial range extraction circuit300, in an embodiment of the present invention. The functional block of the spatial range extraction circuit300depicts the video data106coupled to a minimum intensity detector302and a maximum intensity detector304. A difference processor306may be coupled between a minimum bus308and a maximum bus310. The output of the difference processor306may be an intensity range312. The intensity range312may provide an intensity range value to the next level of logic, not shown.

As the block204, ofFIG. 2, is processed the data from each of the pixels202passes through the minimum intensity detector302and the maximum intensity detector304. When the block204has been completely measured, the intensity range is gated out of the difference processor306. An equation for the intensity range312may include:
BKrange=PMAX−PMIN(Equation 1)

Where the intensity range (BKrange) is equal to the maximum intensity pixel value (PMAX) minus the minimum intensity pixel value (PMIN).

Referring now toFIG. 4, therein is shown a block diagram of a transition change detector400, in an embodiment of the present invention. The block diagram of the transition change detector400depicts an adder402coupled to the maximum bus310. The adder402may also be coupled to a vertical transition array404and a horizontal transition array406. The adder402provides control logic that stores the intensity change between the pixels202, ofFIG. 2, in the proper location for the block204, ofFIG. 2, being analyzed. The transition change between the pixels202may be identified by a horizontal spatial variation408and a vertical spatial variation410. The adder402may provide the logic required to solve the array calculation of equations 2 and 3 as shown below:

The number of the pixels202aligned on the edge of the block is represented by M. The vertical spatial variation may be represented by Varverand the horizontal spatial variation may be represented by Varhor. The pixel202intensity may be represented by p(i, j) where i represents the vertical position and j represents the horizontal position in the block204. In this example the size of the block204is equal in the horizontal and vertical directions, but this is not a requirement and the dimensions may differ in some compression algorithms.

By storing the variation in intensity between the pixels202, any significant changes may be quickly recognized. Since one of the characteristics of a blocking artifact was discovered to be an abrupt change in the intensity, the transition change detector400may quickly identify possible locations of the blocking artifact.

Referring now toFIG. 5, therein is shown a block diagram of a dominant direction detection circuit500, in an embodiment of the present invention. The block diagram of the dominant direction detection circuit500depicts a comparator502that may be coupled to the transition change detector400by the horizontal spatial variation408and the vertical spatial variation410. The comparator may monitor the horizontal spatial variation408and the vertical spatial variation410in order to determine which has the smaller spatial variation. A dominant direction output504may indicate whether the vertical direction or the horizontal direction is dominant.

Referring now toFIG. 6, therein is shown a block diagram of a blocking artifact detection circuit600, in an embodiment of the present invention. The block diagram of the blocking artifact detection circuit600depicts a selector602that may be controlled by the dominant direction output504. The selector602may be designed to pass the horizontal spatial variation408or the vertical spatial variation410based on the value of the dominant direction output504. A magnitude comparator604may manipulate the horizontal spatial variation408or the vertical spatial variation410, depending on which is selected, in order to detect the presence of a blocking artifact. If the selected input meets the following criteria, it means that no blocking artifact is present in the block204, ofFIG. 2, and the block204, in the video data106, is sent on without any additional filtering. The criteria for determining the presence of the blocking artifact is shown in Equation 4 below.
Var>2* BKrange·BKrange(Equation 4)

If the criteria of equation 4 are not satisfied, it is still possible that no blocking artifact is present. It can be further analyzed by the equations 5 through 9 as shown below:

Equations 5 and 6 may be used to establish a dominance value of the vertical direction. The dominance of the horizontal direction may be determined by the equations 7 and 8 as shown below.

Equation 9 may be used to determine whether a blocking artifact is present. If the resulting Blockingfactoris less than 2, it would indicate that no blocking artifact is present.
Blockingfactor=Diff/2M−Var/M*(M−1)   (Equation 9)

If no blocking artifact is present as determined by equations 5 through 9, the block204is sent on without any additional filtering. In equation 9, the value of Diff may be determined by equation 5 or equation 7 whichever is determined to be dominant. The value of Var is determined by equation 2 or equation 3 also selected by the dominant direction. The value of M is dependent on the number of the pixels202, ofFIG. 2, used to form the block204by the compression algorithm. In equation 4, the Var represents the value of the selected input and the BKrangeis the output of equation 1 as shown above. The implementation of the blocking artifact detector circuit600is an example only and other implementations are possible.

A video demultiplexer606may be used to guide the video data106to the video output bus114or a video filter bus608for further processing. If the magnitude comparator604determines that no blocking artifact is present, the video data106is passed to the video output bus114. If the magnitude comparator604determines that a blocking artifact is present, the video data106is passed to a video filter bus608for further processing.

Referring now toFIG. 7, therein is shown a block diagram of an artifact level estimation circuit700, in an embodiment of the present invention. The block diagram of the artifact level estimation circuit700depicts a blocking level calculator702that receives a blocking factor704and an intensity range706from the dominant direction decision. The blocking level is determined in the blocking level calculator702per equation 10 as shown below.

In equation 10 the values of C1 and C2 are experimentally derived and are constants. In order to properly determine the level of blocking artifacts, a threshold must be determined. The threshold value may be used to adjust the response to the blocking level determined by equation 10. The equation used to evaluate the blocking level threshold is shown in equation 11 below.

The values of T1 and T2 are constants that were experimentally derived. If the block204, ofFIG. 2, is determined to be a non-blocking artifacts block, the block204is sent on without any additional filtering. A blocking filter bus708may provide the output of the blocking level calculator702to the next level system, not shown.

Referring now toFIG. 8, therein is shown a block diagram of a block correlation processor800, in an embodiment of the present invention. The block diagram of the block correlation processor800depicts an execution array (XAR)802, such as a math processor or dedicated logic capable of performing mean squared error (MSE) calculations, coupled to a storage device804. The storage device804may be a random access memory that is capable of holding many of the blocks204, ofFIG. 2. The storage device804may be managed by a controller806for providing information on the blocks204to the execution array802. The controller806may receive the video filter bus608for mapping the storage device804with the blocks204. A correlation output808may be sourced from the execution array802as an indication of the similarity between the blocks204that may be adjacent within the full picture, not shown.

Referring now toFIG. 9, therein is shown a diagram of a segment900of a video picture during a block-wise similarity measurement. The diagram of the segment900depicts the blocks204arranged in an adjacent pattern as in the video picture of which they are a part. During the block-wise similarity measurement the correlation processor800, ofFIG. 8, may overlay the current unit of the block204on the blocks204that are adjacent to it. The correlation of the blocks204may be judged by applying equation 12 as shown below.

In equation 12, bm,n(i, j) denotes the pixels202in the block204in position (i,j), k denotes the vertical shifting position relative to the block204being analyzed, l denotes the horizontal shifting position relative to the block204being analyzed, and M denotes the block side length as shown inFIG. 2. For instance, M=8 for 8×8 block. An analysis area902indicates that the block-wise similarity measurement has stepped a count of 1 in the vertical and horizontal directions. During the block-wise similarity measurement, the analysis area902will be stepped in the vertical and horizontal directions independently. The step will continue between 1 and M in both the vertical and horizontal directions until the adjacent blocks are measured.

Referring now toFIG. 10, therein is shown a diagram of a two dimensional cross filter1000, in an embodiment of the present invention. The diagram of the two dimensional cross filter1000depicts a center tap1002, which may have 12 taps in addition to the center tap1002to support either horizontal or vertical filtering of the video data106, ofFIG. 1. The center tap1002may have vertical taps1004positioned symmetrically above and below the center tap1002in a vertical orientation. Likewise, there are horizontal taps1006positioned symmetrically right and left of the center tap1002in a horizontal orientation. There may be six of the vertical taps1004and six of the horizontal taps1006in an embodiment of the present invention. Other numbers of the vertical taps1004and the horizontal taps1006may be possible. The two dimensional cross filter1000may also be known as a symmetrical filter due to the position of the taps and the weighting of a filter coefficient1008established for the vertical taps1004and the horizontal taps1006.

Let C(k,1) denote the filter coefficient1008for the pixel202, ofFIG. 2, in position (i+k, j+l), the initial coefficient value can be obtained by following procedure.

In the present invention, symmetrical filtering is designed for horizontal and vertical filtering. At most seven tap filtering is applied along each direction. Therefore, we have a 13 tap cross filtering as shown inFIG. 10. Before the filtering, we first calculate the horizontal filter coefficients as following two steps:

First calculate C(0, 1) according to the above procedure for l=−3 . . . 3. Then adjust the horizontal filter coefficient values as defined by equation 18 below.
C(0,l)=min(C(0,l),C(0,−l))   (Equation 18)

The vertical filter coefficients may then be calculated by applying the following two steps. First calculate C(k,0) according to the above procedure for k=−3 . . . 3. Then adjust the vertical filter coefficient values as defined by equation 19 below.
C(k,0)=min(C(k,0),C(−k,0))   (Equation 19)

When analyzing the block204, ofFIG. 2, for the blocking artifacts, it is understood that only the pixels202, ofFIG. 2, in the boundary positions, such as the first column212, ofFIG. 2, the last column214, ofFIG. 2, the first row216, ofFIG. 2, or the last row218, ofFIG. 2, may be affected. The orientation of the blocking artifact, if it is present, will be determined by the dominant direction as calculated in equations 6 and 8 above. If the dominant direction is not vertical, indicated by Domiancelevel≠0, a one dimensional seven tap filter may be applied to the pixels202in the first column and the last column respectively as defined in equation 20 below.

If the dominant direction is not horizontal, indicated by Domiancelevel≠1, a one dimensional seven tap filter may be applied to the pixels202in the first row and the last row respectively as defined in equation 21 below.

In both equation 20 and equation 21, the filter coefficients for the boundary may be calculated as defined in equation 22 as defined below.

All of the pixels202remaining in the block204may be filtered using the 13 tap two dimensional cross filter1000as applied by equation 23, which is defined below.

The resulting filtering process is based on the analysis of the video data106, ofFIG. 1, and may be applied to each of the pixels202in the block204based on its' position and the relative intensity within the block204as compared to the blocks204in adjacent positions.

Referring now toFIG. 11, therein is shown a flow chart of a video system with blocking artifact filtering1100for applying the video system with blocking artifact filtering100in an embodiment of the present invention. The system1100includes analyzing video data, having a block, in a block1102; performing a transition change detection for determining a spatial intensity transition within the block in a block1104; performing a block-wise similarity measurement on the block in the video data for identifying a blocking artifact in a block1106; and filtering with a two dimensional cross filter every pixel in the block for removing the blocking artifact in a block1108.

It has been discovered that the present invention thus has numerous aspects.

A principle aspect that has been unexpectedly discovered is that the present invention can effectively estimate the blocking artifacts level from the raw data

Another aspect is that the present invention can take care of the pixels far away from the picture boundary. In doing so, the general encoding artifacts are also reduced

Thus, it has been discovered that the blocking artifact filter system of the present invention furnishes important and heretofore unknown and unavailable solutions, capabilities, and functional aspects for improving the visual quality of low definition signals displayed on High Definition Televisions (HDTV). The resulting processes and configurations are straightforward, cost-effective, uncomplicated, highly versatile and effective, can be surprisingly and unobviously implemented by adapting known technologies, and are thus readily suited for efficiently and economically manufacturing HDTV devices fully compatible with conventional manufacturing processes and technologies.