Motion detection in video fields

A method for detecting motion in video fields of video data, comprises the steps of: calculating texture information for a pixel in the video fields; determining a threshold value as a function of the calculated texture information; calculating a differential value for the pixel; and detecting motion in the video fields as a function of the determined threshold value and the calculated differential value.

FIELD OF INVENTION

This invention generally relates to video processing, and, in particular, to motion detection in video fields for deinterlacing of the video fields for display.

BACKGROUND

Video frames are typically encoded in an interlaced format comprising a first video field (e.g., a top video field) and a second video field (e.g., a bottom video field), each video field having alternating lines of the video frame and each field being temporally separated. Video images are typically encoded and transmitted to a receiver in such an interlaced format as a compromise between bandwidth and video image resolution. Since interlaced video frames are displayed using only half the lines of a full video frame, less system bandwidth is required to process and display these video frames. However, since the human eye typically cannot resolve a single video field, but rather, blends the first field and the second field, the perceived image has the vertical resolution of both fields combined.

Some types of receivers, including computers, televisions, mobile phones, computing tablets, etc., may require the use of de-interlaced video frames instead of interlaced video frames. For such receivers, the video frames encoded in an interlaced format must be de-interlaced prior to display. Typically, any missing pixels from the video frame are interpolated using the pixels of the first video field and the second video field.

There are several well-known methods to construct de-interlaced video frames. One such method is commonly referred to as the “bob” method in which a de-interlaced video frame is constructed from a single video field that is vertically interpolated. Whether to rely on a spatial or a temporal interpolation to interpolate an image data is decided by detecting the motion of a subject in the picture. Specifically, spatial interpolation is used to interpolate image data for pixels that are sensing a subject in motion, and temporal interpolation is used to interpolate image data for pixels that are sensing a motionless subject. In this way, by switching interpolation methods according to the state of motion of the subject being sensed by individual pixels, it is possible to faithfully reproduce the sensed subject in each field of the picture being played back.

Conventionally, such detection of motion is achieved by calculating differences of the image data of identical pixels among even-numbered and odd-numbered fields, and then comparing those differences with a predetermined threshold value. If the differences are greater than the threshold value, the subject being sensed by the pixels in question is recognized to be in motion.

In this way, by comparing the field-to-field differences of the image data of identical pixels with a constant, predefined threshold value, whether the subject being sensed by the pixels for which image data is going to be interpolated is in motion or not is judged. However, as long as such a threshold level is kept constant, for example, in a case where motion was present up to the field immediately previous to the one currently being reproduced but no motion is present any more in the current field, the motion that had been recognized just up to the previous field leads to an erroneous judgment that the motion is still present in the current field. In addition, the predefined threshold value has no relationship between any of the other pixels in the current video field/frame, which can lead to inaccurate results during interpolation.

This makes faithful reproduction of the real image impossible, and sometimes causes flickering or the like while a motion picture is being played back. Therefore, it would be desirable to provide new methods and systems for motion detection in video fields that can use an adaptive threshold value, conserve processing power, and increase system bandwidth.

SUMMARY OF INVENTION

An object of this invention is to provide methods and systems for motion detection in video fields that are adaptive to texture information and combing information of the video fields.

Another object of this invention is to provide methods and systems for motion detection in video fields that minimize the use of memory.

Yet another object of this invention is to provide methods and systems for motion detection in video fields that can be used for deinterlacing.

Briefly, the present invention discloses methods and systems for detecting motion in video fields of video data, comprising the steps of: calculating texture information for a pixel in the video fields; determining a threshold value as a function of the calculated texture information; calculating a differential value for the pixel; and detecting motion in the video fields as a function of the determined threshold value and the calculated differential value.

An advantage of this invention is that methods and systems for motion detection in video fields are provided that are adaptive to texture information and combing information of the video fields.

Another advantage of this invention is that methods and systems for motion detection in video fields are provided that minimize the use of memory.

Yet another advantage of this invention is that methods and systems for motion detection in video fields that can be used for deinterlacing.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description of the embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration of specific embodiments in which the present invention may be practiced.

FIG. 1illustrates a flow chart of the present invention for motion detection in video fields. In a video stream, multiple video fields can be decoded for display. Motion detection can be used to improve performance in decoding of the video fields.

Texture information can be calculated10for a pixel in the video fields to detect motion for the pixel. For instance, texture information can include a vertical infield texture value txt1and a horizontal intra-field texture value txt3. The vertical infield texture value txt1can be calculated for a pixel having a position (x,y) in a current video field at time t by applying a low pass filter (“lpf”) to an absolute difference in luma values f(a,b,c) of neighboring vertical pixels (i.e., pixels at (x, y−1) and (x, y+1) in a previous field (i.e., time=t−1). The following equation can be used to calculate txt1:
txt1(y,x,t−1)=lpf*|f(y−1,x,t−1)−f(y+1,x,t−1)|  Equation [1]

The horizontal intra-field texture value txt3can be calculated for the pixel having position (x,y) in the current video field at the time t by applying a min, max, or average function G(d,e,f) on the difference of textures in different video fields. Each texture txt is based on a single video field, where the video fields can be of the same polarity. Same polarity can mean that the two video fields are both in top fields or both in bottom fields. The texture txt3can be calculated using the following equation:
txt3(y,x,t−1)=G(txt(y,x,t),txt(y,x,t−2)|)  Equation [2]
where txt(a,b,c)=lpf*G(|f(a−2,b,c)−f(a,b,c)|,|f(a+2,b,c)−f(a,b,c)|).

An adaptive threshold thd can be calculated for motion detection21as a function of the calculated texture information. For instance, the adaptive threshold can equal:
thd=α*txt1+β*txt3  Equation [3]
where α and β are predefined constants. The predefined constants α and β can be determined based upon empirical and/or statistical analysis. For instance, α can be set to 0.3 and β can be set to 0.5 for luma components.

In addition, differential values for the pixel can be calculated14for the pixel in the various video fields. The differential value for the pixel is calculated based on the difference of the luma values f(a,b,c) for that pixel position in two or more video fields. Alternatively, chroma values for the pixel position can be used, as well as other video characteristics for the pixel position. Typically, the difference in luma values can be based on the luma value in the current field (e.g., time=t) and the luma value in the previous field that has the same polarity as the current field (e.g., two fields away, time=t−2). This differential value can be referred to as Dif02for the pixel, which can equal:
Dif02(y,x,t,t−1)=f(y,x,t)−f(y,x,t−2)  Equation [4]

Additionally other differential values can be calculated depending on the number of video fields used to detect motion for a particular pixel of the current video field. In the current example, there are a total of three video fields, including the current video field, the previous field, and the previous, previous video field. Thus, the other differentials can equal:
Dif01(y,x,t,t−1)=f(y,x,t)−f(y,x,t−1)  Equation [5]
Dif21(y,x,t−2,t−1)=f(y,x,t−2)−fi(y,x,t−1)  Equation [6]
where fi(a,b,c) is an interpolated value for the pixel at the given time. The other differentials Dif01and Dif21can also be used for determining an adaptive combing value (not shown in the flow chart, but illustrated inFIG. 4).

Motion detection can be determined for the pixel in the video fields16as a function of the determined threshold12and the calculated differential values14. For instance, the differential value Dif02and the adaptive threshold are compared to determine whether motion is detected for the respective pixel. If the differential value Dif02is greater than the adaptive threshold, then motion is detected for the pixel; else, motion is not detected for the pixel.

FIG. 2illustrates a video field at a predefined time. A video field20of a video stream can comprise pixels in an array format of rows and columns. The video field20can comprise a pixel location22at row y and column x. The methods and systems of the present invention can be used to generate a one bit motion flag for each pixel in the video field20, and in particular for the pixel location22.

FIG. 3illustrates various video fields at various times. The video field20of the video stream can be at time t. A video field30of the video stream can be at time t−1, and be the previous field from the video field20. A video field34can be at time t−2 and be two previous fields from the video field20. The pixel location22can be at row y and column x in video field20. A pixel location32can be at row y and column x in video field30. The pixel location36can be at row y and column x in video field34. The methods and systems of the present invention can use multiple video fields to generate one bit motion values for each pixel in the video fields for decoding of the video fields.

FIG. 4illustrates a block diagram of the present invention for motion detection in video fields. A motion detector of the present invention comprises a differential calculator40, a textual information calculator42, an adaptive combing block44, an adaptive threshold calculator46, a motion comparison block48, a field motion calculator50, a sure motion calculator52, a motion field delay block54, and an alpha calculator56.

The luma functions for a current field f(t), a first previous field f(t−1), and a second previous field f(t−2) are inputted to the differential calculator40. The differential calculator calculates various differential values for a pixel position in the various fields, including the differential values Dif02, Dif01, and Dif21. The differential values Dif01and Dif21are inputted to the adaptive combing block44for determining an adaptive combing value cmb. The adaptive combing value can be equal to the following:
cmb=G(Dif(x,y,t t−1),Dif(x,y,t,t−2))  Equation (7)
The differential value Dif02is inputted to the motion comparison block48and the field motion calculator50.

The luma functions for the current field f(t), the first previous field f(t−1), and the second previous field f(t−2) are also inputted to the textual information calculator42. The textual information calculator42determines textual information for the video fields, including txt1and txt3. The determined textural information is inputted to the adaptive threshold calculator46and the field motion calculator50.

The adaptive threshold calculator46receives the textual information and the combing value cmb, and calculates an adaptive threshold. The adaptive threshold can be updated as needed or required. For instance, the threshold result of Equation [3] can be compared with a scaled combing value. The larger of two values can be selected as a new adaptive threshold value.

The adaptive threshold is outputted to the motion comparison block48. The motion comparison block48compares the differential value Dif02and the adaptive threshold to detect motion for the respective pixel. The motion comparison block48can output a one-bit motion value (e.g., the motion value can be as follows mot(x,y,t)) to indicate whether a motion has been detected for the respective pixel. For instance, a 0 motion value can indicate that there is no motion detected for the pixel, and a 1 motion value can indicate that motion is detected for the pixel. Thus, the motion value is equal to zero when the Dif02is greater than the adaptive threshold; and the motion value is equal to one for any other cases. Next, the motion value can be stored by the motion field delay54, and inputted to the sure motion calculator52.

The motion field delay54can store a motion value for each of the pixels of the current field and other fields. A motion history (“hist”) can be stored as the following equation:
hist(y,x,t)=L(hist,mot),  Equation [8]
where hist is an iterative function for indicating the motion/static history of a video field. For instance, assume hist(y,x,t−2)=[mot(y,x,t−8),mot(y,x,t−6),mot(y,x,t−4),mot(y,x,t−2)], then hist(y,x,t) can comprise 4-bits for each pixel since hist(y,x,t)=L(hist(y,x,t−2), mot(y,x,t)), where L is a function operator for the various motion values.

The motion values and hist can be read, and then used by the sure motion calculator52to confirm that motion is being detected for the current field (i.e., mot(y,x,t)) and/or other fields (e.g., from the motion field delay54, including hist(y,x,t−2), hist(y−1,x,t−1) and hist(y+1,x,t−1)). If motion is confirmed, then the sure motion calculator52can send a sure motion flag to the alpha calculator56. If motion is not confirmed, then the sure motion calculator52can send a still motion flag to the alpha calculator56.

The field motion calculator50can receive the differential value Dif02, the adaptive combing cmb and the textual information for generating a field motion value for use by the alpha calculator56. The field motion calculator50calculates a quantified field motion (“fMt0”) between the current video field (time=t) and the previous, previous video field (time=t−2), which can equal:
fMt0=|Dif(y,x,t,t−2)|−H(I(txt1,txt3),cmb),  Equation [9]
where H(k,l)=a1*k+a2*l+a3and I(m,n)=b1*m+b2*n+b3. The values a1, a2, a3, b1, b2, and b3can be predefined and can be implemented by programmable registers.

The field motion calculator50also calculates a quantified field motion (“fMt1”) between the previous, previous video field (time=t−2) and the previous video field (time=t−1), which can equal:
fMt1=|dif(y,x,t−1,t−2)|K(I(txt1,txt3),cmb),  Equation [10]
whereK(k,l)=a1*k+a2*l+a3.

The alpha calculator56uses the field motion calculator and the motion flags to determine an alpha for use by a blender during the decoding of the respective fields. Alpha can equal the following:
alpha=M(fMt0,mot,pSM,pSS),  Equation[11]
where pSM is the sure motion flag, pSS is the sure still flag, and M(a,b,c,d) function is as follows:

if pSM(y,x,t−1) is true, then there is motion and alpha=a maximum value (e.g. 15 for 4-bit value);

if pSS(y,x,t−1) is true, there there is no motion and is static, thus alpha=a minimum value (e.g. 0); and

otherwise, alpha is scaled as a function of the fMt0and mot.

While the present invention has been described with reference to certain preferred embodiments or methods, it is to be understood that the present invention is not limited to such specific embodiments or methods. Rather, it is the inventor's contention that the invention be understood and construed in its broadest meaning as reflected by the following claims. Thus, these claims are to be understood as incorporating not only the preferred apparatuses, methods, and systems described herein, but all those other and further alterations and modifications as would be apparent to those of ordinary skilled in the art.