Pixel interpolation with edge detection based on cross-correlation

A pixel interpolation process is based on detection of a potential edge in proximity to a pixel being estimated, and the angle thereof. The potential edge and its angle is determined based on filtering of offset or overlapping sets of lines from a pixel window centered around the pixel being estimated and then cross-correlating the filter results. The highest value in the correlation result values represents a potential edge in proximity to the pixel being estimated and the index of the highest value represents the angle of the potential edge. This information is used in conjunction with other information from the cross-correlation and analysis of the differences between pixels in proximity to verify the validity of the potential edge. If determined to be valid, a diagonal interpolation based on the edge and its angle is used to estimate the pixel value of the pixel. Otherwise, an alternate interpolation process, such as vertical interpolation, is used to estimate the pixel value for the pixel.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to video processing and more particularly to video deinterlacing or video upscaling.

BACKGROUND

Video systems often estimate pixel values for pixels that are not available in the original video data, such as when converting interlaced video to deinterlaced video or when upscaling video to a higher resolution. Conventional techniques for estimating a pixel value for a missing pixel typically rely on some form of interpolation between lines above and below the line on which the missing pixel will be located. Often, such interpolation processes utilize edge detection to identify whether a pixel value being estimated lies along an edge in the content of the frame, and interpolate for the pixel value accordingly. However, many of these edge-dependent interpolation processes fail to account for the direction of the edge, which can lead to significant interpolation errors and thus introduce undesirable visual artifacts, and those conventional interpolation techniques that do account for the direction of the edge often require considerable processing effort to do so, such as requiring analysis over many successive fields. Moreover, conventional edge-dependent interpolation techniques fail to properly evaluate the validity of the detected edge, thereby frequently calculating incorrect pixel values based on a falsely-detected edge.

DETAILED DESCRIPTION

FIGS. 1-4illustrate exemplary techniques for pixel interpolation in a video processing device. The pixel interpolation is based on detection of a potential edge in proximity to a pixel being estimated, and the angle thereof, based on filtering (vertical high-pass/horizontal low-pass) of offset or overlapping sets of lines from a pixel window centered around the pixel being estimated and then cross-correlating the filter results. The highest value in the correlation result values represents a potential edge in proximity to the pixel being estimated and the index of the highest value within the correlation result values represents the angle of the potential edge. This information is used in conjunction with other information from the cross-correlation and analysis of the differences between pixels in proximity to the pixel being estimated to verify the validity of the potential edge (i.e., whether the potential edge is a true edge). If the potential edge is determined to be valid, the video processing device uses diagonal interpolation based on the edge and its angle to estimate the pixel value of the pixel. Otherwise, if the potential edge is determined to be invalid, the video processing device uses an alternate interpolation process to estimate the pixel value for the pixel, such as a vertical interpolation between pixels above and below the pixel being estimated. The video processing device implementing these techniques can include a deinterlacer and the pixel interpolation therefore can be for the purposes of interpolating the pixels of one field based on the pixel data of another field to create a video frame. Alternately, the video processing device can include a video scaler and the pixel interpolation therefore may be for the purposes of upscaling an original video frame to generate an upscaled video frame.

FIG. 1illustrates a video processing device100in accordance with at least one embodiment of the present disclosure. The video processing device100includes a pixel window buffer102, an edge detection module104, a pixel interpolation module106, and an output buffer108. The edge detection module104includes a spatial filter module110, a spatial filter module112, a correlator module114, and a pick module116. The functionality of the various modules of the video processing device110as illustrated inFIGS. 1-3can be implemented as hardware, firmware, one or more processors and software, or a combination thereof. To illustrate, the functionality of certain components can be implemented as discrete logic, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and the like, while other functions of certain components can be implemented as software instructions executed by one or more processors of a processing system. Further, some or all of the components can implemented in a processing device designed based on a set of register-transfer-level (RTL) or Verilog instructions that define the functionality of the processing device and which are synthesized to generate the electrical and electronic design of the processing device.

In operation, the video processing device100receives input video data via an input118and the pixel window buffer102buffers pixel data for pixels of a plurality of lines of the input video data. The input video data may include an interlaced field for an implementation of the video processing device100as a deinterlacer or an original frame of video for an implementation of the video processing device100as a video scaler. The pixel data may be provided or obtained from any of a variety of sources, such as a frame buffer at the output of a video decoder. The lines of the pixel window are centered around or otherwise encompass the estimated pixel. In the case of a deinterlacing application, the plurality of lines of the pixel window buffer102includes lines from the field other than the field of the estimated pixel. To illustrate, if the estimated pixel is from an even field, the pixel window buffered in the pixel window buffer102is composed of the lines of a temporally-proximate odd field that would be centered around the estimated pixel as though the even field and the odd field were combined into a single frame. In the case of an upsampling application, the plurality of lines of the pixel window are composed of adjacent lines of an original video frame and the pixel being estimated is for a line to be inserted between two lines from the original video frame so as to upscale the original video image to an upscaled video image.

The spatial filter module110is configured to apply a filter matrix to the pixel values of one set of lines of the pixel window to generate one set of filter results and the spatial filter module112is configured to apply the filter matrix to the pixel values of another set of lines of the pixel window (whereby the two sets are offset or overlap by at least one line) to generate another set of filter results. The filter matrix is configured so as to provide high-pass filtering in the vertical direction so as to obtain edge information and to provide low-pass filtering in the horizontal direction so as to filter out noise. The correlator module114cross-correlates the two sets of filter results to generate a set of correlation result values and the pick module116identifies an angle of a potential edge in proximity to the estimated pixel based on the correlation result values of the correlator module114. The pixel interpolation module106then uses the angle of the potential edge indicated by the pick module116, as well as other information determined from the correlation result values and other analyses of pixel values from the pixel window to determine whether to calculate the pixel value of the estimated pixel based on a diagonal interpolation using the angle of the potential edge or to use an alternate interpolation process independent of the angle, such as a direct vertical interpolation. After estimating the pixel value via the selected interpolation method, the pixel value for the estimated pixel is buffered in the output buffer108(which may include, for example, a frame buffer). The original pixel data and the estimated pixel values generated by the video processing device100then may be provided via an output120as output video data to another video processing device (e.g., a display controller, an encoder, etc.) as deinterlaced video (for a deinterlacing application) or as upscaled video (for an upscaling application).

FIG. 2illustrates an example implementation of the edge detection module104of the video processing device100ofFIG. 1in accordance with at least one embodiment of the present disclosure. As illustrated, the edge detection module104includes spatial filter modules210and212(corresponding to the spatial filter modules110and112,FIG. 1), a correlation weighting module213, a correlation module214(corresponding to the correlation module114,FIG. 1), and a pick module216(corresponding to the pick module116,FIG. 1). The pick module216includes a max positive select module204, a max negative select module206, a multiplexer (MUX)208, and a pick control module209.

In the depicted example, the pixel window buffer102(FIG. 1) buffers a pixel window202composed of four (4) adjacent lines, each line having nineteen (19) pixel values. Those pixels having pixel values included in the pixel window202are represented inFIG. 2as filled circles, and those pixels to be estimated or those pixels that have been estimated are represented inFIG. 2as unfilled circles. Further, the pixel to be estimated in the processes described below (estimated pixel203) is depicted in the pixel window202as a larger circle with an “X” located in the center of the circle. To illustrate, the estimated pixel203could be located at line3of an even field, and thus the lines in the pixel window202can include the lines of an associated odd field that would encompass, or be centered around, the estimated pixel203at line3(e.g., lines0,2,4, and6) for a deinterlacing application. Although the pixel window202is illustrated as a 4×19 pixel window, different size pixel windows may be used without departing from the scope of the present disclosure. To illustrate, while a width of nineteen pixels has been found to be sufficient for estimating the angle of a potential edge to a satisfactory degree, a wider pixel window may be used to increase the angle resolution, or a narrower window may be used to reduce the computational effort. Likewise, in the illustrated example, the spatial filters110and112employ two 3-tap vertical filters (with example coefficients of {−1, 2, −1}), thereby resulting in the use of four lines for the filtering process. However, while 3-tap filters are illustrated in view of hardware efficiency, filters with more taps can be used in accordance with the scope of the present disclosure. To illustrate, a larger order filter response can be implemented with, for example, a 5-tap filter, which would then result in the use of six lines in the pixel window202for the filtering process.

As the pixel window202includes four lines (e.g., lines0,2,4, and6) in the illustrated example, the pixel window202can be logically partitioned into two sets of lines that are offset by one line (and overlap by two lines): a first set composed of lines0,2, and4; and a second set composed of lines2,4, and6. In this example, the spatial filter module210applies a filter matrix to the first set to generate a set222of filter results (also identified herein as filter results PA( )) and the spatial filter MODULE212applies a filter matrix to the second set to generate a set224of filter results (also identified herein as filter results PB( )). The applied filter matrix selected or configured to high-pass filter in the vertical direction so as to identify high-frequency edges. The filter matrix further takes the results of the high-pass filtering and applies a low-pass filter to reduce noise. To illustrate, the filter matrix can employ the coefficients {−1, 2, −1} as an approximation of a second-derivative operator, and thus the filter matrix can take the form of:

H=[-12-1]⁡[111]⁡[-1-1-1222-1-1-1]
where H represents the filter matrix applied by the spatial filter modules210and212. In this example, the second-derivative operator will provide a zero-crossing in the center of an edge, if any, and the low-pass filtering aspect takes three adjacent high-pass results and averages them. Thus, each of the two spatial filters110and112generate edge profiles that the correlation module214attempts to match, as described below. Although an example filter matrix is described above, the present disclosure is not limited to this example, but rather any of a variety of filter matrix configurations advantageous for edge detection.

As illustrated inFIG. 2, the application of the filter matrix to the first three lines of 19 pixels from the pixel window202and to the overlapping last three lines of 19 pixels from the pixel window202results in sets222and224, respectively, each having seventeen (17) filter result values, which are indexed in this example from −8 to 8. Note that althoughFIG. 2depicts sets222and224with only positive filter result values for ease of illustration, a filter result value also may be negative depending on the particular pixel values in the pixel window202and the values implemented in the filter matrix. The correlator module214then cross-correlates sets222and224to generate a set226of correlation result values (also identified herein as correlation result values PC(n)). Any of a variety of cross-correlation techniques may be implemented by the correlator module214. To illustrate, in one embodiment, the cross-correlation technique implemented by the correlator module214is represented by the function:

PC⁡(n)={∑k⁢⁢PA⁡(k)*PB⁡(n+k);n⁢⁢even-∞;otherwise}
As illustrated by the above function, the odd set of Pc(n) are set to—infinity for ease of implementation. This approach limits the potential angle so four positive and four negative angles, and thus reduces the number of multiply and add operations. However, in other embodiments the odd set can be calculated in the same manner as the even set so as to provide a finer resolution for the angle determination process. In the example ofFIG. 2, the cross-correlation of the sets222and224(each having seventeen filter result values) results in the set226of correlation result values having thirty-three (33) values, indexed in the example ofFIG. 2from −16 to 16 (noting that the odd-indexed values of—infinity are not depicted inFIG. 2).

The application of the filter matrix to each set of lines in the pixel window202has the effect of identifying changes in the contrast within the corresponding sets of pixels, and thus indicative of a possible edge in the corresponding set of pixels. The cross-correlation of the sets222and224generated by the applications of the filter matrix to these sets of lines thus has the effect of comparing the two sets of pixels to indicate whether the same potential edge runs through both sets of lines in proximity to the estimated pixel, and if so, the angle of the potential edge. The potential edge within a pixel window thus is identified as the highest correlation result value in the set226of correlation result values, and the index of this highest correlation result value represents the angle of the potential edge. To illustrate, the cross-correlation of the values of the illustrated sets222and224can result in the values of the illustrated set226of correlation result values, whereby the highest correlation result value is located at a position indexed as −4, and thus the angle of the potential edge can be identified as:

θ=a⁢⁢tan⁡(y2)*180π
where θ represents the angle of the potential edge and y represents the index of the highest correlation result value in the set226.

Further, in one embodiment, the set226of correlation result values is weighted by the weighting module213to generate a weighted set227of correlation result values (Pc′(n)) so as to emphasize the correlation result values representative of steeper angles over those representative of narrow angles. In one embodiment, the weighting module213accomplishes this weighting by using a set of weight values corresponding to the position of the correlation result values (e.g., a different weight value corresponding to each position between −16 and +16), whereby the weight values increase as they approach the center of the set226(i.e., as the corresponding position approaches 0).

While the highest correlation result value may represent a potential edge, it also may simply be a result of noise or some other artifact. Accordingly, as discussed in greater detail below, the relative magnitude of the next highest correlation result value in the other half of the weighted set227of correlation result values may be used as an indicator of the likelihood that the potential edge is a true edge. To facilitate identification of the highest overall correlation result value and the next-highest correlation result value in the other half of the weighted set227of correlation result values, the correlation result values of a first half of the weighted set227of correlation result values (e.g., the correlation result values at indices −16 to −1, or at indices 0 to 16 for an indexing from 0 to 32) are provided to the max negative select module206, which identifies the highest correlation result value of the first half and its index as values max_neg and neg_index, respectively. The correlation result values of the second half of the set226(e.g., the correlation result values at indices 0 to 16, or at indices 17 to 32 for an indexing from 0 to 32) are provided to the max positive select module206, which identifies the highest correlation result value of the second half and its index as values max_pos and pos_index, respectively. The pick control209determines the highest value between max_pos and max_neg and directs the MUX208via signaling SEL to provide the identified highest value of values max_pos and max_neg as the value max_best and its index as value max_index, as well as to provide the other value as max_other (and its index as value other_index). Thus, the value max_best identifies the highest overall correlation result value in the weighted set227, the value max_index identifies its index, or position, within the weighted set227, the value max_other identifies the highest correlation result value in the half of the weighted set227in that does not include highest overall correlation result for the weighted set227and the value other_index identifies its index, or position, within the weighted set227. The values max_best, max_index, and max_other then are provided to the pixel interpolation module106for use in determining whether the potential edge represented by the value max_best is a valid edge, and thus whether to use a diagonal interpolation process based on an angle of the valid edge as represented by the value max_index or another interpolation process that does not rely on the potential edge. Alternately, rather than weighting the correlation results and using the weighted set227, the unweighted set226of correlation result values may be used in the process described above.

FIG. 3illustrates an example implementation of the pixel interpolation module106of the video processing device100ofFIG. 1in accordance with at least one embodiment of the present disclosure. In the depicted example, the pixel interpolation module106includes a threshold selector module302, a weighting value data store304, a variance module306, a vertical difference module308, a pick difference module310, an edge measure module314, a selector module316, and an interpolator module318. The pixel interpolation module106further can include a fade module320and a median filter module322.

In operation, the pixel interpolation module106determines whether the potential edge determined by the edge detection module104(FIG. 1) is a valid edge and, based on the validity or the invalidity of the potential edge, selects either a diagonal interpolation process based on the angle of the detected potential edge or an alternate interpolation process that is independent of the potential edge. In one embodiment, the pixel interpolation module106determines the validity of the potential edge via analyses and comparisons of the various differences between pixel values in proximity to the estimated pixel, whereby one or more of these differences is weighted or otherwise adjusted based on a threshold weight value determined based on the values max_best, max_index, and max_other determined by the edge detection module104as described above.

In one embodiment, the differences between pixels in proximity to the estimated pixel include a variance Var, a vertical difference D90, and a pick difference DP(i.e., a diagonal difference). Accordingly, the variance module306calculates the variance Var (as a statistical variance or as a mean squared error) between a set of pixels of the pixel window202that are in proximity to the estimated pixel. For example, the variance can be calculated for a set330composed of three pixels from the line above the estimated pixel, three pixels from the line below the estimated pixel, the pixel directly above the estimated pixel in the second line above the estimated pixel, and the pixel directly below the estimated pixel in the second line below the estimated pixel, as illustrated inFIG. 3. It will be appreciated that the variance Var is a measurement of the difference among the pixels in proximity to the estimated pixel. As such, if the variance is small, the region around the estimated pixel is relatively smooth and thus a diagonal interpolation is less likely to produce a spurious result. Conversely, if the variance is large, there could be a sharp edge in the region of interest, and thus diagonal interpolation becomes more risky. To illustrate, a black line on a white background would result in a large variance. If two black values are not used in the interpolation process, the interpolated result will be incorrect and highly noticeable upon display (e.g., interpolating between black and white values would result in a visually distinct grey pixel value). However, a darker grey line on a lighter grey background would result in a smaller variance. Even if a dark grey value and a light grey value were incorrectly used in the interpolation process, the resulting interpolated pixel value would still be a shade of gray and thus less visually distinct from either the darker grey line or the lighter grey background.

Ideally, the vertical difference module308calculates the difference between pixels along a line that is perpendicular to the potential edge. However, the size of the pixel window202(FIG. 2) often does not provide sufficient dimensions to identify pixels that are adequately aligned with the perpendicular line. In such instances, the vertical difference module308instead calculates a vertical difference D90between pixels below and above the estimated pixel, such as, e.g., the vertical difference in a set332between three pixels above the estimated pixel and the corresponding three pixels below the estimated pixel. In one embodiment, the vertical difference D90is calculated as a weighted average of the vertical differences, whereby the weighting coefficients can be programmed or otherwise set by a user, manufacturer, provider, and the like. To illustrate using the set332ofFIG. 3, the vertical difference D90can be calculated as:
D90=(coef 1*(pal−pbl)+coef 2*(pad−pbd)+coef 3*(par−pbr))/3
where pal and pbl represent the pixels to the left of the estimated pixel in the line above and the line below, respectively, the estimated pixel, pad and pbd represent the pixels directly above and directly below, respectively, the estimated pixel, and par and pbr represent the pixels to the right of the estimated pixel in the line above and the line below, respectively, the estimated pixel. In this case, the vertical difference between the pixels directly above and below the estimated pixel typically is given greater weighting than the vertical differences between the pixels to the right and to the left of the estimated pixel (e.g., for a total weighting of 1, coef 1=coef 3=0.25 and coef 2=0.5). It will be appreciated that the vertical difference D90indicates the variation between pixels on one side of the potential edge and pixels on the other side of the potential edge. As such, a larger value for the vertical difference D90is more indicative of the validity of the potential edge, and vice versa.

The pick difference module310calculates the pick, or diagonal, difference DPbetween pixels of the pixel window202in accordance with the angle of the potential edge. To calculate the diagonal difference, the pick difference module310calculates a weighted sum of differences of two pixels along a line through the estimated pixel and differences of two pixels along one or more other lines parallel to this line. To illustrate, in the set334of pixels from the pixel window202, the pick difference310determines the difference between pixels335and336along a line through the estimated pixel having the same angle as the potential edge, the difference between pixels337and338along a parallel line to the left of the estimated pixel, and the difference between pixels339and340along a parallel line to the right of the estimated pixel. The pick difference module310then calculates the pick difference DPas a weighted average of these differences in a manner similar to the process described above for calculating the vertical difference D90using the same weighting coefficients or different weighting coefficients. It will be appreciated that the pick difference DPindicates the variation between pixels along the potential edge. As such, a smaller value for the pick difference DPis more indicative of the validity of the potential edge, and vice versa.

In one embodiment, a threshold weight value TW is determined based on the correlation result values and then used to weight one or more of the variance Var, the vertical difference D90, or the pick difference DP. The threshold weight datastore304(e.g., a register file, a cache, a memory, etc.) can be programmed or otherwise configured to store a plurality of threshold weight values between 0 to 1, such as, e.g., four threshold weight values TW1, TW2, TW3, and TW4, as well as a null threshold weight value (0). The threshold selection module302receives the values max_best, max_index, and max_other and selects one of the plurality of threshold weight values for output as the threshold weight value TW. The selection process employed by the threshold selection module302can be represented by the following algorithm of Table 1 for the example 4×19 pixel window context described above:

As illustrated by the algorithm of Table 1, if the value max_best (representing the highest correlation result value) is less than or equal to zero, there is no correlation between the two sets of lines, and thus no valid edge. Accordingly, the threshold weight value TW is set to a very high value (e.g., 1×106) to ensure that an alternate interpolation process is selected by the interpolator module318for the estimated pixel. Further, if the value max_other is greater than 0 (thereby indicating some correlation in the opposite direction), and the angle represented by the value other_index is 14 degrees, the threshold weight value TW also is set to the very high value (meaning that the 14 degree angle is cautious in diagonal picks). Otherwise, if the value max_other is greater than 0 and the angle represented by the value other_index is 45, 26.6, or 18.4, the threshold weight value TW is selected from one of the four programmed threshold weight values TW1, TW2, TW3, and TW4(which, in this example, increase in value) based on the angle/index (max_index) of the potential edge. Accordingly, the threshold weight selection process described above has the effect of selecting a greater value for the threshold weight value TW as the angle of the potential edge increases. As such, a smaller angle indicated by the value max_index results in a smaller value for the threshold weight value TW, thereby resulting in a more conservative approach to validating the potential edge.

The threshold weight value TW, once selected by the threshold selection module302, is used to weight one or more of the variance Var, the vertical difference D90, or the pick difference DP. In the example ofFIG. 3, the threshold weight value TW is used to weight the variance Var via a weighting module312(e.g., a multiplier) to generate a weighted variance WV (WV=TW*Var). The edge measure module314then calculates an edge measure value EM that represents a measure of the validity of the potential edge based on a difference between the vertical difference D90and the pick difference DP, such as by calculating the square of the difference using the following equation:
EM=(D90−DP)2
For a valid edge proximate to the estimated pixel, the vertical difference D90would be relatively large and the pick difference DPwould be relatively small or even zero.

The selector module316receives the weighted variance WV and the edge measure value EM and determines whether the potential edge is valid based on a comparison of these two values. In the event that the edge measure value EM is greater than or equal to the weighted variance WV, the selector module316identifies the potential edge as valid and configures a signal336to reflect the valid status of the potential edge. Otherwise, in the event that the weighted variance WV is greater than the edge measure value EM, the selector module316identifies the potential edge as invalid and configures the signal336to reflect the invalid status of the potential edge.

The interpolator module318receives the signal336from the selector module316and selects an interpolation process for determining the pixel value of the estimated pixel based on the validity of the potential edge indicated by the signal336. In response to the signal336indicating that the potential edge is valid, the interpolator module318implements a diagonal interpolation process based on the angle of the potential edge (represented by the value max_index) to calculate the pixel value of the estimated pixel. To illustrate, the interpolator module318can interpolate the pixel value of the estimated pixel from the pixel338located along a potential edge340in the line above the estimated pixel and the pixel342located along the potential edge340in the line below the estimated pixel. Other diagonal interpolation calculations using the angle of the potential edge also may be used. However, in response to the signal336indicating that the potential edge is invalid, the interpolator module318uses an alternate interpolation process that does not rely on the potential edge or its angle, such as, e.g., a vertical interpolation between the pixel immediately above the estimated pixel and the pixel immediately below the estimated pixel.

In one embodiment, the resulting interpolated pixel value PVA(either from the diagonal interpolation process or an alternate interpolation process) is output to the output buffer108as the final pixel value for the estimated pixel. However, in certain instances, additional processing of the interpolated pixel value PVAmay be appropriate before the final pixel value is reached. In one embodiment, the fade module320calculates the processed pixel value PVBusing interpolated pixel value PVA, the vertical difference value D90, and the pick difference value DP. To illustrate, the fade module320can use the following equations to determine the processed pixel value PVB:

k=0.5*DP2*D90PVB=(1-k)*PVA+k*y_linear⁢_avg
where y_linear_avg represents the average of the pixel values of the pixels directly above and directly below the pixel to be interpolated.

Rather than directly output the interpolated pixel value PVAor the processed pixel value PVBderived from the interpolated pixel value PVAas the final pixel value for the estimated pixel, additional protective schemes may be implemented using one or both of these pixel values to further protect against the use of a pixel value for the estimated pixel that is determined based an erroneous evaluation of a potential edge as valid. As one technique for further validating the potential edge, one or both of the interpolated pixel value PVAor the processed pixel value PVBcan be input into the median filter module322along with one or more other values that represent alternate potential pixel values for the estimated pixel, whereby the median filter module322selects the median values of these plurality of input values as the median pixel value PVC, which then may be provided to the output buffer108as the final pixel value for the estimated pixel, or which may be subjected to further processing for determining the final pixel value for the estimated pixel. This grouping of the interpolated pixel value PVA(or a representation thereof) without other pixel values that potentially represent the estimated pixel and then selecting the pixel value of the estimated pixel as the median of this group acts as a final evaluation of the validity of the potential edge. If the interpolated pixel value PVAis far from the median value (i.e., an outlier) of the potential values for the estimated pixel, it was most likely an incorrect interpolation and thus should not have been used as the final pixel value for the estimated pixel regardless. Conversely, the greater the number of potential values that are close to the interpolated pixel value PVA, the more likely the potential edge was correctly judged as valid, and thus the more likely the interpolated pixel value PVAor another potential pixel value close in value to the interpolated pixel value PVAwill be the median value of the group and thus selected as the final pixel value for the estimated pixel.

Any of a variety of potential pixel values for the estimated pixel value can be input to the median filter332. To illustrate, in addition to one or both of the interpolated pixel value PVAor the processed pixel value PVB, the input pixels can include: the pixel value of the pixel immediately above the estimated pixel (P2,4); the pixel value of the pixel immediately below the estimated pixel (P4,4); the pixel value of the pixel in the same position as the estimated pixel from an earlier or later frame or field (P3,4); the pixel value of the pixel along the potential edge in the line above the estimated pixel (P2,k); the pixel value of the pixel along the potential edge in the line below the estimated pixel (P2,−k); and a value V representing a spatial/temporal relationship calculated using, for example, the equation:

v=12⁢(P2,4+P4,4)+12⁢(12⁢P3,4-14⁢(P1,4+P5,4))
where P1,4represents the pixel value of the pixel two lines above the estimated pixel from an earlier or a later frame or field, and P5,4represents the pixel value of the pixel two lines below the estimated pixel from an earlier or a later frame or field.

As described above, the edge detection module104identifies a potential edge in proximity to the estimated pixel using cross-correlation of the results of filtering offset sets of lines of a pixel window202and the pixel interpolation module106can use the angle of the potential edge to diagonally interpolate a pixel value for the estimated pixel. However, in at least on embodiment, the pixel interpolation module160uses one or more protective schemes to evaluate the likely validity of the potential edge before utilizing the potential edge in the interpolation process. Through this approach, the processing effort needed to identify a potential edge can be reduced while controlling the risk that an invalid edge does not result in an inaccurate pixel value for the estimated pixel, which could introduce undesirable visual artifacts into the display of the corresponding image.

As noted above, the modules of the video processing device100ofFIGS. 1-3can be implemented as hardware. However, in addition to hardware implementations adapted to perform the functionality in accordance with one of the embodiments of the present disclosure, such modules may also be embodied in one or more processors configured to execute instructions in software disposed, for example, in a computer usable (e.g., readable) medium configured to store the software (e.g., a computer readable program code). The program code causes the enablement of embodiments of the present invention, including the following embodiments: (i) the functions of the devices and methods disclosed herein (such as devices and methods upscaling/deinterlacing video); (ii) the fabrication of the devices and methods disclosed herein (such as the fabrication of devices that are enabled to upscale or deinterlace video); or (iii) a combination of the functions and fabrication of the devices and methods disclosed herein.

For example, this can be accomplished through the use of general programming languages (such as C or C++), hardware description languages (HDL) including Verilog, Verilog-A, HDL, VHDL, Altera HDL (AHDL) and so on, or other available programming and/or schematic capture tools (such as circuit capture tools). The program code can be disposed in any known computer usable medium including semiconductor, magnetic disk, optical disk (such as CD-ROM, DVD-ROM) and as a computer data signal embodied in a computer usable (e.g., readable) transmission medium (such as a digital, optical, or analog-based medium). It is understood that the functions accomplished and/or structure provided by the systems and techniques described above can be represented in a core (such as a GPU core) that is embodied in program code and may be transformed to hardware as part of the production of integrated circuits.

FIG. 4illustrates a processor device400in accordance with at least one embodiment of the present disclosure. The processor device400can include a set of instructions that can be executed to manipulate the processor device400to perform any one or more of the methods or functions disclosed herein. The processor device400may operate as a standalone device or may be connected, e.g., using a network, to other processor devices or peripheral devices.

In a networked deployment, the processor device may operate in the capacity of a server or as a client user computer in a server-client user network environment, or as a peer processor device in a peer-to-peer (or distributed) network environment. The processor device400can also be implemented as or incorporated into, for example, a portable display device. Further, while a single processor device400is illustrated, the term “system” shall also be taken to include any collection of systems or sub-systems that individually or jointly execute a set, or multiple sets, of instructions to perform one or more computer functions.

The processor device400may include a processor402, e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both. Moreover, the processor device400can include a main memory404and a static memory406that can communicate with each other via a bus408. As shown, the processor device400may further include a video display unit410, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid state display, or a cathode ray tube (CRT). Additionally, the processor device400may include an input device412, such as a keyboard, and a cursor control device414, such as a mouse. The processor device400can also include a disk drive unit416, a signal generation device418, such as a speaker, and a network interface device420.

In a particular embodiment, as depicted inFIG. 4, the disk drive unit416may include a computer readable storage device422in which one or more sets of instructions424, e.g. software, can be embedded. Further, the instructions424may embody one or more of the methods or logic as described herein. In a particular embodiment, the instructions424may reside completely, or at least partially, within the main memory404, the static memory406, and/or within the processor402during execution by the processor device400. The main memory404and the processor402also may include computer readable media. The network interface device420can provide connectivity to a network426, e.g., a wide area network (WAN), a local area network (LAN), or other network.

In accordance with various embodiments of the present disclosure, the methods described herein may be implemented, in whole or in part, by software programs executable by a processor device. The present disclosure contemplates a computer readable storage device that includes instructions or receives and provides instructions for execution responsive to a propagated signal, so that a device connected to a network can communicate voice, video or data over the network426. Further, the instructions424may be transmitted or received over the network426via the network interface device420.

In one embodiment, rather than being software instructions that directly implement the functionality described herein, the instructions424instead can implement design instructions representative of a hardware implementation of the above-described functionality that are then synthesized to determine the electrical and electronic design for a processing device that implements the above-described invention. To illustrate, these hardware-design instructions can include register transfer level (RTL) instructions, Verilog instructions, and the like.

While the computer readable storage device is shown to be a single storage device, the term “computer readable storage device” includes a single storage device or multiple storage devices, such as a centralized or distributed database, and/or associated caches and servers that store one or more sets of instructions. The term “computer readable storage device” shall also include any storage device that is capable of storing a set of instructions for execution by a processor or that cause a processor device to perform any one or more of the methods or operations disclosed herein.

In a particular embodiment, the computer readable storage device can include a solid-state memory such as a memory card or other package that houses one or more non-volatile read-only memories. Further, the computer readable storage device can be a random access memory or other volatile re-writeable memory. Additionally, the computer readable storage device can include a magneto-optical or optical medium. Accordingly, the disclosure is considered to include any one or more of a computer readable storage device or a distribution medium and other equivalents and successor media, in which data or instructions may be stored.

The term “another”, as used herein, is defined as at least a second or more. The terms “including”, “having”, or any variation thereof, as used herein, are defined as comprising. The term “coupled”, as used herein with reference to electro-optical technology, is defined as connected, although not necessarily directly, and not necessarily mechanically.

Other embodiments, uses, and advantages of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. The specification and drawings should be considered exemplary only, and the scope of the disclosure is accordingly intended to be limited only by the following claims and equivalents thereof.