Patent Description:
Video content contains many kinds of variations in different characteristics, such as luminance variations. The variations may introduce bias into the inter prediction. For example, the luminance variation in one of the inter predicted blocks from the current block may reduce the prediction accuracy, which reduces the final compression efficiency. If there is a change in luminance in one of the reference blocks, that reference block may not be a good prediction of the current block. When a reference block is not a good prediction, the residual of the difference between the current block and the prediction block is larger, which uses more bandwidth in the encoded bitstream.

<NPL>, describes the Joint Exploration Model <NUM> (JEM <NUM>) algorithm. The document describes inter alia Local Illumination Compensation (LIC) based on a linear model for illumination changes, using a scaling factor and an offset. A least square error method is employed to derive these parameters by using the neighboring samples of the current coding unit and their corresponding reference samples.

<NPL>, describes a compression method for handling local brightness variations in video sequences. The method does not explicitly code and transmit the weighting parameters used to predict the current block. Instead, the correlation of the current block with its spatial neighbors and temporal predictors.

<CIT> describes a video encoding method including selecting a plurality of reference blocks based on a plurality of motion vectors and setting weights assigned to the plurality of reference blocks independently for each of a plurality of regions of a current block in order to predict and encode the current block.

The claimed invention relates to template based adaptive weighting method, the related computer-readable storage medium, and the related apparatus, as defined by the independent claims. Particular embodiments are defined by the dependent claims. Other methods mentioned in the application help to understand the invention.

Described herein are techniques for a video coding system. In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of some embodiments. Some embodiments as defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

Some embodiments use a bi-prediction process that weights the reference blocks differently when generating a prediction block that is used to predict a current block of pixels. A first template region, such as an L-shaped region, of decoded neighboring pixels to a current block and template regions for the reference blocks that are being used to predict the current block are identified. The templates are used to calculate the weights that are used to blend the reference blocks to generate a prediction block when using bi-prediction. The weights may be calculated by a transcoder and signaled to the decoder, or may be calculated by the decoder.

When decoding a current block, the pixels of the current block are not known by the decoder and cannot be used to generate the weights. However, since the templates are close to the current block and the reference blocks, characteristics of the templates, such as the luminance of the templates, may be a good representation of the luminance of the current block and the reference blocks, respectively. The luminance may be the intensity of light and may change across the current block and at least one of the reference blocks. As a result, the transcoder or decoder can change the weights adaptively according to the luminance difference between the current block and each of the reference blocks. Specifically, a weight for a reference block may be smaller if there is bigger luminance gap between the current block and a first reference block compared to a second reference block and the current block. And, vice versa, a weight for a first reference block may be larger if there is smaller luminance gap between the current block and a second reference block compared to the first reference block and the current block. By adjusting the weights adaptively, the accuracy of bi-prediction can be enhanced and the efficiency of inter prediction may be improved. As a result, a better overall video compression performance can be obtained, which will save bandwidth in the video transmission.

<FIG> depicts a simplified system <NUM> for using weighted bi-prediction according to some embodiments. System <NUM> transcodes a source video asset, which may be any type of video, such as for a television show, movie, or video clip. The source video may need to be transcoded into one or more formats, such as one or more bitrates. In some embodiments, a server system <NUM> sends an encoded bitstream to client <NUM>. For example, server system <NUM> may be sending a video to a client <NUM> for playback.

Server system <NUM> includes a transcoder <NUM> that transcodes video into an encoded bitstream. Transcoder <NUM> may be a software video processor/transcoder configured on a computer processing unit (CPU), a hardware accelerated video processor/transcoder with a graphical processing unit (GPU), a field programmable gate array (FPGA), and/or a hardware processor/transcoder implemented in an application-specific integrated circuit (ASIC). Transcoding may be the conversion from one digital format to another digital format. Transcoding may involve decoding the source format and encoding the source video into another digital format, or converting the source content into videos with a specific resolution, framerate, bitrate, codec, etc. Also, encoding may be the conversion of analog source content and to a digital format. As used, the term transcoding may include encoding.

During the transcoding process, a transcoder bi-prediction block <NUM> performs a bi-prediction process for a current block of a current frame. Inter-prediction uses reference blocks from frames other than the current frame. Bi-prediction uses a first reference block from a first frame and a second reference block from a second frame to predict the current block. In some embodiments, the first frame is before the current frame and the second frame is after the current frame; however, the first frame and the second frame may be both before the current frame or both after the current frame.

Transcoder bi-prediction block <NUM> identifies a first reference block in a first reference frame and a second reference block in a second reference frame using a motion search process. After identifying the first reference block and the second reference block, transcoder bi-prediction block <NUM> may perform different processes based on whether the adaptive weight calculation will be performed on the transcoder side or the decoder side. If on the decoder side, transcoder bi-prediction block <NUM> determines signaling values for the bi-prediction mode. The signaling values may be the values for a first motion vector that points from the current block to the first reference block and a second motion vector that points from the current block to the second reference block. Transcoder bi-prediction block <NUM> calculates a prediction block by blending pixel values from the reference blocks with equal weight. Then, transcoder bi-prediction block <NUM> calculates a residual of the difference between the current block and the prediction block and inserts the residual into the encoded bitstream. The number of bits for the residual is expected to be less than the number of bits required to signal pixel values for the current block. Also, transcoder bi-prediction block <NUM> inserts a flag with a value that indicates that the adaptive weight bi-prediction mode should be used in the decoding process.

Client <NUM> includes a decoder <NUM> that decodes the encoded bitstream. During the decoding process, a decoder bi-prediction block <NUM> may blend pixels of reference blocks to generate a prediction block for a current block being decoded. Decoder <NUM> then uses the prediction block to decode the current block by applying the residual to the prediction block to calculate the pixel values for the current block.

If the adaptive weight calculation is performed on the decoder side, decoder bi-prediction block <NUM> may adaptively determine weights that are used to blend pixels of the reference blocks. Then, decoder bi-prediction block <NUM> generates the prediction block using the weights and pixel values for the reference blocks. Decoder bi-prediction block <NUM> decodes the current block by applying the residual to the prediction block.

As discussed above, the adaptive weight calculation may be performed on the transcoder side or the decoder side. If the adaptive weight calculation is performed on the transcoder side, transcoder bi-prediction block <NUM> may adaptively determine weights that are used to blend pixels of reference blocks for a current block being decoded. Then, transcoder bi-prediction block <NUM> inserts the values for the weights into the encoded bitstream in addition to the motion vectors. Transcoder bi-prediction block <NUM> may insert a value for the flag that indicates adaptive weight bi-prediction is not performed in the decoding process. Decoder <NUM> uses the values to adjust the weights to blend pixels of references blocks for a current block being decoded.

<FIG> shows an example of bi-prediction according to some embodiments. Transcoder <NUM> transcodes a current frame <NUM>. In the transcoding process, transcoder <NUM> decodes previously transcoded frames to use in the transcoded process of other frames. Here, transcoder <NUM> has already transcoded and then decoded reference frame <NUM> (reference frame <NUM>) and reference frame <NUM> (reference frame <NUM>). Transcoder <NUM> selects motion vectors (MV) to reference the positions of the reference blocks that are used to predict the current block C. Transcoder <NUM> may use various motion search methods to select the motion vectors for the reference blocks. Then, transcoder <NUM> may insert the motion vectors to use in the encoded bitstream along with a flag with a value that indicates whether or not to use adaptive weight bi-prediction on the decoder side. As discussed above, transcoder <NUM> also generates a prediction block and a residual. The prediction block and resulting residual may be different depending on whether the adaptive weight calculation is performed on the decoder side or the transcoder side.

Decoder <NUM> receives the encoded bitstream and starts decoding frames. Using the example in <FIG>, decoder <NUM> is decoding a current frame <NUM>. Decoder <NUM> has already decoded reference frame <NUM> (reference frame <NUM>) and reference frame <NUM> (reference frame <NUM>). Decoder <NUM> uses motion vectors (MV) to select the positions of the reference blocks that are used to predict the current block C. For example, transcoder <NUM> may have inserted the motion vectors for the current block in the encoded bitstream. A motion vector MV0 <NUM>-<NUM> points to a reference block R0 <NUM>-<NUM> in reference frame <NUM> and a motion vector MV1 <NUM>-<NUM> points to a reference block R1 <NUM>-<NUM> in reference frame <NUM>. Decoder <NUM> generates a prediction block from reference block R0 and reference block R1, and applies the residual to the prediction block to decode the current block.

Transcoder <NUM> or decoder <NUM> uses the pixels of reference block R0 and reference block R1 to predict the pixels of current block C <NUM>. For example, conventionally, an average blending pixel by pixel is used: <MAT> where C is the pixels of the current block, R0 is the pixels of the reference block R0, and R1 is the pixels of reference block R1. The values of "½" in the equation weight the pixels of reference block R0 and reference block R1 equally. Accordingly, the pixels of reference block R0 and reference block R1 are given equal weight to predict the pixels of current block C.

If characteristics, such as luminance conditions, change across current block and reference blocks, bias will be introduced by the fixed weight blending of the pixels. In other examples, other characteristics may be used, such as local complexity (e.g., pixel variance, pixel local entropy, amount of edges, and volume of gradient), texture similarity (e.g., edge directions), color difference, temporal distance (e.g., a difference of frame counter in playback order, or difference of time stamps), coding parameters such as a quantization parameter (QP), block size, coding mode (e.g., inter coding motion information, and intra coding motion information), etc. For example, a change in luminance across the current block and the reference block is not captured by the equal weighted blending of pixels of the two reference blocks. This causes prediction accuracy to go down and compression performance to go down. Luminance variation is common in video, such as changes in light conditions (e.g., flashes, fluorescent changes), movement of objects (e.g., from sunshine to shadows), and fade in and fade out shot and scene changes that are added in post processing. Given the frequency of luminance variation, the possibility of the luminance variation affecting prediction accuracy is high.

Transcoder <NUM> or decoder <NUM> may use an adaptive weighted blending of the weights in the bi-prediction process. <FIG> depicts an example of an equation that uses adaptive weighted blending of the weights in the bi-prediction process according to some embodiments. At <NUM>-<NUM>, a first weight w<NUM> is a weight for pixels of the first reference block R0 and at <NUM>-<NUM>, a second weight w<NUM> is a weight for pixels of the second reference block R1. The prediction block Ci,j is a combination of the weight w<NUM> multiplied by the pixels of the reference block R0 and the weight w<NUM> multiplied by the pixels of the reference block R1.

Transcoder <NUM> or decoder <NUM> may adaptively adjust the weights of w<NUM> and w<NUM> based on the variation of a characteristic between reference block R0 and the current block and between reference block R1 and the current block. For example, transcoder <NUM> or decoder <NUM> determines if the luminance of reference block R0 is closer to the luminance of current block C than the luminance of reference block R1 to the luminance of current block C. If so, then transcoder <NUM> or decoder <NUM> increases the value for weight w<NUM> and decreases the weight w<NUM>. If the luminance of reference block R1 is closer to the luminance of current block C than the luminance of reference block R0 to the luminance of current block C, transcoder <NUM> or decoder <NUM> decreases the value for weight w<NUM> and increases the weight w<NUM>. The change in the value of the weights may make the blended luminance of the current block C more accurate because the luminance of the reference block that is closer to the luminance of the current block is weighted higher. When the blended luminance is closer to the luminance of the current block, the bi-prediction accuracy and compression efficiency increases because the prediction block includes values that are closer to the current block.

Transcoder <NUM> or decoder <NUM> determines the weights adaptively using templates. Both scenarios are described with respect to <FIG> depicts an example of using templates to determine the weights according to some embodiments. In the transcoder side scenario, transcoder <NUM> is transcoding a current frame <NUM>. Transcoder <NUM> has transcoded and then decoded some blocks in current frame <NUM> that are represented with shading at <NUM> and has not decoded some blocks that are represented without shading at <NUM>. Transcoder <NUM> has already decoded reference frame <NUM> (reference frame <NUM>) and reference frame <NUM> (reference frame <NUM>). Transcoder <NUM> determines a shape, such as a L shape of decoded existing pixels at <NUM>. The L shaped region is a template of a width W and a height H. The L shaped region may be neighboring pixels to a current block <NUM> of MxN size being decoded. Although an L shaped region is described, other types of shapes may be used, such as the width of the template may not go beyond the top side of the current block.

Transcoder <NUM> identifies a template <NUM>-<NUM> in the reference frame <NUM> based on reference block <NUM> and a template <NUM>-<NUM> in the reference frame <NUM> based on reference block <NUM>. Template <NUM>-<NUM> and template <NUM>-<NUM> may have the same dimensions as template <NUM>, such as the WxH dimensions. Also, template <NUM>-<NUM> and template <NUM>-<NUM> may also be positioned the same relative to reference blocks <NUM> and <NUM>, respectively, such as forming an L-shape template next to the left and top sides of the reference blocks <NUM> and <NUM>, respectively.

Transcoder <NUM> uses template <NUM>-<NUM> in the reference frame <NUM>, template <NUM> for the current block, and template <NUM>-<NUM> in the reference frame <NUM> to adaptively determine the weights. Transcoder <NUM> uses the templates because current block <NUM> has not been transcoded yet. Transcoder <NUM> may use the templates to generate the weights, but may not need to use templates because transcoder <NUM> knows the decoded values for the current block and the reference blocks. This ensures that both transcoder <NUM> and decoder <NUM> use the same process and generating identical results. Thus, transcoder <NUM> uses decoded pixels in the current frame to determine the values of the weights. In the process, transcoder <NUM> selects the reference frames to use, such as reference frame <NUM> and reference frame <NUM>. Also, transcoder <NUM> selects the motion vectors for the current block being transcoded. The motion vectors identify the positions of the reference blocks R0 and R1. Transcoder <NUM> then selects templates. For example, transcoder <NUM> selects an L shaped region around the reference block R0 and R1 as the templates <NUM>-<NUM> and <NUM>-<NUM>, respectively. Also, transcoder <NUM> selects a similarly shaped template <NUM> for the current block.

Transcoder <NUM> then uses templates <NUM>, <NUM>-<NUM>, and <NUM>-<NUM> to determine the values of the weights. For example, transcoder <NUM> uses the templates <NUM>, <NUM>-<NUM>, and <NUM>-<NUM> to determine a luminance variation. As mentioned above, transcoder <NUM> uses the L shaped region instead of the pixels inside of the current block being decoded because transcoder <NUM> does not know the luminance of the current block due to the current block not having been transcoded yet. However, the luminance of the neighboring pixels in the L shaped region may be similar to the luminance of the current block in most cases.

Transcoder <NUM> then adaptively calculates the weights for the reference blocks, the process of which will be described in more detail below. Transcoder <NUM> may insert values for the weights in the encoded bitstream along with the motion vectors and a flag with a value that indicates the adaptive weighted bi-prediction is being used. The flag may be set to a first value when the adaptive weighted bi-prediction is being used and a second value when fixed weighted bi-prediction is being used, wherein the fixed weighted bi-prediction is not covered by the claimed invention.

When decoder <NUM> receives the encoded bitstream, decoder <NUM> uses the weights to blend the pixels of the reference blocks. For example, pixels of one of the reference blocks may be weighted higher than pixels of the other reference block to create the prediction block.

Decoder <NUM> also calculates the weights in the decoder side scenario. In this case, decoder <NUM> receives the encoded bitstream and is decoding a current frame <NUM>. Decoder <NUM> has decoded some blocks in current frame <NUM> that are represented with shading at <NUM> and has not decoded some blocks that are represented without shading at <NUM>. Decoder <NUM> has already decoded reference frame <NUM> (reference frame <NUM>) and reference frame <NUM> (reference frame <NUM>). Decoder <NUM> determines a shape, such as a L shape of decoded existing pixels at <NUM>. The L shaped region is a template of a width W and a height H. The L shaped region may be neighboring pixels to a current block <NUM> of MxN size being decoded. Although an L shaped region is described, other types of shapes may be used, such as the width of the template may not go beyond the top side of the current block.

Decoder <NUM> identifies a template <NUM>-<NUM> in the reference frame <NUM> based on reference block <NUM> and a template <NUM>-<NUM> in the reference frame <NUM> based on reference block <NUM>. Template <NUM>-<NUM> and template <NUM>-<NUM> may have the same dimensions as template <NUM>, such as the WxH dimensions. Also, template <NUM>-<NUM> and template <NUM>-<NUM> may also be positioned the same relative to reference blocks <NUM> and <NUM>, such as forming an L-shape template next to the left and top sides of the reference blocks <NUM> and <NUM>, respectively.

Decoder <NUM> uses template <NUM>-<NUM> in the reference frame <NUM>, template <NUM> for the current block, and template <NUM>-<NUM> in the reference frame <NUM> to adaptively determine the weights. The templates are used because current block <NUM> has not been decoded yet. Thus, decoder <NUM> uses decoded pixels in the current frame to determine the values of the weights. In the process, decoder <NUM> receives the encoded bitstream, which indicates the reference frames to use via motion vectors, such as reference frame <NUM> and reference frame <NUM>. Then, decoder <NUM> uses the motion vectors to select the positions of the reference blocks R0 and R1. After determining the reference blocks, decoder <NUM> selects templates. For example, decoder <NUM> selects an L shaped region around the reference block R0 and the reference block R1 as the templates <NUM>-<NUM> and <NUM>-<NUM>, respectively. Also, decoder <NUM> selects a similarly shaped template <NUM> for the current block.

Decoder <NUM> then uses templates <NUM>, <NUM>-<NUM>, and <NUM>-<NUM> to determine the values of the weights. For example, decoder <NUM> uses the templates <NUM>, <NUM>-<NUM>, and <NUM>-<NUM> to determine the luminance variation. As mentioned above, decoder <NUM> uses the L shaped region instead of the pixels inside of the current block being decoded because decoder <NUM> does not know the luminance of the current block due to the current block having not yet been decoded. However, the luminance of the neighboring pixels in the L shaped region may be similar to the luminance of the current block in most cases.

The weights may be calculated in different ways. As mentioned above, transcoder <NUM> or decoder <NUM> calculates the weights. <FIG> depicts a simplified flowchart <NUM> of a method for calculating the weights according to some embodiments. At <NUM>, transcoder <NUM> or decoder <NUM> calculates the distance dist0 between the luminance of reference template T0 of reference block R0 and the reference template T of the current block using: <MAT> where the distance dist0 is the sum of the differences between the luminance of each pixel in template T0 <NUM>-<NUM> and template T <NUM> (e.g., per-pixel distance). Although luminance is described, transcoder <NUM> or decoder <NUM> may use other characteristics to determine the weights. Also, other methods for calculating the distance may be used.

At <NUM>, transcoder <NUM> or decoder <NUM> calculates the distance dist0 between the luminance of reference template T0 of reference block R0 and the reference template T of the current block using: <MAT> where the distance distl is the sum of the differences between the luminance of each pixel in template T1 and template T (e.g., per-pixel distance). Also, other methods for calculating the distance may be used.

At <NUM>, transcoder <NUM> or decoder <NUM> calculates the weight w<NUM>. Transcoder <NUM> or decoder <NUM> calculates the weight wo using a contribution of the distance distl to the total distance of dist0 and distl. In some examples, transcoder <NUM> or decoder <NUM> uses the following to calculate weight w<NUM>: <MAT>.

In the above equation, the distance distl is divided by the sum of the distance distl and distance dist0. If the distance distl is larger, then the weight w<NUM> will be larger. This means that when the distance between the current template T and the template T1 of reference block R1 is larger, then the weight of reference block R0 is larger because reference block R0 is closer in luminance to the current block.

At <NUM>, transcoder <NUM> or decoder <NUM> calculates the weight w<NUM>. Transcoder <NUM> or decoder <NUM> calculates the weight w<NUM> using a contribution of the distance dist0 to the total distance of distance dist0 and distance distl. In some examples, transcoder <NUM> or decoder <NUM> uses the following to calculate weight wi: <MAT>
In the above equation, the distance dist1 is divided by the sum of the distance dist1 and distance dist0. If the distance dist0 is larger, then the weight w<NUM> may be larger. This means that when the distance between the current template T and the template T0 of reference block R0 is larger, then the weight of reference block R1 is larger because reference block R1 is closer in luminance to the current block. Although the above equations are described, other methods of calculating the weights may be used, such as a winner take all process described in <FIG>.

<FIG> depicts a simplified flowchart <NUM> of another example of calculating the weights according to some embodiments. In this example, transcoder <NUM> or decoder <NUM> selects one weight of w<NUM> or w<NUM> instead of adaptively calculating both the weight values. This may occur when one of the differences between the reference block and the current block is large, such as above a threshold value. The large difference indicates that the luminance of the reference block may not be close to the current block and thus may not be used. This results in a single direction prediction being used instead of bi-prediction. In some embodiments, decoder <NUM> makes the decision to use a uni-prediction even though transcoder <NUM> used bi-prediction in the transcoding process.

At <NUM>, transcoder <NUM> or decoder <NUM> determines a first distance dist0 between template T0 of reference block R0 and template T of the current block. Transcoder <NUM> or decoder <NUM> may calculate the distance dist0 similar to described above. At <NUM>, transcoder <NUM> or decoder <NUM> determines a second distance dist1 between template T1 of reference block R1 and template T of the current block. Transcoder <NUM> or decoder <NUM> may calculate the distance dist1 similar to described above.

At <NUM>, transcoder <NUM> or decoder <NUM> determines if a ratio of dist0/dist1 is less than a low threshold value, thresholdLow. At <NUM>, if the ratio of dist0/dist1 is less than the low threshold thresholdLow, then transcoder <NUM> or decoder <NUM> calculates the weights as w<NUM>=<NUM> and w<NUM>= <NUM>. In this example, the distance dist1 is large and distance dist0 is small, causing the resulting value of the fraction to be low. This indicates that reference block R1 is not close to the current block (e.g., the luminance of reference block R1 may be very different from the luminance of the current block). At <NUM>, transcoder <NUM> or decoder <NUM> determines if a ratio of dist0/dist1 is greater than a high threshold value. At <NUM>, if a ratio of dist0/dist1 is greater than a high threshold, thresholdHigh, then transcoder <NUM> or decoder <NUM> calculates the weights as w<NUM>=<NUM> and wi= <NUM>. In this example, the distance dist1 is small and distance dist0 is large, causing the resulting value of the fraction to be high. This indicates that reference block R0 is not close to the current block (e.g., the luminance of reference block R0 may be very different from the luminance of the current block). At <NUM>, if neither threshold is met, then transcoder <NUM> or decoder <NUM> calculates the weights as described in <NUM> and <NUM> in <FIG>, which adaptively calculates both weights for both reference blocks.

Using luminance values for pixels that are close to the current block being decoded may be a more accurate prediction of luminance of the current block compared to using a frame level luminance. The frame level luminance may use a single value of luminance for a frame because a comparison between pixels in the current block and pixels in the reference blocks cannot be used because the current block has not been decoded yet. Transcoder <NUM> or decoder <NUM> uses the L-shape template to perform a transcoder side or a decoder side search for the neighboring pixels and use the weighted prediction to change the blending process of bi-prediction.

Accordingly, a prediction block may be more representative of the current block. The use of the templates allows transcoder <NUM> or decoder <NUM> to reflect the luminance differences at the block level and not the frame level. This improves the prediction and thus the compression of the current block.

<FIG> depicts an example of a transcoding system according to some embodiments. A video codec framework includes a set of fundamental components: block partitioning, inter and intra prediction, transform and quantization, and entropy coding.

Transcoder <NUM> receives a frame of a video, which is firstly split into non-overlapping coding blocks for further processing. To cope with different video content characteristics, complex regions will be covered by partitions with smaller sizes, while simple regions will be covered by larger partitions. Multiple block patterns and shapes are may be both used together, for example quad-tree pattern, triple-tree pattern, and binary-tree pattern can be all used together, while square blocks and rectangular blocks can also be used together.

Prediction is used to remove the redundancy of a video signal. By subtracting the predicted pixel values from the pixels being processed, the amplitude of a residual signal can be significantly reduced, thus the resulting bitstream size can be reduced. An intra prediction block <NUM>, which is using reference pixels in the current frame, aims to reduce the spatial redundancy within the frame. An inter prediction block <NUM>, which is using reference pixels from neighboring frames, attempts to remove the temporal redundancy between frames. a motion estimation and compensation block <NUM> may be a sub-module of inter prediction at the transcoder side, which captures the motion trace of objects among adjacent frames and generates reference pixels for inter prediction.

A transform and quantization block <NUM> uses the residual pixels after intra or inter prediction. Transform and quantization block <NUM> performs a transform operation that represents the residual signal in a frequency domain. Considering the human visual system is more sensitive on low frequency components of video signal than the high frequency components, quantization is designed to further compress the residual signal by reducing the precision on high frequency signals.

To avoid the out-of-sync issue between transcoder <NUM> and decoder <NUM>, transcoder <NUM> contains decoding modules to make sure both transcoder <NUM> and decoder <NUM> are using identical mathematical processes. Thus, an inverse transform and inverse quantization block <NUM> is similar to the same block on the decoder side. Inverse transform and inverse quantization block <NUM> reconstructs pixels using the intra and inter prediction.

An in-loop filter <NUM> removes any visual artifacts that are introduced by the above-mentioned processes. Various filtering methods are applied on the reconstructed frame in a cascaded way to reduce different artifacts, including but not limited to the blocking artifacts, mosquito artifacts, color banding effects, etc..

An entropy encoding block <NUM> may further compress the bitstream using a model-based method. Transcoder <NUM> transmits the resulting encoded bitstream to decoder <NUM> over a network or other types of medium.

<FIG> depicts an example of a decoding system according to some embodiments. Decoder <NUM> receives the encoded bitstream and inputs it into an entropy decoding block <NUM> to recover the information needed for decoding process. As above-mentioned, a decoded frame can be decoded by using an inverse transform and inverse quantization block <NUM>, an intra prediction block <NUM> or inter prediction block <NUM>, motion compensation block <NUM>, and in-loop filtering block <NUM> in the same way to build a decoded frame.

Some embodiments may be implemented in a non-transitory computer-readable storage medium for use by or in connection with the instruction execution system, apparatus, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform a method described by some embodiments. The computer system may include one or more computing devices. The instructions, when executed by one or more computer processors, may be configured to perform that which is described in some embodiments.

As used in the description herein and throughout the claims that follow, "a", "an", and "the" includes plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.

Claim 1:
A method comprising:
selecting, by a computing device, a first reference block (<NUM>-<NUM>) in a first reference frame (<NUM>) and a second reference block (<NUM>-<NUM>) in a second reference frame (<NUM>), the first reference block (<NUM>-<NUM>) and the second reference block (<NUM>-<NUM>) being used to predict a current block (<NUM>) in a current frame (<NUM>);
selecting, by the computing device, a first template (<NUM>-<NUM>) in the first reference frame (<NUM>) comprising decoded neighboring pixels of the first reference block (<NUM>-<NUM>), a second template (<NUM>-<NUM>) in the second reference frame (<NUM>) comprising decoded neighboring pixels of the second reference block (<NUM>-<NUM>), and a third template (<NUM>) in the current frame (<NUM>) comprising decoded neighboring pixels of the current block (<NUM>);
comparing, by the computing device, a characteristic of the first template (<NUM>-<NUM>) to a characteristic of the third template (<NUM>) to generate a first difference value dis0;
comparing a characteristic of the second template (<NUM>-<NUM>) to the characteristic of the third template (<NUM>) to generate a second difference value dist1;
calculating, by the computing device, a first weight w<NUM> for the first reference block (<NUM>-<NUM>) and a second weight w<NUM> for the second reference block ( <NUM>-<NUM>) based on the first difference value and the second difference value;
adaptively, by the computing device, calculating a prediction block for the current block (<NUM>) using the first weight and the second weight; and
using, by the computing device, the prediction block as a prediction for the current block (<NUM>) in a video transcoding process or a video decoding process associated with the current block (<NUM>), characterized in that the step of calculating, by the computing device, the first weight w0 and the second weight w1 comprises:
calculating the weight w<NUM> using a contribution of the distance dist1 to the total distance of dist0 and dist1, and
calculating the weight w<NUM> using a contribution of the distance dist0 to the total distance of dist0 and distance dist1.