Patent Description:
Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of pictures, each picture having a spatial dimension of, for example, <NUM> x <NUM> luminance samples and associated chrominance samples. The series of pictures can have a fixed or variable picture rate (informally also known as frame rate) of, for example, <NUM> pictures per second or <NUM>. Uncompressed video has significant bitrate requirements. For example, 1080p60 <NUM>:<NUM>:<NUM> video at <NUM> bit per sample (1920x1080 luminance sample resolution at <NUM> frame rate) requires close to <NUM> Gbit/s bandwidth. An hour of such video requires more than <NUM> GBytes of storage space.

A video encoder and decoder can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

Video codec technologies can include techniques known as intra coding. In intra coding, sample values are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, the picture is spatially subdivided into blocks of samples. When all blocks of samples are coded in intra mode, that picture can be an intra picture. Intra pictures and their derivations such as independent decoder refresh pictures, can be used to reset the decoder state and can, therefore, be used as the first picture in a coded video bitstream and a video session, or as a still image. The samples of an intra block can be exposed to a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction can be a technique that minimizes sample values in the pre-transform domain. In some cases, the smaller the DC value after a transform is, and the smaller the AC coefficients are, the fewer the bits that are required at a given quantization step size to represent the block after entropy coding.

Traditional intra coding such as known from, for example MPEG-<NUM> generation coding technologies, does not use intra prediction. However, some newer video compression technologies include techniques that attempt, from, for example, surrounding sample data and/or metadata obtained during the encoding/decoding of spatially neighboring, and preceding in decoding order, blocks of data. Such techniques are henceforth called "intra prediction" techniques. Note that in at least some cases, intra prediction is only using reference data from the current picture under reconstruction and not from reference pictures.

There can be many different forms of intra prediction. When more than one of such techniques can be used in a given video coding technology, the technique in use can be coded in an intra prediction mode. In certain cases, modes can have submodes and/or parameters, and those can be coded individually or included in the mode codeword. Which codeword to use for a given mode/submode/parameter combination can have an impact in the coding efficiency gain through intra prediction, and so can the entropy coding technology used to translate the codewords into a bitstream.

A certain mode of intra prediction was introduced with H. <NUM>, refined in H. <NUM>, and further refined in newer coding technologies such as joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). A predictor block can be formed using neighboring sample values belonging to already available samples. Sample values of neighboring samples are copied into the predictor block according to a direction. A reference to the direction in use can be coded in the bitstream or may be predicted itself.

Referring to <FIG>, depicted in the lower right is a subset of nine predictor directions known from H. <NUM>'s <NUM> possible predictor directions (corresponding to the <NUM> angular modes of the <NUM> intra modes). The point where the arrows converge (<NUM>) represents the sample being predicted. The arrows represent the direction from which the sample is being predicted. For example, arrow (<NUM>) indicates that sample (<NUM>) is predicted from a sample or samples to the upper right, at a <NUM> degree angle from the horizontal. Similarly, arrow (<NUM>) indicates that sample (<NUM>) is predicted from a sample or samples to the lower left of sample (<NUM>), in a <NUM> degree angle from the horizontal.

Still referring to <FIG>, on the top left there is depicted a square block (<NUM>) of <NUM> x <NUM> samples (indicated by a dashed, boldface line). The square block (<NUM>) includes <NUM> samples, each labelled with an "S", its position in the Y dimension (e.g., row index) and its position in the X dimension (e.g., column index). For example, sample S21 is the second sample in the Y dimension (from the top) and the first (from the left) sample in the X dimension. Similarly, sample S44 is the fourth sample in block (<NUM>) in both the Y and X dimensions. As the block is <NUM> x <NUM> samples in size, S44 is at the bottom right. Further shown are reference samples that follow a similar numbering scheme. A reference sample is labelled with an R, its Y position (e.g., row index) and X position (column index) relative to block (<NUM>). <NUM> and H. <NUM>, prediction samples neighbor the block under reconstruction; therefore no negative values need to be used.

Intra picture prediction can work by copying reference sample values from the neighboring samples as appropriated by the signaled prediction direction. For example, assume the coded video bitstream includes signaling that, for this block, indicates a prediction direction consistent with arrow (<NUM>)-that is, samples are predicted from a prediction sample or samples to the upper right, at a <NUM> degree angle from the horizontal. In that case, samples S41, S32, S23, and S14 are predicted from the same reference sample R05. Sample S44 is then predicted from reference sample R08.

In certain cases, the values of multiple reference samples may be combined, for example through interpolation, in order to calculate a reference sample; especially when the directions are not evenly divisible by <NUM> degrees.

The number of possible directions has increased as video coding technology has developed. <NUM> (year <NUM>), nine different direction could be represented. That increased to <NUM> in H. <NUM> (year <NUM>), and JEM/VVC/BMS, at the time of disclosure, can support up to <NUM> directions. Experiments have been conducted to identify the most likely directions, and certain techniques in the entropy coding are used to represent those likely directions in a small number of bits, accepting a certain penalty for less likely directions. Further, the directions themselves can sometimes be predicted from neighboring directions used in neighboring, already decoded, blocks.

<FIG> shows a schematic (<NUM>) that depicts <NUM> intra prediction directions according to JEM to illustrate the increasing number of prediction directions over time.

The mapping of intra prediction directions bits in the coded video bitstream that represent the direction can be different from video coding technology to video coding technology; and can range, for example, from simple direct mappings of prediction direction to intra prediction mode, to codewords, to complex adaptive schemes involving most probable modes, and similar techniques. In all cases, however, there can be certain directions that are statistically less likely to occur in video content than certain other directions. As the goal of video compression is the reduction of redundancy, those less likely directions will, in a well working video coding technology, be represented by a larger number of bits than more likely directions.

In some video compression techniques, an MV applicable to a certain area of sample data can be predicted from other MVs, for example from those related to another area of sample data spatially adjacent to the area under reconstruction, and preceding that MV in decoding order. Doing so can substantially reduce the amount of data required for coding the MV, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, because when coding an input video signal derived from a camera (known as natural video) there is a statistical likelihood that areas larger than the area to which a single MV is applicable move in a similar direction and, therefore, can in some cases be predicted using a similar MV derived from MVs of a neighboring area. That results in the MV found for a given area to be similar or the same as the MV predicted from the surrounding MVs, and that in turn can be represented, after entropy coding, in a smaller number of bits than what would be used if coding the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (namely: the MVs) derived from the original signal (namely: the sample stream). In other cases, MV prediction itself can be lossy, for example because of rounding errors when calculating a predictor from several surrounding MVs.

Various MV prediction mechanisms are described in <NPL>). Out of the many MV prediction mechanisms that H. <NUM> offers, described herein is a technique henceforth referred to as "spatial merge.

Referring to <FIG>, a current block (<NUM>) can include samples that have been found by the encoder during the motion search process to be predictable from a previous block of the same size that has been spatially shifted. Instead of coding that MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example from the most recent (in decoding order) reference picture, using the MV associated with either one of five surrounding samples, denoted A0, A1, and B0, B1, B2 (<NUM> through <NUM>, respectively). <NUM>, the MV prediction can use predictors from the same reference picture that the neighboring block is using.

<CIT> provides an example method for cross-component prediction of video data. The method includes: downsampling, by at least one hardware processor, a reconstructed luma block to obtain a downsampled luma block, wherein the reconstructed luma block corresponds to a chroma block; generating, by the at least one hardware processor, parameters of a linear model (LM); and generating, by at least one hardware processor, predicted chroma values of the chroma block based on the parameters and the downsampled luma block.

<NPL>), proposes to perform repetitive padding for unavailable luma samples, and applies the same <NUM>-taps filter to generate all the down-sampled luma samples.

<NPL>), proposes that unavailable neighbouring samples are substituted by the available sample values using reference sample generation process, so as to simplify processes of the CCLM, which is already defined in WD.

<CIT> discloses a device for decoding video data. The device determines that a current block of video data is coded using a linear model prediction mode; for the luma component of the current block, the device determines reconstructed luma samples; based on luma samples in a luma component of one or more already decoded neighboring blocks and chroma samples in a chroma component of the one or more already decoded neighboring blocks, the device determines values for linear parameters, wherein the luma samples in the luma component of the one or more already decoded neighboring blocks comprises luma samples from a starting line in the luma component of the one or more already decoded neighboring blocks, wherein the starting line in the luma component of the one or more already decoded neighboring blocks is at least one line removed from a border line of the luma component of the current block.

Aspects of the disclosure provide apparatuses for video encoding/decoding.

The disclosed subject matter can be equally applicable to other video enabled applications, including, for example, video conferencing, digital TV, storing of compressed video on digital media including CD, DVD, memory stick, and the like.

A streaming system may include a capture subsystem (<NUM>) that can include a video source (<NUM>), for example a digital camera, creating for example a stream of video pictures (<NUM>) that are uncompressed. In an example, the stream of video pictures (<NUM>) includes samples that are taken by the digital camera. The stream of video pictures (<NUM>), depicted as a bold line to emphasize a high data volume when compared to encoded video data (<NUM>) (or coded video bitstreams), can be processed by an electronic device (<NUM>) that includes a video encoder (<NUM>) coupled to the video source (<NUM>). The video encoder (<NUM>) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (<NUM>) (or encoded video bitstream (<NUM>)), depicted as a thin line to emphasize the lower data volume when compared to the stream of video pictures (<NUM>), can be stored on a streaming server (<NUM>) for future use. One or more streaming client subsystems, such as client subsystems (<NUM>) and (<NUM>) in <FIG> can access the streaming server (<NUM>) to retrieve copies (<NUM>) and (<NUM>) of the encoded video data (<NUM>). A client subsystem (<NUM>) can include a video decoder (<NUM>), for example, in an electronic device (<NUM>). The video decoder (<NUM>) decodes the incoming copy (<NUM>) of the encoded video data and creates an outgoing stream of video pictures (<NUM>) that can be rendered on a display (<NUM>) (e.g., display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (<NUM>), (<NUM>), and (<NUM>) (e.g., video bitstreams) can be encoded according to certain video coding/compression standards. Examples of those standards include ITU-T Recommendation H. In an example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

The parser (<NUM>) may also extract from the coded video sequence information such as transform coefficients, quantizer parameter values, MVs, and so forth.

The scaler / inverse transform unit (<NUM>) can output blocks comprising sample values that can be input into aggregator (<NUM>).

The aggregator (<NUM>), in some cases, adds, on a per sample basis, the prediction information that the intra prediction unit (<NUM>) has generated to the output sample information as provided by the scaler / inverse transform unit (<NUM>).

The addresses within the reference picture memory (<NUM>) from where the motion compensation prediction unit (<NUM>) fetches prediction samples can be controlled by MVs, available to the motion compensation prediction unit (<NUM>) in the form of symbols (<NUM>) that can have, for example X, Y, and reference picture components. Motion compensation also can include interpolation of sample values as fetched from the reference picture memory (<NUM>) when sub-sample exact MVs are in use, MV prediction mechanisms, and so forth.

), picture size, group of pictures (GOP) layout, maximum MV allowed reference area, and so forth.

Briefly referring also to <FIG>, however, as symbols are available and encoding/decoding of symbols to a coded video sequence by an entropy coder (<NUM>) and the parser (<NUM>) can be lossless, the entropy decoding parts of the video decoder (<NUM>), including the buffer memory (<NUM>) and the parser (<NUM>) may not be fully implemented in the local decoder (<NUM>).

That is, for a new picture to be coded, the predictor (<NUM>) may search the reference picture memory (<NUM>) for sample data (as candidate reference pixel blocks) or certain metadata such as reference picture MVs, block shapes, and so on, that may serve as an appropriate prediction reference for the new pictures.

A predictive picture (P picture) may be one that may be coded and decoded using intra prediction or inter prediction using at most one MV and reference index to predict the sample values of each block.

A bi-directionally predictive picture (B Picture) may be one that may be coded and decoded using intra prediction or inter prediction using at most two MVs and reference indices to predict the sample values of each block.

When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector that is referred to as a MV. The MV points to the reference block in the reference picture, and can have a third dimension identifying the reference picture, in case multiple reference pictures are in use.

A block in the current picture can be coded by a first MV that points to a first reference block in the first reference picture, and a second MV that points to a second reference block in the second reference picture.

Each CTU can be recursively quad-tree split into one or multiple coding units (CUs).

In certain video coding technologies, merge mode can be an inter picture prediction submode where the MV is derived from one or more MV predictors without the benefit of a coded MV component outside the predictors. In certain other video coding technologies, a MV component applicable to the subject block may be present.

The inter encoder (<NUM>) is configured to receive the samples of the current block (e.g., a processing block), compare the block to one or more reference blocks in reference pictures (e.g., blocks in previous pictures and later pictures), generate inter prediction information (e.g., description of redundant information according to inter encoding technique, MVs, merge mode information), and calculate inter prediction results (e.g., prediction block) based on the inter prediction information using any suitable technique.

In an example, the intra encoder (<NUM>) also calculates intra prediction results (e.g., prediction block) based on the intra prediction information and reference blocks in the same picture.

The entropy encoder (<NUM>) is configured to include various information according to a suitable standard such as HEVC.

In some related examples such as VVC, to reduce a cross-component redundancy, a cross-component linear model (CCLM) prediction mode is used. In the CCLM prediction mode, chroma samples of a CU can be predicted based on reconstructed luma samples of the same CU by using a linear model as follows: <MAT> where pred_C (i,j) represents the predicted chroma samples in the CU and rec_L (i,j) represents down-sampled reconstructed luma samples of the CU.

<FIG> shows exemplary locations of neighboring chroma samples and corresponding neighboring luma samples used for the derivation of the CCLM parameters (e.g., α and β). In <FIG>, a luma block (<NUM>) is already reconstructed and a chroma block (<NUM>) corresponding to the luma block is being predicted by using the CCLM prediction mode. The CCLM parameters are derived based the neighboring chroma samples of the chroma block (<NUM>) and the corresponding neighboring luma samples of the luma block (<NUM>). For example, a neighboring chroma sample (<NUM>) and a corresponding neighboring luma sample (<NUM>) can be used for the derivation of the CCLM parameters.

In some examples, the CCLM parameters can be derived with at most four neighboring chroma samples of a CU and corresponding neighboring luma samples of the CU. <FIG> also shows down-sampled luma samples corresponding to the neighboring chroma samples. The corresponding neighboring luma samples can be used directly or padded for a down-sample filtering process. For the neighboring luma samples located in the top (or above), left, and top-left areas of the CU, whenever the neighboring luma samples are reconstructed and available, they can be used in the down-sample filtering process.

<FIG> show exemplary down-sample filtering processes for left and top neighboring luma samples, respectively. In both <FIG>, an SPS chroma vertical collocated flag such as SPS_chroma_vertical_collocated_flag is used to indicate whether a chroma sample and a corresponding luma sample in a CU are vertical collocated. In some examples, if the SPS chroma vertical collocated flag is true, it indicates that the chroma sample and the corresponding luma sample in the CU are vertical collocated. If the SPS chroma vertical collocated flag is false, it indicates that the chroma sample and the corresponding luma sample in the CU are not vertical collocated. For example, for a picture with a chroma sampling pattern of <NUM>:<NUM>:<NUM>, the SPS chroma vertical collocated flag can be true.

In <FIG>, for the left neighboring luma samples of the CU, if the SPS chroma vertical collocated flag is true, a <NUM>-tap down-sample filter can be used for the left neighboring luma samples. If the SPS chroma vertical collocated flag is false, a <NUM>-tap down-sample filter can be used for the left neighboring luma samples.

In <FIG>, for the top (or above) neighboring luma samples of the CU, the down-sample filter can be selected based on the SPS chroma vertical collocated flag and a location of the CU. If the CU is adjacent to a boundary of a CTU in which the CU is located, a <NUM>-tap down-sample filter can be used for the top neighboring luma samples. If the CU is not adjacent to the CTU boundary, the down-sample filter can be determined based on whether the SPS chroma vertical collocated flag is true. If the SPS chroma vertical collocated flag is true, the <NUM>-tap down-sample filter can be used for the top neighboring luma samples. If the SPS chroma vertical collocated flag is false, the <NUM>-tap down-sample filter can be used for the top neighboring luma samples.

As described above, in some cases such as for the top neighboring luma samples, the down-sample filter process depends on the location of the CU, leading to a more complicated process. This disclosure includes improvements to the CCLM prediction mode.

According to aspects of the disclosure, when a neighboring luma sample of a CU used in a derivation of the CCLM parameters is unavailable, a sample value of the unavailable neighboring luma sample can be determined based on a sample value of an available neighboring luma sample of the CU. For example, the unavailable neighboring luma sample can be padded using the available neighboring luma sample.

<FIG> show two examples of padding top neighboring luma samples for the derivation of the CCLM parameters. The top neighboring luma samples include available and unavailable neighboring luma samples. The unavailable top neighboring luma samples can be padded using the available neighboring luma samples.

In some embodiments, when M rows of top neighboring luma samples are needed for the derivation of the CCLM parameters, and N (which is less than M) rows of top neighboring luma samples are available, M-N row(s) is(are) padded using one of the N rows, for example the row that is closest to the M rows of top neighboring luma samples. In an embodiment, M can be predetermined.

In <FIG>, M and N are equal to <NUM> and <NUM>, respectively. Thus, for a current luma block (<NUM>), top rows Line <NUM> - Line <NUM> are needed for the derivation of the CCLM parameters. Line <NUM> is a row of available top neighboring luma samples and Line <NUM> - Line <NUM> are three rows of unavailable top neighboring luma samples. Line <NUM> - Line <NUM> can be padded using Line <NUM> row by row. For example, three unavailable top neighboring luma samples (<NUM>) - (<NUM>) in Line <NUM> - Line <NUM> can be padded using an available top neighboring luma sample (<NUM>) in Line <NUM>. Thus, all the three unavailable top neighboring luma samples (<NUM>) - (<NUM>) can have the same sample value of K, which is a sample value of the available top neighboring luma sample (<NUM>).

In <FIG>, M and N are equal to <NUM> and <NUM>, respectively. Thus, for a current luma block (<NUM>), top rows Line <NUM> - Line <NUM> are needed for the derivation of the CCLM parameters. Line <NUM> - Line <NUM> are two rows of available top neighboring luma samples and Line <NUM> - Line <NUM> are two rows of unavailable top neighboring luma samples. Line <NUM> - Line <NUM> can be padded using Line <NUM> row by row. For example, two unavailable top neighboring luma samples (<NUM>) - (<NUM>) in Line <NUM> - Line <NUM> can be padded using an available top neighboring luma sample (<NUM>) in Line <NUM>, respectively. Thus, both unavailable top neighboring luma samples (<NUM>) - (<NUM>) can have the same sample value of A1, which is a sample value of the available top neighboring luma sample (<NUM>).

As described above, by padding unavailable neighboring luma samples using available neighboring luma samples, the down-sample filter process for top neighboring luma samples can be independent of a CTU boundary. That is, a location of a CU is not needed to be examined in the down-sample filter process for the top neighboring luma samples. Thus, the down-sample filter process for the top neighboring luma samples can only depend on the SPS chroma vertical collocated flag for example. In an embodiment, if the SPS chroma vertical collocated flag is true, a <NUM>-tap filter such as the existing <NUM> tap filter in <FIG> can be used for the derivation of the CCLM parameters. If the SPS chroma vertical collocated flag is false, a <NUM>-tap filter such as the existing <NUM> tap filter in <FIG> can be used for the derivation of the CCLM parameters.

According to aspects of the disclosure, if a reconstructed top-left neighboring luma sample of a CU is used in the derivation of the CCLM parameters, instead of using a reconstructed sample value of the top-left neighboring luma sample, the sample value of the top-left neighboring luma sample can be determined based on a sample value of a top neighboring luma sample or a sample value of a left neighboring luma sample of the CU. For example, the top-left neighboring luma sample can be padded using the top neighboring luma sample or the left neighboring luma sample. Using the padded sample value instead of the reconstructed sample value for the top-left neighboring luma sample can be beneficial for a hardware implementation since the reconstructed top-left neighboring luma sample may be stored in a register separated from the reconstructed top or left neighboring luma samples in some embodiments.

<FIG> show two examples of padding top-left neighboring luma samples for the derivation of the CCLM parameters. In both <FIG>, the top-left neighboring luma samples can be reconstructed when deriving the CCLM parameters.

In an embodiment, the reconstructed top-left neighboring luma samples are not used in the down-sample filter process of the top neighboring luma samples when deriving the CCLM parameters. For the down-sample filter process of the top neighboring luma samples, the availability of the left and top-left neighboring luma samples can be marked as unavailable. For example, only reconstructed top neighboring luma samples can be used in the down-sample filter process of the top neighboring luma samples. Whenever the down-sample filter process of the top neighboring luma samples requires an unavailable luma sample located in N[x, y], where x<<NUM>, the unavailable luma sample can be padded with another neighboring luma sample (e.g., nearest neighboring luma sample). For example, the unavailable luma sample in N[x, y] can be padded with a nearest reconstructed top neighboring luma sample M[j, k], where k=y and j>=<NUM>.

In <FIG>, for a current luma block (<NUM>), an unavailable top-left neighboring luma sample (<NUM>) located in [-<NUM>, -<NUM>] can be padded using an available top neighboring luma sample (<NUM>) located in [<NUM>, -<NUM>] with a sample value of B0. Thus, a sample value of the unavailable top-left neighboring luma sample (<NUM>) can be B0. The unavailable top-left neighboring luma sample (<NUM>) is located adjacent to the available top neighboring luma sample (<NUM>). Both luma samples (<NUM>) and (<NUM>) are in one row above the current luma block (<NUM>).

In an embodiment, the reconstructed top-left neighboring luma samples are not used in the down-sample filter process of the left neighboring luma samples when deriving the CCLM parameters. For the down-sample filter process of the left neighboring luma samples, the availability of the top and top-left neighboring luma samples can be marked as unavailable. For example, only reconstructed left neighboring luma samples can be used in the down-sample filter process of the left neighboring luma samples. Whenever the down-sample filter process of the left neighboring luma samples requires an unavailable luma sample in located in O[x, y], where y<<NUM>, the unavailable luma sample can be padded with another neighboring luma sample (e.g., nearest neighboring luma sample). For example, the unavailable luma sample in O[x, y] can be padded with a nearest reconstructed left neighboring luma sample P[j, k], where j=x and k>=<NUM>.

In <FIG>, for a current luma block (<NUM>), an unavailable top-left neighboring luma sample (<NUM>) located in [-<NUM>, -<NUM>] can be padded using an available left neighboring luma sample (<NUM>) located in [-<NUM>, <NUM>] with a sample value of B0. The unavailable top-left neighboring luma sample (<NUM>) is located adjacent to the available left neighboring luma sample (<NUM>). Both luma samples (<NUM>) and (<NUM>) are in one column left to the current block (<NUM>).

In an embodiment, the reconstructed top-left neighboring luma samples are not used in the down-sample filter process of the top-left neighboring luma samples when deriving the CCLM parameters. For the down-sample filter process of the top-left neighboring luma samples, the availability of the top-left neighboring luma samples can be marked as unavailable. For example, one of the reconstructed top and left neighboring luma samples can be used in the down-sample filter process of the top-left neighboring luma samples. Use of the top or left neighboring luma samples can be based on the down-sample filter process. In an example, if the down-sample filter process is first performed on the top neighboring luma samples and then followed by the left neighboring luma samples, the top-left neighboring luma samples can be determined based on the top neighboring luma samples. In another example, if the down-sample filter process is first performed on the left neighboring luma samples and then followed by the top neighboring luma samples, the top-left neighboring luma samples can be determined based on the left neighboring luma samples.

<FIG> show two examples of padding top-left neighboring luma samples for the derivation of the CCLM parameters. In both <FIG>, the top-left neighboring luma samples can be reconstructed but marked as unavailable when deriving the CCLM parameters. The down-sample filter process can require <NUM> rows of top neighboring luma samples. In both <FIG>, <FIG> rows of top neighboring luma samples are available and the other <NUM> rows of top neighboring luma samples are unavailable. The unavailable rows can be padded using a nearest available row of top neighboring luma samples.

In <FIG>, the down-sample filter process can be first performed on the top neighboring luma samples and then followed by the left neighboring luma samples. For example, for a current luma block (<NUM>), in the down-sample filter process of the top neighboring luma samples, unavailable top-left neighboring luma samples (<NUM>) and (<NUM>) can be padded with available top neighboring luma samples (<NUM>) and (<NUM>), respectively. Both unavailable top neighboring luma samples (<NUM>) and (<NUM>) can be padded with the available top neighboring luma sample (<NUM>). For example, the neighboring luma sample (<NUM>) has a sample value of D and each of the neighboring luma samples (<NUM>) - (<NUM>) has a sample value of D1.

In <FIG>, the down-sample filter process can be first performed on the left neighboring luma samples and then followed by the top neighboring luma samples. For example, for a current luma block (<NUM>), in the down-sample filter process of the left neighboring luma samples, unavailable top-left neighboring luma samples (<NUM>) and (<NUM>) can be padded with available left neighboring luma samples (<NUM>) and (<NUM>), respectively. Then, in the down-sample filter process of the top neighboring luma samples, both unavailable top neighboring luma samples (<NUM>) and (<NUM>) can be padded with an available top neighboring luma sample (<NUM>). Thus, the neighboring luma sample (<NUM>) has a sample value of B0, the neighboring luma sample (<NUM>) has a sample value of B, and each of the neighboring luma samples (<NUM>) and (<NUM>) has a sample value of D1.

<FIG> shows a flow chart outlining an exemplary process (<NUM>) according to an embodiment of the disclosure. In various embodiments, the process (<NUM>) is executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>) and (<NUM>), the processing circuitry that performs functions of the video encoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the intra prediction module (<NUM>), the processing circuitry that performs functions of the video encoder (<NUM>), the processing circuitry that performs functions of the predictor (<NUM>), the processing circuitry that performs functions of the intra encoder (<NUM>), the processing circuitry that performs functions of the intra decoder (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented in software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>).

The process (<NUM>) may generally start at step (S1310), where the process (<NUM>) decodes prediction information for a current block in a current picture that is a part of a coded video sequence. The prediction information indicates a CCLM prediction mode for the current block. Then, the process (<NUM>) proceeds to step (S1320).

At step (S1320), the process (<NUM>) determines a sample value of a first unavailable neighboring luma sample of the current block based on at least one luma sample used in the CCLM prediction mode not being available. The sample value of the first unavailable neighboring luma sample can be determined based on a sample value of an available neighboring luma sample. Then, the process (<NUM>) proceeds to step (S1330).

At step (S1330), the process (<NUM>) calculates a parameter of the CCLM prediction mode based on the sample value of the first unavailable neighboring luma sample of the current block. Then, the process (<NUM>) proceeds to step (S1340).

At step (S1340), the process (<NUM>) reconstructs the current block based on the calculated parameter of the CCLM prediction mode. Then, the process (<NUM>) terminates.

In an embodiment, the available neighboring luma sample is located in a row above the current block and the first unavailable neighboring luma sample is adjacent to the available neighboring luma sample and in a row adjacent to the row in which the available neighboring luma sample is located.

In an embodiment, the available neighboring luma sample is located in a row above the current block and the first unavailable neighboring luma sample is adjacent to the available neighboring luma sample in the same row and top-left of the current block.

In an embodiment, the available neighboring luma sample is located in a column left of the current block and the first unavailable neighboring luma sample is adjacent to the available neighboring luma sample in the same column and top-left of the current block.

In an embodiment, the processing circuitry determines a sample value of a second unavailable neighboring luma sample of the current block based on the sample value of the available neighboring luma sample. The processing circuitry calculates the parameter of the CCLM prediction mode based on the sample value of the first unavailable neighboring luma sample and the sample value of the second unavailable neighboring luma sample of the current block.

In an embodiment, the available neighboring luma sample, the first unavailable neighboring luma sample, and the second unavailable neighboring luma sample are located in a same column and different rows above the current block.

In an embodiment, the processing circuitry performs a down-sample filter on the sample value of the first unavailable neighboring luma sample of the current block. The processing circuitry calculates the parameter of the CCLM prediction mode based on a result of the down-sample filter.

In an embodiment, the first unavailable neighboring luma sample of the current block is located above the current block, and the processing circuitry performs an N-tap filter on the first unavailable neighboring luma sample. N is determined based on whether the first unavailable neighboring luma sample and a corresponding chroma sample are vertical collocated.

These visual output devices (such as screens (<NUM>)) can be connected to a system bus (<NUM>) through a graphics adapter (<NUM>).

Computer system (<NUM>) can also include a network interface (<NUM>) to one or more communication networks (<NUM>). The one or more communication networks (<NUM>) can for example be wireless, wireline, optical. The one or more communication networks (<NUM>) can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of the one or more communication networks (<NUM>) include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, <NUM>, <NUM>, <NUM>, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CANBus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general purpose data ports or peripheral buses (<NUM>) (such as, for example USB ports of the computer system (<NUM>)); others are commonly integrated into the core of the computer system (<NUM>) by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system). Using any of these networks, computer system (<NUM>) can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV), uni-directional send-only (for example CANbus to certain CANbus devices), or bi-directional, for example to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

These devices, along with Read-only memory (ROM) (<NUM>), Random-access memory (<NUM>), internal mass storage (<NUM>) such as internal non-user accessible hard drives, SSDs, and the like, may be connected through the system bus (<NUM>).

As an example and not by way of limitation, the computer system having architecture (<NUM>), and specifically the core (<NUM>) can provide functionality as a result of processor(s) (including CPUs, GPUs, FPGA, accelerators, and the like) executing software embodied in one or more tangible, computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as introduced above, as well as certain storage of the core (<NUM>) that are of non-transitory nature, such as core-internal mass storage (<NUM>) or ROM (<NUM>). The software implementing various embodiments of the present disclosure can be stored in such devices and executed by core (<NUM>). A computer-readable medium can include one or more memory devices or chips, according to particular needs. The software can cause the core (<NUM>) and specifically the processors therein (including CPU, GPU, FPGA, and the like) to execute particular processes or particular parts of particular processes described herein, including defining data structures stored in RAM (<NUM>) and modifying such data structures according to the processes defined by the software. In addition or as an alternative, the computer system can provide functionality as a result of logic hardwired or otherwise embodied in a circuit (for example: accelerator (<NUM>)), which can operate in place of or together with software to execute particular processes or particular parts of particular processes described herein. Reference to software can encompass logic, and vice versa, where appropriate. Reference to a computer-readable media can encompass a circuit (such as an integrated circuit (IC)) storing software for execution, a circuit embodying logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.

Claim 1:
A method of video decoding in a decoder, comprising:
decoding (S1310) prediction information for a current block in a current picture that is a part of a coded video sequence, the prediction information indicating a cross-component linear model, CCLM, prediction mode for the current block;
applying a downsampling filtering process to luma samples before using the luma samples for determining CCLM parameters, wherein a downsampling filter involves neighboring luma samples located in top, left and top-left areas of a coding unit, CU, and the luma samples comprise unavailable top-left neighboring luma samples, wherein:
if the downsampling filtering process is first applied on top neighboring luma samples and then followed by left neighboring luma samples, then unavailable top-left neighboring luma samples (<NUM>, <NUM>) are determined row-wise using nearest adjacent available top neighboring luma samples (<NUM>, <NUM>), and unavailable top neighboring luma samples (<NUM>, <NUM>) are determined using a nearest adjacent available top neighboring luma sample (<NUM>), and
if the downsampling filtering process is first applied on left neighboring luma samples and then followed by the top neighboring luma samples, then unavailable top-left neighboring luma samples (<NUM>, <NUM>) are determined column- wise using nearest adjacent available left neighboring luma samples (<NUM>, <NUM>), and then in the downsampling filtering process on the top neighboring luma samples, unavailable top neighboring luma samples (<NUM>, <NUM>) are determined using a nearest available top neighboring luma sample (<NUM>);
calculating (S1330) a parameter of the CCLM prediction mode based on a result of applying the downsampling filtering process to the unavailable neighboring luma samples; and
reconstructing (S1340) the current block based on the calculated parameter of the CCLM prediction mode;
wherein applying the down-sample filter includes performing an N-tap filter on the at least one unavailable neighboring luma sample, N being determined based on an SPS chroma vertical collocated flag used to indicate whether the at least one unavailable neighboring luma sample and a corresponding chroma sample are vertical collocated, and if the SPS chroma vertical collocated flag is true, N assumes a value of <NUM>, and if the SPS chroma vertical collocated flag is false, N assumes a value of <NUM>.