Patent Description:
Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of pictures, each picture having a spatial dimension of, for example, <NUM> x <NUM> luminance samples and associated chrominance samples. The series of pictures can have a fixed or variable picture rate (informally also known as frame rate), of, for example <NUM> pictures per second or <NUM>. Uncompressed video has significant bitrate requirements. For example, 1080p60 <NUM>:<NUM>:<NUM> video at <NUM> bit per sample (1920x1080 luminance sample resolution at <NUM> frame rate) requires close to <NUM> Gbit/s bandwidth. An hour of such video requires more than <NUM> GBytes of storage space.

In some video compression techniques, an MV applicable to a certain area of sample data can be predicted from other MVs, for example from those related to another area of sample data spatially adjacent to the area under reconstruction, and preceding that MV in decoding order. Doing so can substantially reduce the amount of data required for coding the MV, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, because when coding an input video signal derived from a camera (known as natural video) there is a statistical likelihood that areas larger than the area to which a single MV is applicable move in a similar direction and, therefore, can in some cases be predicted using a similar motion vector derived from MVs of neighboring area. That results in the MV found for a given area to be similar or the same as the MV predicted from the surrounding MVs, and that in turn can be represented, after entropy coding, in a smaller number of bits than what would be used if coding the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (namely: the MVs) derived from the original signal (namely: the sample stream). In other cases, MV prediction itself can be lossy, for example because of rounding errors when calculating a predictor from several surrounding MVs.

Various MV prediction mechanisms are described in H. <NUM>/HEVC (ITU-T Rec. <NUM>, "High Efficiency Video Coding", December <NUM>). Out of the many MV prediction mechanisms that H. <NUM> offers, described here is a technique henceforth referred to as "spatial merge".

Referring to <FIG>, a current block (<NUM>) comprises samples that have been found by the encoder during the motion search process to be predictable from a previous block of the same size that has been spatially shifted. Instead of coding that MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example from the most recent (in decoding order) reference picture, using the MV associated with either one of five surrounding samples, denoted A0, A1, and B0, B1, B2 (<NUM> through <NUM>, respectively). <NUM>, the MV prediction can use predictors from the same reference picture that the neighboring block is using.

<NPL>, discloses a method for intra block copy within one LCU depending on intra motion vector. The intra motion vector is restricted to be within current LCT plus <NUM> columns of left LCU and <NUM> rows of LCU above.

Aspects of the disclosure providing methods and apparatuses for video encoding/decoding are set out in the appended set of claims.

In the <FIG> example, the terminal devices (<NUM>), (<NUM>), (<NUM>) and (<NUM>) may be illustrated as servers, personal computers, and smart phones, but the principles of the present disclosure may be not so limited. Embodiments of the present disclosure find application with laptop computers, tablet computers, media players, and/or dedicated video conferencing equipment. The network (<NUM>) represents any number of networks that convey coded video data among the terminal devices (<NUM>), (<NUM>), (<NUM>), and (<NUM>), including for example wireline (wired) and/or wireless communication networks. Representative networks include telecommunications networks, local area networks, wide area networks, and/or the Internet.

A streaming system may include a capture subsystem (<NUM>), that can include a video source (<NUM>), for example a digital camera, creating for example a stream of video pictures (<NUM>) that are uncompressed. In an example, the stream of video pictures (<NUM>) includes samples that are taken by the digital camera. The stream of video pictures (<NUM>), depicted as a bold line to emphasize a high data volume when compared to encoded video data (<NUM>) (or coded video bitstreams), can be processed by an electronic device (<NUM>) that includes a video encoder (<NUM>) coupled to the video source (<NUM>). The video encoder (<NUM>) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (<NUM>) (or encoded video bitstream (<NUM>)), depicted as a thin line to emphasize the lower data volume when compared to the stream of video pictures (<NUM>), can be stored on a streaming server (<NUM>) for future use. One or more streaming client subsystems, such as client subsystems (<NUM>) and (<NUM>) in <FIG> can access the streaming server (<NUM>) to retrieve copies (<NUM>) and (<NUM>) of the encoded video data (<NUM>). A client subsystem (<NUM>) can include a video decoder (<NUM>), for example, in an electronic device (<NUM>). The video decoder (<NUM>) decodes the incoming copy (<NUM>) of the encoded video data and creates an outgoing stream of video pictures (<NUM>) that can be rendered on a display (<NUM>) (e.g., display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (<NUM>), (<NUM>), and (<NUM>) (e.g., video bitstreams) can be encoded according to certain video coding/compression standards. Examples of those standards include ITU-T Recommendation H. In an example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

Subgroups can include Groups of Pictures (GOPs), pictures, tiles, slices, macroblocks, Coding Units (CUs), blocks, Transform Units (TUs), Prediction Units (PUs), and so forth.

As the decoding of a symbol stream leads to bit-exact results independent of decoder location (local or remote), the content in the reference picture memory.

(<NUM>) is also bit exact between the local encoder and remote encoder.

For example, according to the HEVC standard, a picture in a sequence of video pictures is partitioned into coding tree units (CTU) for compression, the CTUs in a picture have the same size, such as <NUM>×<NUM> pixels, 32x32 pixels, or 16x16 pixels. For example, a CTU of <NUM>×<NUM> pixels can be split into one CU of <NUM>×<NUM> pixels, or <NUM> CUs of 32x32 pixels, or <NUM> CUs of 16x16 pixels.

<FIG> is a schematic illustration of a current block (<NUM>) in a current picture (<NUM>) to be coded using intra block copy (IBC) in accordance with an embodiment.

In some examples, a block may be coded using a reference block from a different picture, which is also referred to as motion compensation. In some examples, a block may be coded using a reference block from a previously reconstructed area within the same picture, which is also referred to as intra picture block compensation, current picture referencing (CPR), or intra block copy (IBC). A displacement vector that indicates the offset between the current block and the reference block is referred to as a block vector (or BV for short). Different from a motion vector in motion compensation, which can be at any value (positive or negative, at either x or y direction), a block vector is subject to constraints to ensure that the reference block has already been reconstructed and the reconstructed samples thereof are available. In some embodiments, in view of parallel processing constrains, a reference area that is beyond a tile boundary or wavefront ladder shape boundary is also excluded.

The coding of a block vector can be either explicit or implicit. In the explicit mode, the difference between a block vector and its predictor can be signaled in a manner similar to an AMVP mode in inter coding. In the implicit mode, the block vector can be recovered from a predictor, in a similar way as a motion vector in merge mode. The resolution of a block vector, in some implementations, is set to integer positions or, in some examples, to fractional positions.

The use of IBC at the block level can be signaled using a block level flag. In some examples, this flag can be signaled when the current block is not coded in merge mode. In some examples, this flag can be signaled by a reference index approach. This is done by treating the current decoded picture as a reference picture. In HEVC Screen Content Coding (HEVC SCC), such a reference picture is placed in the last position of the list. This special reference picture is also managed together with other temporal reference pictures in the Decoded Picture Buffer (DPB).

There are also some variations for implementing IBC, such as flipped intra block copy (where the reference block is flipped horizontally or vertically before being used to predict current block), or line based intra block copy (where each compensation unit inside an MxN coding block is an Mx1 or 1xN line).

An example of using IBC is shown in <FIG>, where the current picture (<NUM>) includes <NUM> blocks arranged into <NUM> rows and <NUM> columns. In some examples, each block corresponds to a Coding Tree Unit (CTU). The current block (<NUM>) includes a sub-block (<NUM>) (e.g., a coding block in the CTU) that has a block vector (<NUM>) pointing to a reference sub-block (<NUM>) in the current picture (<NUM>).

The reconstructed samples of the current picture can be stored in a dedicated memory. In consideration of implementation cost, the reference area where the reconstructed samples for reference blocks remain available may not be as large as an entire frame, depending on a memory size of the dedicated memory. Therefore, for a current sub-block using IBC, in some examples, an IBC reference sub-block may be limited to only certain neighboring areas, but not the entire picture.

In some embodiments, the dedicated memory to store reference samples of previously coded CUs for future intra block copy reference is referred to as a reference sample memory. In one example, the memory size is one CTU, such as for storing up to one previously coded CTU or one left CTU. In another example, the memory size is two CTUs, such as two previously coded CTUs or two left CTUs, or one current CTU together with one left CTU. In some embodiments, each CTU requires a memory size for storing <NUM>×<NUM> luma samples, together with corresponding chroma samples. When a reference block is outside the stored, reconstructed areas, the reference block cannot be used for IBC.

In some embodiments, when starting a new CTU (i.e., a current CTU), the reference sample memory allocates space for storing the reconstructed sample of the entire current CTU. In some examples, the memory size is one CTU, and the allocated space for the current CTU can still be partially used to store the reconstructed samples from a previously coded CTU, hence the allocation of space for the current CTU is not completed at the beginning of the current CTU. Therefore, a portion (or location) of the reference sample memory that stores the reconstructed samples from the previously coded CTU can be used in IBC mode to predict a current coding block in the current CTU until this portion is updated by the reconstructed samples of the current coding block in the current CTU. After that, the data in this portion can still be used for IBC reference for providing reconstructed samples from the current CTU, but no longer for providing the reconstructed samples from the previously coded CTU that have just been overwritten.

In some embodiments, the current CTU is divided into a number of partitions based on one or more predefined grid patterns. For example, into <NUM>×<NUM> partitions, into <NUM>×<NUM> partitions, etc. If the location of the current coding block in the current CTU falls into one of the predefined partitions, this will indicate the reconstructed samples for the whole corresponding partition stored in the reference sample memory will be updated with reconstructed samples from the current CTU, and the old reconstructed samples from a previously coded CTU in that partition in the reference sample memory cannot be used for IBC reference purposes. In some examples, the partition size is at least as large as a largest possible IBC code block size. For example, if the maximum reference block size for IBC is <NUM>×<NUM>, then the CTU can be divided into as small as <NUM>×<NUM> partitions.

In some alternative embodiments, when a reference block in a previously coded CTU and its collocated block in the current CTU share the same location in the reference sample memory, the location of memory will be updated with the data from the current CTU when this collocated block in the current CTU is coded. During the coding process of the current CTU, for a coding block in IBC mode, its reference block in a previously coded CTU is found, whose reference samples are stored in the reference sample memory. For this reference block, if none of the samples in its collocated block in the current CTU has been coded, the location in reference sample memory has not been updated with the data from the current CTU, and this reference block, which contains reference samples from the previously coded CTU, can still be used for IBC. Otherwise, according to an embodiment, if at least one sample of the collocated block in the current CTU has been reconstructed, this reference block in the previously coded CTU can be indicated as overwritten and cannot be used for IBC reference.

In some embodiments, the memory size is two CTUs. When starting a new CTU (i.e., a current CTU), the reconstructed samples from a most recently coded CTU may be left as is, and the allocated space for the current CTU can be partially used to store the reconstructed samples from a previously coded CTU that is coded before the most recently coded CTU. Therefore, depending on the coding order, the block partitioning structure, and the availability of the reconstructed samples in the memory, the allowable area for reference samples used in IBC mode can be extended to the reconstructed part of the current CTU, the entire most recently coded CTU, and/or a portion of the previously coded CTU that can be indicated as not overwritten by the reconstructed samples of the current block.

In a different embodiment, the memory size is two CTUs. When starting a new CTU (i.e., a current CTU), the reconstructed samples from a most recently coded CTU may be left as is. Therefore, depending on the coding order, the block partitioning structure, and the availability of the reconstructed samples in the memory, the allowable area for reference samples used in IBC mode can be extended to the reconstructed part of the current CTU and the entire most recently coded CTU.

In some examples, the size of an IBC reference sub-block can be as large as a regular inter coded block. In order to utilize the reference sample memory more efficiently, the size of an IBC reference sub-block can be limited to not greater than <NUM> luma samples at either width or height edge, where corresponding size constraints apply to chroma samples, depending on the color format. For example, in <NUM>:<NUM>:<NUM> format, the size of a chroma block in IBC mode can be limited to not greater than <NUM> samples on each side. In some embodiments, lower limits, such as <NUM> luma samples each side can be used.

In the following non-limiting examples, for the purposes of illustrating various embodiments, the maximum IBC reference sub-block size is set to <NUM>×<NUM> luma samples. Therefore, in a CTU size of <NUM>×<NUM> luma samples, for luma samples, sub-blocks of <NUM>×<NUM>, <NUM>×<NUM>, <NUM>×<NUM>, <NUM>×<NUM>, <NUM>×<NUM>, etc., cannot use intra block copy mode. For chroma samples, depending on the color format, similar to the constraints for luma samples, the corresponding sizes for chroma samples apply.

<FIG> is a schematic illustration of a current block (CTU, <NUM>) and a neighboring block (CTU, <NUM>) in a current picture using IBC in accordance with an embodiment.

In some embodiments, two sub-blocks from different CTUs are referred to as collocated sub-blocks when these two sub-blocks have the same size and have a same location offset value relative to an upper-left corner of the respective CTU. <FIG> shows a current sub-block (<NUM>) in the current block (i.e., CTU) (<NUM>) and three of its possible reference sub-blocks (<NUM>, <NUM>, and <NUM>) in a left, previously coded block (<NUM>) that are identifiable by respective block vectors (<NUM>, <NUM>, and <NUM>). In this example, if the reference sample memory size is one CTU, reference sub-block (<NUM>) can be found from the memory because its collocated sub-block (<NUM>) in the current block (<NUM>) has not yet been reconstructed (white area). Therefore the location of the reference sample memory still stores the reference samples from the previously coded block (<NUM>). On the contrary, reference sub-block (<NUM>) cannot be used, as its collocated sub-block (<NUM>) in the current block (<NUM>) has been reconstructed completed (grey area). The location of reference sample memory for reference sub-block (<NUM>) has been overwritten with the reconstructed samples from the sub-block (<NUM>) in the current block (<NUM>). Similarly, reference sub-block (<NUM>) cannot be a valid reference sub-block, because part of its collocated sub-block (<NUM>) in the current block (<NUM>) has been reconstructed, and therefore that part of the memory has been partially overwritten with the data in the current block (<NUM>).

In order to efficiently utilize the stored reconstructed samples while sharing a memory space between CTUs, an encoder or a decoder can determine whether a reference sub-block from a previously coded block is overwritten (or is otherwise considered to be overwritten) based on a partition structure, a coding order, and/or a position of a current sub-block in a current block.

For example, when a reference sub-block in a previously coded block and its collocated sub-block in the current block share the same location in the reference sample memory, the location of memory can be indicated as updated (e.g., overwritten or otherwise considered as overwritten) with the data in the current block when any part of this collocated sub-block in the current block is coded. During the coding process of the current block, for a sub-block in IBC mode, its reference sub-block in a previously coded block is found, whose reference samples are stored in the reference sample memory. For this reference sub-block, if none of the samples in its collocated sub-block in the current block has been coded, the location in reference sample memory has not been updated with the data from the current block, this reference sub-block, which contains reference samples from a previously coded block, can be used for IBC reference. Otherwise, when at least one sample of the collocated sub-block in the current block is coded, the corresponding location in reference sample memory has been updated by the data in the current block, and this reference sub-block cannot be used for IBC reference.

The above general solution is based on checking availability of different locations during the process of encoding and/or decoding a current block. Such an availability checking process, in some examples, can be simplified to only checking availabilities at a few pre-set locations. In some examples, the determination of the availability of a <NUM>×<NUM> luma block from the previous coded block can be based on whether any part of its collocated <NUM>×<NUM> block of the current block has been coded or not. In this case, only the upper-left position of each <NUM>×<NUM> block in the current block may need to be checked. Other positions may be checked in other embodiments. The proposed methods/solutions can be extended to smaller block sizes, such as the evaluation based on <NUM>×<NUM> blocks.

Different determination factors for different partitioning structures will be further described based on the following two partitioning scenarios. Based on different partitioning structures, such availability determination may be made with no or limited checking of individual samples in order to improve IBC performance by increasing the available reference range without using extra reference sample memory.

Under a first scenario, each of the four <NUM>×<NUM> luma partitions (<NUM>×<NUM> chroma partition in <NUM>:<NUM>:<NUM> format) in the current CTU will be contained completely in a coding block (also referred to as a sub-block); or, each coding block in the current CTU will be contained completely in one of the four <NUM>×<NUM> luma partitions (<NUM>×<NUM> chroma partition in <NUM>:<NUM>:<NUM> format).

According to the first scenario, at a <NUM>×<NUM> CTU level, this block can be coded as is (<NUM>×<NUM>), or split into four <NUM>×<NUM> blocks and with a potential further split, or split into two <NUM>×<NUM> blocks and with a potential further split, or split into two <NUM>×<NUM> blocks and with a potential further split.

In some variations, ternary-tree split for a block with either edge (width or height) larger than <NUM> luma samples is not allowed; otherwise the resulting block will not be contained in one of the four <NUM>×<NUM> partitions, or contains one of the four <NUM>×<NUM> partitions completely.

In one example, if the coding block is <NUM>×<NUM> in size, and the max IBC block size is <NUM>×<NUM>, then this <NUM>×<NUM> block will not be coded in IBC mode.

In one example, if the coding blocks are four <NUM>×<NUM> blocks, the availability of reference samples can be illustrated with reference to <FIG>.

<FIG> is a schematic illustration of how reconstructed samples in a neighboring block are to be indicated as overwritten based on a position of a current sub-block that is coded using IBC in accordance with one embodiment.

In <FIG>, a current block (<NUM>) corresponds to a current CTU that includes four <NUM>×<NUM> partitions (<NUM>, <NUM>, <NUM>, and <NUM>). A previously coded block (<NUM>) corresponds to a left CTU that includes four <NUM>×<NUM> partitions (<NUM>, <NUM>, <NUM>, and <NUM>). The coding order for processing coding blocks in the current block (<NUM>) starts from the upper-left partition (<NUM>), then the upper-right partition (<NUM>), then the lower-left partition (<NUM>), and finally the lower-right partition (<NUM>). The <NUM>×<NUM> partition with vertical stripes is where the current coding block is located (the current coding block can be smaller than <NUM>×<NUM> in size). The shaded grey blocks are the reconstructed blocks. The ones marked "X" are not available for IBC reference since they should be or have been overwritten with the reconstructed samples from the current block in the corresponding locations.

Therefore, if the current coding block falls into the upper-left <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), then in addition to the already reconstructed samples in the current CTU, the reconstructed samples in the upper-right, lower-left, and lower-right <NUM>×<NUM> partitions (<NUM>, <NUM>, and <NUM>) of the left CTU (block <NUM>), can be referenced using the IBC mode. Partition (<NUM>) is indicated as overwritten and thus unavailable.

If the current block falls into the upper-right <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), then in addition to the already reconstructed samples in the current CTU, the reconstructed samples in the lower-left and lower-right <NUM>×<NUM> partitions (<NUM> and <NUM>) of the left CTU (block <NUM>), can be referenced using the IBC mode in an embodiment. Partitions (<NUM> and <NUM>) are indicated as overwritten and thus unavailable.

If the current block falls into the lower-left <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), then in addition to the already reconstructed samples in the current CTU, the reference samples in the lower-right <NUM>×<NUM> partition of the left CTU (block <NUM>), can be referenced using the IBC mode in an embodiment. Partitions (<NUM>, <NUM>, and <NUM>) are indicated as overwritten and thus unavailable.

If the current block falls into the lower-right <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), only the already reconstructed samples in the current CTU can be referenced using the IBC mode in an embodiment. Partitions (<NUM>, <NUM>, <NUM>, and <NUM>) are indicated as overwritten and thus unavailable.

The above assumption works for the case that the CTU will be split in quad-tree at a first level (if there is any split at <NUM>×<NUM> level), such as when a separate luma/chroma coding tree (dual-tree) is used.

In one example, if the coding blocks are two <NUM>×<NUM> blocks, it is not allowed to apply horizontal binary-tree split at the next level. Otherwise, the resulting <NUM>×<NUM> block will be contained by two <NUM>×<NUM> partitions, which violates the assumption under the first scenario. Therefore, each <NUM>×<NUM> block will be coded as is, or split into two <NUM>×<NUM> blocks, or may be split by quad-tree into four 64x16 blocks.

In one example, if the coding blocks are two <NUM>×<NUM> blocks, it is not allowed to apply vertical binary-tree split at the next level. Otherwise, the resulting <NUM>×<NUM> block will be contained by two <NUM>×<NUM> partitions, which violates the assumption under the first scenario. Therefore, each <NUM>×<NUM> block will be coded as is, or split into two <NUM>×<NUM> blocks, or may be split by quad-tree into four 16x64 blocks.

In some examples, the VVC standard allows flexible block partitioning strategies with quad-tree, binary-tree, and ternary-tree. If the first level split is not quad-tree, it can still be binary-tree split (not ternary split), such as when dual-tree is not used. If vertical binary-tree split is applied at the first level from the CTU, such as having two <NUM>×<NUM> blocks or two <NUM>×<NUM> blocks, then the coding order of 2nd and 3rd of the four <NUM>×<NUM> partitions in <FIG> may be different.

<FIG> is a schematic illustration of how reconstructed samples in a neighboring block are to be indicated as overwritten based on a position of a current sub-block that is coded using IBC in accordance with another embodiment. In <FIG>, the availability of reference samples for an upper-right <NUM>×<NUM> partition and a lower-left <NUM>×<NUM> partition are shown, when the vertical binary-tree split is applied at a <NUM>×<NUM> level from the CTU. When the horizontal binary-tree split is applied at a <NUM>×<NUM> level from the CTU, the coding order and availability of reference samples for an upper-right <NUM>×<NUM> block and a lower-left <NUM>×<NUM> block are the same as in <FIG>.

Therefore, if a current coding block falls into the upper-left <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), then in addition to the already reconstructed samples in the current CTU, the reconstructed samples in the upper-right, lower-left, and lower-right <NUM>×<NUM> partitions (<NUM>, <NUM>, and <NUM>) of the left CTU (block <NUM>), can be referenced using the IBC mode according to an embodiment. Partition (<NUM>) is indicated as overwritten and thus unavailable.

If the current block falls into the lower-left <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), then in addition to the already reconstructed samples in the current CTU, the reconstructed samples in the upper-right and lower-right <NUM>×<NUM> partitions (<NUM> and <NUM>) of the left CTU (block <NUM>), can be referenced using the IBC mode according to an embodiment. Partitions (<NUM> and <NUM>) are indicated as overwritten and thus unavailable.

If the current block falls into the upper-right <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), then in addition to the already reconstructed samples in the current CTU, the reference samples in the lower-right <NUM>×<NUM> partition of the left CTU (block <NUM>), can be referenced using the IBC mode. Partitions (<NUM>, <NUM>, and <NUM>) are indicated as overwritten and thus unavailable.

If the current block falls into the lower-right <NUM>×<NUM> partition (<NUM>) of the current block (<NUM>), only the already reconstructed samples in the current CTU can be referenced using the IBC mode according to an embodiment. Partitions (<NUM>, <NUM>, <NUM>, and <NUM>) are indicated as overwritten and thus unavailable.

For the discussions with reference to <FIG> and <FIG>, two exemplary solutions are summarized as follows.

A first exemplary solution is for fully reusing the reference sample memory when possible. More specifically, depending on the location of a current sub-block (e.g., a coding block) relative to a current block (e.g., a CTU), the following can apply:.

Table I below summarizes the availabilities of reconstructed samples from a left block for a first exemplary solution. UL, UR, LL, and LR refer to upper-left, upper-right, lower-left, and lower-right, respectively. Mark "X" means not available, mark "Y" means available.

A second exemplary solution is for a simplified process regardless of the adopted block partitioning strategy. More specifically, depending on the location of a current sub-block (e.g., a coding block) relative to a current block (e.g., a CTU), the following can apply:.

Table II below summarizes the availabilities of reconstructed samples from a left block for a second exemplary solution. UL, UR, LL, and LR refer to upper-left, upper-right, lower-left, and lower-right, respectively. Mark "X" means not available, mark "Y" means available.

Under a second scenario, at CTU root (<NUM>×<NUM> luma samples), only a quad-tree and a binary-tree split is allowed. After that, a binary split or a ternary split can be applied to either side of each <NUM>×<NUM>, <NUM>×<NUM>, or <NUM>×<NUM> block.

According to the second scenario, it is only guaranteed that if the current sub-block in CPR mode falls into the upper-left <NUM>×<NUM> partition, then all the coding units in the upper-left <NUM>×<NUM> partition will be coded prior to coding blocks in the lower-right <NUM>×<NUM> partition. In this case, the lower-right <NUM>×<NUM> partition of left block has not yet been updated while processing coding units in the upper-left <NUM>×<NUM> partition of current block. Reconstructed samples in this reference area (the lower-right <NUM>×<NUM> partition of left block) can be used for CPR referencing.

For coding blocks in the other three <NUM>×<NUM> partitions of the current block, there is no guarantee that a complete <NUM>×<NUM> partition that contains reference samples of the left block will not be updated during the processing of coding units in the corresponding <NUM>×<NUM> partitions. In some examples, no special operation is proposed.

Under the second scenario, two exemplary solutions are summarized as follows.

A third exemplary solution allows the lower-right <NUM>×<NUM> partition of left block to be used as reference for CPR mode, if current sub-block is inside the upper-left <NUM>×<NUM> partition of the current block. For sub-blocks inside other three <NUM>×<NUM> partitions of the current block, they can only refer to reconstructed samples within the current block.

Table III below summarizes the availabilities of reconstructed samples from a left block for a third exemplary solution. UL, UR, LL, and LR refer to upper-left, upper-right, lower-left, and lower-right, respectively. Mark "X" means not available, mark "Y" means available.

To further improve upon the third exemplary solution, for each sub-block in the current block, the availability of each <NUM>×<NUM> partition in left block can be evaluated by checking the top-left corner's availability of each <NUM>×<NUM> partition in the current block. For example, when a current sub-block is in an upper-right <NUM>×<NUM> partition of the current block, if the upper-left corner of the lower-left <NUM>×<NUM> partition in the current block has not yet been reconstructed, that means the upper-left and upper-right <NUM>×<NUM> partitions of current block will be processed prior to the lower-left and lower-right <NUM>×<NUM> partitions of current block. So the reference sample memory locations that store reference samples in the lower-left and lower-right <NUM>×<NUM> partition of the left block have not yet been updated. They can be used as references for the current sub-block in CPR mode. Similar checks for upper-left corner of upper-right <NUM>×<NUM> partition will apply.

Accordingly, the fourth exemplary solution allows for more fully reusing the reference sample memory when possible. More specifically, depending on the location of the current sub-block relative to the current block, the following can apply:.

Table IV below summarizes the availabilities of reconstructed samples from left block for a fourth exemplary solution. UL, UR, LL, and LR refer to upper-left, upper-right, lower-left, and lower-right, respectively. Mark "X" means not available, mark "Y" means available.

In the above discussed solutions, the reconstructed samples in the left block are divided into four <NUM>×<NUM> partitions. Each one of the <NUM>×<NUM> partitions is considered as a whole unit to determine if reconstructed samples in this partition can be used in coding a current sub-block in CPR mode. The proposed solutions as discussed above can also be applicable to finer partitioning settings, such as, to evaluate each <NUM>×<NUM> partition in the reference sample memory.

Furthermore, the evaluation of whether a reference sub-block for the current sub-block in IBC mode is in the left block can be done by determining whether (a) all the samples in the reference sub-block are from the left block; or (b) any reconstructed sample in the reference sub-block is from the left block.

An example of the block vector constraints based on the fourth solution discussed above is used as a non-limiting example illustrated as follows. Also, in the example illustrated below, a CTU has a size of <NUM>×<NUM> and the reference sample memory has a size of storing one CTU. In the following example, there is no chroma interpolation for IBC mode.

In this non-limiting example, assume the following:.

Therefore, the upper left location of the current sub-block is (xCb, yCb), the lower right location of the current sub-block is (xCb + cbWidth - <NUM>, yCb + cbHeight - <NUM>). Also, the upper left location of the reference sub-block is (xCb + bVx, yCb + bVy), and the lower right location of the reference sub-block is (xCb + bVx + cbWidth - <NUM>, yCb + bVy + cbHeight - <NUM>).

In this non-limiting example, a valid block vector satisfies the following conditions:.

If (xCb + bVx) / ctuSize equals (xCb / ctuSize) - <NUM>, which means at least part of the reference sub-block is in the left block, the followings apply:.

The proposed solutions as described above can be extended also to other configurations of the reference sample memory. Further, while the determination of which reconstructed samples are available is described above with respect to certain processing orders and partition size (e.g., left to right, or top to bottom), it is noted that the determination can be modified in accordance with other processing orders and/or partition sizes in other embodiments.

<FIG> is a schematic illustration of a current block (CTU, <NUM>), a first reference block (CTU, <NUM>), and a second reference block (CTU, <NUM>) between the current block (CTU, <NUM>) and the first reference block (CTU, <NUM>) in a current picture using IBC in accordance with an embodiment. <FIG> shows a current sub-block (<NUM>) in the current block (i.e., CTU) (<NUM>), a first possible reference sub-block (<NUM>) in the first reference block (<NUM>) that is identifiable by a block vector (<NUM>), and a second possible reference sub-block (<NUM>) in the second reference block (<NUM>) that is identifiable by a block vector (<NUM>). Because the memory space for storing the reconstructed samples of block (<NUM>) remains intact when generating the reconstructed samples of the current block (<NUM>), all reconstructed samples of the second reference block (<NUM>) can be available for determining a reference sub-block for the current sub-block (<NUM>) in the IBC mode. However, the memory space for storing the reconstructed samples of the first reference block (<NUM>) is allocated for storing the reconstructed samples of the current block (<NUM>), the availability of the reference sub-block within the first reference block (<NUM>) would depend on whether the collocated block in the current block (<NUM>) has been reconstructed, in a manner similar to those discussed above with reference to <FIG>.

<FIG> shows a flow chart outlining a decoding process (<NUM>) according to an embodiment of the disclosure. The process (<NUM>) can be used in the reconstruction of a block (i.e., a current block) of a picture coded using IBC mode. In some embodiments, one or more operations are performed before or after process (<NUM>), and some of the operations illustrated in <FIG> may be reordered or omitted.

In various embodiments, the process (<NUM>) is executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>), and (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), (<NUM>), or (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented by software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1201) and proceeds to (S1210).

At (S1210), reconstructed samples of a reconstructed block of a picture are stored in a memory. The reconstructed samples of the reconstructed block are reconstructed according to an encoded video bitstream. In some examples, the reconstructed block corresponds to the block (<NUM>) in <FIG> or block (<NUM>) in <FIG>. In some examples, the reconstructed samples of the reconstructed block can be generated using the system or decoders illustrated in <FIG>, <FIG>, and <FIG>.

At (S1220), whether a current sub-block in a current block of the picture is to be reconstructed using intra block copy (IBC) based on a reference sub-block in the reconstructed block is determined. If it is determined that the current sub-block is to be reconstructed using IBC, the process proceeds to (S1230). Otherwise, the current sub-block can be reconstructed using another process, and the process proceeds to (S1299) and terminates for the purposes of coding using IBC mode.

At (S1230), whether the reconstructed samples of the reference sub-block stored in the memory are not overwritten (or otherwise indicated as overwritten) is determined based on a position of the current sub-block. In some examples, the reconstructed samples of the reference sub-block stored in the memory are determined as overwritten as described above, for example with reference to <FIG>. When it is determined that the reconstructed samples of the reference sub-block stored in the memory are indicated as not overwritten, the process proceeds to (S1240). Otherwise, the current sub-block is to be reconstructed without using the reconstructed samples of the reference sub-block or by another process, and the process proceeds to (S1299) and terminates for the purposes of coding using IBC mode. In some examples, the reconstructed samples of the reconstructed block can be generated using the system or decoders illustrated in <FIG>, <FIG>, and <FIG>.

At (S1240), the reconstructed samples of the current sub-block are generated for output based on the reconstructed samples of the reference sub-block when the reconstructed samples of the reference sub-block stored in the memory are determined to be indicated as not overwritten. At (S1250), the reconstructed samples of a collocated sub-block in the reconstructed block stored in the memory are overwritten with the generated reconstructed samples of the current sub-block. In some examples, the reconstructed samples of the reconstructed block can be generated using the system or decoders illustrated in <FIG>, <FIG>, and <FIG>.

After (S1250), the process proceeds to (S1299) and terminates.

<FIG> shows a flow chart outlining an encoding process (<NUM>) according to an embodiment of the disclosure. The process (<NUM>) can be used to encode a block (i.e., a current block) of a picture using IBC mode. In some embodiments, one or more operations are performed before or after process (<NUM>), and some of the operations illustrated in <FIG> may be reordered or omitted.

In various embodiments, the process (<NUM>) is executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>), and (<NUM>), the processing circuitry that performs functions of the video encoder (<NUM>), (<NUM>), or (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented by software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1301) and proceeds to (S1310).

At (S1310), reconstructed samples of a reconstructed block of a picture are stored in a memory. The reconstructed samples of the reconstructed block are reconstructed according to encoded prediction information. In some examples, the reconstructed block corresponds to the block (<NUM>) in <FIG> or block (<NUM>) in <FIG>. In some examples, the reconstructed samples of the reconstructed block can be generated using the system or encoders illustrated in <FIG>, <FIG>, and <FIG>.

At (S1320), whether a current sub-block in a current block of the picture is to be coded using intra block copy (IBC) based on a reference sub-block in the reconstructed block is determined. If it is determined that the current sub-block is to be coded using IBC, the process proceeds to (S1330). Otherwise, the current sub-block can be coded using a process not fully described in this disclosure, and the process proceeds to (S1399) and terminates for the purposes of coding using IBC mode.

At (S1330), a range of reconstructed samples stored in the memory that are not overwritten (or otherwise indicated as overwritten) is determined based on at least a position of the current sub-block. In some examples, the range of reconstructed samples stored in the memory can be determined as overwritten or not as illustrated described above, for example with reference to <FIG>. In some examples, the reconstructed samples of the reconstructed block can be generated using the system or encoders illustrated in <FIG>, <FIG>, and <FIG>.

At (S1340), a reference sub-block within the range of reconstructed samples that are not indicated as overwritten is determined. At (S1350), reconstructed samples of the current sub-block are generated based on the reconstructed samples of the reference sub-block. At (S1360), the reconstructed samples of a collocated sub-block in the reconstructed block stored in the memory are overwritten with the generated reconstructed samples of the current sub-block. In some examples, the reconstructed samples of the reconstructed block can be generated using the system or encoders illustrated in <FIG>, <FIG>, and <FIG>.

After (S1360), the process proceeds to (S1399) and terminates.

Computer system (<NUM>) can also include an interface to one or more communication networks. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, <NUM>, <NUM>, <NUM>, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CANBus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general purpose data ports or peripheral buses.

(<NUM>) (such as, for example USB ports of the computer system (<NUM>)); others are commonly integrated into the core of the computer system (<NUM>) by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system). Using any of these networks, computer system (<NUM>) can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV), uni-directional send-only (for example CANbus to certain CANbus devices), or bi-directional, for example to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

As an example and not by way of limitation, the computer system having architecture (<NUM>), and specifically the core (<NUM>) can provide functionality as a result of processor(s) (including CPUs, GPUs, FPGA, accelerators, and the like) executing software embodied in one or more tangible, computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as introduced above, as well as certain storage of the core (<NUM>) that are of non-transitory nature, such as core-internal mass storage.

(<NUM>) or ROM (<NUM>). The software implementing various embodiments of the present disclosure can be stored in such devices and executed by core (<NUM>). A computer-readable medium can include one or more memory devices or chips, according to particular needs. The software can cause the core (<NUM>) and specifically the processors therein (including CPU, GPU, FPGA, and the like) to execute particular processes or particular parts of particular processes described herein, including defining data structures stored in RAM (<NUM>) and modifying such data structures according to the processes defined by the software. In addition or as an alternative, the computer system can provide functionality as a result of logic hardwired or otherwise embodied in a circuit (for example: accelerator (<NUM>)), which can operate in place of or together with software to execute particular processes or particular parts of particular processes described herein. Reference to software can encompass logic, and vice versa, where appropriate. Reference to a computer-readable media can encompass a circuit (such as an integrated circuit (IC)) storing software for execution, a circuit embodying logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.

Claim 1:
A method for video decoding in a decoder, comprising:
storing (S1210) reconstructed samples of a reconstructed block of a picture in a memory, the reconstructed samples of the reconstructed block being reconstructed according to a Versatile Video Coding, VVC, encoded video bitstream; and
when a current sub-block in a current block of the picture is to be reconstructed using (S1220) intra block copy, IBC, based on a reference sub-block in the reconstructed block, wherein the current block having a size of <NUM>×<NUM> luma samples includes at least four non-overlapping partitions, including a current partition in which the current sub-block is located, and the at least four non-overlapping partitions each having a maximum size of <NUM>×<NUM> luma samples, the reconstructed block having a size of <NUM>×<NUM> luma samples includes at least four non-overlapping partitions that are collocated with the at least four partitions of the current block respectively,
determining (S1230) whether the reconstructed samples of the reference sub-block stored in the memory are indicated as overwritten based on a position of the current sub-block,
generating (S1240) reconstructed samples of the current sub-block for output based on the reconstructed samples of the reference sub-block when the reconstructed samples of the reference sub-block stored in the memory are determined to be indicated as not overwritten, and
overwriting (S1250) the reconstructed samples of a collocated sub-block in the reconstructed block stored in the memory with the generated reconstructed samples of the current sub-block,
generating the reconstructed samples of the current sub-block without using the reference sub-block when the reconstructed samples of the reference sub-block stored in the memory are determined to be indicated as overwritten.