Patent Description:
In spite of the advances in video compression, digital video still accounts for the largest bandwidth use on the internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, it is expected that the bandwidth demand for digital video usage will continue to grow.

Devices, systems and methods related to digital video coding, and specifically, to video and image coding and decoding in which adaptive loop filtering is used are disclosed in accordance with the appended claims.

Embodiments of the disclosed technology may be applied to existing video coding standards (e.g., HEVC, H. <NUM>) and future standards to improve compression performance. Section headings are used in the present document to improve readability of the description and do not in any way limit the discussion or the embodiments (and/or implementations) to the respective sections only.

This document is related to video coding technologies. Specifically, it is related to adaptive loop filter in video coding or decoding. It may be applied to the existing video coding standard like HEVC, or the standard (Versatile Video Coding) to be finalized. It may be also applicable to future video coding standards or video codec.

Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. The ITU-T produced H. <NUM> and H. <NUM>, ISO/IEC produced MPEG-<NUM> and MPEG-<NUM> Visual, and the two organizations jointly produced the H. <NUM>/MPEG-<NUM> Video and H. <NUM>/MPEG-<NUM> Advanced Video Coding (AVC) and H. <NUM>/HEVC standards. <NUM>, the video coding standards are based on the hybrid video coding structure wherein temporal prediction plus transform coding are utilized. To explore the future video coding technologies beyond HEVC, Joint Video Exploration Team (JVET) was founded by VCEG and MPEG jointly in <NUM>. Since then, many new methods have been adopted by JVET and put into the reference software named Joint Exploration Model (JEM). In April <NUM>, the Joint Video Expert Team (JVET) between VCEG (Q6/<NUM>) and ISO/IEC JTC1 SC29/WG11 (MPEG) was created to work on the VVC standard targeting at <NUM>% bitrate reduction compared to HEVC.

The latest version of VVC draft, i.e., Versatile Video Coding (Draft <NUM>) could be found at:
http://phenix. it-sudparis. eu/jvet/doc_end_user/documents/15_Gothenburg/wg11/JVET-O2001-v14.

The latest reference software of VVC, named VTM, could be found at:
https://vcgit. fraunhofer. de/jvet/VVCSoftware_VTM/tags/VTM-<NUM>.

Color space, also known as the color model (or color system), is an abstract mathematical model which simply describes the range of colors as tuples of numbers, typically as <NUM> or <NUM> values or color components (e.g. RGB). Basically speaking, color space is an elaboration of the coordinate system and sub-space.

For video compression, the most frequently used color spaces are YCbCr and RGB.

YCbCr, Y'CbCr, or Y Pb/Cb Pr/Cr, also written as YCBCR or Y'CBCR, is a family of color spaces used as a part of the color image pipeline in video and digital photography systems. Y' is the luma component and CB and CR are the blue-difference and red-difference chroma components. Y' (with prime) is distinguished from Y, which is luminance, meaning that light intensity is nonlinearly encoded based on gamma corrected RGB primaries.

Chroma subsampling is the practice of encoding images by implementing less resolution for chroma information than for luma information, taking advantage of the human visual system's lower acuity for color differences than for luminance.

<NUM>:<NUM>:<NUM>
Each of the three Y'CbCr components have the same sample rate, thus there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and cinematic post production.

<NUM>:<NUM>:<NUM>
The two chroma components are sampled at half the sample rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of an uncompressed video signal by one-third with little to no visual difference.

<NUM>:<NUM>:<NUM>
In <NUM>:<NUM>:<NUM>, the horizontal sampling is doubled compared to <NUM>:<NUM>:<NUM>, but as
the Cb and Cr channels are only sampled on each alternate line in this scheme, the vertical resolution is halved. The data rate is thus the same. Cb and Cr are each subsampled at a factor of <NUM> both horizontally and vertically. There are three variants of <NUM>:<NUM>:<NUM> schemes, having different horizontal and vertical siting.

<FIG> shows an example of encoder block diagram of VVC, which contains three in-loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO) and ALF. Unlike DF, which uses predefined filters, SAO and ALF utilize the original samples of the current picture to reduce the mean square errors between the original samples and the reconstructed samples by adding an offset and by applying a finite impulse response (FIR) filter, respectively, with coded side information signaling the offsets and filter coefficients. ALF is located at the last processing stage of each picture and can be regarded as a tool trying to catch and fix artifacts created by the previous stages.

In the JEM, a geometry transformation-based adaptive loop filter (GALF) with block-based filter adaption is applied. For the luma component, one among <NUM> filters is selected for each <NUM>×<NUM> block, based on the direction and activity of local gradients.

In the JEM, up to three diamond filter shapes (as shown in <FIG>) can be selected for the luma component. An index is signalled at the picture level to indicate the filter shape used for the luma component.

<FIG> shows examples of GALF filter shapes (left: <NUM>×<NUM> diamond, middle: <NUM>×<NUM> diamond, right: <NUM>×<NUM> diamond).

For chroma components in a picture, the <NUM>×<NUM> diamond shape is always used.

Each <NUM> × <NUM> block is categorized into one out of <NUM> classes. The classification index C is derived based on its directionality D and a quantized value of activity Â, as follows: <MAT>.

To calculate D and Â, gradients of the horizontal, vertical and two diagonal direction are first calculated using <NUM>-D Laplacian: <MAT> <MAT> <MAT> <MAT>.

Indices i and j refer to the coordinates of the upper left sample in the <NUM> × <NUM> block and R(i,j) indicates a reconstructed sample at coordinate (i, j).

Then D maximum and minimum values of the gradients of horizontal and vertical directions are set as: <MAT>
and the maximum and minimum values of the gradient of two diagonal directions are set as: <MAT>.

To derive the value of the directionality D, these values are compared against each other and with two thresholds t<NUM> and t<NUM>:.

The activity value A is calculated as: <MAT>.

A is further quantized to the range of <NUM> to <NUM>, inclusively, and the quantized value is denoted as Â.

For both chroma components in a picture, no classification method is applied, i.e. a single set of ALF coefficients is applied for each chroma component.

Before filtering each <NUM>×<NUM> block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k, l) depending on gradient values calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks to which ALF is applied more similar by aligning their directionality.

Three geometric transformations, including diagonal, vertical flip and rotation are introduced: <MAT>
where K is the size of the filter and <NUM> ≤ k, l ≤ K - <NUM> are coefficients coordinates, such that location (<NUM>,<NUM>) is at the upper left corner and location (K - <NUM>, K - <NUM>) is at the lower right corner. The transformations are applied to the filter coefficients f(k, l) depending on gradient values calculated for that block. The relationship between the transformation and the four gradients of the four directions are summarized in Table <NUM>.

In the JEM, GALF filter parameters are signalled for the first CTU, i.e., after the slice header and before the SAO parameters of the first CTU. Up to <NUM> sets of luma filter coefficients could be signalled. To reduce bits overhead, filter coefficients of different classification can be merged. Also, the GALF coefficients of reference pictures are stored and allowed to be reused as GALF coefficients of a current picture. The current picture may choose to use GALF coefficients stored for the reference pictures, and bypass the GALF coefficients signalling. In this case, only an index to one of the reference pictures is signalled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture.

To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of decoding a new sequence, the candidate list is empty. After decoding one picture, the corresponding set of filters may be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e., <NUM> in current JEM), a new set of filters overwrites the oldest set in decoding order, and that is, first-in-first-out (FIFO) rule is applied to update the candidate list. To avoid duplications, a set could only be added to the list when the corresponding picture doesn't use GALF temporal prediction. To support temporal scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a temporal layer. More specifically, each array assigned by temporal layer index (TempIdx) may compose filter sets of previously decoded pictures with equal to lower TempIdx. For example, the k-th array is assigned to be associated with TempIdx equal to k, and it only contains filter sets from pictures with TempIdx smaller than or equal to k. After coding a certain picture, the filter sets associated with the picture will be used to update those arrays associated with equal or higher TempIdx.

Temporal prediction of GALF coefficients is used for inter coded frames to minimize signalling overhead. For intra frames, temporal prediction is not available, and a set of <NUM> fixed filters is assigned to each class. To indicate the usage of the fixed filter, a flag for each class is signalled and if required, the index of the chosen fixed filter. Even when the fixed filter is selected for a given class, the coefficients of the adaptive filter f(k, l) can still be sent for this class in which case the coefficients of the filter which will be applied to the reconstructed image are sum of both sets of coefficients.

The filtering process of luma component can controlled at CU level. A flag is signalled to indicate whether GALF is applied to the luma component of a CU. For chroma component, whether GALF is applied or not is indicated at picture level only.

At decoder side, when GALF is enabled for a block, each sample R(i, j) within the block is filtered, resulting in sample value R'(i, j) as shown below, where L denotes filter length, fm,n represents filter coefficient, and f(k, l) denotes the decoded filter coefficients.

Overall encoder decision process for GALF is illustrated in <FIG>. For luma samples of each CU, the encoder makes a decision on whether or not the GALF is applied and the appropriate signalling flag is included in the slice header. For chroma samples, the decision to apply the filter is done based on the picture-level rather than CU-level. Furthermore, chroma GALF for a picture is checked only when luma GALF is enabled for the picture.

The current design of GALF in VVC has the following major changes compared to that in JEM:.

<FIG> shows examples of subsampled Laplacian calculation for CE2. Top left (a) Subsampled positions for vertical gradient, top right (b) Subsampled positions for horizontal gradient, bottom left (c) Subsampled positions for diagonal gradient and bottom right (d) Subsampled positions for diagonal gradient.

In the latest version of VVC draft, ALF parameters can be signaled in Adaptation Parameter Set (APS) and can be selected by each CTU adaptively.

The detailed signaling of ALF (in JVET-O2001-vE) is as follows.

Each APS RBSP shall be available to the decoding process prior to it being referred, included in at least one access unit with TemporalId less than or equal to the TemporalId of the coded slice NAL unit that refers it or provided through external means.

Let aspLayerId be the nuh_layer_id of an APS NAL unit. If the layer with nuh_layer_id equal to aspLayerId is an independent layer (i.e., vps_independent_layer_flag[ GeneralLayerIdx[ aspLayerId ] ] is equal to <NUM>), the APS NAL unit containing the APS RBSP shall have nuh_layer_id equal to the nuh_layer_id of a coded slice NAL unit that referrs it. Otherwise, the APS NAL unit containing the APS RBSP shall have nuh_layer_id either equal to the nuh_layer_id of a coded slice NAL unit that referrs it, or equal to the nuh_layer_id of a direct dependent layer of the layer containing a coded slice NAL unit that referrs it. All APS NAL units with a particular value of adaptation_parameter_set_id and a particular value of aps_params_type within an access unit shall have the same content. adaptation_parameter_set_id provides an identifier for the APS for reference by other syntax elements.

When aps_params_type is equal to ALF_APS or SCALING_APS, the value of adaptation_parameter_set_id shall be in the range of <NUM> to <NUM>, inclusive.

When aps_params_type is equal to LMCS_APS, the value of adaptation_parameter_set_id shall be in the range of <NUM> to <NUM>, inclusive.

aps_params_type specifies the type of APS parameters carried in the APS as specified in Table <NUM>-<NUM>. When aps_params_type is equal to <NUM> (LMCS_APS), the value of adaptation_parameter_set_id shall be in the range of <NUM> to <NUM>, inclusive.

aps_extension_flag equal to <NUM> specifies that no aps_extension_data_flag syntax elements are present in the APS RBSP syntax structure. aps_extension_flag equal to <NUM> specifies that there are aps_extension_data_flag syntax elements present in the APS RBSP syntax structure.

aps_extension_data_flag may have any value. Its presence and value do not affect decoder conformance to profiles specified in this version of this Specification. Decoders conforming to this version of this Specification shall ignore all aps_extension_data_flag syntax elements.

alf_luma_filter_signal_flag equal to <NUM> specifies that a luma filter set is signalled.

alf_luma_filter_signal_flag equal to <NUM> specifies that a luma filter set is not signalled.

alf_chroma_filter_signal_flag equal to <NUM> specifies that a chroma filter is signalled.

alf_chroma_filter_signal_flag equal to <NUM> specifies that a chroma filter is not signalled. When ChromaArrayType is equal to <NUM>, alf_chroma_filter_signal_flag shall be equal to <NUM>.

The variable NumAlfFilters specifying the number of different adaptive loop filters is set equal to <NUM>.

alf_luma_clip_flag equal to <NUM> specifies that linear adaptive loop filtering is applied on luma component. alf_luma_clip_flag equal to <NUM> specifies that non-linear adaptive loop filtering may be applied on luma component.

alf_luma_num_filters_signalled_minus1 plus <NUM> specifies the number of adpative loop filter classes for which luma coefficients can be signalled. The value of alf_luma_num_filters_signalled_minus1 shall be in the range of <NUM> to NumAlfFilters - <NUM>, inclusive.

alf_luma_coeff_delta_idx[ filtIdx ] specifies the indices of the signalled adaptive loop filter luma coefficient deltas for the filter class indicated by filtIdx ranging from <NUM> to NumAlfFilters - <NUM>. When alf_luma_coeff_delta_idx[ filtIdx ] is not present, it is inferred to be equal to <NUM>. The length of alf_luma_coeff_delta_idx[ filtIdx ] is.

Ceil( Log2( alf_luma_num_filters_signalled_minus1 + <NUM> ) ) bits.

alf_luma_coeff_signalled_flag equal to <NUM> indicates that alf_luma_coeff_flag[ sfIdx ] is signalled. alf_luma_coeff_signalled_flag equal to <NUM> indicates that alf_luma_coeff_flag[ sfIdx ] is not signalled.

alf_luma_coeff_flag[ sfIdx ] equal <NUM> specifies that the coefficients of the luma filter indicated by sfIdx are signalled. alf_luma_coeff_flag[ sfIdx ] equal to <NUM> specifies that all filter coefficients of the luma filter indicated by sfIdx are set equal to <NUM>. When not present, alf_luma_coeff_flag[ sfIdx ] is set equal to <NUM>.

alf_luma_coeff_abs[ sfIdx ][ j ] specifies the absolute value of the j-th coefficient of the signalled luma filter indicated by sfIdx. When alf_luma_coeff_abs[ sfIdx ][ j ] is not present, it is inferred to be equal <NUM>.

The order k of the exp-Golomb binarization uek(v) is set equal to <NUM>.

alf_luma_coeff_sign[ sfIdx ][ j ] specifies the sign of the j-th luma coefficient of the filter indicated by sfIdx as follows:.

When alf_luma_coeff_sign[ sfIdx ][ j ] is not present, it is inferred to be equal to <NUM>.

The variable filtCoeff[ sfIdx ][ j ] with sfIdx = <NUM>. alf_luma_num_filters_signalled_minus1, j = <NUM>. <NUM> is initialized as follows: <MAT>.

The luma filter coefficients AlfCoeffL[ adaptation_parameter_set_id ] with elements AlfCoeffL[ adaptation_parameter_set_id ][ filtIdx ][ j ], with filtIdx = <NUM>. NumAlfFilters - <NUM> and j = <NUM>. <NUM> are derived as follows: <MAT>.

The fixed filter coefficients AlfFixFiltCoeff[ i ][ j ] with i = <NUM>. <NUM>, j = <NUM>. <NUM> and the class to filter mapping AlfClassToFiltMap[ m ][ n ] with m = <NUM>. <NUM> and n = <NUM>. <NUM> are derived as follows:
<IMG>
<IMG>
<IMG>.

It is a requirement of bitstream conformance that the values of AlfCoeffL[ adaptation_parameter_set_id ][ filtIdx ][ j ] with filtIdx = <NUM>. NumAlfFilters - <NUM>, j = <NUM>. <NUM> shall be in the range of -<NUM><NUM> to <NUM><NUM> - <NUM>, inclusive.

alf_luma_clip_idx[ sfIdx ][ j ] specifies the clipping index of the clipping value to use before multiplying by the j-th coefficient of the signalled luma filter indicated by sfIdx. It is a requirement of bitstream conformance that the values of alf_luma_clip_idx[ sfIdx ][ j ] with sfIdx = <NUM>. alf_luma_num_filters_signalled_minus1 and j = <NUM>. <NUM> shall be in the range of <NUM> to <NUM>, inclusive.

The luma filter clipping values AlfClipL[ adaptation_parameter_set_id ] with elements AlfClipL[adaptation_parameter_set_id ][ filtIdx ][ j ], with filtIdx = <NUM>. NumAlfFilters - <NUM> and j = <NUM>. <NUM> are derived as specified in Table <NUM>-<NUM> depending on bitDepth set equal to BitDepthY and clipIdx set equal to alf_luma_clip_idx[ alf_luma_coeff_delta_idx[ filtIdx ] ][ j ].

alf_chroma_num_alt_filters_minus1 plus <NUM> specifies the number of alternative filters for chroma components.

alf_chroma_clip_flag[ altIdx ] equal to <NUM> specifies that linear adaptive loop filtering is applied on chroma components when using the chroma filter with index altIdx;
alf_chroma_clip_flag[ altIdx ] equal to <NUM> specifies that non-linear adaptive loop filtering is applied on chroma components when using the chroma filter with index altIdx. When not present, alf_chroma_clip_flag[ altIdx ] is inferred to be equal to <NUM>.

alf_chroma_coeff_abs[ altIdx ][ j ] specifies the absolute value of the j-th chroma filter coefficient for the alternative chroma filter with index altIdx. When alf_chroma coeff_abs[ altIdx ][ j ] is not present, it is inferred to be equal <NUM>. It is a requirement of bitstream conformance that the values of alf_chroma coeff_abs[ altIdx ][ j ] shall be in the range of <NUM> to <NUM><NUM> - <NUM>, inclusive.

alf_chroma_coeff_sign[ altIdx ][ j ] specifies the sign of the j-th chroma filter coefficient for the alternative chroma filter with index altIdx as follows:.

When alf_chroma_coeff_sign[ altIdx ][ j ] is not present, it is inferred to be equal to <NUM>.

The chroma filter coefficients AlfCoeffC[ adaptation_parameter_set_id ][ altIdx ] with elements AlfCoeffC[ adaptation_parameter_set_id ][ altIdx ][ j ], with
altIdx = <NUM>. alf_chroma_num_alt_filters_minus1, j = <NUM>. <NUM> are derived as follows:
AlfCoeffC[ adaptation_parameter_set_id ][ altIdx ][ j ] = alf_chroma coeff_abs[ altIdx ][ j ] * (<NUM>-<NUM>)
( <NUM> - <NUM> * alf_chroma coeff_sign[ altIdx ][ j ] ).

It is a requirement of bitstream conformance that the values of.

alf_chroma_clip_idx[ altIdx ][ j ] specifies the clipping index of the clipping value to use before multiplying by the j-th coefficient of the alternative chroma filter with index altIdx. It is a requirement of bitstream conformance that the values of alf_chroma_clip_idx[ altIdx ][ j ] with altIdx = <NUM>. alf_chroma_num_alt_filters_minus1, j = <NUM>. <NUM> shall be in the range of <NUM> to <NUM>, inclusive. The chroma filter clipping values AlfClipC[ adaptation_parameter_set_id ][ altIdx ] with elements AlfClipC[ adaptation_parameter_set_id ][ altIdx ][ j ], with.

In the VTM6, ALF filter parameters are signalled in Adaptation Parameter Set (APS). In one APS, up to <NUM> sets of luma filter coefficients and clipping value indexes, and up to <NUM> sets of chroma filter coefficients and clipping value indexes could be signalled. To reduce bits overhead, filter coefficients of different classification for luma component can be merged. In slice header, the indices of the APSs used for the current slice are signaled.

Clipping value indexes, which are decoded from the APS, allow determining clipping values using a Luma table of clipping values and a Chroma table of clipping values. These clipping values are dependent of the internal bitdepth. More precisely, the Luma table of clipping values and Chroma table of clipping values are obtained by the following formulas: <MAT> <MAT>
with B equal to the internal bitdepth and N equal to <NUM> which is the number of allowed clipping values in VTM6.

In slice header, up to <NUM> APS indices can be signaled to specify the luma filter sets that are used for the current slice. The filtering process can be further controlled at CTB level. A flag is always signalled to indicate whether ALF is applied to a luma CTB. A luma CTB can choose a filter set among <NUM> fixed filter sets and the filter sets from APSs. A filter set index is signaled for a luma CTB to indicate which filter set is applied. The <NUM> fixed filter sets are pre-defined and hard-coded in both the encoder and the decoder.

For chroma component, an APS index is signaled in slice header to indicate the chroma filter sets being used for the current slice. At CTB level, a filter index is signaled for each chroma CTB if there is more than one chroma filter set in the APS.

More specifically, the followings apply:.

Slice on/off control flags are firstly coded to indicate whether at least one CTU in the slice applies ALF. When it is true, for each CTU, the following are checked and signaled in order:.

slice: An integer number of bricks of a picture that are exclusively contained in a single NAL unit.

NOTE - A slice consists of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.

NOTE - A tile may be partitioned into multiple bricks, each of which consisting of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks is also referred to as a brick. However, a brick that is a true subset of a tile is not referred to as a tile. brick scan: A specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture.

A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs that covers a rectangular region of a picture.

A tile is divided into one or more bricks, each of which consisting of a number of CTU rows within the tile.

A tile that is not partitioned into multiple bricks is also referred to as a brick. However, a brick that is a true subset of a tile is not referred to as a tile.

A slice either contains a number of tiles of a picture or a number of bricks of a tile.

A subpicture contains one or more slices that collectively cover a rectangular region of a picture.

Two modes of slices are supported, namely the raster-scan slice mode and the rectangular slice mode. In the raster-scan slice mode, a slice contains a sequence of tiles in a tile raster scan of a picture. In the rectangular slice mode, a slice contains a number of bricks of a picture that collectively form a rectangular region of the picture. The bricks within a rectangular slice are in the order of brick raster scan of the slice.

<FIG> shows an example of raster-scan slice partitioning of a picture, where the picture is divided into <NUM> tiles and <NUM> raster-scan slices.

<FIG> shows an example of rectangular slice partitioning of a picture, where the picture is divided into <NUM> tiles (<NUM> tile columns and <NUM> tile rows) and <NUM> rectangular slices.

<FIG> shows an example of a picture partitioned into tiles, bricks, and rectangular slices, where the picture is divided into <NUM> tiles (<NUM> tile columns and <NUM> tile rows), <NUM> bricks (the top-left tile contains <NUM> brick, the top-right tile contains <NUM> bricks, the bottom-left tile contains <NUM> bricks, and the bottom-right tile contain <NUM> bricks), and <NUM> rectangular slices.

<FIG> shows an example of subpicture partitioning of a picture, where a picture is partitioned into <NUM> subpictures of varying dimensions.

When a picture is coded using three separate colour planes (separate_colour_plane_flag is equal to <NUM>), a slice contains only CTUs of one colour component being identified by the corresponding value of colour_plane_id, and each colour component array of a picture consists of slices having the same colour_plane_id value. Coded slices with different values of colour_plane_id within a picture may be interleaved with each other under the constraint that for each value of colour_plane_id, the coded slice NAL units with that value of colour_plane_id shall be in the order of increasing CTU address in brick scan order for the first CTU of each coded slice NAL unit.

NOTE <NUM> - When separate_colour_plane_flag is equal to <NUM>, each CTU of a picture is contained in exactly one slice. When separate _colour_plane _flag is equal to <NUM>, each CTU of a colour component is contained in exactly one slice (i.e., information for each CTU of a picture is present in exactly three slices and these three slices have different values of colour_plane_id).

AVC and HEVC does not have the ability to change resolution without having to introduce an IDR or intra random access point (IRAP) picture; such ability can be referred to as adaptive resolution change (ARC). There are use cases or application scenarios that would benefit from an ARC feature, such as Rate adaption in video telephony and conferencing. ARC is also known as Dynamic resolution conversion.

ARC may also be regarded as a special case of Reference Picture Resampling (RPR) such as H. <NUM> Annex P.

In VVC, the ARC, a. RPR (Reference Picture Resampling) is incorporated in JVET-O2001-v14. With RPR in JVET-O2001-v14, TMVP is disabled if the collocated picture has a different resolution to the current picture. Besides, BDOF and DMVR are disabled when the reference picture has a different resolution to the current picture. In SPS, the maximum picture resolution is defined. And for each picture in PPS, its resolution (including picture width and height in luma samples) are defined. When the picture resolution is different, the RPR is enabled.

Conformance window in VVC defines a rectangle. Samples inside the conformance window belongs to the image of interest. Samples outside the conformance window may be discarded when output.

When conformance window is applied, the scaling ration in RPR is derived based on conformance windows.

When subpics_present _flag is equal to <NUM>, the value of pic_width_in_luma_samples shall be equal to pic_width_max_in_luma_samples. pic_height_in_luma_samples specifies the height of each decoded picture referring to the PPS in units of luma samples. pic_height_in_luma_samples shall not be equal to <NUM> and shall be an integer multiple of Max( <NUM>, MinCbSizeY ), and shall be less than or equal to pic_height_max_in_luma_samples.

When subpics_present _flag is equal to <NUM>, the value of pic_height_in_luma_samples shall be equal to pic_height_max_in_luma_samples.

Let refPicWidthInLumaSamples and refPicHeightInLumaSamples be the pic_width_in_luma_samples and pic_height_in_luma_samples, respectively, of a reference picture of a current picture referring to this PPS. Is a requirement of bitstream conformance that all of the following conditions are satisfied:.

The conformance cropping window contains the luma samples with horizontal picture coordinates from SubWidthC * conf_win _left_offset to pic_width_in_luma_samples - ( SubWidthC * conf_win_right_offset + <NUM> ) and vertical picture coordinates from SubHeightC * conf_win_top_offset to pic_height_in_luma_samples - ( SubHeightC * conf_win_bottom_offset + <NUM> ), inclusive.

The value of SubWidthC * ( conf_win_left_offset + conf_win_right_offset ) shall be less than pic_width_in_luma_samples, and the value of SubHeightC * ( conf_win_top_offset + conf_win_bottom_offset ) shall be less than pic_height_in_luma_samples.

The variables PicOutputWidthL and PicOutputHeightL are derived as follows:
PicOutputWidthL = pic_width_in_luma_samples - (<NUM>-<NUM>)
SubWidthC * ( conf_win_right_offset + conf_win_left_offset ) PicOutputHeightL = pic_height_in_pic_size_units - (<NUM>-<NUM>) SubHeightC * ( conf_win_bottom_offset + conf_win_top_offset ).

When ChromaArrayType is not equal to <NUM>, the corresponding specified samples of the two chroma arrays are the samples having picture coordinates ( x / SubWidthC, y / SubHeightC ), where ( x, y ) are the picture coordinates of the specified luma samples.

NOTE - The conformance cropping window offset parameters are only applied at the output.

All internal decoding processes are applied to the uncropped picture size.

Let ppsA and ppsB be any two PPSs referring to the same SPS. It is a requirement of bitstream conformance that, when ppsA and ppsB have the same the values of pic_width_in_luma_samples and pic_height_in_luma_samples, respectively, ppsA and ppsB shall have the same values of conf_win_left_offset, conf_win_right_offset, conf_win_top_offset, and conf_win_bottom_offset, respectively.

The ALF data in APS has the following problem:.

The listing below should be considered as examples to explain general concepts. These items should not be interpreted in a narrow way.

In this document, the resolution (or dimensions, or width/height, or size) of a picture may refer to the resolution (or dimensions, or width/height, or size) of the coded/decoded picture, or may refer to the resolution (or dimensions, or width/height, or size) of the conformance window in the coded/decoded picture.

Examples of the Present Technology The deleted parts are enclosed in double bolded brackets (e.g., [[a]] indicates that "a" has been deleted) and the newly added parts are enclosed in double bolded braces (e.g., {{a}} indicates that "a" has been added). The embodiment is on top of JVET-O2001-vE.

This part gives some examples on how to signal ALF parameters in ALF APS.

alf_luma_filter_signal_flag equal to <NUM> specifies that a luma filter set is signalled. alf_luma_filter_signal_flag equal to <NUM> specifies that a luma filter set is not signalled. alf_chroma_filter_signal_flag equal to <NUM> specifies that a chroma filter is signalled. alf_chroma_filter_signal_flag equal to <NUM> specifies that a chroma filter is not signalled. When ChromaArrayType is equal to <NUM>, alf_chroma_filter_signal_flag shall be equal to <NUM>.

alf_luma_coeff_delta_idx[ filtIdx ] specifies the indices of the signalled adaptive loop filter luma coefficient deltas for the filter class indicated by filtIdx ranging from <NUM> to NumAlfFilters - <NUM>. When alf_luma_coeff_delta_idx[ filtIdx ] is not present, it is inferred to be equal to <NUM>. The length of alf_luma_coeff_delta_idx[ filtIdx ] is Ceil( Log2( alf_luma_num_filters_signalled_minus1 + <NUM> ) ) bits.

[[ alf_luma_coeff_signalled_flag equal to <NUM> indicates that alf_luma_coeff_flag[ sfIdx ] is signalled. alf_luma_coeff_signalled_flag equal to <NUM> indicates that alf_luma_coeff_flag[ sfIdx ] is not signalled.

alf_luma_coeff_flag[ sfIdx ] equal <NUM> specifies that the coefficients of the luma filter indicated by sfIdx are signalled. alf_luma_coeff_flag[ sfIdx ] equal to <NUM> specifies that all filter coefficients of the luma filter indicated by sfIdx are set equal to <NUM>. When not present,.

This embodiment, which is claimed, gives an example on how to signal ALF parameters for luma CTBs.

[[ alf_ctb_use_first_aps_flag equal to <NUM> sepcifies that the filter information in APS with adaptive_parameter_set_id equal to slice_alf_aps_id_luma[ <NUM> ] is used.

alf_ctb_use_first_aps_flag equal to <NUM> specifies that the luma CTB does not use the filter information in APS with adaptive_parameter_set_id equal to slice_alf_aps_id_luma[ <NUM> ]. When alf_ctb_use_first_aps_flag is not present, it is inferred to be equal to <NUM>. ]]
alf_use_aps_flag equal to <NUM> specifies that one of the fixed filter sets is applied to the luma CTB. alf_use_aps_flag equal to <NUM> specifies that a filter set from an APS is applied to the luma CTB. When alf_use_aps_flag is not present, it is inferred to be equal to <NUM>.

alf_luma_prev_filter_idx[[_minus1 plus <NUM>]] specifies the previous filter that is applied to the luma CTB. The value of alf_luma_prev_filter_idx[[_minus1]] shall be in a range of <NUM> to slice_num_alf_aps_ids_luma - {{<NUM>}}[[<NUM>]], inclusive. When alf_luma_prev_filter_idx[[_minus1]] is not present, it is inferred to be equal to <NUM>.

The variable AlfCtbFiltSetIdxY[ xCtb >> CtbLog2SizeY ][ yCtb >> CtbLog2SizeY ] specifying the filter set index for the luma CTB at location ( xCtb, yCtb ) is derived as follows:.

alf_luma_fixed_filter_idx specifies the fixed filter that is applied to the luma CTB. The value of alf_luma_fixed_filter_idx shall be in a range of <NUM> to <NUM>, inclusive.

<FIG> is a block diagram showing an example video processing system <NUM> in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system <NUM>. The system <NUM> may include input <NUM> for receiving video content. The video content may be received in a raw or uncompressed format, e.g., <NUM> or <NUM> bit multi-component pixel values, or may be in a compressed or encoded format. The input <NUM> may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interface include wired interfaces such as Ethernet, passive optical network (PON), etc. and wireless interfaces such as Wi-Fi or cellular interfaces.

The system <NUM> may include a coding component <NUM> that may implement the various coding or encoding methods described in the present document. The coding component <NUM> may reduce the average bitrate of video from the input <NUM> to the output of the coding component <NUM> to produce a coded representation of the video. The coding techniques are therefore sometimes called video compression or video transcoding techniques. The output of the coding component <NUM> may be either stored, or transmitted via a communication connected, as represented by the component <NUM>. The stored or communicated bitstream (or coded) representation of the video received at the input <NUM> may be used by the component <NUM> for generating pixel values or displayable video that is sent to a display interface <NUM>. The process of generating user-viewable video from the bitstream representation is sometimes called video decompression. Furthermore, while certain video processing operations are referred to as "coding" operations or tools, it will be appreciated that the coding tools or operations are used at an encoder and corresponding decoding tools or operations that reverse the results of the coding will be performed by a decoder.

Examples of a peripheral bus interface or a display interface may include universal serial bus (USB) or high definition multimedia interface (HDMI) or Displayport, and so on. Examples of storage interfaces include SATA (serial advanced technology attachment), PCI, IDE interface, and the like. The techniques described in the present document may be embodied in various electronic devices such as mobile phones, laptops, smartphones or other devices that are capable of performing digital data processing and/or video display.

<FIG> is a block diagram of a video processing apparatus <NUM>. The apparatus <NUM> may be used to implement one or more of the methods described herein. The apparatus <NUM> may be embodied in a smartphone, tablet, computer, Internet of Things (IoT) receiver, and so on. The apparatus <NUM> may include one or more processors <NUM>, one or more memories <NUM> and video processing hardware <NUM>. The processor(s) <NUM> may be configured to implement one or more methods described in the present document. The memory (memories) <NUM> may be used for storing data and code used for implementing the methods and techniques described herein. The video processing hardware <NUM> may be used to implement, in hardware circuitry, some techniques described in the present document. In some implementations, the hardware <NUM> may be partially or completely a part of the processor <NUM>, e.g., a graphics processor.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, determining, for a conversion between a current region of a video and a bitstream representation of the video, whether a luma adaptive loop filter is used during the conversion and whether luma adaptive loop filter coefficients are included in the bitstream representation, such that a single syntax element in the bitstream representation is indicative of use of the luma adaptive loop filter and signaling of the luma adaptive loop filter coefficients.

The method <NUM> includes, at operation <NUM>, performing, based on the determining, the conversion.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, performing a conversion between a current region of a video and a bitstream representation of the video, such that an adaptive loop filter is used during the conversion, and the bitstream representation conforms to a syntax rule that specifies that coefficients of the adaptive loop filter signaled in the bitstream representation include zero-valued adaptive loop filter coefficients.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, determining, for a conversion between a current region of a video and a bitstream representation of the video, that zero-valued adaptive loop filter coefficients of a previous region of the video signaled in the bitstream representation are not used in the conversion.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, performing a conversion between a current region of a video and a bitstream representation of the video, such that the bitstream representation conforms to a syntax rule that specifies a flag indicating whether in-loop filtering is used for the conversion is included in the bitstream representation at a video unit level comprising the current region that is smaller than a slice level of the video.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, performing a conversion between a current region of a video and a bitstream representation of the video, such that the conversion comprises using an adaptive loop filter, and the bitstream representation is configured to indicate the adaptive loop filter using a two-part signaling including a first part indicative of a technique of determining the adaptive loop filter, and a second part indicative of an index used by the technique.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, determining, based on a property of a video, a size of a current region of the video that shares a common loop filtering setting for a conversion between the current region and a bitstream representation of the video.

<FIG> is a flowchart of an example method of video processing. The method <NUM> includes, at operation <NUM>, performing a lossless conversion between a current region of the video and a bitstream representation of the video, such that the bitstream representation conforms to a syntax rule that restricts a value of a syntax field associated with the current region in the bitstream representation due to the conversion being lossless.

The disclosed and other solutions, examples, embodiments, modules and the functional operations described in this document can be implemented in digital electronic circuitry, or in computer software, firmware, or The disclosed and other embodiments can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus.

Claim 1:
A method of processing video data, comprising:
performing a conversion between a current block of a current region of a video and a bitstream of the video, wherein the conversion comprises using an adaptive loop filter for the current block, and wherein the bitstream is configured to indicate the adaptive loop filter using multiple syntax elements including:
a first syntax element of the multiple syntax elements, the first syntax element being alf_use_aps_flag, and the first syntax element indicating whether a fixed filter set or a filter set from an adaptation parameter set (APS) is applied to the current block,
a second syntax element of the multiple syntax elements, the second syntax element being alf_luma_prev_filter_idx, and the second syntax element indicative of an index of the filter set from the APS, and specifies the previous filter that is applied to the luma coding tree block CTB, and
a third syntax element of the multiple syntax elements, the third syntax element being alf_luma_fixed_filter_idx, and the third syntax element indicative of an index of the fixed filter set,
wherein the bitstream comprises a fourth syntax element in a slice header, the fourth syntax element being slice_ num_alf_aps_ids_luma, and the fourth syntax element indicating a number of adaptive loop filter APSs used for the current region,
wherein the first syntax element is included in the bitstream when a value of the fourth syntax element in the slice header is greater than one and the first syntax element is included in the bitstream when the value of the fourth syntax element in the slice header is equal to one,
wherein the second syntax element indicative of the index of the filter set from the APS is included in the bitstream when a value of the first syntax element is equal to one and the value of the fourth syntax element in the slice header is greater than one, and
wherein the third syntax element is included in the bitstream when the value of the first syntax element is equal to zero.