Patent Description:
Video coding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video can include a series of pictures, each picture having a spatial dimension of, for example, <NUM> × <NUM> luminance samples and associated chrominance samples. The series of pictures can have a fixed or variable picture rate (informally also known as frame rate), of, for example <NUM> pictures per second or <NUM>. Uncompressed video has significant bitrate requirements. For example, 1080p60 <NUM>:<NUM>:<NUM> video at <NUM> bit per sample (1920x1080 luminance sample resolution at <NUM> frame rate) requires close to <NUM> Gbit/s bandwidth. An hour of such video requires more than <NUM> GBytes of storage space.

A video encoder and decoder can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

Video codec technologies can include techniques known as intra coding. In intra coding, sample values are represented without reference to samples or other data from previously reconstructed reference pictures. In some video codecs, the picture is spatially subdivided into blocks of samples. When all blocks of samples are coded in intra mode, that picture can be an intra picture. Intra pictures and their derivations such as independent decoder refresh pictures, can be used to reset the decoder state and can, therefore, be used as the first picture in a coded video bitstream and a video session, or as a still image. The samples of an intra block can be exposed to a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction can be a technique that minimizes sample values in the pre-transform domain. In some cases, the smaller the DC value after a transform is, and the smaller the AC coefficients are, the fewer the bits that are required at a given quantization step size to represent the block after entropy coding.

Traditional intra coding such as known from, for example MPEG-<NUM> generation coding technologies, does not use intra prediction. However, some newer video compression technologies include techniques that attempt, from, for example, surrounding sample data and/or metadata obtained during the encoding/decoding of spatially neighboring, and preceding in decoding order, blocks of data. Such techniques are henceforth called "intra prediction" techniques. Note that in at least some cases, intra prediction is only using reference data from the current picture under reconstruction and not from reference pictures.

There can be many different forms of intra prediction. When more than one of such techniques can be used in a given video coding technology, the technique in use can be coded in an intra prediction mode. In certain cases, modes can have submodes and/or parameters, and those can be coded individually or included in the mode codeword. Which codeword to use for a given mode/submode/parameter combination can have an impact in the coding efficiency gain through intra prediction, and so can the entropy coding technology used to translate the codewords into a bitstream.

A certain mode of intra prediction was introduced with H. <NUM>, refined in H. <NUM>, and further refined in newer coding technologies such as joint exploration model (JEM), versatile video coding (VVC), and benchmark set (BMS). A predictor block can be formed using neighboring sample values belonging to already available samples. Sample values of neighboring samples are copied into the predictor block according to a direction. A reference to the direction in use can be coded in the bitstream or may itself be predicted.

In some video compression techniques, an MV applicable to a certain area of sample data can be predicted from other MVs, for example from those related to another area of sample data spatially adjacent to the area under reconstruction, and preceding that MV in decoding order. Doing so can substantially reduce the amount of data required for coding the MV, thereby removing redundancy and increasing compression. MV prediction can work effectively, for example, because when coding an input video signal derived from a camera (known as natural video) there is a statistical likelihood that areas larger than the area to which a single MV is applicable move in a similar direction and, therefore, can in some cases be predicted using a similar motion vector derived from MVs of a neighboring area. That results in the MV found for a given area to be similar or the same as the MV predicted from the surrounding MVs, and that in turn can be represented, after entropy coding, in a smaller number of bits than what would be used if coding the MV directly. In some cases, MV prediction can be an example of lossless compression of a signal (namely: the MVs) derived from the original signal (namely: the sample stream). In other cases, MV prediction itself can be lossy, for example because of rounding errors when calculating a predictor from several surrounding MVs.

Various MV prediction mechanisms are described in H. <NUM>/HEVC (ITU-T Rec. <NUM>, "High Efficiency Video Coding," December <NUM>).

Referring to <FIG>, a current block (<NUM>) comprises samples that have been found by the encoder during the motion search process to be predictable from a previous block of the same size that has been spatially shifted. Instead of coding that MV directly, the MV can be derived from metadata associated with one or more reference pictures, for example from the most recent (in decoding order) reference picture, using the MV associated with either one of five surrounding samples, denoted A0, A1, and B0, B1, B2 (<NUM> through <NUM>, respectively). <NUM>, the MV prediction can use predictors from the same reference picture that the neighboring block is using. State of the art of VVC coding is disclosed in <NPL>. The coding of transform skipped blocks is disclosed also in <NPL>; <NPL>, and <CIT>.

The invention is set out in the set of appended claims. Aspects of the disclosure provide methods and apparatuses for video coding at a decoder. In some examples, a bit stream including bins of syntax elements is received. The syntax elements correspond to coefficients of a region of a transform skipped block in a coded picture. The syntax elements include a first flag indicating whether an absolute coefficient level of one of the coefficients is greater than a first threshold value, and a second flag indicating a parity of the absolute coefficient level. The second flag is decoded in a pass. The pass satisfies at least one of: (<NUM>) no other syntax elements is decoded in the pass; (<NUM>) a third flag indicating whether the absolute coefficient level is greater than a second threshold value is decoded in the pass; and (<NUM>) a fourth flag indicating sign information of the coefficient level of the one of the coefficients is decoded in the pass. The second threshold value being greater than the first threshold value.

In an embodiment, the first threshold value is <NUM> and the second threshold value is <NUM>.

In an embodiment, a current block corresponding to the transform skipped block is coded with a block differential pulse-code modulation mode.

In an embodiment, the second flag is decoded in the pass without decoding other syntax elements after decoding the first flag and a fifth flag indicating whether the absolute coefficient level is greater than <NUM> in a previous pass.

In an embodiment, the second flag is decoded in the pass without decoding other syntax elements after decoding the third flag in a previous pass.

In an embodiment, the second flag is decoded in the same pass with the third flag.

In an embodiment, the second flag is decoded in the same pass with the fourth flag.

In some example, a bit stream including bins of syntax elements is received. The syntax elements correspond to coefficients of a region of a transform skipped block in a coded picture. The syntax elements include a first flag indicating whether an absolute coefficient level of one of the coefficients is greater than a first threshold value, and a second flag indicating whether the absolute coefficient level of the one of the coefficients is greater than a second threshold value. The second threshold value is greater than the first threshold value. A number of the bins of the second flag that are context coded is determined based on coded information of the coefficients. Context modeling is performed to determine a context model for each of the number of the bins of the second flag. The number of the bins of the second flag is decoded based on the determined context models.

In an embodiment, the coded information of the coefficients includes a number of the first flag in a current coefficient group (CG) including the coefficients, a number the first flag in a previous CG, whether a current block corresponding to the transform skipped block is intra coded or inter coded, a size of the transform skipped block, a width of the transformed skipped block, a height of the transformed skipped block, or an aspect ratio of the transformed skipped block.

In an embodiment, when the current block is intra coded, the number of the bins of the second flag is more than the number of the bins of the second flag when the current block is inter coded.

In some examples, a bit stream including bins of syntax elements is received. The syntax elements correspond to coefficients included in a coefficient group (CG) of a transform skipped block in a coded picture. A maximum number of context coded bins of the CG or a maximum average number of context coded bins of the CG is determined based on coded information of the coefficients. Context modeling is performed to determine a context model for each of a number of the bins of syntax elements in the CG. The number of the bins of syntax elements in the CG being context coded does not exceed the maximum number of context coded bins of the CG or the maximum average number of context coded bins of the CG. The number of the bins of syntax elements is decoded based on the determined context models.

In an embodiment, the coded information of the coefficients includes a number of the first flag in the CG including the coefficients, a number the first flag in a previous CG, whether a current block corresponding to the transform skipped block is intra coded or inter coded, a size of the transform skipped block, a width of the transformed skipped block, a height of the transformed skipped block, or an aspect ratio of the transformed skipped block.

Aspects of the disclosure also provide non-transitory computer-readable storage mediums storing instructions which when executed by a computer cause the computer to perform any of the above methods.

A streaming system may include a capture subsystem (<NUM>) that can include a video source (<NUM>), for example a digital camera, creating for example a stream of video pictures (<NUM>) that are uncompressed. In an example, the stream of video pictures (<NUM>) includes samples that are taken by the digital camera. The stream of video pictures (<NUM>), depicted as a bold line to emphasize a high data volume when compared to encoded video data (<NUM>) (or coded video bitstreams), can be processed by an electronic device (<NUM>) that includes a video encoder (<NUM>) coupled to the video source (<NUM>). The video encoder (<NUM>) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The encoded video data (<NUM>) (or encoded video bitstream (<NUM>)), depicted as a thin line to emphasize the lower data volume when compared to the stream of video pictures (<NUM>), can be stored on a streaming server (<NUM>) for future use. One or more streaming client subsystems, such as client subsystems (<NUM>) and (<NUM>) in <FIG> can access the streaming server (<NUM>) to retrieve copies (<NUM>) and (<NUM>) of the encoded video data (<NUM>). A client subsystem (<NUM>) can include a video decoder (<NUM>), for example, in an electronic device (<NUM>). The video decoder (<NUM>) decodes the incoming copy (<NUM>) of the encoded video data and creates an outgoing stream of video pictures (<NUM>) that can be rendered on a display (<NUM>) (e.g., display screen) or other rendering device (not depicted). In some streaming systems, the encoded video data (<NUM>), (<NUM>), and (<NUM>) (e.g., video bitstreams) can be encoded according to certain video coding/compression standards. Examples of those standards include ITU-T Recommendation H. In an example, a video coding standard under development is informally known as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

The scaler / inverse transform unit (<NUM>) can output blocks comprising sample values, which can be input into aggregator (<NUM>).

The residue decoder (<NUM>) may also require certain control information to include the Quantizer Parameter (QP), and that information may be provided by the entropy decoder (<NUM>) (data path not depicted as this may be low volume control information only).

In an embodiment of the present disclosure, a transform skip (TS) mode can be applied for coding both intra and inter prediction residuals. For a luma or chroma coding block with less than or equal to <NUM> samples, a flag may be signaled to indicate whether the TS mode is applied for current block.

When the TS mode is applied, the prediction process is the same as the prediction process in the regular transform mode. In some examples, intra or inter prediction may be applied. For transform skipping TUs, a scaling process may be used so that transform skipping coefficients can have similar magnitudes as other transform coefficients. In an embodiment, a scaling-down process may be performed, and the scaling factor may be the same as the scaling associated with other transforms (versus standard floating point transform with norm <NUM>) of the same size. Moreover, when the TS mode is applied, de-quantization and scaling are the same as the de-quantization and scaling in the regular transform mode. Deblocking, sample adaptive offset (SAO), and adaptive loop filtering (ALF) also are the same when the TS mode is applied, but a flag may be signaled to indicate whether transform is bypassed. Further, a flag may be signaled in the SPS to indicate whether the TS mode is enabled or not.

In an example, the related spec text of the TS mode in a VVC draft is described in Table <NUM> below:.

In Table <NUM>, transform_skip_flag[ x0 ][ y0 ][ cIdx ] specifies whether a transform is applied to the associated transform block or not. When transform_skip_flag[ x0 ][ y0 ][ cIdx ] is equal to <NUM>, it specifies that no transform is applied to the current transform block. When transform_skip_flag[ x0 ][ y0 ][ cIdx ] is equal to <NUM>, it specifies that whether transform is applied to the current transform block depends on other syntax elements. When transform_skip_flag[ x0 ][ y0 ][ cIdx ] is not present, it is inferred to be equal to <NUM>.

The array indices x0, y0 specify the location (x0, y0) of a top-left luma sample of a transform block relative to a top-left luma sample of a picture. The array index cIdx specifies an indicator for the color component. When the array index cIdx is equal to <NUM>, the color component is luma. When the array index cIdx is equal to <NUM>, the color component is Cb. When the array index cIdx is equal to <NUM>, the color component is Cr.

Next, an example of the scaling and the transformation process is described below.

The output of this process may be the (nTbW)x(nTbH) array of residual samples resSamples[ x ][ y ] with x = <NUM>. nTbW - <NUM>, y = <NUM>. nTbH - <NUM>.

In the scaling process, a variable bitDepth is a bit depth of the current color component, a variable bdShift is a scaling shift factor, and a variable tsShift is a transform skip shift. The variables bitDepth, bdShift, and tsShift may be derived as follows: <MAT> <MAT> <MAT>.

The scaling process for transform coefficients may be invoked with the transform block location (xTbY, yTbY), the transform width nTbW and the transform height nTbH, the color component variable cIdx, and the bit depth of the current color component bitDepth as the inputs, and an (nTbW)x(nTbH) array of scaled transform coefficients d as the output.

The (nTbW)x(nTbH) array of residual samples r is the quantized coefficients and may be derived as follows:.

If transform_skip_flag[ xTbY ][ yTbY ][ cIdx ] is equal to <NUM>, the residual sample array values r[ x ][ y ] with with x = <NUM>. nTbW - <NUM>, y = <NUM>. nTbH - <NUM> may be derived as follows: <MAT>.

When transform_skip_flag[ xTbY ][ yTbY ][ cIdx ] is equal to <NUM>, the transformation process for scaled transform coefficients may be invoked with the transform block location (xTbY, yTbY), the transform width nTbW and the transform height nTbH, the color component variable cIdx and the (nTbW)x(nTbH) array of scaled transform coefficients d as the inputs, and an (nTbW)x(nTbH) array of residual samples r as the output.

The residual samples resSamples[ x ][ y ] with x = <NUM>. nTbW - <NUM>, y = <NUM>. nTbH - <NUM> can be derived as follows: <MAT>.

Block differential pulse-code modulation (BDPCM) is an intra-coding tool that uses a differential pulse-code modulation (DPCM) approach at the block level. A bdpcm_flag may be transmitted at the CU level whenever the CU is a luma intra coded CU having each dimension smaller or equal to <NUM>. This flag indicates whether regular intra coding or DPCM is used and encoded using a single Context-based adaptive binary arithmetic coding (CABAC) context.

BDPCM may use the Median Edge Detector of LOCO-I (e.g., used in JPEG-LS). Specifically, for a current pixel X having pixel A as a left neighbor, pixel B as a top neighbor, and C as a top-left neighbor, the prediction of the current pixel X P(X) is determined by the following formula: <MAT>.

The pixel predictor uses unfiltered reference pixels when predicting from the top row and left column of the CU. The predictor then uses reconstructed pixels for the rest of the CU. Pixels are processed in a raster-scan order inside the CU. The prediction error may be quantized in the spatial domain, after rescaling, in a way identical to the transform skip quantizer. Each pixel can be reconstructed by adding the dequantized prediction error to the prediction. Thus, the reconstructed pixels can be used to predict the next pixels in the raster-scan order. Amplitude and signs of the quantized prediction error are encoded separately. A cbf_bdpcm _flag is coded. If cbf_bdpcm _flag is equal to <NUM>, all amplitudes of the block are to be decoded as zero. If cbf_bdpcm _flag is equal to <NUM>, all amplitudes of the block are encoded individually in raster-scan order. In order to keep complexity low, the amplitude can be limited to at most <NUM> (inclusive). The amplitude may be encoded using unary binarization, with three contexts for the first bin, then one context for each additional bin until the 12th bin, and one context for all remaining bins. A sign may be encoded in bypass model for each zero residue.

In order to maintain the coherence of the regular intra mode prediction, the first mode in the most probable mode (MPM) list of the intra mode prediction is associated with a BDPCM CU (without being transmitted) and is available for generating the MPM for subsequent blocks.

The deblocking filter may be deactivated on a border/boundary between two BDPCM coded blocks because neither of the BDPCM coded blocks performs a transform, which is usually responsible for blocking artifacts. Further, BDPCM may not use any other step except the ones described here. In particular, BDPCM may not perform any transform in residual coding as described above.

Several tests regarding BDPCM have been conducted to investigate the throughput improvements of BDPCM and the interaction with other Screen Content Coding (SCC) tools.

<FIG> shows examples of BDPCM coded blocks according to an embodiment. The examples shown in <FIG> are related to Test <NUM>. As shown in <FIG>, in order to increase throughput, smaller blocks (e.g., having sizes of 4x4, 4x8, and 8x4) may be divided into two independently decodable areas, using a diagonal which effectively divides the block into two halves (e.g., stair-case shaped partition).

In an embodiment, pixels from one area of a first half may not be allowed to use pixels from another area of a second half to compute the prediction. If pixels from one area need to use pixels from another area to compute the prediction, reference pixels are used instead. For example, the pixels from the other area may be replaced by the closest reference pixels. For example, a left neighbor may be replaced with a left reference pixel from the same row, a top neighbor may be replaced with a left reference pixel from the same column, and a top-left neighbor may be replaced with the closest reference pixel. Thus, the two areas can be processed in parallel.

<FIG> also provides an exemplary throughput of each block having a different size. For example, for a <NUM>×<NUM> block with two independently decodable areas, the throughput may be <NUM> pixels per cycle. For a <NUM>×<NUM> or <NUM>×<NUM> block with two independently decodable areas, the throughput may be <NUM> pixels per cycle. For an 8x8 block without independently decodable areas, the throughput may be <NUM> pixels per cycle. For an 8x8 block without independently decodable areas, the throughput may be <NUM> pixels per cycle. For a 16x16 block without independently decodable areas, the throughput may be <NUM> pixels per cycle.

<FIG> shows examples of BDPCM coded blocks according to an embodiment. The examples shown in <FIG> are related to Test <NUM>. In <FIG>, the block may be divided using a vertical or horizontal predictor to replace a JPEG-LS predictor. The vertical or horizontal predictor may be chosen and signaled at a block level. The shape of the independently decodable regions reflects the geometry of the predictor. Due to the shape of the horizontal or vertical predictors, which use a left or a top pixel for prediction of the current pixel, the most throughput-efficient way of processing the block may be to process all the pixels of one column or row in parallel, and to process these columns or rows sequentially. For example, in order to increase throughput, a block of width <NUM> is divided into two halves with a horizontal boundary when the predictor chosen on this block is vertical, and a block of height <NUM> is divided into two halves with a vertical boundary when the predictor chosen on this block is horizontal. For a <NUM>×<NUM> block, an 8x4, or a <NUM>×<NUM> block with two independently decodable areas, the throughput may be <NUM> pixels per cycle. For a <NUM>×<NUM> block, an 8x4 block, or an 8x8 block without independently decodable areas, the throughput may be <NUM> pixels per cycle. For a 16x16 block without independently decodable areas, the throughput may be <NUM> pixels per cycle.

In Test <NUM>, according to an embodiment of the present disclosure, the BDPCM residue amplitude is limited to <NUM>, and the amplitude is encoded with truncated unary binarization for the first <NUM> bins, followed by order-<NUM> Exp-Golomb equal probability bins for the remainder (e.g., using an encodeRemAbsEP() function).

Entropy coding can be performed at a last stage of video coding (or a first stage of video decoding) after a video signal is reduced to a series of syntax elements. Entropy coding can be a lossless compression scheme that uses statistic properties to compress data such that a number of bits used to represent the data are logarithmically proportional to the probability of the data. For example, by performing entropy coding over a set of syntax elements, bits representing the syntax elements (referred to as bins) can be converted to fewer bits (referred to as coded bits) in a bit stream. CABAC is one form of entropy coding. In CABAC, a context model providing a probability estimate can be determined for each bin in a sequence of bins based on a context associated with the respective bin. Subsequently, a binary arithmetic coding process can be performed using the probability estimates to encode the sequence of bins to coded bits in a bit stream. In addition, the context model is updated with a new probability estimate based on the coded bin.

<FIG> shows an exemplary CABAC based entropy encoder (900A) in accordance with an embodiment. For example, the entropy encoder (900A) can be implemented in the entropy coder (<NUM>) in the <FIG> example, or the entropy encoder (<NUM>) in the <FIG> example. The entropy encoder (900A) can include a context modeler (<NUM>) and a binary arithmetic encoder (<NUM>). In an example, various types of syntax elements are provided as input to the entropy encoder (900A). For example, a bin of a binary valued syntax element can be directly input to the context modeler (<NUM>), while a non-binary valued syntax element can be binarized to a bin string before bins of the bin string are input to the context modeler (<NUM>).

In an example, the context modeler (<NUM>) receives bins of syntax elements, and performs a context modeling process to select a context model for each received bin. For example, a bin of a binary syntax element of a transform coefficient in a transform block is received. The transform block may be a transform skipped block when the current block is coded with BDPCM for prediction. A context model can accordingly be determined for this bin based, for example, on a type of the syntax element, a color component type of the transform component, a location of the transform coefficient, and previously processed neighboring transform coefficients, and the like. The context model can provide a probability estimate for this bin.

In an example, a set of context models can be configured for one or more types of syntax elements. Those context models can be arranged in a context model list (<NUM>) that is stored in a memory (<NUM>) as shown in <FIG>. Each entry in the context model list (<NUM>) can represent a context model. Each context model in the list can be assigned an index, referred to as a context model index, or context index. In addition, each context model can include a probability estimate, or parameters indicating a probability estimate. The probability estimate can indicate a likelihood of a bin being <NUM> or <NUM>. For example, during the context modeling, the context modeler (<NUM>) can calculate a context index for a bin, and a context model can accordingly be selected according to the context index from the context model list (<NUM>) and assigned to the bin.

Moreover, probability estimates in the context model list can be initialized at the start of the operation of the entropy encoder (900A). After a context model in the context model list (<NUM>) is assigned to a bin and used for encoding the bin, the context model can subsequently be updated according to a value of the bin with an updated probability estimate.

In an example, the binary arithmetic encoder (<NUM>) receives bins and context models (e.g., probability estimates) assigned to the bins, and accordingly performs a binary arithmetic coding process. As a result, coded bits are generated and transmitted in a bit stream.

<FIG> shows an exemplary CABAC based entropy decoder (900B) in accordance with an embodiment. For example, the entropy decoder (900B) can be implemented in the parser (<NUM>) in the <FIG> example, or the entropy decoder (<NUM>) in the <FIG> example. The entropy decoder (900B) can include a binary arithmetic decoder (<NUM>), and a context modeler (<NUM>). The binary arithmetic decoder (<NUM>) receives coded bits from a bit stream, and performs a binary arithmetic decoding process to recover bins from the coded bits. The context modeler (<NUM>) can operate similarly to the context modeler (<NUM>). For example, the context modeler (<NUM>) can select context models in a context model list (<NUM>) stored in a memory (<NUM>), and provide the selected context models to the binary arithmetic decoder (<NUM>). The context modeler (<NUM>) can determine the context models based on the recovered bins from the binary arithmetic decoder (<NUM>). For example, based on the recovered bins, the context modeler (<NUM>) can know a type of a syntax element of a next to-be-decoded bin, and values of previously decoded syntax elements. That information is used for determining a context model for the next to-be-decoded bin.

In an embodiment, residual signals of a transform block are first transformed from spatial domain to frequency domain resulting in a block of transform coefficients. Then, a quantization is performed to quantize the block of transform coefficients into a block of transform coefficient levels. In various embodiments, different techniques may be used for converting residual signals into transform coefficient levels. The block of transform coefficient levels is further processed to generate syntax elements that can be provided to an entropy encoder and encoded into bits of a bit stream. In an embodiment, a process of generating the syntax elements from the transform coefficient levels can be performed in the following way.

The block of transform coefficient levels can be first split into sub-blocks, for example, with a size of 4x4 positions. Those sub-blocks can be processed according to a predefined scan order. <FIG> shows an example of the sub-block scan order, referred to as an inverse diagonal scan order. As shown, a block (<NUM>) is partitioned into sixteen sub-blocks.

Each sub-block (<NUM>) may be a coefficient group (CG). Before a position in the sub-block (<NUM>) is processed or scanned, a flag may be signaled to indicate whether the CG includes at least one non-zero transform coefficient level. When the flag indicates the CG includes at least one non-zero transform coefficient level, the sub-block at the bottom-right corner is first processed, and the sub-block at the top-left corner is last processed. For a sub-block within which the transform coefficient levels are all zero, the sub-block can be skipped without processing in an example. In a TS mode, the scan order may be the opposite of the inverse diagonal scan order. That is, the sub-block at the top-left corner may be first processed, and the sub-block at the bottom-right corner may be last processed.

For sub-blocks each having at least one non-zero transform coefficient level, four passes of a scan can be performed in each sub-block. During each pass, the <NUM> positions in the respective sub-block can be scanned in the inverse diagonal scan order. <FIG> shows an example of a sub-block scanning process (<NUM>) from which different types of syntax elements of transform coefficients can be generated.

Sixteen coefficient positions (<NUM>) inside a sub-block are shown in one dimension at the bottom of <FIG>. The positions (<NUM>) are numbered from <NUM> to <NUM> reflecting the respective scan order. During a first pass, the scan positions (<NUM>) are scanned over, and three types of syntax elements (<NUM>-<NUM>) may be generated at each scan position (<NUM>) as follows:.

During a second pass, a fourth type of binary syntax elements (<NUM>) may be generated. The fourth type of syntax elements (<NUM>) is referred to as greater than <NUM> flags and denoted by rem_abs_gt2_flag. The fourth type of syntax elements (<NUM>) indicates whether the absolute transform coefficient level of the respective transform coefficient is greater than <NUM>. The greater than <NUM> flags are generated only when (absLevel - <NUM>) >> <NUM> is greater than <NUM> for the respective transform coefficient.

During a third pass, a fifth type of non-binary syntax elements (<NUM>) can possibly be generated. A remainder generally refers to a remaining value of the absolute transform coefficient level absLevel (e.g., abs_remainder). In coefficient coding, at least a first signal is generated to indicate whether the coefficient level is greater than a predetermined value X. Subsequently, a second signal is generated corresponding to the remainder value of the absolute transform coefficient level (e.g., absLevel-X). For example, here, the fifth type of syntax elements (<NUM>) may be denoted by abs_remainder, and indicates a remaining value of the absolute transform coefficient level of the respective transform coefficient that is greater than <NUM>. The fifth type of syntax elements (<NUM>) are generated only when the absolute transform coefficient level of the respective transform coefficient is greater than <NUM>.

During a fourth pass, a sixth type of syntax elements (<NUM>) can be generated at each scan position (<NUM>) with a non-zero coefficient level indicating a sign of the respective transform coefficient level.

In a TS mode, the first pass may include the significance flags, the parity flags, the greater than <NUM> flags, and the greater than <NUM> flags. Moreover, additional types of syntax elements such as greater than x flags may be generated during a separate pass. The greater than x flags may be denoted by rem_abs_gtx_flag and indicate whether the absolute transform coefficient level of the respective transform coefficient is greater than x. In some examples, x can be <NUM>, <NUM>, <NUM>, or <NUM>.

The above described various types of syntax elements can be provided to an entropy encoder according to the order of the passes and the scan order in each pass. Different entropy encoding schemes can be employed for encoding different types of syntax elements. For example, in an embodiment, the significance flags, parity flags, greater than <NUM> flags, and greater than <NUM> flags can be encoded with a CABAC based entropy encoder, as described in <FIG>. In contrast, the syntax elements generated during the third and fourth passes can be encoded with a CABAC-bypassed entropy encoder (e.g., a binary arithmetic encoder with fixed probability estimates for input bins).

Context modeling can be performed to determine context models for bins of some types of transform coefficient syntax elements. In an embodiment, in order to exploit the correlation among the transform coefficients, the context models can be determined according to a local template and a diagonal position of each current coefficient (e.g., a coefficient currently under processing) possibly in combination with other factors.

<FIG> shows an example of a local template (<NUM>) used for context selection for current coefficients. The local template (<NUM>) can cover a set of neighboring positions or coefficients of a current coefficient (<NUM>) in a coefficient block (<NUM>). The coefficient block.

(<NUM>) can have a size of 8x8 positions, and include coefficient levels at the <NUM> positions. The coefficient block (<NUM>) is partitioned into <NUM> sub-blocks each with a size of 4x4 positions. Each sub-block may be a CG that can include 4x4 coefficient positions. The CG (<NUM>) includes the current coefficient (<NUM>). A flag may be signaled to indicate whether the CG (<NUM>) includes only zero coefficient levels. In the <FIG> example, the local template (<NUM>) is defined to be a <NUM> position template covering <NUM> coefficient levels at the bottom-right side of the current coefficient (<NUM>). When an inverse diagonal scan order is used for multiple passes over the scan positions within the coefficient block (<NUM>), the neighboring positions within the local template (<NUM>) are processed prior to the current coefficient (<NUM>). In a TS mode, the scan order may be the opposite of the inverse diagonal scan order and the local template may be a <NUM> position template cover <NUM> coefficient levels at the top-left side of the current coefficient.

During the context modeling, information of the coefficient levels within the local template (<NUM>) can be used to determine a context model. For this purpose, a measure, referred to as a template magnitude, is defined in some embodiments to measure or indicate magnitudes of the transform coefficients or transform coefficient levels within the local template (<NUM>). The template magnitude can then be used as the basis for selecting the context model.

In one example, the template magnitude is defined to be a sum, denoted by sumAbs1, of partially reconstructed absolute transform coefficient levels inside the local template (<NUM>). A partially reconstructed absolute transform coefficient level can be determined according to bins of the syntax elements, sig_coeff_flag, par_level_flag, and rem_abs_gt1_flag of the respective transform coefficient. Those three types of syntax elements may be obtained after a first pass over scan positions of a sub-block performed in an entropy encoder or an entropy decoder. In an embodiment, numSig is a number of non-zero coefficients in the local template (<NUM>). Moreover, a diagonal position of a scan position (x, y) d is defined according to: d = x +y, where x and y are coordinates of the respective position. The context model index may be selected based on sumAbs1 and the diagonal position d as described below.

In an embodiment, during a context modeling process in an entropy encoder or decoder, context indices can be determined as described below for the context coded binary syntax elements of the current coefficient (<NUM>). The determination can be based on the local template (<NUM>) and the diagonal position of the current coefficient (<NUM>).

When coding sig_coeff_flag of the current coefficient (<NUM>), the context index can be selected depending on sumAbs1 and diagonal position d of the current coefficient (<NUM>). For example, for luma component, the context index is determined according to:<MAT> where ctxSig represents the context index of the significance flag syntax element, and "state" specifies a state of a scaler quantizer of a dependent quantization scheme where the state can have a value of <NUM>, <NUM>, <NUM>, or <NUM>.

The Eq. <NUM> is equivalent to the following formulas: <MAT> <MAT>.

In Eq. <NUM> and Eq. <NUM>, ctxIdBase represents a context index base. The context index base can be determined based on the state and the diagonal position d. For example, the state can have a value of <NUM>, <NUM>, <NUM>, or <NUM>, and accordingly max(<NUM>, state-<NUM>) can have one of three possible values, <NUM>, <NUM>, or <NUM>. For example, (d < <NUM> ? <NUM> : (d < <NUM> ? <NUM> : <NUM>)) can take a value of <NUM>, <NUM>, or <NUM>, corresponding to different ranges of d: d<<NUM>, <NUM><= d <<NUM>, or <NUM><= d.

In Eq. <NUM> and Eq. <NUM>, ctxIdSigTable[ ] can represent an array data structure, and can store context index offsets of significance flags with respect to the ctxIdBase. For example, for different sumAbs1 values, min(sumAbs1, <NUM>) clips a sumAbs1 value to be smaller than or equal to <NUM>. Then, the clipped value is mapped to a context index offset. For example, under the definition of ctxIdSigTable[<NUM>~<NUM>] = {<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>}, the clipped value <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> are mapped to <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, or <NUM>, respectively.

For chroma component, the context index can be determined according to: <MAT> which is equivalent to the following formulas: <MAT> <MAT>.

When coding par_level_flag of the current coefficient (<NUM>), the context index can be selected depending on sumAbs1, numSig, and diagonal position d. For example, for luma component, if the current coefficient is the first non-zero coefficient in decoding order, the context index ctxPar is assigned to be <NUM>; otherwise the context index can be determined according to: <MAT> which is equivalent to the following formulas: <MAT> <MAT> where ctxPar represents the context index of the parity flag, and ctxIdTable[ ] represents another array data structure, and stores context index offsets with respect to the respective ctxIdBase. For example, ctxIdTable[<NUM>~<NUM>] = {<NUM>, <NUM>, <NUM>, <NUM>, <NUM>}.

For chroma, if the current coefficient is the first non-zero coefficient in decoding order, the context index ctxPar is assigned to be <NUM>; otherwise, the context index can be determined according to: <MAT> which is equivalent to the following formulas: <MAT> <MAT>.

When coding rem_abs_gt1_flag and rem_abs_gt2_flag of the current coefficient (<NUM>), the context model indices can be determined in the same way as par_level_flag: <MAT> <MAT> where ctxGt1 and ctxGt2 represent the context indices of the greater than <NUM> and greater than <NUM> flags, respectively.

It is noted that different sets of context models are used for different types of the syntax elements, sig_coeff_flag, par_level_flag, rem_abs_gt1_flag and rem_abs_gt2_flag. For example, the context model used for rem_abs_gt1_flag is different from that of rem_abs_gt2_flag, even though a value of ctxGt1 is equal to that of ctxGt2.

The residual coding for both the TS mode and the BDPCM mode is processed in the spatial domain without transform. Therefore, a shared module between the TS mode and the BDPCM mode can be used for a simpler design of coefficient coding.

Moreover, the coefficients of the TS mode and the BDPCM mode show different characteristics from regular transform coefficients which are associated with transform and quantization. Therefore, a different coefficient coding scheme may show better coding performance.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, the parity bit flag (e.g., par_level_flag) which indicates the parity of the absolute transform coefficient level (e.g., absLevel) may be bypass coded without using context modeling in arithmetic coding. In an example, the parity bit flag can be coded in a separate pass. In another example, the parity bit flag can be coded together with abs_remainder in a same pass.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, the parity bit flag (e.g., par_level_flag) which indicates the parity of the absolute transform coefficient level (e.g., absLevel) may be coded in a separate pass or coded together with other syntax elements in one pass with a certain order or combination. In an example, the parity bit flag can be coded in a separate pass after the coding pass which codes sig_coeff_flag and rem_abs_gt1_flag. In another example, the parity bit flag can be coded in a separate pass after the coding pass which codes rem_abs_gt2_flag or a greater than x flag (e.g., rem_abs_gtx_flag), where x can be <NUM>, <NUM>, <NUM>, or <NUM>. In an example, the parity bit flag can be coded together with rem_abs_gt2_flag or the greater than x flag in a same coding pass. In an example, the parity bit flag can be coded together with sign information in a same coding pass.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, when coding the syntax element rem abs_gt2_flag, the number of context coded bins per coefficient for coding rem abs_gt2_flag may depend on coded information. The coded information may include the number of nonzero coefficients (e.g., number of coded sig_coeff_flag in the current CG), the number of nonzero coefficients (e.g., number of coded sig_coeff_flag) in the previously coded CG, whether the current block is coded by intra prediction or inter prediction or IBC mode, the current coefficient block size, coefficient block width, coefficient block height, and coefficient block aspect ratio. In some examples, the current CG includes the coefficients.

In an example, if a current block is intra coded, the number of context coded bins per coefficient for coding rem_abs_gt2_flag may be more than the number of context coded bins per coefficient for coding rem_abs_gt2_flag when current block is inter coded and/or IBC coded. Exemplary values of the number of context coded bins per coefficient for coding rem_abs_gt2_flag when current block is inter coded include <NUM>, <NUM>, <NUM>, and <NUM>. Exemplary values of the number of context coded bins per coefficient for coding rem_abs_gt2_flag when current block is intra coded include <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, the maximum number of context coded bins of the syntax elements per CG or the maximum average number of context coded bins of the syntax elements per CG depends on coded information. The coded information includes the number of nonzero coefficients (e.g., number of coded sig_coeff_flag in the current CG), the number of nonzero coefficients (e.g., number of coded sig_coeff_flag) in the previously coded CG, whether the current block is coded by intra prediction or inter prediction or IBC mode, the current coefficient block size, coefficient block width, coefficient block height, and coefficient block aspect ratio. Setting a maximum number of context coded bins not only boosts the coding speed, but also reduces required memory size and cost of maintaining the context models.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, the parity bit flag (e.g., par_level_flag) which indicates the parity of absLevel may not be coded. Instead, the syntax elements sig_coeff_flag, abs_gt1_flag, abs_gt2_flag, the sign information, and abs_remainder are coded.

In an example, the syntax elements sig_coeff_flag and abs_gt1_flag may be coded in the first pass. The syntax element abs_gt2 _flag may be coded in the second pass. The syntax element abs_remainder may be coded in the third pass. If necessary, the sign information may be coded in the fourth pass.

In another example, the syntax elements sig_coeff_flag, abs_gt1_flag, and sign information may be coded in the first pass. The syntax element abs_gt2_flag may be coded in the second pass. The syntax element abs_remainder may be coded in the fourth pass.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, in addition to the syntax elements rem_abs_gt1_flag (e.g., which indicates absLevel is greater than <NUM>), and rem_abs_gt2_flag (e.g., which indicates absLevel is greater than <NUM>), additional syntax elements rem_abs_gt3_flag (e.g., which indicates absLevel is greater than <NUM>) and/or rem_abs_gt4_flag (e.g., which indicates absLevel is greater than <NUM>) may be also signaled. The syntax elements rem_abs_gt3_flag and/or rem_abs_gt4_flag may use separate contexts for entropy coding.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, when coding the sign information for each non-zero coefficient, the context may depend on previously coded sign bit values.

In an example, <FIG> shows that the context used for coding the sign information includes the above block (<NUM>) and the left block (<NUM>) of the current coefficient (marked as X). In an example, <FIG> shows that the context used for coding the sign information includes multiple above and left blocks (<NUM>, <NUM>, <NUM>, <NUM>, and <NUM>) of the current coefficient (marked as X).

In an embodiment, the context used for coding the sign information depends on the previously coded N sign bits. Exemplary values of N include <NUM>, <NUM>, <NUM>, and <NUM>.

In an embodiment, instead of coding the sign information, a sign residual is coded, and the sign residual bit indicates whether the current sign bit is equal to a predicted sign bit value. In an example, the predicted sign bit is derived using previous scanned N sign bits. Exemplary values of N include <NUM>, <NUM>, <NUM>, and <NUM>. In another example, the predicted sign bit is derived using left and/or top neighboring sign bit values.

In an embodiment, for coefficient coding of the TS mode and/or the BDPCM mode, when coding the magnitude of coefficients, one or more primary level values may be first signaled, then the residual of each non-zero coefficient minus one of these primary level values are signaled. In an embodiment, the primary level values are limited to a given threshold. In an embodiment, the number of primary level values is limited to a given threshold, such as <NUM>, <NUM>, <NUM>, or <NUM>.

In an embodiment, when multiple primary level values are signaled, then the primary level values are arranged in an ascending order, then for each coefficient, starting from the smallest primary level value, a flag indicating whether the current coefficient has a level value greater than the current primary level value is signaled. In an example, when the current coefficient has a level value which is not greater than the current primary level value, then the difference between the current level value and the previous primary level value is signaled.

<FIG> shows a flow chart outlining a coefficient decoding process (<NUM>) according to some embodiments of the disclosure. The process (<NUM>) can be used in entropy decoding of several types of coefficient syntax elements. In various embodiments, the process (<NUM>) can be executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>) and (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented in software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1401) and proceeds to (S1410).

At (S1410), a bit stream including coded bits of bins of syntax elements is received. The syntax elements correspond to coefficients of a region of a transform skipped block in a coded picture, and include a first flag and a second flag. The first flag indicates whether an absolute coefficient level of one of the coefficients is greater than a first threshold (e.g., <NUM>), and the second flag indicates a parity of the absolute coefficient level. For example, the first flag may be a significance syntax element (e.g., sig_coeff_flag) that indicates that an absolute value of the current coefficient (e.g., absLevel) is greater than the first threshold. The second flag may be a parity syntax element (e.g., par_level_flag) that indicates the parity of absLevel. The transform skipped block may indicate that a transform was not performed on the transform block. For example, when the current block is coded with BDPCM, a transform is not performed on the transform block.

At (S1420), the second flag is decoded in a separate pass. For example, the pass satisfies at least one of: (<NUM>) no other syntax elements is decoded in the pass, (<NUM>) a third flag indicating whether the absolute coefficient level is greater than a second threshold (e.g., <NUM>) is decoded in the pass, and (<NUM>) a fourth flag indicating sign information of the coefficient level of the one of the coefficients is decoded in the pass. The second threshold is greater than the first threshold in some embodiments. In an example, the second flag, which is a parity syntax element, may be decoded in a separate pass after the first flag and a fifth flag are decoded. The fifth flag indicates whether the absolute coefficient level is greater than a third threshold (e.g., <NUM>). In an example, the second flag is decoded in a separate pass after the third flag is decoded in a previous pass. In an example, the second flag is decoded in the same pass with the third flag or the fourth flag. The process (<NUM>) proceeds to and terminates at (S1499).

<FIG> shows a flow chart outlining a coefficient decoding process (<NUM>) according to some embodiments of the disclosure. The process (<NUM>) can be used in entropy decoding of several types of coefficient syntax elements. In various embodiments, the process (<NUM>) can be executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>) and (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented in software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1501) and proceeds to (S1510).

At (S1510), a bit stream including coded bits of bins of syntax elements is received. The syntax elements correspond to coefficients of a region of a transform skipped block in a coded picture, and include a first flag and a second flag. The first flag indicates whether an absolute coefficient level of one of the coefficients is greater than a first threshold (e.g., <NUM>), and the second flag indicates whether the absolute coefficient level of the one of the coefficients is greater than a second threshold (e.g., <NUM>). The second threshold is greater than the first threshold in some embodiments. For example, the first flag may be a significance syntax element (e.g., sig_coeff_flag) that indicates that an absolute value of the current coefficient (e.g., absLevel) is greater than the first threshold. The second flag may be a greater than <NUM> syntax element (e.g., rem_abs_gt2_flag), which indicates whether the absolute transform coefficient level of the respective transform coefficient is greater than the second threshold. The transform skipped block may indicate that a transform was not performed on the transform block. For example, when the current block is coded with BDPCM, a transform is not performed on the transform block.

At (S1520), a number of the bins of the second flag that are context coded is determined based on coded information of the coefficients. For example, the coded information of the coefficients includes a number of the first flag in a current coefficient group (CG) including the coefficients, a number the first flag in a previous CG, whether a current block corresponding to the transform skipped block is intra coded or inter coded, a size of the transform skipped block, a width of the transformed skipped block, a height of the transformed skipped block, or an aspect ratio of the transformed skipped block. The previous CG may be a neighboring CG that has been scanned. In one example, when the current block is intra coded, the number of the bins of the second flag is more than the number of the bins of the second flag when the current block is inter coded.

At (S1530), context modeling is performed to determine a context model for each of the number of the bins of the second flag. The number of the bins of the second flag that are context coded may not exceed the number determined at S1520.

At (S1540), the coded bits of the number of the bins of the second flag are decoded based on the determined context models. The coded bits of the remaining total number of the bins of the second flag for the region of the transform skipped block may be decoded based on an EP model (i.e., a bypass model). Based on the recovered bins, coefficient levels of the coefficients can be reconstructed. The process (<NUM>) proceeds to and terminates at (S1599).

<FIG> shows a flow chart outlining a coefficient decoding process (<NUM>) according to some embodiments of the disclosure. The process (<NUM>) can be used in entropy decoding of several types of coefficient syntax elements. In various embodiments, the process (<NUM>) can be executed by processing circuitry, such as the processing circuitry in the terminal devices (<NUM>), (<NUM>), (<NUM>) and (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), the processing circuitry that performs functions of the video decoder (<NUM>), and the like. In some embodiments, the process (<NUM>) is implemented in software instructions, thus when the processing circuitry executes the software instructions, the processing circuitry performs the process (<NUM>). The process starts at (S1601) and proceeds to (S1610).

At (S1610), a bit stream including coded bits of bins of syntax elements is received. The syntax elements correspond to coefficients included in a coefficient group (CG) of a transform skipped block in a coded picture. The CG may be partitioned by a transform skipped block/coefficient block and may include 4x4 coefficient positions. The transform skipped block may indicate that a transform was not performed on the transform block. For example, when the current block is coded with BDPCM, a transform is not performed on the transform block.

At (S1620), a maximum number of context coded bins of the CG or a maximum average number of context coded bins of the CG is determined based on coded information of the coefficients. For example, the coded information of the coefficients includes a number of the first flag in the CG including the coefficients, a number the first flag in a previous CG, whether a current block corresponding to the transform skipped block is intra coded or inter coded, a size of the transform skipped block, a width of the transformed skipped block, a height of the transformed skipped block, or an aspect ratio of the transformed skipped block. The previous CG may be a neighboring CG that has been scanned.

At (S1630), context modeling is performed to determine a context model for each of a number of the bins of syntax elements in the CG. The number of the bins of syntax elements in the CG being context coded may not exceed the maximum number of context coded bins of the CG or the maximum average number of context coded bins of the CG determined at S1620.

At (S1640), the coded bits of the number of the bins of syntax elements are decoded based on the determined context models. The coded bits of the remaining total number of the bins of syntax elements in the CG may be decoded based on an EP model (i.e., a bypass model). Based on the recovered bins, coefficient levels of the coefficients can be reconstructed. The process (<NUM>) proceeds to and terminates at (S1699).

Computer system (<NUM>) can also include an interface to one or more communication networks. Networks can for example be wireless, wireline, optical. Networks can further be local, wide-area, metropolitan, vehicular and industrial, real-time, delay-tolerant, and so on. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks to include GSM, <NUM>, <NUM>, <NUM>, LTE and the like, TV wireline or wireless wide area digital networks to include cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial to include CANBus, and so forth. Certain networks commonly require external network interface adapters that attached to certain general purpose data ports or peripheral buses (<NUM>) (such as, for example USB ports of the computer system (<NUM>)); others are commonly integrated into the core of the computer system (<NUM>) by attachment to a system bus as described below (for example Ethernet interface into a PC computer system or cellular network interface into a smartphone computer system). Using any of these networks, computer system (<NUM>) can communicate with other entities. Such communication can be uni-directional, receive only (for example, broadcast TV), uni-directional send-only (for example CANbus to certain CANbus devices), or bi-directional, for example to other computer systems using local or wide area digital networks. Certain protocols and protocol stacks can be used on each of those networks and network interfaces as described above.

Claim 1:
A method (<NUM>) of video decoding performed in a video decoder, the method comprising:
receiving (S1510)a bit stream including bins of syntax elements, the syntax elements corresponding to coefficients of a region of a transform skipped block in a coded picture, the syntax elements including a first flag indicating whether an absolute coefficient level of one of the coefficients is greater than a first threshold value, and a second flag indicating whether the absolute coefficient level of the one of the coefficients is greater than a second threshold value, the second threshold value being greater than the first threshold value; characterized in that the method further comprises:
determining (S1520) a number of the bins of the second flag that are context coded based on coded information of the coefficients;
performing (S1530) context modeling to determine a context model for each of the number of the bins of the second flag; and
decoding (S1540) the number of the bins of the second flag based on the determined context models
wherein the coded information of the coefficients includes a number of the first flag in a current coefficient group, CG, including the coefficients, a number the first flag in a previous CG, whether a current block corresponding to the transform skipped block is intra coded or inter coded, a size of the transform skipped block, a width of the transformed skipped block, a height of the transformed skipped block, or an aspect ratio of the transformed skipped block.