Office Action 1: Notice of Allowance (2025-02-20)

CLAIMS:
CLAIMS

1\. A method for making a smooth prediction block of samples in a picture in an image or video encoder or decoder, the method comprising:
determining parameters r of a polynomial model by r = (BTB)-l\*(BT\*x), where B is a matrix of bases and x is a source of samples in vector form; and
predicting a block x' based on the parameters r and the matrix of bases B.
2\. The method of claim 1, wherein the smooth prediction block x' comprises r(i)\*b(i) \], where r(i) refers to the i-th component of r and b(i) refers to the i-th basis vector of B, and the sum is taken over the number of basis vectors in B.SVG {EMBED HERE} 0.1430.46
3\. The method of claim 1, wherein the smooth prediction block x' comprises x'= r(i)\*b(i) \] + 2N-1)>> N, where N is a scaling factor, r(i) refers to the i-th component of r and b(i) refers to the i-th basis vector of B, and the sum is taken over the number of basis vectors inB.SVG {EMBED HERE} 0.1430.217

4\. The method of any one of claims 1-3, wherein each basis vector b(i) in B has its mean removed such that b(i) := b(i) - mean (b(i)).

5\. The method of any one of claims 1-4, wherein B at least includes three bases B = { b1, b2, b3}, where b1 = {1,1,1..,1}T, b2 = {0,1,..M-1,.., 0,1,..,M}T, and b3={0,..,0,1,..,1,
M-1,..,M-1}T, where M is equal to a height of the block

6\. The method of claim 5, wherein B further includes bases (b4, b5, b6) so that B={ b1, b2, b3, b4, b5, b6}, where b4 = {0, .., 0, 1,2,..M-1, .., M-1, 2\*M-1, .., (M-1)\*(M-1)}, b5 = {0,1,4,..,(M-1)\*(M-1), .., 0,1,4,..,(M-1)\*(M-1)} and b6 = {0, ..,0, 1, .., 1, 4, .., 4, .., (M- 1)\*(M-1)}.

7\. The method of any one of claims 1-6, wherein (BTB)1 is precomputed.

8\. The method of any one of claims 1-7, wherein the source of samples x is determined by taking a subset of samples (e.g., every second sample horizontally and vertically).

9\. The method of any one of claims 1-8, wherein each basis vector b(i) in B is taken at sample positions which are multiples of two.

10\. The method of any one of claims 1-9, wherein a height of the block (M) is equal to the maximum transform size used in the image or video encoder/decoder.

11\. A method for adjusting a quantized parameter, the method comprising:
determining that a block in an image is smooth;
determining that the quantized parameter (QP) would reduce the quality of the block after compression; and
as a result of determining that the block in the image is smooth and determining that the QP would reduce the quality of the block after compression, decreasing the QP for the block so that it is encoded with higher quality.

12\. The method of claims 11, wherein determining that a block in an image is smooth comprises:
determining parameters for a model based on low-frequency base functions;
predicting predicted samples with the determined parameters;
comparing the predicted samples with source samples of the block to determine an error; and
determining that the error is below a threshold.

13\. The method of claim 12, wherein the low-frequency base functions comprise polynomial functions, and wherein comparing the predicted block with source samples to determine an error comprises computing one or more of a sum-of-absolute-differences (SAD) value and a sum-of-squared-differences (SSD) value for the differences between the source samples of the block and the predicted samples.

14\. The method of any one of claims 12-13, wherein decreasing the QP for the block so that it is encoded with higher quality comprises determining the decrease of the QP based on a linear model such that the decrease is larger for higher QPs and that there is no decrease of the QP below a specified value.

15\. The method of claim 14, wherein the decrease of the QP is determined by min(O,QPscale\*QP+QPoffset), where QPscale and QPoffset are parameters.

16\. The method of any one of claims 14-15, wherein the decrease of QP is limited by a limiting value.

17\. The method of claim 14, wherein the decrease of the QP is determined by max(QPlimit, min(O,QPscale\*QP+QPoffset)), where QPscale, QPoffset and QPlimit are parameters.

18\. The method of any one of claims 11-17, further comprising determining that the image is an intra-coded picture type, and wherein decreasing the quantized parameter (QP)

for the block so that it is encoded with higher quality is performed as a result of determining that the image is an intra-coded picture type.

19\. The method of any one of claims 11-17, further comprising determining that the image has a sequence number (e.g. picture order count (POC)) within a sequence of images that is a multiple of a value M, and wherein decreasing the quantized parameter (QP) for the block so that it is encoded with higher quality is performed as a result of determining that the image has a sequence number that is a multiple of M.

20\. The method of claim 19, wherein M is one of 2, 4, 8, 16, and 32.

21\. The method of any one of claims 11-20, wherein determining that a block in an image is smooth is based on information at a sub-block level of granularity.

22\. A computer program comprising instructions which when executed by processing circuitry of a node, causes the node to perform the method of any one of claims 1-21.

23\. A carrier containing the computer program of claim 22, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

24\. An encoder, the encoder comprising:
processing circuitry; and
a memory, the memory containing instructions executable by the processing circuitry, whereby the encoder is configured to perform the method of any one the claims 1-21.

25\. A decoder, the decoder comprising:
processing circuitry; and
a memory, the memory containing instructions executable by the processing circuitry, whereby the decoder is configured to perform the method of any one the claims 1-10.

26\. An encoder configured to make a smooth prediction block of samples in a picture, the encoder being further configured to:determine parameters r of a polynomial model by r = (BTB)-SVG {EMBED HERE} 0.160.577 where B is a matrix of bases and x is a source of samples in vector form; and
predict a block x' based on the parameters r and the matrix of bases B.

27\. The encoder of claim 26, wherein the encoder is further configured to perform the method of any one of claims 2-10.

28\. An encoder configured to adjust a quantized parameter, the encoder being further configured to:
determine that a block in an image is smooth;
determine that the quantized parameter (QP) would reduce the quality of the block after compression; and
as a result of determining that the block in the image is smooth and determining that the QP would reduce the quality of the block after compression, decrease the QP for the block so that it is encoded with higher quality.

29\. The encoder of claim 28, wherein the encoder is further configured to perform the method of any one of claims 12-21.

30\. A decoder configured to make a smooth prediction block of samples in a picture, the decoder being further configured to:SVG {EMBED HERE} 0.4030.997determine parameters r of a polynomial model by r = where B is a matrix of bases and x is a source of samples in vector form; a predict a block x' based on the parameters r and the matrix of bases B.

31\. The decoder of claim 30, wherein the decoder is further configured to perform the method of any one of claims 2-10.

OFFICE ACTION:
EXAMINER'S AMENDMENT
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
An examiner’s amendment to the record appears below. Should the changes and/or additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 1.312. To ensure consideration of such an amendment, it MUST be submitted no later than the payment of the issue fee.
Authorization for this examiner’s amendment was given in an interview with Mr. Brian Rosenbloom on 1/10/25.
The application has been amended as follows:
In the Claims:
1-10 (Cancelled)
11. (Currently Amended)  A method for adjusting a quantization parameter (QP) for a block in an image, the method comprising:
determining that the block in the image is smooth, wherein determining that the block in the image is smooth is based on information at a sub-block level of granularity;
determining thatthe QP for the block would reduce the quality of the block after compression;
determining that the image is an intra-coded picture type; and
as a result of determining that: i) the block in the image is smooth, ii) , and iii) the image is an intra-coded picture type, decreasing the QP for the block so that the block is encoded with higher quality.
12. (Previously Presented)  The method of claim 11, wherein determining that a block in an image is smooth comprises:
determining parameters for a model based on low-frequency base functions;
predicting predicted samples with the determined parameters;
comparing the predicted samples with source samples of the block to determine an error; and
determining that the error is below a threshold.
13. (Previously Presented)  The method of claim 12, wherein
the low-frequency base functions comprise polynomial functions, and
comparing the predicted block with source samples to determine an error comprises computing one or more of a sum-of-absolute-differences (SAD) value and a sum-of-squared-differences (SSD) value for the differences between the source samples of the block and the predicted samples.
14. (Previously Presented)  The method of claim 12, wherein decreasing the QP for the block so that it is encoded with higher quality comprises determining the decrease of the QP based on a linear model such that the decrease is larger for higher QPs and that there is no decrease of the QP below a specified value.
15. (Previously Presented)  The method of claim 14, wherein the decrease of the QP is determined by min(0,QPscale*QP+QPoffset), where QPscale is a first parameter and QPoffset is a second parameter.
16. (Previously Presented)  The method of claim 14, wherein the decrease of QP is limited by a limiting value.
17. (Previously Presented)  The method of claim 14, wherein the decrease of the QP is determined by max(QPlimit, min(0,QPscale*QP+QPoffset)), where QPscale is a first parameter, QPoffset is a second parameter, and QPlimit is a third parameter.
18.
19. (Previously Presented)  The method of claim 11, further comprising determining that the image has a sequence number within a sequence of images that is a multiple of a value M, wherein
decreasing the quantized parameter (QP) for the block so that it is encoded with higher quality is performed as a result of determining that the image has a sequence number that is a multiple of M.
20. (Previously Presented)  The method of claim 19, wherein M is 2, 4, 8, 16, or 32.
21.
22. (Currently Amended) A non-transitory computer readable storage medium storing a computer program comprising instructions which when executed by processing circuitry of a node, causes the node to perform the method ofclaim 11.
23. (Cancelled)
24. (Currently Amended) An apparatus, the apparatus comprising:
processing circuitry; and
a memory, the memory containing instructions executable by the processing circuitry, wherein the apparatus is configured to perform a method for adjusting a quantization parameter (QP) for a block in an image, the method comprising:
determining that the block in the image is smooth, wherein determining that the block in the image is smooth is based on information at a sub-block level of granularity;
determining that the QP for the block would reduce the quality of the block after compression;
determining that the image is an intra-coded picture type; and
as a result of determining that: i) the block in the image is smooth, ii) the QP would reduce the quality of the block after compression, and iii) the image is an intra-coded picture type, decreasing the QP for the block so that the block is encoded with higher quality.
25-31. (Cancelled)
Reasons for Allowance
The following is an examiner’s statement of reasons for allowance: Claims 11-22 and 24 are considered allowable over the prior art because none of the references of record alone or in combination suggest, disclose, or teach a method for adjusting a quantization parameter (QP) for a block in an image, the method comprising determining that the block in the image is smooth, wherein determining that the block in the image is smooth is based on information at a sub-block level of granularity; determining that the QP for the block would reduce the quality of the block after compression; determining that the image is an intra-coded picture type; and as a result of determining that: i) the block in the image is smooth, ii) the QP would reduce the quality of the block after compression, and iii) the image is an intra-coded picture type, decreasing the QP for the block so that the block is encoded with higher quality all together as claimed.  The closest prior art, Kwon et al (10,897,625), discloses a conventional quantization method, either singularly or in combination, fails to anticipate or render the above limitations obvious.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Zuo et al discloses a conventional method for adjusting a quantization parameter.
Conclusion



CITED REFERENCE SUMMARIES: Ref US20190045192A1: Claims:
A system to perform efficient motion based video processing using global motion, comprising: a global motion analyzer, the global motion analyzer including one or more substrates and logic coupled to the one or more substrates, wherein the logic is to: obtain a plurality of block motion vectors for a plurality of blocks of a current frame with respect to a reference frame; modify the plurality of block motion vectors, wherein the modification of the plurality of block motion vectors includes one or more of the following operations: smoothing of at least a portion of the plurality of block motion vectors, merging of at least a portion of the plurality of block motion vectors, and discarding of at least a portion of the plurality of block motion vectors; restrict the modified plurality of block motion vectors by excluding a portion of the frame in some instances; compute a plurality of candidate global motion models based on the restricted-modified plurality of block motion vectors for the current frame with respect to the reference frame, wherein each candidate global motion model comprises a set of candidate global motion model parameters representing global motion of the current frame; determine a best global motion model from the plurality of candidate global motion models on a frame-by-frame basis, wherein each best global motion model comprises a set of best global motion model parameters representing global motion of the current frame; modify a precision of the best global motion model parameters in response to one or more application parameters; map the modified-precision best global motion model parameters to a pixel-based coordinate system to determine a plurality of mapped global motion warping vectors for a plurality of reference frame control-grid points; predict and encode the plurality of mapped global motion warping vectors for the current frame with respect to a plurality of previous mapped global motion warping vectors; determine a best sub-pel filter to use for interpolation at an â th pel location or a 1/16 th pel location from among two or more sub-pel filter choices per frame; apply the plurality of mapped global motion warping vectors at sub-pel locations to the reference frame and perform interpolation of pixels based on the determined best sub-pel filter to generate a global motion compensated warped reference frame; and a power supply to provide power to the global motion analyzer.
a global motion analyzer, the global motion analyzer including one or more substrates and logic coupled to the one or more substrates, wherein the logic is to: obtain a plurality of block motion vectors for a plurality of blocks of a current frame with respect to a reference frame; modify the plurality of block motion vectors, wherein the modification of the plurality of block motion vectors includes one or more of the following operations: smoothing of at least a portion of the plurality of block motion vectors, merging of at least a portion of the plurality of block motion vectors, and discarding of at least a portion of the plurality of block motion vectors; restrict the modified plurality of block motion vectors by excluding a portion of the frame in some instances; compute a plurality of candidate global motion models based on the restricted-modified plurality of block motion vectors for the current frame with respect to the reference frame, wherein each candidate global motion model comprises a set of candidate global motion model parameters representing global motion of the current frame; determine a best global motion model from the plurality of candidate global motion models on a frame-by-frame basis, wherein each best global motion model comprises a set of best global motion model parameters representing global motion of the current frame; modify a precision of the best global motion model parameters in response to one or more application parameters; map the modified-precision best global motion model parameters to a pixel-based coordinate system to determine a plurality of mapped global motion warping vectors for a plurality of reference frame control-grid points; predict and encode the plurality of mapped global motion warping vectors for the current frame with respect to a plurality of previous mapped global motion warping vectors; determine a best sub-pel filter to use for interpolation at an â th pel location or a 1/16 th pel location from among two or more sub-pel filter choices per frame; apply the plurality of mapped global motion warping vectors at sub-pel locations to the reference frame and perform interpolation of pixels based on the determined best sub-pel filter to generate a global motion compensated warped reference frame; and
obtain a plurality of block motion vectors for a plurality of blocks of a current frame with respect to a reference frame;
modify the plurality of block motion vectors, wherein the modification of the plurality of block motion vectors includes one or more of the following operations: smoothing of at least a portion of the plurality of block motion vectors, merging of at least a portion of the plurality of block motion vectors, and discarding of at least a portion of the plurality of block motion vectors;
restrict the modified plurality of block motion vectors by excluding a portion of the frame in some instances;
compute a plurality of candidate global motion models based on the restricted-modified plurality of block motion vectors for the current frame with respect to the reference frame, wherein each candidate global motion model comprises a set of candidate global motion model parameters representing global motion of the current frame;
determine a best global motion model from the plurality of candidate global motion models on a frame-by-frame basis, wherein each best global motion model comprises a set of best global motion model parameters representing global motion of the current frame;
modify a precision of the best global motion model parameters in response to one or more application parameters;
map the modified-precision best global motion model parameters to a pixel-based coordinate system to determine a plurality of mapped global motion warping vectors for a plurality of reference frame control-grid points;
predict and encode the plurality of mapped global motion warping vectors for the current frame with respect to a plurality of previous mapped global motion warping vectors;
determine a best sub-pel filter to use for interpolation at an â th pel location or a 1/16 th pel location from among two or more sub-pel filter choices per frame;
apply the plurality of mapped global motion warping vectors at sub-pel locations to the reference frame and perform interpolation of pixels based on the determined best sub-pel filter to generate a global motion compensated warped reference frame; and
a power supply to provide power to the global motion analyzer.
The system of claim 1 , wherein the modification of the plurality of block motion vectors further comprises operations to: calculate a set of initial global motion model parameters for an initial global motion model for the current frame; apply the initial global motion model to a center of each of the plurality of blocks of the current frame to determine an initial global motion vector of each block; calculate a global/local difference between the initial global motion vector of each block and a local motion vector of each block; calculate an adaptive threshold to apply to the global/local motion difference, wherein the adaptive threshold adaptively changes based on the content of the particular current frame; and segment the current frame into a global motion region and a local motion region based on a size of the global/local difference as applied to the adaptive threshold.
calculate a set of initial global motion model parameters for an initial global motion model for the current frame;
apply the initial global motion model to a center of each of the plurality of blocks of the current frame to determine an initial global motion vector of each block;
calculate a global/local difference between the initial global motion vector of each block and a local motion vector of each block;
calculate an adaptive threshold to apply to the global/local motion difference, wherein the adaptive threshold adaptively changes based on the content of the particular current frame; and
segment the current frame into a global motion region and a local motion region based on a size of the global/local difference as applied to the adaptive threshold.
The system of claim 2 , wherein calculation of initial global motion model parameters further comprises operations to: use random sampling through a plurality of iterations to selects a set of three linearly independent motion vectors at a time per iteration, wherein each set of three linearly independent motion vectors is linearly independent motion vectors is used to calculate a sampled six parameter global motion model; and generate a histogram for each of the sampled six parameter global motion model to find a best model parameter from a peak value of each parameter, wherein a set of best model parameters describes an initial global motion equation.
use random sampling through a plurality of iterations to selects a set of three linearly independent motion vectors at a time per iteration, wherein each set of three linearly independent motion vectors is linearly independent motion vectors is used to calculate a sampled six parameter global motion model; and
generate a histogram for each of the sampled six parameter global motion model to find a best model parameter from a peak value of each parameter, wherein a set of best model parameters describes an initial global motion equation.
The system of claim 2 , wherein the global motion region and the local motion region segmentation is performed in at least some instances using several thresholds to create multiple alternate segmentations.
The system of claim 2 , wherein the global motion region and the local motion region segmentation is performed in at least some instances by morphologically operation of erosion and dilation to form one or more revised segmentations; wherein the selection of a best global motion region and the local motion region segmentation is made between all such available segmentations, alternate segmentations, and revised segmentations to find the segmentation that generates a least residual between the current frame an the reference frame; wherein the best global motion region and the local motion region segmentation is further refined by determining and discarding low-detail portions of the current frame that is detected by a texture measure; wherein the best global motion region and the local motion region segmentation is further refined by determining and discarding all other blocks except blocks containing object corners as well as blocks containing high texture as detected by a texture measure; and wherein the restriction of the modified plurality of block motion vectors further comprises operations to: determine and remove certain block motion vectors as they may belong to an inactive static region, wherein the one or more inactive static areas include one or more of the following inactive static area types: black bar-type inactive static areas, black boarder-type inactive static areas, letterbox-type inactive static areas, logo overlay-type inactive static areas, and text overlay-type inactive static areas.
wherein the selection of a best global motion region and the local motion region segmentation is made between all such available segmentations, alternate segmentations, and revised segmentations to find the segmentation that generates a least residual between the current frame an the reference frame;
wherein the best global motion region and the local motion region segmentation is further refined by determining and discarding low-detail portions of the current frame that is detected by a texture measure;
wherein the best global motion region and the local motion region segmentation is further refined by determining and discarding all other blocks except blocks containing object corners as well as blocks containing high texture as detected by a texture measure; and
wherein the restriction of the modified plurality of block motion vectors further comprises operations to: determine and remove certain block motion vectors as they may belong to an inactive static region, wherein the one or more inactive static areas include one or more of the following inactive static area types: black bar-type inactive static areas, black boarder-type inactive static areas, letterbox-type inactive static areas, logo overlay-type inactive static areas, and text overlay-type inactive static areas.
determine and remove certain block motion vectors as they may belong to an inactive static region, wherein the one or more inactive static areas include one or more of the following inactive static area types: black bar-type inactive static areas, black boarder-type inactive static areas, letterbox-type inactive static areas, logo overlay-type inactive static areas, and text overlay-type inactive static areas.
The system of claim 1 , wherein the computation of the plurality of candidate global motion models further comprises operations to: choose a global motion region and a local motion region segmentation for selection of a valid region for choosing candidate motion vectors for global motion model computation; choose a set of global motion models in a first mode selected from among four parameter models, six parameter models, and eight parameter models as well as in a second mode selected from among six parameter models, eight parameter models, and twelve parameter models, wherein the first mode is selected for low definition scene sequences and the second mode is selected for high definition scene sequences; choose a method for computing each individual global motion model of the set of global motion models selected from among least square and Levenberg Marquardt (LMA); and choose one or more convergence parameters for the chosen least square and Levenberg Marquardt method.
choose a global motion region and a local motion region segmentation for selection of a valid region for choosing candidate motion vectors for global motion model computation;
choose a set of global motion models in a first mode selected from among four parameter models, six parameter models, and eight parameter models as well as in a second mode selected from among six parameter models, eight parameter models, and twelve parameter models, wherein the first mode is selected for low definition scene sequences and the second mode is selected for high definition scene sequences;
choose a method for computing each individual global motion model of the set of global motion models selected from among least square and Levenberg Marquardt (LMA); and
choose one or more convergence parameters for the chosen least square and Levenberg Marquardt method.
The system of claim 6 , further comprising operations to: select a method for computing each individual global motion model depending on the order of the model including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method; perform computation of the each global motion model using the related chosen method; and select a best model based on lowest modified distortion.
select a method for computing each individual global motion model depending on the order of the model including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method;
perform computation of the each global motion model using the related chosen method; and
select a best model based on lowest modified distortion.
The system of claim 6 , further comprising operations to: select a method for computing each individual global motion model depending on the order of the model including for four and six parameter model using the least square method, and for eight and twelve parameter model using the Levenberg Marquardt method; perform computation of the each global motion model using the related chosen method; and select a best model based on a best Rate Distortion Optimization tradeoff that takes into account both distortion as well as rate.
select a best model based on a best Rate Distortion Optimization tradeoff that takes into account both distortion as well as rate.
The system of claim 1 wherein the modification of the precision of the best global motion model parameters further comprises operations to: determine the significance of each model parameter of the best global motion model parameters to define an active range; determine the application parameters including one or more of the following application parameter types: coding bit-rate, resolution, and required quality; and assign a different accuracy to each model parameter of the best global motion model parameters based on the determined significance in some instances, based on the determined application parameter in other instances, and based on the determined significance and the determined application parameter in further instances.
determine the significance of each model parameter of the best global motion model parameters to define an active range;
determine the application parameters including one or more of the following application parameter types: coding bit-rate, resolution, and required quality; and
assign a different accuracy to each model parameter of the best global motion model parameters based on the determined significance in some instances, based on the determined application parameter in other instances, and based on the determined significance and the determined application parameter in further instances.
The system of claim 1 , wherein the map of the modified-precision best global motion model parameters to the pixel-based coordinate system to determine the plurality of mapped global motion warping vectors for the plurality of reference frame control-grid points further comprises operations to: map modified precision global motion model parameters to pixel-domain based mapped global motion warping vectors as applied to control-grid points, wherein the control-grid points comprise two vertices of a frame for four parameters, three vertices of a frame for six parameters, all four vertices of a frame for eight parameters, and four vertices of a frame plus two negative-mirror vertices of a frame for twelve parameters.
map modified precision global motion model parameters to pixel-domain based mapped global motion warping vectors as applied to control-grid points, wherein the control-grid points comprise two vertices of a frame for four parameters, three vertices of a frame for six parameters, all four vertices of a frame for eight parameters, and four vertices of a frame plus two negative-mirror vertices of a frame for twelve parameters.
The system of claim 1 , wherein the prediction and encode of the plurality of mapped global motion warping vectors further comprises operations to: predict the warping vectors of the current frame based on one or more previously stored warping vectors to generate first predicted warping vectors, wherein the previously stored warping vectors are scaled to adjust for frame distance; predict the warping vectors of the current frame based on multiple codebook warping vectors to generate second predicted warping vectors, wherein the codebook warping vectors are scaled to adjust for frame distance; compute a difference of the warping vectors of the current frame with the first and second predicted warping vectors to generate residual warping vectors; choose a best one of the residual warping vectors based on minimal residual warping vectors, of the fir
[Truncated to input limit]