Office Action 3: Notice of Allowance (2025-02-12)

CLAIMS:
AMENDMENTS TO THE CLAIMS

Please amend the claims as follows:

1\. (Currently Amended) A method for image processing, comprising:
decoding, from a coded bitstream that carries a compressed image, an index that points to a value in a set of values of a parameter, a value change of the parameter adjusting a compression rate of the compressed image, the compressed image being generated by a neural network based encoder based on the parameter, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss;
inputting the value of the parameter to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), the multi- rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and
generating, by the multi-rate compression domain computer vision task decoder, a computer vision task result according to the compressed image in the coded bitstream which is compressed at a corresponding compression rate from the multiple different compression rates, the compression rate beingbased on the value of the parameter, wherein the generating the computer vision task result includes:converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor;
inputting the tensor to one or more layers in a second neural network in the multi- rate compression domain computer vision task decoder; and
generating, by the second neural network, the computer vision task result according to the compressed image and the tensor.

2\. (Canceled).

3\. (Currently Amended) The method of claim 1,wherein the first neural network comprises one or more convolution layers.

4\. (Currently Amended) The method of claim 1,wherein the first neural network comprises a convolution layer with an activation function.

5\. (Currently Amended) The method of claim 1,wherein the second neural network is configured to generate the computer vision task result without generating a reconstructed image from the compressed image.

6\. (Currently Amended) The method of claim 1,wherein the second neural network is configured to generate a reconstructed image from the compressed image and generate the computer vision task result from the reconstructed image.

7\. (Original) The method of claim 1, wherein the neural network based encoder is based on an encoder model in a neural image compression (NIC) framework, and the multi-rate compression domain computer vision task decoder is based on a decoder model in the NIC framework, the NIC framework is trained end-to-end.

8\. (Previously Presented) The method of claim 1, wherein a decoder model of the multi- rate compression domain computer vision task decoder is trained separately from an encoder model of the neural network based encoder.

9\. (Previously Presented) The method of claim 1, wherein the parameter is a hyperparameter.

10\. (Original) The method of claim 1, wherein the computer vision task comprises at least one of image classification, image denoising, object detection and super resolution.

11\. (Currently Amended) An apparatus for image processing, comprising processing circuitry configured to:
input a value of a parameter to a multi-rate image compression encoder, the multi-rate image compression encoder comprising one or more neural networks for encoding images

according to a set of values of the parameter that are used for generating compressed images at multiple different compression rates, a value change of the parameter adjusting a compression rate of a respective compressed image, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss;encode, by the multi-rate image compression encoder, an input image into a compressed image according to a corresponding compression rate from the multiple different compression rates, the compression rate being based on the value of the parameter; and
encode an index that points to the value in the set of values of the parameter, wherein the value of the parameter is converted to a tensor by a first neural network in the multi- rate image compression encoder;
the tensor is input to one or more layers in a second neural network in the multi-rate image compression encoder; and
the compressed image is encoded by the second neural network according to the input image and the tensor.

12\. (Canceled).

13\. (Currently Amended) The apparatus of claim 11,wherein the first neural network comprises one or more convolution layers.

14\. (Currently Amended) The apparatus of claim 11,wherein the first neural network comprises a convolution layer with an activation function.

15-16. (Canceled).

17\. (Previously Presented) The apparatus of claim 11, wherein the multi-rate image compression encoder is based on an encoder model in a neural image compression (NIC) framework, and a multi-rate compression domain computer vision task decoder that corresponds to the multi-rate image compression encoder is based on a decoder model in the NIC framework, the NIC framework is trained end-to-end.

18\. (Previously Presented) The apparatus of claim 17, wherein the decoder model of the multi-rate compression domain computer vision task decoder is trained separately from the encoder model of the multi-rate image compression encoder.

19\. (Previously Presented) The apparatus of claim 11, wherein the parameter is a hyperparameter.

20\. (Canceled).

21\. (Currently Amended) A method of processing visual media data, the method comprising:
processing a bitstream of the visual media data according to a format rule, wherein the bitstream includes a syntax element indicating an index that points to a value in a set of values of a parameter, a value change of the parameter adjusting a compression rate of a compressed image in the bitstream, the compressed image being generated based on the parameter, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss; and
the format rule specifies that:the value of the parameter is input to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; \[\[and\]\] the value of the parameter is converted to a tensor by a first neural network in the multi-rate compression domain computer vision task decoder;
the tensor is input to one or more layers in a second neural network in the multi- rate compression domain computer vision task decoder; and
a computer vision task result is generated by the second neural network according to thecompressed image and the tensor, the compressed image in the bitstream being compressed at a corresponding

compression rate from the multiple different compression rates, the compression rate being based on the value of the parameter.

OFFICE ACTION:
DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 1, 3-11, 13-14, 17-19, and 21 are allowed.
The following is an examiner’s statement of reasons for allowance:
The closest prior art of record determined from an Examiner’s search is Anderson et al. (US 20220224914 A1) (hereinafter Anderson).  Anderson discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using a compression model.  The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.
Additional prior art Besenbruch (US 20230154055 A1) (hereinafter Besenbruch) discloses a loss used for neural network training, being L=d + (lambda)*R, where D is a distortion function, (lambda) is a weighting factor, and R is the rate loss.  R is related to entropy, and both D and R are differentiable functions.
However, the cited prior art fails to teach or render obvious the following combination of elements of independent claims 1, 11, and 21: “generating the computer vision task result includes: converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor; inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and generating, by the second neural network, the computer vision task result according to the compressed image and the tensor.”
Thus, Applicant’s amendments have been considered persuasive, in light of the claim limitations as well as the enabling portions of the specification. The cited prior art found as a result of an exhaustive search cannot reasonably be combined to arrive at the level of specificity of the instant claim language. Therefore, the various claimed limitations mentioned in the claims are not taught or suggested by the prior art taken either singly or in combination, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements that make them allowable over the prior art.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

APPLICANT REBUTTAL: REMARKS Favorable reconsideration of this application, as presently amended and in light of the following discussion, is respectfully requested. Claims 1, 3-11, 13-14, 17-19, and 21 are currently pending. Claims 1, 3-6, 11, 13-14, and 21 have been amended; and Claims 2 and 12 have been canceled without prejudice or disclaimer by the present Amendment. The changes to the claims are supported by the originally filed specification and claims.1 Thus, no new matter has been added. In the outstanding Office Action, Claims 1, 10-11, and 21 were rejected under 35 U.S.C. § 103 as being unpatentable over U.S. Publication No. 2020/0250003 (hereinafter "Yang") in view of U.S. Publication No. 2022/0224914 (hereinafter "Anderson") and U.S. Publication No. 2023/0154055 (hereinafter "Besenbruch"); Claims 2-8, 12-14, and 17-18 were rejected under 35 U.S.C. § 103 as being unpatentable over Yang in view of Anderson, Besenbruch, and U.S. Publication No. 2020/0280730 (hereinafter "Wang"); and Claims 9 and 19 were rejected under 35 U.S.C. § 103 as being unpatentable over Yang in view of Anderson, Besenbruch, and U.S. Publication No. 2020/0280717 (hereinafter "Li"). Substance of Interview Applicant and Applicant's representative thank Examiner Demosky for the courtesy of the interview with Applicant's representative on January 6, 2025, at which time proposed amendments incorporating features of the previously presented Claim 2 into Claim 1 were discussed. During the interview, the examiner indicated that the proposed amendments appear to place the application in condition for allowance. Amendments based on the discussion during the interview are presented herewith. Rejection under 35 U.S.C. Q 103 Independent Claim 1 is directed to a method for image processing, comprising: decoding, from a coded bitstream that carries a compressed image, an index that points to a value in a set of values of a parameter, a value change of the parameter adjusting a compression rate of the compressed image, the compressed image being generated by a neural network based 1 See, e.g., [0149] in the published application. encoder based on the parameter, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss; inputting the value of the parameter to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and generating, by the multi-rate compression domain computer vision task decoder, a computer vision task result according to the compressed image in the coded bitstream which is compressed at a corresponding compression rate from the multiple different compression rates, the compression rate being based on the value of the parameter, wherein the generating the computer vision task result includes: converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor; inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and generating, by the second neural network, the computer vision task result according to the compressed image and the tensor. (Emphasis added) Regarding the rejection of Claim 1 under 35 U.S.C. § 103, the Office Action acknowledges that Yang, Anderson, and Besenbruch fail to disclose "converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor; inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and generating, by the second neural network, the computer vision task result according to the compressed image and the tensor."2 Rather, the Office Action cites Wang for such a teaching. However, it is respectfully submitted that Wang fails to disclose "converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor; inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and generating, by the second 2 Office Action, at page 14. neural network, the computer vision task result according to the compressed image and the tensor," as discussed during the interview. Accordingly, it is respectfully submitted that Claim 1 (and all associated dependent claims) patentably defines over Yang, Anderson, Besenbruch, and Wang. Independent Claims 11 and 21, although differing in scope and/or statutory class, patentably define over Yang, Anderson, Besenbruch, and Wang at least for reasons analogous to the reasons stated above for the patentability of Claim 1. Accordingly, it is respectfully submitted that Claims 11 and 21 (and all associated dependent claims) patentably define over Yang, Anderson, Besenbruch, and Wang. Regarding the rejections of Claims 9 and 10 under 35 U.S.C. § 103, it is respectfully submitted that Claims 9 and 10 patentably define over Yang, Anderson, Besenbruch, and Wang at least for the reasons stated above for the patentability of Claim 1, from which Claims 9 and 10 depend. Further, the Office Action does not cite Li for such a teaching. Accordingly, it is respectfully submitted that Claims 9 and 10 patentably define over Yang, Anderson, Besenbruch, Wang, and Li. Conclusion If for any reason the examiner feels the application is not now in condition for allowance, it is respectfully requested that the examiner contact the undersigned practitioner to arrange for an interview to expedite the disposition of this case. If this communication is filed after the shortened statutory time period has elapsed and no separate Petition is enclosed, Applicant hereby petitions, under 37 C.F.R. § 1.136(a), to extend the time for filing a response to the outstanding Office Action by the number of months necessary to avoid abandonment under 37 C.F.R. § 1.135. Please charge any such fee, or any other fee due under 37 C.F.R. § 1.17, to our Deposit Account No. 01-2300, under 043380.01224, from which the undersigned is authorized to draw. Customer No.: 4372 Respectfully submitted, Atty Direct Line: 202.350.3626 Peifang.Tian@afslaw. com /Peifang Tian/ By Peifang Tian Registration No.: 79249Johnny Ma Registration No.: 59,976


CITED REFERENCE SUMMARIES: Ref US20230154055A1: Claims:
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system analyzing the quantized latent to produce parameters; (viii) the second computer system using the produced parameters to modify weights of a second trained neural network; (ix) the second computer system using the second trained neural network including the modified weights to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
(i) receiving an input image at a first computer system;
(ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation;
(iii) quantizing the latent representation using the first computer system to produce a quantized latent;
(iv) entropy encoding the quantized latent into a bitstream, using the first computer system;
(v) transmitting the bitstream to a second computer system;
(vi) the second computer system entropy decoding the bitstream to produce the quantized latent;
(vii) the second computer system analyzing the quantized latent to produce parameters;
(viii) the second computer system using the produced parameters to modify weights of a second trained neural network;
(ix) the second computer system using the second trained neural network including the modified weights to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
The method of claim 1 , wherein the parameters are a discrete perturbation of the weights of the second trained neural network.
The method of claim 1 , wherein the weights of the second trained neural network are perturbed by a perturbation function that is a function of the parameters, using the parameters in the perturbation function.
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) the first computer system optimizing a binary mask using the quantized latent; (v) entropy encoding the quantized latent and the binary mask into a bitstream, using the first computer system; (vi) transmitting the bitstream to a second computer system; (vii) the second computer system entropy decoding the bitstream to produce the quantized latent, and to produce the binary mask; (viii) the second computer system using the binary mask to modify a convolutional network of a second trained neural network; (ix) the second computer system using the second trained neural network including the modified a convolutional network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
(iv) the first computer system optimizing a binary mask using the quantized latent;
(v) entropy encoding the quantized latent and the binary mask into a bitstream, using the first computer system;
(vi) transmitting the bitstream to a second computer system;
(vii) the second computer system entropy decoding the bitstream to produce the quantized latent, and to produce the binary mask;
(viii) the second computer system using the binary mask to modify a convolutional network of a second trained neural network;
(ix) the second computer system using the second trained neural network including the modified a convolutional network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
The method of claim 4 , wherein in step (iv), the binary mask is optimized using a ranking based method, or using a stochastic method, or using a sparsity regularization method.
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image; wherein one of the input image, latent representation or quantized latent is, in a loop, modified so as to progressively reduce a finetuning loss after step (i), (ii) or (iii) respectively.
(vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image; wherein one of the input image, latent representation or quantized latent is, in a loop, modified so as to progressively reduce a finetuning loss after step (i), (ii) or (iii) respectively.
wherein one of the input image, latent representation or quantized latent is, in a loop, modified so as to progressively reduce a finetuning loss after step (i), (ii) or (iii) respectively.
The method of claim 6 , wherein the finetuning loss measures one of, or a combination of: a rate of the modified quantized latent, or a distortion between the current decoder prediction of the output image and the input image, or a distortion between the current decoder prediction of the output image and a decoder prediction of the output image using the quantized latent from step (iii).
The method of claim 6 , wherein the loop ends when the modified input image, latent or quantized latent satisfies an optimization criterion.
The method of any of claim 6 , wherein in the loop, the input image, latent or quantized latent is modified using a 1st-order optimization method, or using a 2nd-order optimization method, or using Monte-Carlo, Metropolis-Hastings, simulated annealing, or other greedy approaches.
Ref US20220224914A1: Claims:
A method of encoding a video including a set of frames, the method comprising: accessing a machine-learned compression model coupled to receive a frame and a parameter map for the frame, and output compressed code at a codelayer and a reconstructed version of the frame, wherein at least a portion of the compression model is configured as a neural network and the compression model is trained by: obtaining a video including a set of training frames, each training frame associated with a parameter map that indicates one or more values of an encoding parameter for the training frame, and changing weights of the compression model to reduce a loss function, the loss function including a reconstruction loss that indicates a difference between the set of training frames and reconstructed versions of the set of training frames, and a codelength loss that indicates codelength of a compressed code for the training frames, wherein the loss function changes depending on a value of the encoding parameter for the training frames, and wherein the compressed codes for at least two training frames with different values of the encoding parameter are generated at the same codelayer; iteratively performing, for each current frame in the set of frames: obtaining a parameter map for the current frame, and applying at least a portion of the machine-learned compression model to the current frame and the parameter map for the current frame to generate compressed code for the current frame; and transmitting the compressed codes generated for the set of frames to a receiver system.
accessing a machine-learned compression model coupled to receive a frame and a parameter map for the frame, and output compressed code at a codelayer and a reconstructed version of the frame, wherein at least a portion of the compression model is configured as a neural network and the compression model is trained by: obtaining a video including a set of training frames, each training frame associated with a parameter map that indicates one or more values of an encoding parameter for the training frame, and changing weights of the compression model to reduce a loss function, the loss function including a reconstruction loss that indicates a difference between the set of training frames and reconstructed versions of the set of training frames, and a codelength loss that indicates codelength of a compressed code for the training frames, wherein the loss function changes depending on a value of the encoding parameter for the training frames, and wherein the compressed codes for at least two training frames with different values of the encoding parameter are generated at the same codelayer;
obtaining a video including a set of training frames, each training frame associated with a parameter map that indicates one or more values of an encoding parameter for the training frame, and
changing weights of the compression model to reduce a loss function, the loss function including a reconstruction loss that indicates a difference between the set of training frames and reconstructed versions of the set of training frames, and a codelength loss that indicates codelength of a compressed code for the training frames, wherein the loss function changes depending on a value of the encoding parameter for the training frames, and wherein the compressed codes for at least two training frames with different values of the encoding parameter are generated at the same codelayer;
iteratively performing, for each current frame in the set of frames: obtaining a parameter map for the current frame, and applying at least a portion of the machine-learned compression model to the current frame and the parameter map for the current frame to generate compressed code for the current frame; and
obtaining a parameter map for the current frame, and
applying at least a portion of the machine-learned compression model to the current frame and the parameter map for the current frame to generate compressed code for the current frame; and
transmitting the compressed codes generated for the set of frames to a receiver system.
The method of claim 1 , wherein the one or more values of the encoding parameter represent one or more bitrate levels, and wherein the codelength loss increases when a bitrate level for the training frame increases.
The method of claim 2 , further comprising: performing an interpolation operation to generate the parameter map for the current frame that represents an interpolated value for the encoding parameter between the one or more values of the encoding parameter.
performing an interpolation operation to generate the parameter map for the current frame that represents an interpolated value for the encoding parameter between the one or more values of the encoding parameter.
The method of claim 1 , wherein the compression model is further configured to include the parameter map in the compressed code for a decoder at the receiver system configured to obtain the parameter map from the compressed code.
The method of claim 1 , wherein the portion of the machine-learned compression model includes one or more subsets of weights, each subset of weights associated with a respective value for the encoding parameter, and wherein applying the portion of the machine-learned compression model to the current frame and the parameter map for the current frame further comprises selecting a respective subset of weights associated with a value for the encoding parameter for the current frame, and applying the respective subset of weights to the current frame.
The method of claim 1 , wherein the machine-learned compression model includes a flow autoencoder and a residue autoencoder, and applying the portion of the machine-learned compression model to the current frame and the parameter map for the current frame further comprises: obtaining zero or more previously reconstructed frames for the current frame, applying the flow autoencoder to the current frame, the zero or more previously reconstructed frames, and the parameter map for the current frame to generate flow code as a part of the compressed code and a reconstructed flow for the current frame, generating a compensated frame by applying a flow operator to the one or more previously reconstructed frames and the reconstructed flow, generating a residue frame by computing a difference between the current frame and the compensated frame, applying the residue autoencoder to the compensated frame and the residue frame to generate residue code as another part of the compressed code and a reconstructed residue frame, and generating a reconstructed version of the current frame by combining the reconstructed residue frame and the compensated frame.
obtaining zero or more previously reconstructed frames for the current frame,
applying the flow autoencoder to the current frame, the zero or more previously reconstructed frames, and the parameter map for the current frame to generate flow code as a part of the compressed code and a reconstructed flow for the current frame,
generating a compensated frame by applying a flow operator to the one or more previously reconstructed frames and the reconstructed flow,
generating a residue frame by computing a difference between the current frame and the compensated frame,
applying the residue autoencoder to the compensated frame and the residue frame to generate residue code as another part of the compressed code and a reconstructed residue frame, and
generating a reconstructed version of the current frame by combining the reconstructed residue frame and the compensated frame.
The method of claim 1 , wherein the machine-learned compression model includes a flow predictor portion, a flow autoencoder, and a residue autoencoder, and applying the portion of the machine-learned compression model to the current frame and the parameter map for the current frame further comprises: obtaining one or more previously reconstructed frames for the current frame, applying the flow predictor portion to the one or more previously reconstructed frames or previous flow for the one or more previously reconstructed frames to generate a flow prediction for the current frame, generating a predicted frame by applying a flow operator to the one or more previously reconstructed frames and the flow prediction, applying the flow autoencoder to the current frame and the predicted frame, and the parameter map for the current frame to generate flow code as a part of the compressed code and a reconstructed flow for the current frame, generating a compensated frame by applying a flow operator to the one or more previously reconstructed frames and the reconstructed flow, generating a residue frame by computing a difference between the current frame and the compensated frame, applying the residue autoencoder to the compensated frame and the residue frame to generate residue code as another part of the compressed code and a reconstructed residue frame, and generating a reconstructed version of the current frame by combining the reconstructed residue frame and the compensated frame.
obtaining one or more previously reconstructed frames for the current frame,
applying the flow predictor portion to the one or more previously reconstructed frames or previous flow for the one or more previously reconstructed frames to generate a flow prediction for the current frame,
generating a predicted frame by applying a flow operator to the one or more previously reconstructed frames and the flow prediction,
applying the flow autoencoder to the current frame and the predicted frame, and the parameter map for the current frame to generate flow code as a part of the compressed code and a reconstructed flow for the current frame,
The method of claim 1 , wherein the one or more values of the encoding parameter represent one or more bitrate levels, wherein the parameter map for the training frame includes a plurality of encoded vectors for a plurality of regions in the training frame, each encoded vector representing a bitrate level for a respective region in the training frame, and wherein the reconstruction loss for a region in the training frame increases when the respective bitrate level for the region increases.
The method of claim 2 , wherein the machine-learned compression model is further coupled to receive one or more previously reconstructed frames for the frame to generate the compressed code and the reconstructed version of the frame, and wherein the one or more values of the encoding parameter for the training frame further represents: (i) whether the training frame will be processed as a P-frame with a previously reconstructed frame that is placed temporally before the training frame in the video, (ii) whether the training frame will be processed as a B 1 -frame with two or more previously reconstructed frames that were processed as a P-frame or an I-frame, (iii) whether the training frame will be processed as a B 2 -frame with a previously reconstructed frame that was processed as a P-frame or an I-frame and another previously reconstructed frame that was processed as a B 1 -frame, or (iv) whether the training frame will be processed as a B 3 -frame with a previously reconstructed frame that was processed as a B 1 -frame and another previously reconstructed frame that was processed as a B 2-frame.
(i) whether the training frame will be processed as a P-frame with a previously reconstructed frame that is placed temporally before the training frame in the video,
(ii) whether the training frame will be processed as a B 1 -frame with two or more previously reconstructed frames that were processed as a P-frame or an I-frame,
(iii) whether the training frame will be processed as a B 2 -frame with a previously reconstructed frame that was processed as a P-frame or an I-frame and another previously reconstructed frame that was processed as a B 1 -frame, or
(iv) whether the training frame will be processed as a B 3 -frame with a previously reconstructed frame that was processed as a B 1 -frame and another previously reconstructed frame that was processed as a B 2-frame.
The method of claim 1 , wherein the parameter map is represented as a one-hot encoded vector including one or more elements each corresponding to a respective value in the one or more values of the encoding parameter.
A non-transitory computer-readable medium containing instructions for execution on a processor, the instructions comprising: accessing a machine-learned compression model coupled to receive a frame and a parameter map for the frame, and output compressed code at a codelayer and a reconstructed version of the frame, wherein at least a portion of the compression model is configured as a neural network and the compression model is trained by: obtaining a video including a set of training frames, each training frame associated with a parameter map that indicates one or more values of an encoding p
[Truncated to input limit]