Office Action 2: Final Rejection (2024-11-27)
Rejection types: 103 - Obviousness

CLAIMS:
AMENDMENTS TO THE CLAIMS

Please amend the claims as follows:

1\. (Currently Amended) A method for image processing, comprising:
decoding, from a coded bitstream that carries a compressed image, an index that points to a value in a set of values of a parameter, a value change of the parameter adjusting a compression rate of the compressed image, the compressed image being generated by a neural network based encoder based on the parameter, the parameter weighting a distortion and a rate in a calculation of arate-distortion (R-D) loss;
inputting the value of the parameter to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), the multi- rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and
generating, by the multi-rate compression domain computer vision task decoder, a computer vision task result according to the compressed image in the coded bitstream which is compressed at a corresponding compression rate from the multiple different compression rates based on the value of the parameter.

2\. (Original) The method of claim 1, wherein the generating the computer vision task result comprises:
converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor;
inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and
generating, by the second neural network, the computer vision task result according to the compressed image and the tensor.

3\. (Original) The method of claim 2, wherein the first neural network comprises one or more convolution layers.

4\. (Original) The method of claim 2, wherein the first neural network comprises a convolution layer with an activation function.

5\. (Original) The method of claim 2, wherein the second neural network is configured to generate the computer vision task result without generating a reconstructed image from the compressed image.

6\. (Currently Amended) The method of claim 2, wherein the second neural network is configured to generate a reconstructed image from the compressed image\[\[,\]\] and generate the computer vision task result from the reconstructed image.

7\. (Original) The method of claim 1, wherein the neural network based encoder is based on an encoder model in a neural image compression (NIC) framework, and the multi-rate compression domain computer vision task decoder is based on a decoder model in the NIC framework, the NIC framework is trained end-to-end.

8\. (Currently Amended) The method of claim 1, wherein a decoder model of the multi- rate compression domain computer vision task decoder is trained separately from an encoder model of the neural network based encoder.

9\. (Currently Amended) The method of claim 1, wherein the parameter is a hyperparameter.

10\. (Original) The method of claim 1, wherein the computer vision task comprises at least one of image classification, image denoising, object detection and super resolution.

11\. (Currently Amended) An apparatus for image processing, comprising processing circuitry configured to:

input \[\[the\]\] a value of \[\[the\]\] a parameter to a multi-rate image compression encoder, the multi-rate image compression encoder comprising one or more neural networks for encoding images according to a set of values of the parameter that are used for generating \[\[the\]\] compressed images at multiple different compression rates, a value change of the parameter adjusting a compression rate of a respective compressed image, the parameter weighting a distortion and a rate in a calculation of a rate- distortion (R-D) loss; \[\[and\]\] encode, by the multi-rate image compression encoder, an input image into a compressed image according to a corresponding compression rate from the multiple different compression rates based on the value of the parameter; and
encode an index that points to the value in the set of values of the parameter.

12\. (Currently Amended) The apparatus of claim 11, wherein the processing circuitry is configured to:
convert, by a first neural network in the multi-rate image compression encoder, the value of the parameter to a tensor;
input the tensor to one or more layers in a second neural network in the multi-rate image compression encoder; and
 encode, by the second neural network, the compressed image according to the input image and the tensor.

13\. (Original) The apparatus of claim 12, wherein the first neural network comprises one or more convolution layers.

14\. (Original) The apparatus of claim 12, wherein the first neural network comprises a convolution layer with an activation function.

15-16. (Canceled).

17\. (Currently Amended) The apparatus of claim 11, wherein the multi-rate image compression encoder is based on an encoder model in a neural image compression (NIC) framework, and \[\[the\]\] a multi-rate compression domain computer vision task decoder that corresponds to the multi-rate image compression encoder is based on a decoder model in the NIC framework, the NIC framework is trained end-to-end.

18\. (Currently Amended) The apparatus of claim 17,wherein \[\[a\]\] the decoder model of the multi-rate compression domain computer vision task decoder is trained separately from \[\[an\]\] the encoder model of the multi-rate image compression encoder.

19\. (Currently Amended) The apparatus of claim 11, wherein the parameter is a hyperparameter.

20\. (Canceled).

21\. (New) A method of processing visual media data, the method comprising:
processing a bitstream of the visual media data according to a format rule, wherein the bitstream includes a syntax element indicating an index that points to a value in a set of values of a parameter, a value change of the parameter adjusting a compression rate of a compressed image in the bitstream, the compressed image being generated based on the parameter, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R- D) loss; and
the format rule specifies that:the value of the parameter is input to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and

a computer vision task result is generated according to the compressed image in the bitstream which is compressed at a corresponding compression rate from the multiple different compression rates based on the value of the parameter.

OFFICE ACTION:
DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements submitted on 9/15/2023 and 3/22/2023 are in compliance with the provisions of 37 CFR 1.97.  Accordingly, the information disclosure statement(s) are being considered by the examiner.
Response to Arguments
Applicant's arguments filed 11/05/2024 have been fully considered but they are drawn towards newly amended claim language.
Regarding Rejections under 35 U.S.C. § 103,
Applicant contends that the cited prior art fails to disclose newly amended limitations of independent claims 1, 11, and 21 including at least “the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss”.
See the rejection below for how the cited art in light of new/existing references reads on the newly amended language as well as the examiner’s interpretation of the cited art in view of the presented claim set.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA  35 U.S.C. 102 and 103 (or as subject to pre-AIA  35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA  to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary.  Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 10-11, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 20200250003 A1) (hereinafter Yang) in view of Anderson et al. (US 20220224914 A1) (hereinafter Anderson) in view of Besenbruch et al. (US 20230154055 A1; priority to PCT/GB2021/051041, filed 4/29/2021) (hereinafter Besenbruch).
Regarding claim 1, Yang discloses:
A method for image processing, comprising:
decoding, from a coded bitstream that carries a compressed image, [See Yang, ¶ 0281, 0293 discloses compressed visual data provided as input to CNN may first be partially decoded to separate and extract different syntax elements (e.g., motion vectors, macroblock (MB) coding modes, quantized prediction residuals), thus producing a subset of partial compression data.] discloses an index that points a value in a set of values of a parameter, [See Yang, ¶ 0259-0261 discloses an image format schema using an array-based database manager to specify parameters about the array that can be used to arrange image data.]  a value change of the parameter adjusting a compression rate of the compressed image, [See Yang, ¶ 0243 discloses that images can be compressed either in a lossless or lossy manner. Lossless image compression preserves all the information in the image, while lossy compression takes advantage of visual perception and statistical properties to achieve better compression rates, but results in some data being lost.] the compressed image being generated by a neural network based encoder based on the parameter; [See Yang, ¶ 0272 discloses that neural networks can be designed to operate on compressed visual data directly, thus eliminating the need to decompress visual data before it can be processed.  Further, that visual metadata could be compressed by pre-training a convolutional neural network (CNN) to classify visual metadata, replacing long strings of visual metadata with shorter symbols (e.g., pre-defined human codes), performing multi-scale de-duplication on the visual metadata, and finally compressing the resulting visual metadata using a compression algorithm.]
Yang does not appear to explicitly disclose:
inputting the value of the parameter to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF),
the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and
generating, by the multi-rate compression domain computer vision task decoder, a computer vision task result according to the compressed image in the coded bitstream which is compressed at a corresponding compression rate from the multiple different compression rates based on the value of the parameter,
However, Anderson discloses:
inputting the value of the parameter to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), [See Anderson, ¶ 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.  The parameter map may at least in part incorporate the frame type of the target frame, and in one embodiment, may include at least a first parameter map indicating desired bitrate levels and a second parameter map indicating the frame type of the target frame concatenated to the first parameter map. As an example, the second parameter map may be represented as a one-hot encoded vector including four elements, each representing whether a target frame is encoded and decoded as a P-frame, a B1-frame, a B2-frame, or a B3-frame.] the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images [See Anderson, ¶ 0047, 0051, 0058, 0090, 0098 discloses  a frame decoder 414 may be configured as a neural network model.  Furthermore, that at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task.] according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and [See Anderson, ¶ 0005, 0025, 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.  The parameter map may at least in part incorporate the frame type of the target frame, and in one embodiment, may include at least a first parameter map indicating desired bitrate levels and a second parameter map indicating the frame type of the target frame concatenated to the first parameter map.]
generating, by the multi-rate compression domain computer vision task decoder, a computer vision task result according to the compressed image in the coded bitstream which is compressed at a corresponding compression rate from the multiple different compression rates based on the value of the parameter, [See Anderson, ¶ 0047, 0051, 0058, 0090, 0098 discloses at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task; See Anderson, ¶ 0005, 0025, 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang to add the teachings of Anderson in order to improve coding efficiency by permitting multiple rates – allowing encoding and decoding of different B-frame types with a single B-frame compression model.  This makes deployment easier since multiple models with different sets of parameters do not have to be constructed for each B-frame type.
Yang in view of Anderson does not appear to explicitly disclose:
the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss;
However, Besenbruch discloses:
the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss; [See Besenbruch, ¶ 0520 discloses a loss used for neural network training, being L = D + λ*R, where D is a distortion function, λ is a weighting factor, and R is the rate loss.  R is related to entropy, and both D and R are differentiable functions.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang in view of Anderson to add the teachings of Besenbruch in order to permit calculation of loss resembling rate-distortion trade off in neural network training.
Regarding claim 10, Yang in view of Anderson in view of Besenbruch discloses all the limitations of claim 1.
Yang discloses:
wherein the computer vision task comprises at least one of image classification, image denoising, object detection and super resolution. [See Yang, ¶ 0226 discloses image classification is a common visual data operation that uses a neural network to identify the contents of an image. For example, in machine learning, a convolutional neural network (CNN) is a type of feed-forward artificial neural network where the input is generally assumed to be an image. CNNs are commonly used for image classification, where the goal is to determine the contents of an image with some level of confidence. For example, a CNN is first trained for a specific classification task using a set of images whose object classes or features have been labeled, and the CNN can then be used to determine the probability of whether other images contain the respective object classes.]
Regarding claim 11, this claim recites analogous limitations to claim 1 in the form of “an apparatus” rather than “a method”, and is therefore rejected on the same premise.  Please see examiner’s earlier rejection of claim 1 for corresponding motivation statement.
Further, claim 11 recites the following limitations which are not explicitly found from claim 1, but are addressed as follows:
Yang discloses:
An apparatus for image processing, comprising processing circuitry configured to: [See Yang, ¶ 0114, 0148 discloses a processing system includes any system that has a processor, such as, for example; a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.]
encode an index that points to the value in the set of values of the parameter. [See Yang, ¶ 0259-0261 discloses an image format schema using an array-based database manager to specify parameters about the array that can be used to arrange image data.]
Anderson discloses:
input a value of a parameter to a multi-rate image compression encoder, the multi-rate image compression encoder comprising one or more neural networks for encoding images according to a set of values of the parameter that are used for generating compressed images at multiple different compression rates, [See Anderson, ¶ 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.  The parameter map may at least in part incorporate the frame type of the target frame, and in one embodiment, may include at least a first parameter map indicating desired bitrate levels and a second parameter map indicating the frame type of the target frame concatenated to the first parameter map. As an example, the second parameter map may be represented as a one-hot encoded vector including four elements, each representing whether a target frame is encoded and decoded as a P-frame, a B1-frame, a B2-frame, or a B3-frame; See Anderson, ¶ 0047, 0051, 0058, 0090, 0098 discloses  a frame decoder 414 may be configured as a neural network model.  Furthermore, that at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task.]
encode, by the multi-rate image compression encoder an input image into a compressed image according to a corresponding compression rate from the multiple different compression rates based on the value of the parameter; and [See Anderson, ¶ 0047, 0051, 0058, 0090, 0098 discloses at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task; See Anderson, ¶ 0005, 0025, 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang to add the teachings of Anderson in order to improve coding efficiency by permitting multiple rates – allowing encoding and decoding of different B-frame types with a single B-frame compression model.  This makes deployment easier since multiple models with different sets of parameters do not have to be constructed for each B-frame type.
Besenbruch discloses:
the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss; [See Besenbruch, ¶ 0520 discloses a loss used for neural network training, being L = D + λ*R, where D is a distortion function, λ is a weighting factor, and R is the rate loss.  R is related to entropy, and both D and R are differentiable functions.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang in view of Anderson to add the teachings of Besenbruch in order to permit calculation of loss resembling rate-distortion trade off in neural network training.
Regarding claim 21, Yang discloses:
A method of processing visual media data, the method comprising:
processing a bitstream of the visual media data according to a format rule, wherein the bitstream includes a syntax element indicating an index that points to a value in a set of values of a parameter, [See Yang, ¶ 0259-0261 discloses an image format schema using an array-based database manager to specify parameters about the array that can be used to arrange image data.]  a value change of the parameter adjusting a compression rate of a compressed image in the bitstream, the compressed image being generated based on the parameter, [See Yang, ¶ 0243 discloses that images can be compressed either in a lossless or lossy manner. Lossless image compression preserves all the information in the image, while lossy compression takes advantage of visual perception and statistical properties to achieve better compression rates, but results in some data being lost.]
the format rule specifies that:
the value of the parameter is input to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), [See Anderson, ¶ 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.  The parameter map may at least in part incorporate the frame type of the target frame, and in one embodiment, may include at least a first parameter map indicating desired bitrate levels and a second parameter map indicating the frame type of the target frame concatenated to the first parameter map. As an example, the second parameter map may be represented as a one-hot encoded vector including four elements, each representing whether a target frame is encoded and decoded as a P-frame, a B1-frame, a B2-frame, or a B3-frame.] the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images [See Anderson, ¶ 0047, 0051, 0058, 0090, 0098 discloses  a frame decoder 414 may be configured as a neural network model.  Furthermore, that at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task.]  according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and [See Anderson, ¶ 0005, 0025, 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.  The parameter map may at least in part incorporate the frame type of the target frame, and in one embodiment, may include at least a first parameter map indicating desired bitrate levels and a second parameter map indicating the frame type of the target frame concatenated to the first parameter map.]
a computer vision task result is generated according to the compressed image in the bitstream which is compressed at a corresponding compression rate from the multiple different compression rates based on the value of the parameter. [See Anderson, ¶ 0047, 0051, 0058, 0090, 0098 discloses at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task; See Anderson, ¶ 0005, 0025, 0116 discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using the compression model 600. The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang to add the teachings of Anderson in order to improve coding efficiency by permitting multiple rates – allowing encoding and decoding of different B-frame types with a single B-frame compression model.  This makes deployment easier since multiple models with different sets of parameters do not have to be constructed for each B-frame type.
the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss; and  [See Besenbruch, ¶ 0520 discloses a loss used for neural network training, being L = D + λ*R, where D is a distortion function, λ is a weighting factor, and R is the rate loss.  R is related to entropy, and both D and R are differentiable functions.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang in view of Anderson to add the teachings of Besenbruch in order to permit calculation of loss resembling rate-distortion trade off in neural network training.
Claim(s) 2-8, 12-14, 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Anderson in view of Besenbruch in view of Wang et al. (US 20200280730 A1) (hereinafter Wang).
Regarding claim 2, Yang in view of Anderson in view of Besenbruch discloses all the limitations of claim 1.
Yang in view of Anderson in view of Besenbruch does not appear to explicitly disclose:
wherein the generating the computer vision task result comprises:
converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor;
inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and
generating, by the second neural network, the computer vision task result according to the compressed image and the tensor.
However, Wang discloses:
wherein the generating the computer vision task result comprises:
converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor; [See Wang, ¶ 0406, 0407, 0412 discloses a convolutional neural network forming tensors from low and high resolution image parameters.]
inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and [See Wang, ¶ 0072 discloses the method is performed at a first network node. Furthermore, optionally the hierarchical algorithm may be transmitted to a second network node.]
generating, by the second neural network, the computer vision task result according to the compressed image and the tensor. [See Wang, ¶ 0406 discloses The task of the example based model, in some embodiments a single image super resolution network, is to estimate a high resolution image given a low resolution image that is downscaled or downsampled from a corresponding high resolution image. Further, that estimation can be performed, for example by training a classifier or some visual comparison, and optionally this could be parametised for the super resolution process in some embodiments. In general, both the low and high resolution image have C colour channels, thus can be represented as real-valued tensors of size H×W×C and rH×rW×C respectively. In some embodiments, rather than recovering an high resolution image from an upscaled and interpolated version of the low resolution image, instead the low resolution image is used by a three layer convolutional network to avoid upscaling before the low-resolution image is fed into the network. In the network, a sub-pixel convolution layer is applied to upscale the low resolution feature maps to produce a high resolution image using super resolution.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang in view of Anderson in view of Besenbruch to add the teachings of Wang in order to introduce non-linearity to a model and allow the neural network to learn and represent complex patterns.
Regarding claim 3, Yang in view of Anderson in view of Besenbruch in view of Wang discloses all the limitations of claim 2.
Anderson discloses:
wherein the first neural network comprises one or more convolution layers. [See Anderson, ¶ 0098 discloses some components of the compression model 500 may be configured as a convolutional neural network model (CNN) that include a convolutional filter having weights repeatedly applied to a moving window of pixels of an input tensor. In one instance, the convolutional layer may include a set of convolutional filters each corresponding to a respective bitrate level, and different regions of the input tensor may be applied with different convolutional filters depending on the value of the encoding parameter for the region in the parameter map.]
The reasons to combine the cited prior art are applicable to those presented for previously rejected claim 1.
Regarding claim 4, Yang in view of Anderson in view of Besenbruch in view of Wang discloses all the limitations of claim 2.
Although Anderson discloses the use of a compression model configured as a convolutional neural network in combination with a parameter map for specifying multiple rate encoding schemes, Yang in view of Anderson does not appear to explicitly disclose:
wherein the first neural network comprises a convolution layer with an activation function.
However, Wang discloses:
wherein the first neural network comprises a convolution layer with an activation function. [See Wang, ¶ 0199, 0412, 0417 discloses that as a part of a convolutional neural network-based encoding, an activation function can optionally be employed.]
The reasons to combine the cited prior art are applicable to those presented for previously rejected claim 2.
Regarding claim 5, Yang in view of Anderson in view of Besenbruch in view of Wang discloses all the limitations of claim 2.
Anderson discloses:
wherein the second neural network is configured to generate the computer vision task result without generating a reconstructed image from the compressed image. [See Anderson, ¶ 0090-0091 discloses at least a part of the information processed by a compression model (e.g., any inputs or outputs of the compression model 200 of FIG. 2, compression model 500 of FIG. 5, or compression model 600 of FIG. 6) is represented as state information that captures and propagates information used by a neural network component to perform an assigned task. The state information for a neural network component may be uninterpretable in the sense that it can be processed by the neural network component to store and retrieve any type of information necessary for performing the task, rather than having a specific, interpretable meaning on the information.]
The reasons to combine the cited prior art are applicable to those presented for previously rejected claim 1.
Regarding claim 6, Yang in view of Anderson in view of Besenbruch in view of Wang discloses all the limitations of claim 2.
Anderson discloses:
wherein the second neural network is configured to generate a reconstructed image from the compressed image, and generate the computer vision task result from the reconstructed image. [See Anderson, ¶ 0116 discloses a B-frame may be encoded and decoded in a hierarchical manner. For example, a B-frame may be encoded and decoded as a B1-frame, a B2-frame, or a B3-frame. A B1-frame is encoded and decoded using previously reconstructed frames that were reconstructed as an I-frame and/or a P-frame, a B2-frame is encoded and decoded using previously reconstructed frames that include one frame reconstructed as an I-frame or P-frame and another frame reconstructed as a B1-frame, and a B3-frame is encoded and decoded using previously reconstructed frames that were reconstructed as a B1-frame and a B2-frame.]
The reasons to combine the cited prior art are applicable to those presented for previously rejected claim 1.
Regarding claim 7, Yang in view of Anderson in view of Besenbruch discloses all the limitations of claim 1.
Wang discloses:
wherein the neural network based encoder is based on an encoder model in a neural image compression (NIC) framework, and the multi-rate compression domain computer vision task decoder is based on a decoder model in the NIC framework, the NIC framework is trained end-to-end. [See Wang, ¶ 0594-0595 discloses a neural network training process as a full end-to-end system encapsulating how visual data being provided to the encoder 2830 will be decoded by the decoder 2850.]
The reasons to combine the cited prior art are applicable to those presented for previously rejected claim 2.
Regarding claim 8, Yang in view of Anderson in view of Besenbruch discloses all the limitations of claim 1.
Wang discloses:
wherein a decoder model of the multi-rate compression domain computer vision task decoder is trained separately from an encoder model of the neural network based encoder. [See Wang, ¶ 0594-0595 discloses a neural network training process as a full end-to-end system encapsulating how visual data being provided to the encoder 2830 will be decoded by the decoder 2850.  (Examiner’s note: it is within the level of ordinary skill to understand that “end-to-end” routinely and conventionally corresponds with training being separated, which is further evidenced by WO2021/220008A1, which notes on pg. 83, section 1.2.2 “Training of the segmentation module, if it is parameterized as a neural network, may be performed separately or during the training of the compression network itself – referred to as end-to-end.)]
The reasons to combine the cited prior art are applicable to those presented for previously rejected claim 2.
Regarding claim 12, this claim recites analogous limitations to claim 2, and is therefore rejected on the same premise.
Regarding claim 13, this claim recites analogous limitations to claim 3, and is therefore rejected on the same premise.
Regarding claim 14, this claim recites analogous limitations to claim 4, and is therefore rejected on the same premise.
Regarding claim 17, this claim recites analogous limitations to claim 7, and is therefore rejected on the same premise.
Regarding claim 18, this claim recites analogous limitations to claim 8, and is therefore rejected on the same premise.
Claim(s) 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Anderson in view of Besenbruch in view of Li et al. (US 20200280717 A1) (hereinafter Li).
Regarding claim 9, Yang in view of Anderson in view of Besenbruch discloses all the limitations of claim 1.
Yang in view of Anderson does not appear to explicitly disclose:
wherein the parameter is a hyperparameter.
However, Li discloses:
wherein the parameter is a hyperparameter. [See Li, ¶ 0115, 0181 discloses in addition to adjusting the number of features on each convolution layer, the architecture of FIGS. 9 and 12, and their variations described herein can be designed so that a global feature adjustment rate r may be applied across all layers for reducing/expanding the number of features. This feature adjustment rate can be a configurable hyper-parameter. After applying the feature adjustment rate, the number of features can be reduced/expended from n to n*r.]
It would have been obvious to the person having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention disclosed by Yang in view of Anderson in view of Besenbruch to add the teachings of Li in order to identify quantization parameters that can be used to identify the tradeoff between rate and distortion.]
Regarding claim 19, this claim recites analogous limitations to claim 9, and is therefore rejected on the same premise.
Conclusion
THIS ACTION IS MADE FINAL.  Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action.  In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action.  In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.

APPLICANT REBUTTAL: REMARKS Favorable reconsideration of this application, as presently amended and in light of the following discussion, is respectfully requested. Claims 1-14, 17-19, and 21 are currently pending. Claims 1, 6, 8, 9, 11, 12, and 17-19 have been amended; Claims 15-16 and 20 have been canceled without prejudice or disclaimer; and Claim 21 has been added by the present Amendment. The changes and additions to the claims are supported by the originally filed specification and claims.1 Thus, no new matter has been added. In the outstanding Office Action, Claims 1 and 11 were objected to due to certain informalities; Claims 1, 10-11, and 20 were rejected under 35 U.S.C. § 103 as being unpatentable over U.S. Publication No. 2020/0250003 (hereinafter "Yang") in view of U.S. Publication No. 2022/0224914 (hereinafter "Anderson");2 Claims 2-8 and 12-18 were rejected under 35 U.S.C. § 103 as being unpatentable over Yang in view of Anderson and U.S. Publication No. 2020/0280730 (hereinafter "Wang"); and Claims 9 and 19 were rejected under 35 U.S.C. § 103 as being unpatentable over Yang in view of Anderson and U.S. Publication No. 2020/0280717 (hereinafter "Li"). Substance of Interview Applicant and Applicant's representative thank Examiner Demosky for the courtesy of the interview with Applicant's representative on September 17, 2024, at which time proposed amendments to Claim 1 were discussed. During the interview, the examiner indicated that the proposed amendments appear to distinguish over the applied reference but that further search and consideration would be required. Amendments based on the discussion during the interview are presented herewith. 1 See, e.g., [0052] and Figures. 14 and 16 and the discussion related thereto in the originally filed specification. 2 The Office Action appears to have inadvertently listed Claim 12 instead of Claim 20. Claim Objection Regarding the objections to Claims 1 and 11, Claims 1 and 11 have been amended to address the informalities noted in the Office Action. Accordingly, it is respectfully requested that the objections to the claims be withdrawn. Rejection under 35 U.S.C. Q 103 Independent Claim 1 is directed to a method for image processing, comprising: decoding, from a coded bitstream that carries a compressed image, an index that points to a value in a set of values of a parameter, a value change of the parameter adjusting a compression rate of the compressed image, the compressed image being generated by a neural network based encoder based on the parameter, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss; inputting the value of the parameter to a multi-rate compression domain computer vision task decoder in a compression domain computer vision task framework (CDCVTF), the multi-rate compression domain computer vision task decoder comprising one or more neural networks for performing a computer vision task from compressed images according to corresponding values of the parameter that are used for generating the compressed images at multiple different compression rates; and generating, by the multi-rate compression domain computer vision task decoder, a computer vision task result according to the compressed image in the coded bitstream which is compressed at a corresponding compression rate from the multiple different compression rates based on the value of the parameter. (Emphasis added) Regarding the rejection of Claim 1 under 35 U.S.C. § 103, it is respectfully submitted that Yang and Anderson fail to disclose "a value change of the parameter adjusting a compression rate of the compressed image, the compressed image being generated by a neural network based encoder based on the parameter, the parameter weighting a distortion and a rate in a calculation of a rate-distortion (R-D) loss," as discussed during the interview. Accordingly, it is respectfully submitted that Claim 1 (and all associated dependent claims) patentably defines over Yang and Anderson. Independent Claim 11, although differing in scope and statutory class, patentably defines over Yang and Anderson at least for reasons analogous to the reasons stated above for the patentability of Claim 1. Accordingly, it is respectfully submitted that Claim 11 (and all associated dependent claims) patentably defines over Yang and Anderson. Regarding the rejections of Claims 2-9 and 12-19 under 35 U.S.C. § 103, it is respectfully submitted that Claims 2-9 and 12-19 patentably define over the applied references at least for the reasons stated above for the patentability of Claims 1 and 11, from which Claims 2-9 and 12-19 depend, respectively. Further, the Office Action does not cite Wang and Li for such teachings. Accordingly, it is respectfully submitted that Claims 2-9 and 12-19 patentably define over the applied references. Newly Added Claim The present Amendment adds new Claim 21 for examination on the merits. No new matter has been added. It is respectfully submitted that Claim 21 patentably defines over Yang and Anderson at least for reasons analogous to the reasons stated above for the patentability of Claim 1. Conclusion If for any reason the examiner feels the application is not now in condition for allowance, it is respectfully requested that the examiner contact the undersigned practitioner to arrange for an interview to expedite the disposition of this case. If this communication is filed after the shortened statutory time period has elapsed and no separate Petition is enclosed, Applicant hereby petitions, under 37 C.F.R. § 1.136(a), to extend the time for filing a response to the outstanding Office Action by the number of months necessary to avoid abandonment under 37 C.F.R. § 1.135. Please charge any such fee, or any other fee due under 37 C.F.R. § 1.17, to our Deposit Account No. 01-2300, under 043380.01224, from which the undersigned is authorized to draw. Customer No.: 4372 Respectfully submitted, Atty Direct Line: 202.350.3626 SVG {EMBED HERE} 0.733.197 Peifang.Tian@afslaw.com Johnny Ma Registration No.: 59,976

NEXT OFFICE ACTION: DETAILED ACTION
Notice of Pre-AIA  or AIA  Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Allowable Subject Matter
Claims 1, 3-11, 13-14, 17-19, and 21 are allowed.
The following is an examiner’s statement of reasons for allowance:
The closest prior art of record determined from an Examiner’s search is Anderson et al. (US 20220224914 A1) (hereinafter Anderson).  Anderson discloses that a parameter map encodes whether a target frame is processed as a P-frame, a B1-frame, a B2-frame, or a B3-frame using a compression model.  The ability to have multiple rates allows encoding and decoding of different B-frame types (e.g., B1, B2, B3) with a single B-frame compression model that takes as input the rate and frame type in the parameter map.
Additional prior art Besenbruch (US 20230154055 A1) (hereinafter Besenbruch) discloses a loss used for neural network training, being L=d + (lambda)*R, where D is a distortion function, (lambda) is a weighting factor, and R is the rate loss.  R is related to entropy, and both D and R are differentiable functions.
However, the cited prior art fails to teach or render obvious the following combination of elements of independent claims 1, 11, and 21: “generating the computer vision task result includes: converting, by a first neural network in the multi-rate compression domain computer vision task decoder, the value of the parameter to a tensor; inputting the tensor to one or more layers in a second neural network in the multi-rate compression domain computer vision task decoder; and generating, by the second neural network, the computer vision task result according to the compressed image and the tensor.”
Thus, Applicant’s amendments have been considered persuasive, in light of the claim limitations as well as the enabling portions of the specification. The cited prior art found as a result of an exhaustive search cannot reasonably be combined to arrive at the level of specificity of the instant claim language. Therefore, the various claimed limitations mentioned in the claims are not taught or suggested by the prior art taken either singly or in combination, with emphasis that it is each claim, taken as a whole, including the interrelationships and interconnections between various claimed elements that make them allowable over the prior art.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee.  Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion

CITED REFERENCE SUMMARIES: Ref US20230154055A1: Claims:
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system analyzing the quantized latent to produce parameters; (viii) the second computer system using the produced parameters to modify weights of a second trained neural network; (ix) the second computer system using the second trained neural network including the modified weights to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
(i) receiving an input image at a first computer system;
(ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation;
(iii) quantizing the latent representation using the first computer system to produce a quantized latent;
(iv) entropy encoding the quantized latent into a bitstream, using the first computer system;
(v) transmitting the bitstream to a second computer system;
(vi) the second computer system entropy decoding the bitstream to produce the quantized latent;
(vii) the second computer system analyzing the quantized latent to produce parameters;
(viii) the second computer system using the produced parameters to modify weights of a second trained neural network;
(ix) the second computer system using the second trained neural network including the modified weights to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
The method of claim 1 , wherein the parameters are a discrete perturbation of the weights of the second trained neural network.
The method of claim 1 , wherein the weights of the second trained neural network are perturbed by a perturbation function that is a function of the parameters, using the parameters in the perturbation function.
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) the first computer system optimizing a binary mask using the quantized latent; (v) entropy encoding the quantized latent and the binary mask into a bitstream, using the first computer system; (vi) transmitting the bitstream to a second computer system; (vii) the second computer system entropy decoding the bitstream to produce the quantized latent, and to produce the binary mask; (viii) the second computer system using the binary mask to modify a convolutional network of a second trained neural network; (ix) the second computer system using the second trained neural network including the modified a convolutional network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
(iv) the first computer system optimizing a binary mask using the quantized latent;
(v) entropy encoding the quantized latent and the binary mask into a bitstream, using the first computer system;
(vi) transmitting the bitstream to a second computer system;
(vii) the second computer system entropy decoding the bitstream to produce the quantized latent, and to produce the binary mask;
(viii) the second computer system using the binary mask to modify a convolutional network of a second trained neural network;
(ix) the second computer system using the second trained neural network including the modified a convolutional network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image.
The method of claim 4 , wherein in step (iv), the binary mask is optimized using a ranking based method, or using a stochastic method, or using a sparsity regularization method.
A computer-implemented method for lossy image or video compression, transmission and decoding, the method including the steps of: (i) receiving an input image at a first computer system; (ii) encoding the input image using a first trained neural network, using the first computer system, to produce a latent representation; (iii) quantizing the latent representation using the first computer system to produce a quantized latent; (iv) entropy encoding the quantized latent into a bitstream, using the first computer system; (v) transmitting the bitstream to a second computer system; (vi) the second computer system entropy decoding the bitstream to produce the quantized latent; (vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image; wherein one of the input image, latent representation or quantized latent is, in a loop, modified so as to progressively reduce a finetuning loss after step (i), (ii) or (iii) respectively.
(vii) the second computer system using a second trained neural network to produce an output image from the quantized latent, wherein the output image is an approximation of the input image; wherein one of the input image, latent representation or quantized latent is, in a loop, modified so as to progressively reduce a finetuning loss after step (i), (ii) or (iii) respectively.
wherein one of the input image, latent representation or quantized latent is, in a loop, modified so as to progressively reduce a finetuning loss after step (i), (ii) or (iii) respectively.
The method of claim 6 , wherein the finetuning loss measures one of, or a combination of: a rate of the modified quantized latent, or a distortion between the current decoder prediction of the output image and the input image, or a distortion between the current decoder prediction of the output image and a decoder prediction of the output image using the quantized latent from step (iii).
The method of claim 6 , wherein the loop ends when the modified input image, latent or quantized latent satisfies an optimization criterion.
The method of any of claim 6 , wherein in the loop, the input image, latent or quantized latent is modified using a 1st-order optimization method, or using a 2nd-order optimization method, or using Monte-Carlo, Metropolis-Hastings, simulated annealing, or other greedy approaches.
Ref US20200250003A1: Claims:
- 8 . (canceled)
A processing device for scheduling workloads of a visual computing application on a distributed computing infrastructure, comprising: interface circuitry to communicate over a network; and processing circuitry to: receive, via the interface circuitry, a request to perform a workload for the visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within the distributed computing infrastructure; receive, via the interface circuitry, resource telemetry data indicating an available resource capacity for the set of compute resources; identify a collection of vision capability implementations available for performing the workload, wherein the vision capability implementations are for performing one or more vision capabilities of the workload on a plurality of resource types corresponding to the set of compute resources; determine, based on the resource telemetry data and the collection of vision capability implementations, a workload schedule for the workload, wherein the workload schedule comprises a mapping of the workload onto the set of compute resources on the plurality of devices within the distributed computing infrastructure; and send, via the interface circuitry, the workload schedule to the plurality of devices, wherein the workload is to be performed by the set of compute resources on the plurality of devices based on the workload schedule.
interface circuitry to communicate over a network; and
processing circuitry to: receive, via the interface circuitry, a request to perform a workload for the visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within the distributed computing infrastructure; receive, via the interface circuitry, resource telemetry data indicating an available resource capacity for the set of compute resources; identify a collection of vision capability implementations available for performing the workload, wherein the vision capability implementations are for performing one or more vision capabilities of the workload on a plurality of resource types corresponding to the set of compute resources; determine, based on the resource telemetry data and the collection of vision capability implementations, a workload schedule for the workload, wherein the workload schedule comprises a mapping of the workload onto the set of compute resources on the plurality of devices within the distributed computing infrastructure; and send, via the interface circuitry, the workload schedule to the plurality of devices, wherein the workload is to be performed by the set of compute resources on the plurality of devices based on the workload schedule.
receive, via the interface circuitry, a request to perform a workload for the visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within the distributed computing infrastructure;
receive, via the interface circuitry, resource telemetry data indicating an available resource capacity for the set of compute resources;
identify a collection of vision capability implementations available for performing the workload, wherein the vision capability implementations are for performing one or more vision capabilities of the workload on a plurality of resource types corresponding to the set of compute resources;
determine, based on the resource telemetry data and the collection of vision capability implementations, a workload schedule for the workload, wherein the workload schedule comprises a mapping of the workload onto the set of compute resources on the plurality of devices within the distributed computing infrastructure; and
send, via the interface circuitry, the workload schedule to the plurality of devices, wherein the workload is to be performed by the set of compute resources on the plurality of devices based on the workload schedule.
The processing device of claim 9 , wherein the processing circuitry to identify the collection of vision capability implementations available for performing the workload is further to: receive, via the interface circuitry, vision capability data from a vision capability repository, wherein the vision capability repository stores a plurality of vision capability implementations for a plurality of vision capabilities, and wherein the vision capability data indicates the collection of vision capability implementations that are available for performing the workload.
receive, via the interface circuitry, vision capability data from a vision capability repository, wherein the vision capability repository stores a plurality of vision capability implementations for a plurality of vision capabilities, and wherein the vision capability data indicates the collection of vision capability implementations that are available for performing the workload.
The processing device of claim 9 , wherein the one or more vision capabilities of the workload comprise: a facial recognition capability; an object detection capability; or a gesture recognition capability.
a facial recognition capability;
an object detection capability; or
a gesture recognition capability.
The processing device of claim 9 , wherein: the one or more vision capabilities of the workload comprise a first vision capability and a second vision capability; and the collection of vision capability implementations comprises: a first set of vision capability implementations for performing the first vision capability on the plurality of resource types; and a second set of vision capability implementations for performing the second vision capability on the plurality of resource types.
the one or more vision capabilities of the workload comprise a first vision capability and a second vision capability; and
the collection of vision capability implementations comprises: a first set of vision capability implementations for performing the first vision capability on the plurality of resource types; and a second set of vision capability implementations for performing the second vision capability on the plurality of resource types.
a first set of vision capability implementations for performing the first vision capability on the plurality of resource types; and
a second set of vision capability implementations for performing the second vision capability on the plurality of resource types.
The processing device of claim 12 , wherein the first set of vision capability implementations comprises: a first set of software implementations of the first vision capability for a first resource type from the plurality of resource types; and a second set of software implementations of the first vision capability for a second resource type from the plurality of resource types.
a first set of software implementations of the first vision capability for a first resource type from the plurality of resource types; and
a second set of software implementations of the first vision capability for a second resource type from the plurality of resource types.
The processing device of claim 9 , wherein the plurality of resource types comprises central processing units, graphics processing units, and special-purpose accelerators.
The processing device of claim 14 , wherein the special-purpose accelerators comprise hardware video coders and neural network processors.
The processing device of claim 14 , wherein the special-purpose accelerators comprise field-programmable gate array (FPGA) accelerators and application-specific integrated circuit (ASIC) accelerators.
At least one non-transitory machine-readable storage medium having instructions stored thereon, wherein the instructions, when deployed and executed on processing circuitry, cause the processing circuitry to: receive, via interface circuitry, a request to perform a workload for a visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within a distributed computing infrastructure; receive, via the interface circuitry, resource telemetry data indicating an available resource capacity for the set of compute resources; identify a collection of vision capability implementations available for performing the workload, wherein the vision capability implementations are for performing one or more vision capabilities of the workload on a plurality of resource types corresponding to the set of compute resources; determine, based on the resource telemetry data and the collection of vision capability implementations, a workload schedule for the workload, wherein the workload schedule comprises a mapping of the workload onto the set of compute resources on the plurality of devices within the distributed computing infrastructure; and send, via the interface circuitry, the workload schedule to the plurality of devices, wherein the workload is to be performed by the set of compute resources on the plurality of devices based on the workload schedule.
receive, via interface circuitry, a request to perform a workload for a visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within a distributed computing infrastructure;
The storage medium of claim 17 , wherein the instructions that cause the processing circuitry to identify the collection of vision capability implementations available for performing the workload further cause the processing circuitry to: receive, via the interface circuitry, vision capability data from a vision capability repository, wherein the vision capability repository stores a plurality of vision capability implementations for a plurality of vision capabilities, and wherein the vision capability data indicates the collection of vision capability implementations that are available for performing the workload.
The storage medium of claim 17 , wherein the one or more vision capabilities of the workload comprise: a facial recognition capability; an object detection capability; or a gesture recognition capability.
The storage medium of claim 17 , wherein: the one or more vision capabilities of the workload comprise a first vision capability and a second vision capability; and the collection of vision capability implementations comprises: a first set of vision capability implementations for performing the first vision capability on the plurality of resource types; and a second set of vision capability implementations for performing the second vision capability on the plurality of resource types.
The storage medium of claim 20 , wherein the first set of vision capability implementations comprises: a first set of software implementations of the first vision capability for a first resource type from the plurality of resource types; and a second set of software implementations of the first vision capability for a second resource type from the plurality of resource types.
The storage medium of claim 17 , wherein the plurality of resource types comprises central processing units, graphics processing units, and special-purpose accelerators.
The storage medium of claim 22 , wherein the special-purpose accelerators comprise hardware video coders and neural network processors.
A method for scheduling workloads of a visual computing application on a distributed computing infrastructure, comprising: receiving, via interface circuitry, a request to perform a workload for the visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within the distributed computing infrastructure; receiving, via the interface circuitry, resource telemetry data indicating an available resource capacity for the set of compute resources; identifying a collection of vision capability implementations available for performing the workload, wherein the vision capability implementations are for performing one or more vision capabilities of the workload on a plurality of resource types corresponding to the set of compute resources; determining, based on the resource telemetry data and the collection of vision capability implementations, a workload schedule for the workload, wherein the workload schedule comprises a mapping of the workload onto the set of compute resources on the plurality of devices within the distributed computing infrastructure; and sending, via the interface circuitry, the workload schedule to the plurality of devices, wherein the workload is to be performed by the set of compute resources on the plurality of devices based on the workload schedule.
receiving, via interface circuitry, a request to perform a workload for the visual computing application, wherein the workload is to be scheduled across a set of compute resources on a plurality of devices within the distributed computing infrastructure;
receiving, via the interface circuitry, re
[Truncated to input limit]