Patent ID: 11899551
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A system, comprising:
at least one processor;
a memory, storing program instructions that when executed by the at least one processor cause the at least one processor to submit one or more machine learning tasks for processing to an inference accelerator; and
wherein the inference accelerator comprises an integrated circuit, wherein the integrated circuit comprises a software-based activity monitor implemented by firmware executing on a first portion of the integrated circuit, wherein the integrated circuit further comprises a tensor processing unit that comprises a hardware-based activity monitor implemented on a second portion of the integrated circuit and a systolic array implemented on a third portion of the integrated circuit that is configured to perform the one or more machine learning tasks; and
wherein the software-based activity monitor is configured to:
monitor data obtained from a component of the integrated circuit;
determine a modification to throttling criteria used by the hardware-based activity monitor based on the monitored data; and
cause the modification to the throttling criteria used by the hardware-based activity monitor; and

wherein the hardware-based activity monitor is configured to:
monitor processing performed by the systolic array according to the modified throttling criteria; and
cause, according to the monitoring, a modification to a rate at which data enters the systolic array for processing.