Patent ID: 11915139
Assignee: GOOGLE LLC
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A machine learning computations system, comprising:
a data processing apparatus; and
a memory storage apparatus in data communication with the data processing apparatus, the memory storage apparatus storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising:
receiving data of a machine learning model, the data representing operations of the machine learning model;
receiving data specifying characteristics of a memory hierarchy for one or more machine learning processors on which the machine learning model is going to be deployed, the memory hierarchy including multiple memories for storing machine learning data used by the one or more machine learning processors when performing machine learning computations using the machine learning model, the characteristics including a data storage capacity of each memory and a memory bandwidth of each memory, wherein at least one of the memories has a different memory bandwidth than at least one other memory;
generating, based on the data of the machine learning model and the characteristics of the memory hierarchy, an updated machine learning model, the generating comprising:
determining that output data of a given operation of the machine learning model should be stored in a highest bandwidth memory of the multiple memories based on the machine learning model;
determining that the output data of the given operation has a data size that is larger than a data storage capacity of the highest bandwidth memory; and
in response to determining that the output data of the given operation has the data size that is larger than the data storage capacity of the highest bandwidth memory adding, to the updated machine learning model, one or more operations for splitting the output data into multiple portions of output data such that each portion of output data has a data size that is less than or equal to the data storage capacity of the highest bandwidth memory; and

performing machine learning computations using the updated machine learning model.