Patent ID: 11893502
Assignee: ADVANCED MICRO DEVICES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 10:
11. A system, comprising:
a set of hardware devices; and
a system for distributing an expert of a mixture-of-experts system for execution in the set of hardware devices, the system comprising:
a data store configured to store a set of priority data; and
an orchestrator configured to:
execute experts of the mixture-of-experts system on the set of hardware devices using one or more items of parameter data associated with a plurality of execution parameters to obtain the priority data for different combinations of the execution parameters and the experts, the set of priority data ranking hardware devices of the set of hardware devices for executing each expert of the mixture-of-experts system for each execution parameter of the plurality of execution parameters, wherein the plurality of execution parameters includes of execution speed, throughput, latency, and power consumption;
identify one execution parameter of the plurality of execution parameters to act as a basis for selecting a hardware device, of the set of hardware devices, on which to execute the expert, the expert being a component of a mixture-of-experts machine learning model;
retrieve, from the set of priority data stored in the data store, priority for a combination of the expert and the one execution parameter;
identify a hardware device of the set of hardware devices based on the priority data, the identifying including identifying a highest ranked hardware device, for the combination of the expert and the one execution parameter, that is available for processing the expert; and
dispatch the expert to the identified hardware device for execution.