Patent ID: 11893502
Assignee: ADVANCED MICRO DEVICES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A method for distributing an expert of a mixture-of-experts system for execution in a set of hardware devices, the method comprising:
executing experts of the mixture-of-experts system on the set of hardware devices using one or more items of parameter data associated with a plurality of execution parameters to obtain a set of priority data for different combinations of the execution parameters and the experts, the set of priority data ranking hardware devices of the set of hardware devices for executing each expert of the mixture-of-experts system for each execution parameter of the plurality of execution parameters, wherein the plurality of execution parameters includes execution speed, throughput, latency, and power consumption;
identifying one execution parameter of the plurality of execution parameters to act as a basis for selecting a hardware device, of the set of hardware devices, on which to execute the expert, the expert being a component of a mixture-of-experts machine learning model;
retrieving, from the set of priority data, priority data for a combination of the expert and the one execution parameter;
identifying a hardware device of the set of hardware devices based on the priority data, the identifying including identifying a highest ranked hardware device, for the combination of the expert and the one execution parameter, that is available for processing the expert; and
dispatching the expert to the identified hardware device for execution.