Patent ID: 11960935
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 7:
8. A computer-implemented method, comprising:
attaching a first set of one or more accelerator slots of an accelerator appliance to an application instance according to an application instance configuration, the attached application instance attached to the accelerator appliance in a multi-tenant provider network, the accelerator appliance comprising a plurality of accelerators, the plurality of accelerators having a compute capacity, each accelerator slot of the first set of one or more accelerator slots corresponding to a fraction of the compute capacity of the plurality of accelerators;
loading a machine learning model of the attached application instance onto the first set of one or more accelerator slots;
handling a first set of one or more inference calls, made by an application of the attached application instance to the loaded machine learning model, using the first set of one or more accelerator slots; and
based on determining that the first set of one or more accelerator slots no longer meet at least one of a timing requirement or a cost requirement, migrating processing for the attached application instance from the first set of one or more accelerator slots to a second set of one or more accelerator slots including detaching the first set of one or more accelerator slots from the application instance, attaching the second set of one or more accelerator slots to the application instance, and loading the machine learning model of the attached application instance onto the second set of one or more accelerator slots;
handling a second set of inference calls, made by the application of the attached application instance to the loaded machine learning model, using the second set of one or more accelerator slots; and
returning, to the application of the application instance, a result of the second set of one or more inference calls.