Patent ID: 11880726
Assignee: AMAZON TECHNOLOGIES, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 3:
4. A method comprising:
receiving a request to execute a generative operation;
executing the generative operation over a time period to yield an operation result;
sending the operation result;
during the time period, dequeuing, from a first queue, and executing each request task of a first set of request tasks spawned by executing the generative operation;
during the time period, determining a request cost index reflecting an amount of computing resources used to execute the first set of request tasks;
during the time period, determining, based on the request cost index, to queue, in a second queue, subsequent request tasks spawned by executing the generative operation;
during the time period, queuing, in the second queue, a second set of request tasks spawned by executing the generative operation; and
during the time period, dequeuing, from the second queue, and executing each request task of the second set of request tasks.