Patent ID: 11886919
Assignee: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A non-transitory machine readable medium comprising instructions that when executed cause a system to:
receive queries at an Application Programming Interface (API) gateway of a cluster of a container orchestration platform, wherein the cluster includes a host system comprising a first processing resource and one or more hardware accelerators comprising a second processing resource, wherein the host system comprising the first processing resource or a hardware accelerator of the one or more hardware accelerators comprising the second processing resource comprises a primary node of the cluster, and wherein the host system and the one or more hardware accelerators comprise a plurality of worker nodes of the cluster;
determine, by the API gateway, a distribution of the queries among the plurality of worker nodes of the host system and the one or more hardware accelerators based on a queuing model that takes into consideration respective compute capacities of the host system and the one or more hardware accelerators;
perform auto-scaling to run a quantity of instances of an application in the plurality of worker nodes; and, responsive to receipt of the queries, direct the queries to the plurality of worker nodes according to the distribution for processing, by the plurality of worker nodes, the quantity of instances of the application.