Capacity resolver for point of presence (POP) systems

A capacity resolver system for provisioning and management of nodes at point of presence (POP) in a cloud-based multi-tenant system. The capacity resolver system includes a plurality of POPs and a cloud orchestration server. The POPs include hypervisors that include a plurality of nodes. The cloud orchestration receives a request for provisioning a node. The request is provisioned at the POP based on parameters from the hypervisors of the POP. The parameters include Central Processing Unit (CPU) Core utilization, memory utilization, disk utilization and Virtual File System (VFS) availability of the node. A triggering of one or more parameters above their respective threshold values is determined at the POP. Nodes are identified for downsizing or migration based on the triggering of the one or more parameters. The node is provisioned at the hypervisor of the POP in accordance with a priority for the downsizing or the migration of the nodes.

BACKGROUND

This disclosure relates in general to capacity management at point of presence (POP) systems and, not by way of limitation, to provisioning virtual machines at the POP systems based on one or more utilization parameters, among other things.

Data centers or POPs often face problems related to over-provisioning of virtual machines (VMs) which can cause a risk to its operations. Resolving requests for placing new nodes at existing data centers can be burdensome and slow. Further, VM configurations at new data centers are often not optimal. This leads to error-prone and non-deterministic handling of requests at the POPs.

Solutions for capacity management and scheduling are available. However, the complex environment of the data centers demands an ad hoc solution. Previously, methods for resolution of capacity-related problems were handled in a semi-manual process taking into account provisioning metrics, utilization, failover considerations, and dozens of boundary conditions.

Moreover, efficiently allocating virtual machines has become a key cost-saving challenge. In order to increase overall controller and memory utilization, a more optimal and sustainable VM allocation strategy entails to be built. This entails data sets that represent accurate, updated and comprehensive metrics associated with VM provisioning and utilization.

SUMMARY

In one embodiment, the present disclosure provides a capacity resolver system for provisioning and management of nodes at point of presence (POP) in a cloud-based multi-tenant system. The capacity resolver system includes a plurality of POPs and a cloud orchestration server. The POPs include hypervisors that include a plurality of nodes. The cloud orchestration receives a request for provisioning a node. The request is provisioned at the POP based on parameters from the hypervisors of the POP. The parameters include provisioning availability and utilization of Central Processing Unit (CPU) Cores and memory in addition to disk availability and Virtual File System (VFS) availability of the node. A triggering of one or more parameters above their respective threshold values is determined at the POP. Nodes are identified for downsizing or migration based on the triggering of one or more parameters. The node is provisioned at the hypervisor of the POP in accordance with a priority for the downsizing or the migration of the nodes.

In an embodiment, a capacity resolver system for provisioning and management of nodes at point of presence (POP) systems in a cloud-based multi-tenant system. The capacity resolver system includes a plurality of POPs and a cloud orchestration server. The plurality of POPs includes a plurality of hypervisors. The plurality of hypervisors includes a plurality of nodes. The cloud orchestration server is configured to receive a request for provisioning a node in a POP of the plurality of POPs. The request is provisioned at the POP based on a plurality of parameters. The cloud orchestration server receives the plurality of parameters from the plurality of hypervisors of the POP. The plurality of parameters includes Central Processing Unit (CPU) Core utilization, memory utilization, disk utilization and Virtual File System (VFS) availability of the plurality of nodes of a hypervisor. A determination is made if there is a triggering of one or more parameters from the plurality of parameters at the POP. The determination of the triggering of the one or more parameters includes a determination of the one or more parameters that are above respective threshold values. A condition for downsizing or migration of one or more nodes of the plurality of nodes based on the triggering of the one or more parameters is identified. A priority for the downsizing or the migration of the one or more nodes based on the condition is identified. The node is provisioned at the hypervisor of the POP based on the downsizing or the migration of the one or more nodes in accordance with the priority.

In another embodiment, a method for capacity management of nodes at point of presences (POPs) in a cloud-based multi-tenant system. A cloud orchestration server is configured to receive a request for provisioning a node in a POP of a plurality of POPs. The request is provisioned at the POP based on a plurality of parameters. The cloud orchestration server receives the plurality of parameters from the plurality of hypervisors of the POP. The plurality of parameters includes Central Processing Unit (CPU) Core utilization, memory utilization, disk utilization and Virtual File System (VFS) availability of the plurality of nodes of a hypervisor. A determination is made if there is a triggering of one or more parameters from the plurality of parameters at the POP. The determination of the triggering of the one or more parameters includes a determination of the one or more parameters that are above respective threshold values. A condition for downsizing or migration of one or more nodes of the plurality of nodes based on the triggering of the one or more parameters is identified. A priority for the downsizing or the migration of the one or more nodes based on the condition is identified. The node is provisioned at the hypervisor of the POP based on the downsizing or the migration of the one or more nodes in accordance with the priority.

In yet another embodiment, a capacity resolver system for managing capacity of nodes at point of presence (POPs) systems based on boundary conditions, the capacity resolver system comprising a plurality of servers, collectively having code for:receiving a request at a cloud orchestration server for provisioning a node in a POP, wherein: the request is provisioned at the POP based on a plurality of parameters, the cloud orchestration server receives the plurality of parameters from a plurality of hypervisors of the POP, and the plurality of parameters includes Central Processing Unit (CPU) Core utilization, memory utilization, disk utilization and Virtual File System (VFS) availability of a plurality of nodes of a hypervisor;determining if there is a triggering of one or more parameters from the plurality of parameters at the POP, wherein determining triggering of the one or more parameters includes determining if the one or more parameters are above respective threshold values;identifying a condition for downsizing or migration of one or more nodes based on the triggering of the one or more parameters;identifying a priority for the downsizing or the migration of the one or more nodes based on the condition; andprovisioning the node at the hypervisor of the POP based on the downsizing or the migration of the one or more nodes in accordance with the priority.

DETAILED DESCRIPTION

Referring to theFIG.1A, a block diagram of a capacity resolver system100in a cloud-based multi-tenant system/environment is shown. The presence of a multi-tenant environment helps in handling security, quality of service compliance, service level agreement enforcement, service request metering, and other management activities relating to the capacity resolver system100. The capacity resolver system100includes an end-user device(s)102(102-1,102-2,102-3), Point of Presence (POP)104(104-1,104-2,104-3), a public network106, virtual machines108(108-1,108-2,108-3), services120, a cloud orchestration server124, and tenants126(126-1,126-2,126-3). The end-user device(s)102such as smartphones, tablets, PCs and any other computers communicate with the cloud orchestration server124using the public network106. The POP104remotely hosts the software environment that is secured. The POP104are data centers of enterprises. The end-user device(s)102runs on any popular operating system (OS) such as Windows™, iOS™, Android™, Linux, set-top box OSes, and Chromebook™ Third-party apps are applications running on the operating system of the end-user device(s)102. The POP104includes hypervisors (not shown) and the virtual machines108of the hypervisors. The virtual machines may also be referred to as nodes.

Respective POP104-1,104-2, and104-3are present at different geographical locations. The geographical locations can be located in different regions, states/countries. The respective POP104-1,104-2, and104-3are connected and can provide their status to one another. The status of the POP104can include an indicator that the POP104is overloaded, working properly/unavailable/offline/available, etc.

The end-user device(s)102and the POPs104are connected via the public network106. Further, individual POPs104communicate with one another using the public network106. The end-user device(s)102use content and processing for content sites, for example, websites, streaming content, etc., and the services120for example, SaaS tools, databases, cloud service providers, etc.

The capacity resolver system100receives requests for VM placement in the POPs104. VM represent virtual machine(s) (VMs). The capacity resolver100extracts the specifics of the request, reconciles with the existing VM configuration, and provides recommendations for VM placement(s) along with any VM migrations and/or downsizings that are necessary to create sufficient space in the POP104to accommodate the VM108.

Mechanisms for configuration improvement by the capacity resolver system100There are two types of concerns with existing VM configurations: KVMs have over-provisioned resources (CPU cores, RAM, disk, and/or VFS). KVMs have insufficient resource availability for migration or provisioning of new VMs. Correspondingly, there are two mechanisms by which the capacity resolver system100recommends improvements:

Service Downsizing

VM Migration

Referring toFIG.1B, a block diagram of an embodiment of the capacity resolver system100is shown. The capacity resolver system100allows multiple tenants in different domains to communicate with various cloud providers over the public internet106. The capacity resolver system100may be a multi-tenant cloud-based system or a single-tenant cloud-based system. The capacity resolver system100includes a plurality of servers. The capacity resolver system100allows multiple tenants/multi-tenant systems or enterprise(s)198to use the same network separated by domain or some other logical separation. Encryption, leased/encrypted tunnels, firewalls, and/or gateways can be used to keep the data from one enterprise(s)198separate from other enterprise(s)198. Individual end-user device(s)195of an end-user(s)122can communicate with the cloud orchestration server124for services and storage using the public internet106. The cloud orchestration server124provides multi-tenancy control, policies, and node provisioning and placement for individual domain data centers.

The capacity resolver system100may include a first computing environment150-1having end-user devices195-1for a first domain, a second computing environment150-2having end-user devices195-2for a second domain, and a third computing environment150-3having end-user devices195-3for a third domain. Individual domain communicates with its respective enterprise(s)198using a virtual private network (VPN)190over local area networks (LANs), wide area networks (WANs), and/or the public Internet106. Instead of the VPN190as an end-to-end path, tunneling (e.g., Internet Protocol in Internet Protocol (IP-in-IP), Generic Routing Encapsulation (GRE)), policy-based routing (PBR), Border Gateway Protocol (BGP)/Interior Gateway Protocol (IGP) route injection, or proxies could be used. The POP104or the data center provides site-to-site optimized connectivity for critical apps and traffic (especially voice/video). Cloud providers140for providing remote services may include public or private clouds including Web/Software as a service (SaaS), SASE gateway public/private data center, and voice/video connected to the cloud orchestration server124via VPN190. Enterprise(s)198are connected to the cloud orchestration server124using the VPN190. Some examples of the cloud provider(s)140include Amazon Web Services (AWS)®, Google Cloud Platform (GCP)®, and Microsoft Azure®. Some or all of the cloud provider(s)140may be different from each other, for example, the first cloud provider140-1may run Amazon Web Services (AWS)®, the second cloud provider140-2may run Google Cloud Platform (GCP)®, and the third cloud provider140-3may run Microsoft Azure®. Although three cloud provider(s)140are shown, any suitable number of cloud provider(s)140may be provided with some captive to a particular enterprise or otherwise not accessible to multiple domains.

Each of the cloud providers140may communicate with the public Internet using a secure connection. For example, the first cloud provider140-1may communicate with the public Internet106via the VPN190, the second cloud provider140-2may communicate with the public Internet106via a different VPN190, and the third cloud provider140-3may communicate with the public Internet106via yet another VPN190. Some embodiments could use leased connections or physically separated connections to segregate traffic. Although one VPN190is shown, it is to be understood that there are many VPNs to support different end-user devices, tenants, domains, etc.

A plurality of enterprise(s)198may also communicate with the public Internet106and the end-user devices195for their domain via VPNs190. Some examples of the enterprise(s)198may include corporations, educational facilities, governmental entities, and private consumers. Each enterprise may support one or more domains to logically separate its networks. The end-user devices195for each domain may include individual computers, tablets, servers, handhelds, and network infrastructure that are authorized to use computing resources of their respective enterprise(s)198. In an embodiment, the cloud orchestration server124may be present inside the POP104.

Further, the cloud orchestration server124may communicate with the public Internet106via the VPN190. The cloud orchestration server124also provides cloud access security broker (CASB) functionality for cloud security to the enterprises198with data flows of the CASB being regulated with a global cloud traffic controller (GCTC). Communication between the cloud orchestration server124and the cloud provider(s)140for a given enterprise198can be either a VPN connection or tunnel depending on the preference of the enterprise198. The cloud orchestration server124may configure, test, and enforce VM placement and configuration in hypervisors of the POPs104across the capacity resolver system100. For example, the cloud orchestration server124may ensure that the policies for VM provisioning including vertical scaling and VM migration are consistent across the cloud providers140, the enterprises198and computing environments150. The cloud orchestration server124provides proxies to the cloud providers140and may apply various policies. The connection between the end-user devices195and the cloud orchestration server124is over an encrypted VPN190or tunnel.

With reference toFIG.2A, a block diagram of an embodiment of a single-tenant capacity resolver system200where an end-user device102communicates with a cloud provider140is shown. The end-user device102is operated by an end-user122. The cloud provider140is accessible directly or through the cloud orchestration server124depending on the route chosen, services, policies, etc. Included in the cloud provider140are the services120such as storage that enable applications and functionality on the end-user devices102. The end-user devices102uses the virtual machines present at the POP104for deployment and use of the services120. Tenant specific rules and policies are set for the end-user devices102and stored in a rules storage110.

Referring next toFIG.2B, a block diagram of an embodiment of an end-user device102that includes a client210for enabling enhanced routing control is shown. The end-user device102includes applications (apps)212and a browser202that use the client210for communication over the LAN204and in due course to the cloud provider(s)140(not shown). The browser202and the app(s)212can be redirected using domain name services (DNS) to use the client210. Alternatively, the browser202and the app(s)212may natively support the client210to utilize Application Programming Interfaces (APIs) or other communication to select policies, and rules and receive the corresponding resolution of the capacity management of the nodes in the data centers. The policies corresponding to the tenants include rules, preferences, and priorities of provisioning VM based on a set of parameters. The set of parameters includes Central Processing Unit (CPU) cores, memory, disk space, and Virtual File System (VFS) availability of VMs in a Kernel Based Virtual Machine (KVM)/hypervisor of VMs108. The policies are stored in a policy cache208which includes the policies retrieved from the rules storage110. A remote app206is used to access a remote application on a remote instance of a remote server on the end-user device102.

The provisioning of the VM in the data centers and the metrics involved in downsizing the VMs or migrating the VMs are displayed to the end-user(s)122including the end-user(s)122requesting a display214. An Information Technology (IT) module214is used by the end-user(s)122to provide any feedback related to the optimization of VMs and the VM placement and configuration. The feedback is provided to an administrator of the enterprise(s)198of the end-user(s)122. The feedback may include changes in policies or the VM placement conditions or changes in parameters and thresholds. The display214provides graphical depictions of VM capacity to the end-user(s)122and asks for suggestions and/or feedback which may be provided to the IT module214.

Referring next toFIG.3, a block diagram of a cloud orchestration server124is shown. The cloud orchestration server124controls the VM provisioning and VM placement in the hypervisors or KVM of the POP104. The cloud orchestration server124receives the request for provisioning a node or VM108. The cloud orchestration server124considers various parameters to identify whether the request for placing the VM108in the KVM can be fulfilled or whether another KVM is entailed for placing the VM108. The cloud orchestration server124includes a network interface302, an overprovisioning detector304, a controller306, a threshold storage308, an Application Programming Interface (API)310, a resolver312, a provisioning engine314, a rules cache316, and a recommendation engine318.

The network interface302is used to receive requests for VM placement from the end-user device(s)122via a Domain Name System (DNS) interface. The network interface302is used for communication between the cloud orchestration server124and the end-user device(s)122. The VM placement recommendations generated by the cloud orchestration server124are provided on the display216of the end-user device102for the end-users122.

The controller306manages the components of the cloud orchestration server124. The controller306performs VM provisioning that includes suggested placements for new VM(s)108(given the POP name, service group, and VM cores/memory/disk) by the recommendation engine318. The controller306optimizes the VM configurations at existing POP104, that is by resolving over-subscription by the overprovisioning detector304.

The overprovisioning detector304identifies whether the KVMs are overprovisioned. The overprovisioning detector304compares the one or more parameters associated with the VMs for example, CPU cores, memory, disk space, and VFS availability with respective threshold values obtained from the threshold storage308. The overprovisioning detector304identifies a triggering of the one or more parameters above the threshold values. The threshold values are predefined by the administrators of the enterprise(s)198, the end-user(s)122, and/or may be automatically defined by a machine learning algorithm based on current and past provisioning and utilization of the VMs108and stored in the threshold storage308. A prediction to consumption of the one or more parameters may be identified by the machine learning algorithm. The triggering of the parameters is provided to the resolver312and the controller306for processing the requests.

In vertical scaling VMs of a service group that have very low utilization are considered candidates for downsizing. The downsizing refers to the reduction in provisioned cores and/or memory. The VMs may be selected for downsizing if doing so would directly reduce over-subscription or increase available provisioning space for new VMs108.

Downsizing recommendations apply to all nodes/VMs108of a given service concurrently. Services are resized by a factor of one-half and then rounded up to the nearest exponent of 2 (for example, 6 cores are rounded down to 4, 15 GiB memory is rounded down to 8 GiB, etc). CPU provisioning cannot be reduced to below 1 core per VM. Services cannot be downsized more than once in 7 days. Utilization rates are measured over the past (for example 7 days) and are aggregated at a percentile (for example, the 99th percentile for core utilization and the 95th percentile) for memory utilization. Downsizing only applies to the resource that is determined to be underutilized (for example, if a service group is underutilized only in terms of cores, then its memory allocation would not be reduced as well). The terms ‘service’ or ‘service group’ refer to a proxy metric derived from the portion of the VM's hostname before the first period or numeric character.

In migrating VMs, the VMs108may be migrated from one KVM to another, if doing so would directly reduce over-subscription or increase available provisioning space. At existing POPs104, migrating the VMs108is more burdensome than downsizing VMs108, therefore, migrating VMs108is a secondary consideration only after all downsizing options have been exhausted.

On receiving the request for VM placement, the controller302uses the learning algorithm to provision the VM108. The learning algorithm identifies a KVM for a new VM108to be provisioned based on its requisites of CPU cores, memory, and disk space. The learning algorithm identifies any VM migrations or downsizing necessary to create the necessary resource space for the assignment of the VM108.

In a similar request for new VM placement, an upsize request is often made in response by the learning engine to a service whose nodes are overutilized at the POP104. Rather than adding new VMs108, vertical scaling is preferred. The service upsize use case leverages much of the same logic as the VM placement logic. The learning engine performs optimization of VM configuration by identifying VM migrations and downsizings to eliminate over-provisioning and increase the maximum available provisioning space. When a request for VM configuration for new POP104is received the learning algorithm identifies a VM configuration for a new POP104. The learning algorithm would not perform actual migration or downsizing as the POP104is new.

The learning algorithm will suggest only the actions that are directly necessary to resolve the request. For example, when requesting a VM placement, the algorithm will omit an action such as downsizing a VM108, if it does not directly prevent the VM108from being placed without causing an oversubscription not already present. The request is resolved in the minimum number of actions. There are several additional boundary conditions considered by the learning algorithm which include VMs that are to be excluded from downsizing or migration.

Oversubscription: No proposed action will introduce additional oversubscription (with respect to cores, memory, or disk space) to any KVMs in the configuration.

Anti-affinity: One potential provisioning risk is the over-concentration of the VMs108from the same service group within a single KVM. If the KVM were to fail, the entire service within the POP104may be jeopardized. The learning algorithm ensures VMs104are not overly concentrated.

Static VMs: Certain VMs108may not be feasibly migrated and/or downsized. These VMs are specified through configuration YAML Ain′t Markup Language (YAML) files. Additionally, the VMs108with disk sizes greater than a specified threshold, and those with multiple external hard drives are not considered for migration.

Failover considerations: Certain services at certain POPs104may have intentionally low utilization so that they can absorb additional workloads in the event of a failover. These VMs108are treated differently and have separate, stricter utilization thresholds for downsizing.

Inactive VMs: Occasionally, VMs108are deactivated but still provisioned resources. If ignored, this presents a risk that the VMs108will subsequently be reactivated and its KVM may become oversubscribed as a result. The learning algorithm considers inactive VM provisioned memory and disk to be valid, but provisioned CPU cores to be invalid. Additionally, inactive VMs will not be recommended for migration.

The user inputs to the learning algorithm received from the end-user(s)122are:POP NameHostname of requested VMRequested VM CoresRequested VM MemoryRequested VM DiskMaximum iterations (for example, 30 by default)List of KVMs to avoid placement onList of VMs to not migrate and/or downsizeService anti-affinity threshold (for example, 25% by default)Cores and/or memory utilization threshold for downsizing candidates (for example, 12.5% by default)Maximum migrate-able VM disk size (for example, 501 GiB by default)Configuration Evaluation of VM108:

If a configuration cannot accommodate a VM placement request in its initial state, the learning algorithm works to incrementally improve the configuration until it can be accommodated. A cost function is determined that serves to quantify the configuration with respect to the requested placement, essentially evaluating how far off a configuration is from a state that could accommodate the request. The four-dimensional cost function quantifies the resource difference between the current configuration and the closest configuration that would accommodate the requested VM placement.

For a given KVM, the four primary terms of the cost function quantify the insufficiency of each of the four resources necessary to provision the VM (CPU cores, memory, disk, and VFS). Because the four resources each have their units which cannot be directly compared, each term is normalized (indicated by each denominator) by a factor determined by the placement request itself e.g. the cores term is divided by the quantity of requested cores; the result is a unitless ratio that can be added along with other unitless terms. If a KVM can accommodate, for example, sufficient cores to place the VM then that term would be equal to zero, no term can have a negative value. A KVM with a lower cost function value is ‘closer’ to being able to accommodate the VM placement. The minimum aggregation of the equation indicates that each configuration is scored based on its KVM which has the lowest evaluation. If the cost function equates to 0 for any KVM, then the VM placement may occur (provided that anti-affinity or other boundary conditions don't invalidate this KVM). The cost function is:

* Ratio cannot be less than 0, c refers to CPU cores, m for memory, d for disk space, and v for VFS availability.

The specified cost function serves to identify whether each KVM can accommodate the placement request (if its value is equal to 0). Quantify the ‘distance’ between each KVM's current state and a state that could accommodate the placement request. Incorporate all four dimensions of the VM placement problem (cores, memory, disk and VFS) such that each is treated as equally imperative and normalized by a factor associated with the placement request itself. Evaluate whether potential configuration changes would result in being ‘closer’ or ‘further away’ from adequately accommodating the VM placement request.

There are four potential outcomes when attempting to place a VM108.Type 1: Placement successful with existing configuration. The existing configuration can accommodate VM placement without any entailed migrations or downsizings.Type 2: Placement successful through service downsizing. The VM placement can be accommodated through one or more service downsizings. No VM migrations are entailed.Type 3: Placement successful through a combination of downsizing and VM migrations. The VM placement can be accommodated through one or more VM migrations, possibly in addition to service downsizing.Type 4: Placement unsuccessful. VM placement request cannot be accommodated given the boundary conditions of the capacity problem. All possible service downsizings and VM migrations have been considered (or the algorithm has reached its maximum iteration limit).

The outcomes from the learning engine are provided to the resolver312for further processing.

The resolver312uses the rules cache316that includes rules specifying conditions for downsizing and migration of VMs108. The rules cache316includes tenant or enterprise(s)198set rules including policies, parameters, and threshold values for provisioning VMs108. For example, for urgent requests, consider downsizing the VMs108else set a new VM108in another KVM. For normal requests, downsizing VMs or migrating VMs108may be considered. The rules cache316includes the rules from the rules store110and the policy cache208. The resolver312determines whether downsizing or migration of the VMs108will accommodate the VMs108of the request. The determination is provided to the controller302. The controller302uses the determination from the resolver312and the recommendations from the recommendation engine318to instruct the provisioning engine314to place and provision the VM108in the KVM accordingly.

The recommendation engine318provides recommendations to place the VMs108in the KVM of the POP104based on the analysis of the controller306using machine learning models. The machine learning models include machine learning algorithms that use the current and past VM provisioning from the controller302and suggest placements to the end-user(s)122on the display214. The recommendation engine318also proposes initial VM configuration for new POP104. The generation of recommended capacity solutions is fully automated. The recommendation engine318simplifies the resolution process for the end-user(s)122and reduces the amount of time needed to address capacity issues. The machine learning models access a fully developed data pipeline of VM provisioning and utilization data that is updated frequently. The machine learning model provides graphical depictions of VM capacity which informs end-user(s)122of the recommendations in detail.

The recommendation engine318proposes initial VM configuration for new POP104. The controller306resolves the request for VM placement by two processes, vertical scaling or downsizing services and migrating VMs. Learning algorithms including machine learning algorithms or fuzzy logic or other algorithms may be used to resolve the request by vertical scaling or VM migration.

The API interface310can use commands to fetch data from the POP104. For example, the API interface310can be used to collect data from the power supply at the POP104and the temperature sensor at the POP104. The API interface310can also be used to collect data from a router used to receive data from the end-user device(s)102to the POP104. In another embodiment, the API interface310can be used to collect data from the network interface302present at the POP location.

The provisioning engine314places and configures the VM108on the same KVM or another KVM or set up the VM as new VM108in the KVM based on the instructions from the controller302. The provisioning engine314provides the status of placement of the VMs108in the KVM to the controller302which further displays it to the end-user(s)122on the display214.

Referring next toFIG.4, an embodiment illustrating a structural diagram of the POPs104is shown. The POPs104includes POP104-1,104-2, . . .104-N, the POPs104include a controller404(404-1,404-2, . . .404-N), VMs108(108-1,108-2, . . .108-N) and hypervisors402(402-1,402-2, . . .402-N). N number of POPs104may be present. The hypervisor402is the KVM that includes the VMs108and provisions resources like CPU cores and memory for the functioning of the VMs108. The controller404monitors the capacities of the hypervisor402and determines whether the hypervisor402can accommodate more VMs108or a new hypervisor402is needed to accommodate the VM108. The operations and management of the hypervisor402and the VMs108are executed by the controller404.

Referring next toFIGS.5A-5C, a graphical representation of capacity management of VMs in the KVM500is shown. As illustrated inFIG.5A, an adequately provisioned KVM500A and an over-provisioned KVM500B are shown. The adequately provisioned KVM500A includes VMs108placed by downsizing the VMs108in the KVM502such that there is available space for resources504. The available resources504may place more VMs. The over-provisioned KVM500B has provisioned more resources (cores, memory, and/or disk) than it has to provide with the KVM502causing a risk of performance and stability issues. A portion of cores needed with respect to requested VM cores and available resources shows using cost function to assess the VM placement.

Referring next toFIG.5B, an embodiment of downsizing and over-provisioning reduction510is shown. The KVM502including VMs108is shown. Service downsizing or vertical scaling refers to the reduction in provisioned cores and/or memory associated with the nodes of a common service at a given POP104. The downsizing can be applied as a mechanism for creating additional KVM provisioning availability512on the KVM502as shown in KVM510A with respect to KVM510B. Reduction or resolve KVM oversubscription is shown in KVM510C with respect to KVM510D. An over-provisioning reduction514by reducing the overprovisioned VMs108. VMs108of a service group that have very low utilization are considered candidates for downsizing. These VMs108may be selected for downsizing if doing so would directly reduce over-subscription or increase available provisioning space for new VMs108. Service groups are considered candidates for downsizing if and only if all associated nodes have underutilization (core utilization, memory utilization, or both).

Referring next toFIG.5C, an embodiment of a VM migration520in KVM is shown. The KVM502including VMs108is shown. Migration refers to the re-provisioning of a node/VM108from one KVM to another KVM. Similar to VM downsizing, VM migration can be applied to create additional KVM provisioning availability as shown in KVM520A and KVM520B. Increased KVM provisioning availability522shows the additional VM108placement space.

Reducing or resolving KVM oversubscription is shown in KVM520C and KVM520D. Migration of VMs108improves a configuration. However, the process of reprovisioning a VM108to its new KVM can last hours, potentially disrupting a service. In contrast, downsizing a service group can be accomplished without any disruption to the VMs108. As a result, migrations are only a secondary consideration after all downsizing actions have been exhausted.

Referring toFIG.6, a Graphical User Interface (GUI)600illustrates a VM provisioning by the capacity resolver system100displayed to the end-user(s)122and/or the administrator of the enterprise(s)198is shown. The GUI600is a visual representation of one or more parameters including CPU cores, memory, disk, and VFS availability. Each block represents one node. Each row is a KVM and then each of the columns here are the one or more parameters. There is a threshold limit of placing VMs in the hypervisor or KVM. When the threshold limit is exceeded, the capacity resolver100provides remedies in order to resolve the VM placement request.

Referring next toFIG.7, a block diagram of an embodiment of an OSI model700is shown. The cloud OSI model700for cloud computing environments partitions the flow of data in a communication system into six layers of abstraction. The cloud OSI model700for cloud computing environments can include, in order, an application layer710, a service layer715, an image layer720, a software-defined data center layer725, a hypervisor layer730, and an infrastructure layer735. The respective layer serves a class of functionality to the layer above it and is served by the layer below it. Classes of functionality can be realized in software by various communication protocols.

The infrastructure layer735can include hardware, such as physical devices in a data center, that provides the foundation for the rest of the layers. The infrastructure layer735can transmit and receive unstructured raw data between a device and a physical transmission medium. For example, the infrastructure layer735can convert the digital bits into electrical, radio, or optical signals.

The hypervisor layer730can perform virtualization, which can permit the physical devices to be divided into virtual machines that can be bin packed onto physical machines for greater efficiency. The hypervisor layer730can provide virtualized computing, storage, and networking. For example, OpenStack® software that is installed on bare metal servers in a data center can provide virtualization cloud capabilities. The OpenStack® software can provide various infrastructure management capabilities to cloud operators and administrators and can utilize the Infrastructure-as-Code concept for deployment and lifecycle management of a cloud data center. In the Infrastructure-as-Code concept, the infrastructure elements are described in definition files. Changes in the files are reflected in the configuration of data center hosts and cloud services.

The software-defined data center layer725can provide resource pooling, usage tracking, and governance on top of the hypervisor layer730. The software-defined data center layer725can enable the creation of virtualization for the Infrastructure-as-Code concept by using representational state transfer (REST) APIs. The management of block storage devices can be virtualized, and end-users can be provided with a self-service API to request and consume those resources which do not entail any knowledge of where the storage is deployed or on what type of device. Various compute nodes can be balanced for storage.

The image layer720can use various operating systems and other pre-installed software components. Patch management can be used to identify, acquire, install, and verify patches for products and systems. Patches can be used to correct security and functionality problems in software. Patches can also be used to add new features to operating systems, including security capabilities. The image layer720can focus on the compute in place of storage and networking. The instances within the cloud computing environments can be provided at the image layer720.

The service layer715can provide middleware, such as functional components that applications use in tiers. In some examples, the middleware components can include databases, load balancers, web servers, message queues, email services, or other notification methods. The middleware components can be defined at the service layer715on top of particular images from the image layer720. Different cloud computing environment providers can have different middleware components.

The application layer710can interact with software applications that implement a communicating component. The application layer710is the layer that is closest to the end-user. Functions of the application layer710can include identifying communication partners, determining resource availability, and synchronizing communication. Applications within the application layer710can include custom code that makes use of middleware defined in the service layer715.

Various features discussed above can be performed at one or more layers of the cloud OSI model700for cloud computing environments. For example, translating the general policies into specific policies for different cloud computing environments can be performed at the service layer715and the software-defined data center layer725. Various scripts can be updated across the service layer715, the image layer720, and the software-defined data center layer725. Further, APIs and policies can operate at the software-defined data center layer725and the hypervisor layer730.

Respective different cloud computing environments can have different service layers715, image layers720, software-defined data center layers725, hypervisor layers730, and infrastructure layers735. Further, respective different cloud computing environments can have an application layer710that can make calls to the specific policies in the service layer715and the software-defined data center layer725. The application layer710can have noticeably the same format and operation for respective different cloud computing environments. Accordingly, developers for the application layer710cannot need to understand the peculiarities of how respective cloud computing environments operate in the other layers.

Referring next toFIG.8, a flowchart of a VM provisioning process800including provisioning a VM or node at a hypervisor/KVM based on one or more parameters is shown. The VM provisioning process800starts at block802where requests for placing one or more VMs108/nodes at a hypervisor/KVM are received. The request is received from an end-user(s)122via an end-user device(s)102. The request is received at the cloud orchestration server124or at the POP104. At block804, a determination is made if one or more parameters are triggered at a KVM of the POP104where a requested VM108entails to be placed.

The one or more parameters include Central Processing Unit (CPU) core utilization, memory utilization, disk utilization and Virtual File System (VFS) availability associated with the VMs108of the KVM. There are four resources (including the one or more parameters) associated with each VM108and each KVM that define the problem of capacity management. KVMs have a certain amount of each resource which can be provisioned to one or more VMs including CPU cores, memory (typically expressed as GiB), disk space (typically expressed as GiB), Virtual File System (VFS) availability. An over-provisioned KVM has provisioned more resources (cores, memory, and/or disk) than it has to provide.

If the one or more parameters are not triggered at block804, the request for placing the VM108continues with the new KVM in the same POP104or another POP104. At block810, the requested VM108is provisioned in the new KVM. A condition for downsizing or migration is checked at block806.

Over-provisioning introduces the risk of performance and stability issues. The learning algorithm of the cloud orchestration server124performs the VM placement of the request by placing the requested VM108on the KVM of the POP104. At block806, based on the triggering of the one or more parameters, either vertical scaling (downsizing) or VM migration may be performed. At first vertical scaling is considered over VM migration. VMs108of a service group that have very low utilization of CPU cores, memory, disk, and VFS are considered for downsizing. In downsizing, the provisioned cores and/or memory are reduced in size. These VMs108may be selected for downsizing if doing so would directly reduce the overprovisioning or increase available provisioning space for the requested VM108. In VM migration nodes108may be migrated from one KVM to another, if doing so would directly reduce over-subscription or increase available provisioning space.

At block812, a priority of selecting vertical scaling (downsizing) or VM migration is determined. Initially, vertical scaling is considered however, if not possible or feasible, VM migration is considered in priority. The priority is determined by the cloud orchestration server124based on the reduction or downsizing of the parameters.

At block814, the requested VM108is provisioned on the KVM of the existing POP104based on the priority by selecting downsizing or migration. The requested VM108is provisioned in the POP existing104by downsizing or migration.

At block816, the requested VM108is determined to be a new VM108. The VM configuration will depend on whether the requested VM108is new. For the new VM108, the request already includes details for configuration like POP104name, service group, and VM cores/memory/disk. At block818, the new VM108is provisioned on the given POP104based on the configuration details.

At block820, if the VM108is not new, a determination is made whether the request includes configuration on a new POP104. For the new POP104at block822, the recommendation engine318of the cloud orchestration server124, proposes initial configuration for the new POP104based on requisites and preferences of the end-user(s)122or the enterprise(s)198providing the request. If the requested VM108is neither a new VM108nor entails a new POP104, the VM108will be provisioned on the KVM of the existing POP104at block814.

Referring next toFIG.9, a flowchart of determining the triggering of parameters at block804of the VM provisioning process800is shown. The learning algorithm considers triggering of the one or more parameters for resolving the request for VM configuration and placement. However, there are other boundary conditions of the VMs108that the learning algorithm considers. At block902, boundary conditions of VM108are determined including the one or more parameters like CPU cores, memory, disk, and VFS availability. At block904, a low utilization of these parameters of the VMs108in the KVMs is determined by the learning algorithm. If there is low utilization of these parameters then at block908, the VMs108in the KVMs are downsized so that the requested VM108can be accommodated.

If the utilization of the parameters at the KVMs is not low and there is no space for accommodating the requested VM108, then at block906, the requested VM108is migrated to another KVM of another POP104. At block910, other parameters of the boundary conditions of the VMs108are determined. Anti-affinity presence is determined, the anti-affinity includes over-concentration of VMs from the same service group within a single KVM. Each VM108is associated with a service/kind and the learning algorithm ensures the VMs108are not overly concentrated. Within a given POP104, provisioning a large portion of VMs108with a shared service to the same KVM represents an unnecessary risk for that service. In the event that the KVM were to shut down, the service would be significantly impacted. To mitigate this risk, it is pertinent to diversify the KVM assignments for the VMs108with a common service. Multiple VMs108with a shared service cannot be provisioned to the same KVM unless the count of VMs108represents less than a determined amount of all the VMs108associated with the service at that POP104. Based on the presence of the anti-affinity at block912, the VMs108are excluded from downsizing or migration.

At block914, presence of static VMs108is determined and if present excluded from downsizing or migration at block912. The static VMs108may not be feasibly migrated and/or downsized. These VMs108are specified through configuration YAML files. Additionally, VMs108with disk sizes greater than a specified threshold, and those with multiple external hard drives are not considered for migration. Migration of the VMs108with sufficiently large disk space requisites and those with multiple external disks are significantly more challenging and time consuming. For this reason, the learning algorithm will not recommend migrating these VMs108.

At block916, failover considerations are determined and if present excluded from downsizing or migration at block912. The failover considerations include certain services at certain POPs104that have intentionally low utilization so that they can absorb additional workloads in the event of a failover. There are a number of services/kinds that should not be migrated or downsized for various other reasons. These VMs108are treated differently and are not selected for downsizing.

Further, inactive VMs108include VMs108that are deactivated but still provisioned resources. If ignored, this presents a risk that the VMs108will subsequently be reactivated and its KVM may become oversubscribed as a result. The learning algorithm considers inactive VM provisioned memory and disk to be valid, but provisioned CPU cores to be invalid. Additionally, inactive VMs will not be recommended for migration.

At block918, oversubscription is determined. No proposed action will introduce additional oversubscription (with respect to cores, memory, or disk space) to any KVMs in the configuration else the VMs will be excluded from downsizing or migration at block912.

Referring next toFIG.10, a flowchart of a VM placement process1000including VM placement phases is shown. The VM placement process1000begins at block1002where phase 1 initiates. Phase 1 includes the initial placement attempt of requested VM108in the KVM of the POP104. The learning algorithm performs the phase 1 analysis. The learning algorithm initiates with an attempt to place the VM108in the KVM configuration. If the VM placement is possible at the KVM, the placement is made, and no further logic is applied at block1004as VM108is successfully placed in the KVM. If multiple KVMs could accommodate the placement, the algorithm places preference on anti-affinity firstly and ‘completing a KVM’ secondly. If a triggering of the parameters is determined, then the process moves to phase 2 that is downsizing the VMs108.

If an initial configuration cannot accommodate the placement request, the learning algorithm will attempt to resolve the request exclusively through service downsizings in phase 2. At block1006, a determination is made to check if there is any downsize able VMs108are available to accommodate the requested VM108. If there is a downsize able VM108then at block1008, the VM108is downsized that most directly accommodates VM placement request.

At block1010, VM placement is determined and if possible, then at block1012, the requested VM108is successfully placed at the KVM else the process moves to find another downsize able VM at block1006. The algorithm sequentially applies downsizings in order of maximum benefit until the placement can be accommodated or until all candidate services have been downsized. Only once all downsizing options have been exhausted, the learning algorithm considers VM migrations at phase 3. The process ended by pruning unnecessary downsizing at phase 4 at block1024.

At block1014, VM migrations are determined. VMs108are determined for migrations by the learning algorithm and if the VM108is determined for migration then at block1016, VM migration that most directly accommodates placement request is applied. At block1018, after VM migration, the requested VM placement is checked at block1018and finally, the requested VM108is placed at block1020. To avoid recursive loops, the same VM108cannot be moved to a KVM that was already moved off. Migrations are applied until either the placement can be accommodated, the algorithm fails to converge, or a user-specified maximum iteration threshold is reached at block1022or at phase 4 at block1026and at block1028where unnecessary migrations and downsizing are pruned respectively.

Phase 4: Pruning unnecessary actions: If either phases 2 or 3 result in successful VM placement a final pruning step is applied. While the learning algorithm helps to guide the configuration iteratively towards an acceptable state, there are cases where the path is not maximally efficient. This is imperative because configuration changes can be lengthy and disruptive. It is highly desirable to identify the solution with minimal number of actions needed (particularly with respect to VM migrations). After placement is achieved, the algorithm will work iteratively backwards to evaluate whether any individual action could be omitted entirely, or whether any pair of actions could be replaced with a single action. Actions are pruned only if doing so would not cause additional over provisioning in the configuration or otherwise violate a boundary condition such as anti-affinity.

Multiple VM Placements: Often, requests necessitate the placement of two or more nodes of the same service at a given POP104. To accommodate these requests, the learning algorithm applies the logic described above iteratively for each individual VM placement, while updating the configuration to reflect the associated downsizings, migrations and placement at each step. At the end of the placement of all requested nodes (if feasible), the algorithm triggers one additional pruning step to reduce unnecessary actions that have been recommended across different VM placements.

While the principles of the disclosure have been described above in connection with specific apparatuses and methods, it is to be clearly understood that this description is made only by way of example and not as a limitation on the scope of the disclosure.