INTELLIGENTLY SCALING DATABASE AS A SERVICE RESOURCES IN A CLOUD PLATFORM

A computer-implemented method, system and computer program product for scaling a resource of a Database as a Service (DBaaS) cluster in a cloud platform. User service requests from a service cluster to be processed by the DBaaS cluster are received. A first set of tracing data is generated by a service mesh, which facilitates service-to-service communication between the service cluster and the DBaaS cluster, from the user service requests. A second set of tracing data is generated by the DBaaS cluster from handling the user service requests. A dependency tree is then generated to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on these sets of tracing data. The pod(s) of a DBaaS node are then scaled based on the dependency tree, which is used in part, to predict the utilization of the resources of the DBaaS node identified as being a potential bottleneck.

TECHNICAL FIELD

The present disclosure relates generally to Database as a Service (DBaaS), and more particularly to intelligently scaling the pods (encapsulates sidecars and services) used for DBaaS in a cloud platform.

BACKGROUND

Database as a Service (DBaaS) is a cloud computing managed service offering that provides access to a database without requiring the setup of physical hardware, the installation of software or the need to configure the database. Most maintenance and administrative tasks are handled by the service provider thereby freeing up users to quickly benefit from using the database.

Recently, DBaaS is being implemented using a microservices approach as opposed to a monolithic approach. The monolithic approach corresponds to the traditional model of a software program in which the software program is built as a unified unit that is self-contained and independent from other applications. The microservices approach corresponds to a method that relies on a series of independently deployable services. That is, the microservices approach corresponds to an architectural and organizational approach to software development where software is composed of small independent services that communicate over well-defined application programming interfaces. These services have their own business logic and database with a specific goal. Updating, testing, deployment, and scaling occur within each service. Microservices decouple major business, domain-specific concerns into separate, independent code bases.

The microservices approach to implementing DBaaS may utilize containers. A container refers to a standard unit of software that packages up code and all its dependencies so that the application runs quickly and reliably from one computing environment to another. Such microservices may run in their own containers.

In such an environment, these containers may be run in “pods.” All the containers in the pod share an Internet Protocol (IP) address, inter-process communication (IPC), hostname and other resources. A “pod” is a group of one or more containers, which may be deployed to a node, referred to as a “worker node.” A worker node is used to run containerized applications and handle networking to ensure that traffic between applications across the cluster and from outside of the cluster can be properly facilitated. A “cluster” refers to a set of nodes (e.g., worker nodes) that run containerized applications (containerized applications package an application with its dependencies and necessary services). Such a cluster (“DBaaS cluster”) may be used to process DBaaS service requests.

At times, the DBaaS cluster may receive an inordinate amount of service requests to be processed. As a result, the DBaaS resources may be scaled in order to handle the increased workload. For example, in response to the increased load, horizontal scaling may be implemented by a horizontal pod autoscaler to deploy more pods. Such horizontal scaling is different from vertical scaling which assigns more resources (e.g., memory, CPU) to the pods that are already running for the workload.

If the load decreases and the number of pods is above the configured minimum, the horizontal pod autoscaler instructs the workload resource to scale back down.

The DBaaS cluster may receive various types of requests to be processed. For example, the DBaaS cluster may receive a create, read, update or delete request to be processed. Such a request (create, read, update or delete requests are collectively referred to as a “CRUD” request) though may automatically generate numerous downstream requests, such as for indexing and replication. For example, the CRUD request may generate downstream requests for indexing and replication which are processed by the containers of the pods in the DBaaS cluster. For instance, the containers of a particular pod may be utilized to process the downstream requests for indexing and the containers of another particular pod may be utilized to process the downstream requests for replication.

As a result of a request being converted into multiple requests with upstream and downstream relationships, the DBaaS cluster may not be able to service such requests in an efficient manner thereby resulting in a system bottleneck which negatively impacts system performance. A “system bottleneck,” as used herein, refers to an overloaded system in which components of the system, such as the DBaaS cluster, are unable to keep pace with the system thereby slowing overall performance.

Unfortunately, the scaling mechanism discussed above, such as horizontal scaling by a horizontal pod autoscaler, is only able to address such a system bottleneck after the problem has impacted system performance. For example, such scaling occurs after the observed metrics, such as memory and CPU performance, has indicated that scaling is necessary. As a result, the scaling mechanism discussed above does not address such a system bottleneck since the DBaaS throughput cannot be changed in time in the cloud platform.

SUMMARY

In one embodiment of the present disclosure, a computer-implemented method for scaling a resource of a Database as a Service (DBaaS) cluster in a cloud platform comprises receiving user service requests from a service cluster to be processed by the DBaaS cluster, where the DBaaS cluster comprises one or more nodes, and where each of the one or more nodes comprises one or more pods containing a group of one or more containers. The method further comprises generating a first set of tracing data from the user service requests by a service mesh facilitating service-to-service communication between the service cluster and the DBaaS cluster. The method additionally comprises generating a second set of tracing data by the DBaaS cluster from handling the user service requests. Furthermore, the method comprises generating a dependency tree to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on the first and second sets of tracing data. Additionally, the method comprises scaling one or more pods of a node of the DBaaS cluster based on the dependency tree.

In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pods in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring.

In another embodiment of the present disclosure, a computer program product for scaling a resource of a Database as a Service (DBaaS) cluster in a cloud platform, where the computer program product comprises one or more computer readable storage mediums having program code embodied therewith, where the program code comprising programming instructions for receiving user service requests from a service cluster to be processed by the DBaaS cluster, where the DBaaS cluster comprises one or more nodes, and where each of the one or more nodes comprises one or more pods containing a group of one or more containers. The program code further comprises the programming instructions for generating a first set of tracing data from the user service requests by a service mesh facilitating service-to-service communication between the service cluster and the DBaaS cluster. The program code additionally comprises the programming instructions for generating a second set of tracing data by the DBaaS cluster from handling the user service requests. Furthermore, the program code comprises the programming instructions for generating a dependency tree to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on the first and second sets of tracing data. Additionally, the program code comprises the programming instructions for scaling one or more pods of a node of the DBaaS cluster based on the dependency tree.

In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pods in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring.

In a further embodiment of the present disclosure, a system comprises a memory for storing a computer program for scaling a resource of a Database as a Service (DBaaS) cluster in a cloud platform and a processor connected to the memory. The processor is configured to execute program instructions of the computer program comprising receiving user service requests from a service cluster to be processed by the DBaaS cluster, where the DBaaS cluster comprises one or more nodes, and where each of the one or more nodes comprises one or more pods containing a group of one or more containers. The processor is further configured to execute the program instructions of the computer program comprising generating a first set of tracing data from the user service requests by a service mesh facilitating service-to-service communication between the service cluster and the DBaaS cluster. The processor is additionally configured to execute the program instructions of the computer program comprising generating a second set of tracing data by the DBaaS cluster from handling the user service requests. Furthermore, the processor is configured to execute the program instructions of the computer program comprising generating a dependency tree to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on the first and second sets of tracing data. Additionally, the processor is configured to execute the program instructions of the computer program comprising scaling one or more pods of a node of the DBaaS cluster based on the dependency tree.

In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pods in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring.

The foregoing has outlined rather generally the features and technical advantages of one or more embodiments of the present disclosure in order that the detailed description of the present disclosure that follows may be better understood. Additional features and advantages of the present disclosure will be described hereinafter which may form the subject of the claims of the present disclosure.

DETAILED DESCRIPTION

As stated in the Background section, the DBaaS cluster may receive various types of requests to be processed. For example, the DBaaS cluster may receive a create, read, update or delete request to be processed. Such a request (create, read, update or delete requests are collectively referred to as a “CRUD” request) though may automatically generate numerous downstream requests, such as for indexing and replication. For example, the CRUD request may generate downstream requests for indexing and replication which are processed by the containers of the pods in the DBaaS cluster. For instance, the containers of a particular pod may be utilized to process the downstream requests for indexing and the containers of another particular pod may be utilized to process the downstream requests for replication.

As a result of a request being converted into multiple requests with upstream and downstream relationships, the DBaaS cluster may not be able to service such requests in an efficient manner thereby resulting in a system bottleneck which negatively impacts system performance. A “system bottleneck,” as used herein, refers to an overloaded system in which components of the system, such as the DBaaS cluster, are unable to keep pace with the system thereby slowing overall performance.

Unfortunately, the scaling mechanism discussed above, such as horizontal scaling by a horizontal pod autoscaler, is only able to address such a system bottleneck after the problem has impacted system performance. For example, such scaling occurs after the observed metrics, such as memory and CPU performance, has indicated that scaling is necessary. As a result, the scaling mechanism discussed above does not address such a system bottleneck since the DBaaS throughput cannot be changed in time in the cloud platform.

The embodiments of the present disclosure provide a means for addressing system bottlenecks at the DBaaS by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pods in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring. In one embodiment, potential bottlenecks in the nodes of the DBaaS cluster are identified based on discovering the application relationships for handling requests that generate downstream requests, such as for indexing and replication, by the components of the nodes of the DBaaS cluster. In one embodiment, such application relationships may be discovered by generating a dependency tree using tracing data for handling such service requests (tracing data illustrates how the service components of a node of a DBaaS cluster operate, execute and perform in handling service requests). After generating such a dependency tree, potential bottlenecks in the nodes of the DBaaS cluster can be identified. When a service request is received by the DBaaS cluster that corresponds to one of the service requests upon which the dependency tree was generated, a potential bottleneck in handling such a service request in a node of the DBaaS cluster may be identified from the dependency tree. Consumption predictors (e.g., memory utilization, timeline of called components of the node of the DBaaS cluster, traffic generation model, etc.) for the components of the node of the DBaaS cluster identified as being a potential bottleneck may be analyzed so that the utilization of the resources for such components is determined. The predicted utilization of the resources for the components of the DBaaS node identified as being a potential bottleneck is determined based on the determined utilization of the resources of the components of the DBaaS node identified as being a potential bottleneck and a timeline of called components of the DBaaS cluster. A scale operation may then be executed to scale one or more pods in the node of the DBaaS cluster identified as being a potential bottleneck in response to the predicted utilization of the resources being above or below a threshold level. A more detailed description of these and other features will be provided below.

In some embodiments of the present disclosure, the present disclosure comprises a computer-implemented method, system and computer program product for scaling a resource of a Database as a Service (DBaaS) cluster in a cloud platform. In one embodiment of the present disclosure, user service requests from a service cluster to be processed by the DBaaS cluster are received. A “service cluster,” as used herein, refers to a cluster of nodes for receiving and forwarding service requests to the DBaaS cluster. A “DBaaS cluster,” as used herein, refers to a cluster of nodes for handling such service requests. For example, an ingress gateway of the service cluster may receive and forward such requests to a sidecar which invokes a DBaaS service to handle such a service request. The DBaaS cluster and the service cluster each consists of a set of worker machines, called nodes, that run containerized applications (containerized applications package an application with its dependencies and necessary services). Each of the nodes may include one or more pods containing a group of one or more containers. A “container,” as used herein, refers to a standard unit of software that packages up code and all its dependencies so that the application runs quickly and reliably from one computing environment to another. A first set of tracing data from the user service requests is generated by a service mesh facilitating service-to-service communication between the service cluster and the DBaaS cluster. A second set of tracing data is generated by the DBaaS cluster from handling the user service requests. Such tracing data (both first and second sets) illustrates how the service components of a node of a DBaaS cluster operate, execute and perform in handling service requests. A dependency tree is then generated to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on the first and second sets of tracing data. A “dependency tree,” as used herein, refers to a graph illustrating the relationship between the services, such as the service pairs handling a particular type of request (e.g., create request, indexing, replication). One or more pods of a node of the DBaaS cluster are then scaled (scaled up or down) based on the dependency tree, which is used in part, to predict the utilization of the resources of the components of the DBaaS node identified as being a potential bottleneck. When the predicted utilization of the resources is above or below a threshold level, a scale operation is executed to scale the pod(s) of the DBaaS node identified as being a potential bottleneck. In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pod(s) in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring.

In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without such specific details. In other instances, well-known circuits have been shown in block diagram form in order not to obscure the present disclosure in unnecessary detail. For the most part, details considering timing considerations and the like have been omitted inasmuch as such details are not necessary to obtain a complete understanding of the present disclosure and are within the skills of persons of ordinary skill in the relevant art.

Referring now to the Figures in detail,FIG.1illustrates an embodiment of the present disclosure of a communication system100for practicing the principles of the present disclosure. Communication system100includes a computing device101connected to a container orchestration system102via a network103.

Computing device101may be any type of computing device (e.g., portable computing unit, Personal Digital Assistant (PDA), laptop computer, mobile device, tablet personal computer, smartphone, mobile phone, navigation device, gaming unit, desktop computer system, workstation, Internet appliance and the like) configured with the capability of connecting to network103and consequently communicating with other computing devices101and container orchestration system102. It is noted that both computing device101and the user of computing device101may be identified with element number101.

In one embodiment, the user of computing device101issues a request to access a database managed by a Database as a Service (DBaaS) running on container orchestration system102(e.g., Kubernetes®).

Network103may be, for example, a local area network, a wide area network, a wireless wide area network, a circuit-switched telephone network, a Global System for Mobile Communications (GSM) network, a Wireless Application Protocol (WAP) network, a WiFi network, an IEEE 802.11 standards network, various combinations thereof, etc. Other networks, whose descriptions are omitted here for brevity, may also be used in conjunction with system100ofFIG.1without departing from the scope of the present disclosure.

In one embodiment, the DBaaS platform is built on container orchestration system102. In one embodiment, container orchestration system102identifies potential bottlenecks in the nodes of the DBaaS cluster based on discovering the application relationships for handling requests that generate downstream requests, such as for indexing and replication, by the components of the node of the DBaaS cluster. In one embodiment, such application relationships may be discovered by generating a dependency tree using tracing data for handling such service requests (tracing data illustrates how the service components of a node of a DBaaS cluster operate, execute and perform in handling service requests). After generating such a dependency tree, potential bottlenecks in the nodes of the DBaaS cluster can be identified. When a service request is received by the DBaaS cluster that corresponds to one of the service requests upon which the dependency tree was generated, a potential bottleneck in handling such a service request in a node of the DBaaS cluster may be identified from the dependency tree. Consumption predictors (e.g., memory utilization, timeline of called components of the node of the DBaaS cluster, traffic generation model, etc.) for the components of the node of the DBaaS cluster identified as being a potential bottleneck may be analyzed so that the utilization of the resources for such components is determined. The predicted utilization of the resources for the components of the DBaaS node identified as being a potential bottleneck is determined based on the determined utilization of the resources of the components of the DBaaS node identified as being a potential bottleneck and a timeline of called components of the DBaaS cluster. A scale operation may then be executed to scale one or more pods in the node of the DBaaS cluster identified as being a potential bottleneck in response to the predicted utilization of the resources being above or below a threshold level. A more detailed description of these and other features will be provided below.

A description of the architecture of the DBaaS platform built on container orchestration system102is provided below in connection withFIG.2. Furthermore, a description of the hardware configuration of container orchestration system102is provided further below in connection withFIG.6.

System100is not to be limited in scope to any one particular network architecture. System100may include any number of computing devices101, container orchestration systems102and networks103.

Referring now toFIG.2,FIG.2illustrates the architecture of the DBaaS platform built on container orchestration system102in accordance with an embodiment of the present disclosure.

As shown inFIG.2, the architecture of the DBaaS platform includes a data plane201which includes a DBaaS cluster202and a service cluster203. A “service cluster”203, as used herein, refers to a cluster of nodes for receiving and forwarding service requests to the DBaaS cluster. A “DBaaS cluster”202, as used herein, refers to a cluster of nodes for handling such service requests. In one embodiment, DBaaS cluster202and service cluster203each consists of a set of worker machines, called nodes, that run containerized applications (containerized applications package an application with its dependencies and necessary services). For example, DBaaS cluster202consists of a set of nodes204(one or more worker nodes) and service cluster203consists of a set of nodes205(one or more worker nodes). Worker nodes204,205are used to run containerized applications and handle networking to ensure that traffic between applications across the cluster and from outside of the cluster can be properly facilitated.

In one embodiment, worker node(s)204,205host the pods that are components of the application workload. For example, node204hosts pods206A-206D and node205hosts pods207A-207C. Pods206A-206D may collectively or individually be referred to as pods206or pod206, respectively. Furthermore, pods207A-207C may collectively or individually be referred to as pods207or pod207, respectively. Each node204,205may host any number of pods206,207, respectively.

A “pod”206,207, as used herein, is a group of one or more containers, which are deployed to a node (e.g., node204,205). For example, pod206A contains a sidecar container208A and a service209A (identified as “Service A” corresponding to the core engine). Pod206B contains a sidecar container208B and a service209B (identified as “Service B” corresponding to the view engine). Pod206C contains a sidecar container208C and a service209C (identified as “Service C” corresponding to the replication engine). Pod206D contains a sidecar container208D and a service209D (identified as “Service D” corresponding to the search engine). Containers208A-208D of DBaaS cluster202may collectively or individually be referred to as containers208or container208, respectively. Services209A-209D of DBaaS cluster202may collectively or individually be referred to as services209or service209, respectively.

The relationship between such components (containers208, services209) are shown inFIG.2by lines210. Such a relationship may be discovered by generating the dependency tree as discussed in further detail below.

Furthermore, as shown inFIG.2, pod207A contains a sidecar container211A and a service212A (identified as “Service X”). Pod207B contains a sidecar container211B and a service212B (identified as “Service Y”). Furthermore, pod207C contains a sidecar container211C and a service212C (identified as “Service Z”). Containers211A-211C of service cluster203may collectively or individually be referred to as containers211or container211, respectively. Services212A-212C of service cluster203may collectively or individually be referred to as services212or service212, respectively.

Sidecars208,211, as used herein, refer to separate containers that run alongside an application container in a pod.

It is noted that pods206,207may contain any number of containers208,211, respectively, and services209,212, respectively, and thatFIG.2is illustrative.

Furthermore, as shown inFIG.2, service cluster203includes an ingress gateway213for receiving service requests issued by a user, such as a user of computing device101. Such service requests may then be forwarded to a sidecar211which invokes a DBaaS service to handle such a service request.

Additionally, as shown inFIG.2, the architecture of the DBaaS platform includes a control plane214which manages worker nodes204,205and pods206,207in clusters202,203, respectively.

In one embodiment, control plane214includes a tracing server215configured to store tracing data216, where such tracing data216captures data that illustrates how the components (e.g., sidecars208, services209) operate, execute and perform. In one embodiment, tracing data216may be obtained from a service mesh217that facilitates service-to-service communication between service cluster203and DBaaS cluster202. In one embodiment, service mesh217controls how different parts of an application share data with one another. In one embodiment, service mesh217corresponds to a dedicated infrastructure layer for facilitating service-to-service communications between services or microservices, using a proxy. In one embodiment, service mesh217consists of network proxies paired with each service in an application and a set of task management processed. The proxies are called the “data plane”201and the management processes are called the “control plane”214. In one embodiment, data plane201intercepts calls between different services and processes them; whereas, control plane214coordinates the behavior of proxies and provides APIs for operations and maintenance. In one embodiment, the service mesh architecture is implemented using various software tools including, but not limited to, Istio®, Linkerd®, Consul®, Traefik Mesh®, etc.

In one embodiment, service mesh217generates tracing data216by generating distributed traces spans for each service within it (see lines from sidecars208,211to tracing data216as shown inFIG.2). Such traces can be used to follow a single request (user service request received by ingress gateway213) through the mesh across multiple services and proxies.

In one embodiment, service mesh217stores the generated tracing data216in tracing server215.

In one embodiment, tracing data216is further obtained from DBaaS cluster202from handling the user service requests. In one embodiment, DBaaS cluster202utilizes a distributed tracing tool218for obtaining tracing data216from DBaaS cluster202handling the user service requests. In one embodiment, distributed tracing tool218may perform distributed tracing using various software tools, including, but not limited to, SigNoz®, Jaeger, Zipkin, Dynatrace®, New Relic®, Honeycomb®, Lightstep®, Instana®, DataDog®, Splunk®, etc.

In one embodiment, distributed tracing tool218stores the generated tracing data216in tracing server215.

Furthermore, such tracing data216is used to generate a dependency tree, such as shown inFIG.3. A “dependency tree,” as used herein, refers to a graph illustrating the relationship between services (e.g., services209), such as the service pairs handling a particular type of request (e.g., create request, indexing, replication).

Referring toFIG.3,FIG.3illustrates a dependency tree300in accordance with an embodiment of the present disclosure.

As shown inFIG.3, in conjunction withFIG.2, dependency tree300illustrates the chain of requests that are spawned from an initially received user request. For instance, a CRUD request may be received which automatically generates downstream requests for indexing and replication. Such downstream requests are chained together in dependency tree300showing the upstream and downstream relationship. For example, service request301of request type 1 may correspond to a create request, which generates a service request302of request type 2 (e.g., indexing) and a service request303of request type 3 (e.g., replication).

Additionally, as shown inFIG.3, in conjunction withFIG.2, dependency tree300illustrates the various services handling such request types, and the relationship between such services in terms of service pairs. For example, service X212A receives a service request301of request type 1, which is transferred to service A209A to handle. As a result, service X212A and service A209A form a service pair as identified as “X->A.” Furthermore, as shown inFIG.3, the service request302of request type 2 (generated from service request301) is transferred from service A209A to service B209B to handle thereby forming service pair “A->B.” Additionally, as shown inFIG.3, the service request303of request type 3 (generated from service request302) is transferred from service B209B to service C209C to handle thereby forming service pair “B->C.”

Furthermore, tracing data216includes the time duration (referred to herein as simply “time”) for handling such a service request type and the document count, which may be used for determining a potential bottleneck. For instance, as shown inFIG.4,FIG.4illustrates a table400representing the tracing data216that is used for generating dependency tree300ofFIG.3in accordance with an embodiment of the present disclosure. As shown inFIG.4, table400includes the various request types401(e.g., type 1, type 2, type 3), the service pairs402(e.g., X->A”), the time duration403for processing such a request and the document count404(“Doc Count”) corresponding to the number of documents processed during the processing of such a request. For example, as shown inFIG.4, the service pair (“X->A”) processes the request corresponding to request type 1 in 6 seconds involving 0.5 million (0.5M) documents. In another example, the service pair (“A->B”) processes the request corresponding to request type 2 in 8 seconds involving 2.5 million (2.5M) documents. In a further example, the service pair (“B->C”) processes the request corresponding to request type 3 in 6 seconds involving 15 million (15M) documents.

Returning toFIG.2, such information may be used by DBaaS component analyzer219to identify potential bottlenecks in the DBaaS nodes (e.g., DBaaS node204) in DBaaS cluster202. In one embodiment, DBaaS component analyzer219identifies such potential bottlenecks based on information found in tracing data216, such as the time and document count. For example, referring toFIGS.2-4, DBaaS component analyzer219identifies a potential bottleneck304in processing service request303of type 3 by the service pair “B->C” based on time403and/or document count404being above or below a threshold level, which may be established by an expert. For example, a potential bottleneck may be established based on the document count exceeding 14 million over a time frame of 6 seconds. In one embodiment, such relationships between time403and/or document count404corresponding to a potential bottleneck based on exceeding or being less than a threshold level is established by an expert. Such relationships may be stored in a data structure which may be stored in a storage device (e.g., memory, disk unit) of container orchestration system102. In one embodiment, DBaaS component analyzer219accesses the data structure to determine whether a potential bottleneck has been identified in dependency tree300using table400. In one embodiment, DBaaS component analyzer219utilizes a software tool for analyzing the data structure to determine whether a potential bottleneck has been identified in dependency tree300using the information found in tracing data216, such as, but not limited to, IBM® Cognos®, Microsoft® Power BI, Sisense®, Thoughtspot, etc.

Furthermore, as shown inFIG.2, control plane214includes a monitor server220configured to monitor service requests, such as the service requests received by DBaaS cluster202. In one embodiment, monitor server220utilizes various software tools for monitoring service requests, including, but not limited to, New Relic®, Pixie, Google® Kubernetes Engine, Microsoft® Azure Kubernetes Service, etc.

In one embodiment, monitor server220is configured to identify a chain of requests of different types generated from a monitored service request. For example, a CRUD request may be received which automatically generates downstream requests for indexing and replication. Such downstream requests are chained together, such as shown in dependency tree300which illustrates the upstream and downstream relationship. For example, service request301of request type 1 may correspond to a create request, which generates a service request302of request type 2 (e.g., indexing) and a service request303of request type 3 (e.g., replication). In one embodiment, monitor server220identifies the chain of requests of different types generated from a monitored service request based on dependency tree300. For example, if monitor server220receives service request301of type 1, then monitor server220identifies a dependency tree300which is directed to such a service request, which includes the generated downstream requests that are chained together. In one embodiment, monitor server220identifies the appropriate dependency tree300based on matching the received service request with the service request at the root of dependency tree300. In one embodiment, such matching may be accomplished by matching the service request type of the monitored service request with the service request type at the root of dependency tree300.

Furthermore, in one embodiment, monitor server220identifies the services (e.g., services209) in nodes204of DBaaS cluster202to handle the chain of requests from dependency tree300. For example, a service request301of request type 1 (e.g., read request) may generate a service request302of request type 2 (e.g., indexing) and a service request303of request type 3 (e.g., replication), where service pair X->A (services212A,209A) handle request type 1, service pair A->B (services209A,209B) handle request type 2 and service pair B->C (services209B,209C) handle request type 3 as shown in dependency tree300.

Additionally, control plane214includes metrics analyzer221configured to analyze various “consumption predictors” for the components (e.g., services209) of node204of DBaaS cluster202identified as being a potential bottleneck to determine the utilization of the resources for the components (e.g., services209) of node204of DBaaS cluster202identified as being a potential bottleneck.

“Consumption predictors,” as used herein, refer to the metrics that are used to predict utilization of the resources for the components of node204of DBaaS cluster202identified as being a potential bottleneck. For example, such consumption predictors include CPU utilization, memory utilization, disk utilization, input/output utilization, timeline of called components of node204of DBaaS cluster202identified as being a potential bottleneck, a traffic generation model and the relationship of components of node204of DBaaS cluster202identified as being a potential bottleneck.

In one embodiment, metrics analyzer221analyzes the consumption predictors, such as CPU utilization, memory utilization, disk utilization, and input/output utilization, using various software tools, including, but not limited to, Paessler® PRTG, AIDA64 Extreme, Wise System Monitor, Rainmeter, SolarWind® Network Performance Monitor, etc. Based on such an analysis, the utilization of the resources for the components of node204of DBaaS cluster202identified as being a potential bottleneck is obtained.

In one embodiment, the timeline of called components of node204of DBaaS cluster202identified as being a potential bottleneck may be obtained and analyzed by metrics analyzer221based on analyzing tracing data216which includes the timeline of called components of nodes204. For example, such tracing data216includes the time of components (e.g., services209) calling each other, such as the time of service209A calling service209B, etc. In one embodiment, such information may be traced by tracing server215using various software tools, including, but not limited to, Datadog®, Dynatrace®, Retrace®, ContainIQ®, Jaeger, New Relic®, Honeycomb®, etc. In one embodiment, metrics analyzer221analyzes such information in tracing data216using various software tools, including, but not limited to, Dynatrace®, Device42®, Retrace®, ManageEngine® Applications Manager, Datadog®, Extrahop®, AppDynamics®, Pinpoint, etc. Based on identifying the timeline of the called components of node204of DBaaS cluster202identified as being a potential bottleneck, metrics analyzer221determines the extent of utilization of such components, such as whether such components are being utilized to a great extent within a short period of time.

A “traffic generation model,” as used herein, refers to a stochastic model of the packet flow or data sources, such as the traffic flow to DBaaS cluster202. In one embodiment, such a traffic generation model is created by monitor server220using a network traffic generator, such as iperf, bwping and Mausezahn. In one embodiment, such information in the traffic generation model is analyzed by metrics analyzer221via various software tools, including, but not limited to, SolarWinds® Network Traffic Analysis Tool, Auvik®, Wireshark®, Nagios®, etc. Based on analyzing the traffic generation model, metrics analyzer221determines the extent of utilization of such components, such as whether such components are being utilized to a great extent within a short period of time.

In one embodiment, the traffic generation model is created using the directed traffic of service mesh217, which controls the flow of traffic between services, into the mesh, and to outside services. In one embodiment, service mesh217maintains a service registry of all services in the mesh by name and by their respective endpoints. The registry is maintained to manage the flow of traffic (e.g., pod IP addresses). By using this service registry, and by running the proxies side-by-side with the services, service mesh217can direct traffic to the appropriate endpoint. Such directed traffic may be used to generate the traffic generation model.

In one embodiment, the relationship of components of node204of DBaaS cluster202identified as being a potential bottleneck may be identified based on dependency tree300, which illustrates how services (e.g., services209,212) can be paired. Such information may be obtained by metrics analyzer221based on analyzing dependency tree300, such as via various software tools, including, but not limited to, SAS® Visual Analytics, IBM® SPSS® Modeler, Tibco® Spotfire, etc. Such information may be used to determine the resources of which components need to be analyzed to determine their utilization.

Upon determining the utilization of the resources for the components of node204of DBaaS cluster202identified as being a potential bottleneck, consumption predictor222in control plane214predicts future utilization of such resources (e.g., CPU, memory, disk, input/output) based on the current utilization of such resources obtained by metrics analyzer221and the timeline of called components of DBaaS cluster202. In one embodiment, such predicted utilization is based on a machine learning model that is trained to predict the utilization of such resources.

In one embodiment, consumption predictor222uses a machine learning algorithm (e.g., supervised learning) to train a machine learning model to predict utilization of a resource, such as a resource used by the components of node204of DBaaS cluster202identified as being a potential bottleneck, based on the current utilization of the resource and the timeline of called components of DBaaS cluster202. In one embodiment, such training is based on sample data consisting of past utilization data of the resources provided by metrics analyzer221along with tracing data216which identifies the timeline of the called out components as well as the number of service requests handled by such components (e.g., services209) according to such a timeline.

Such sample data is referred to herein as the “training data,” which is used by the machine learning algorithm to make predictions or decisions as to the utilization of the resources used by components based on past utilization of the resources in connection with the timeline of the called out components as well as the number of service requests handled by such components (e.g., services209) according to such a timeline. The algorithm iteratively makes predictions on the training data as to the predicted utilization of the resources until the predictions achieve the desired accuracy as determined by an expert. Examples of such learning algorithms include nearest neighbor, Naïve Bayes, decision trees, linear regression, support vector machines and neural networks.

As a result of training the machine learning model to predict the utilization of a resource, such as a resource used by the components of node204of DBaaS cluster202identified as being a potential bottleneck, based on the timeline of called components of DBaaS cluster202as well as the number of service requests handled by such components (e.g., services209) according to such a timeline, consumption predictor222predicts the future utilization of the resources used by the components of node204of DBaaS cluster202identified as being a potential bottleneck based on the information (the current utilization of resources used by the components of node204of DBaaS cluster202identified as being a potential bottleneck as well as the timeline of called components of DBaaS cluster202) provided by metrics analyzer221as illustrated inFIG.5.

FIG.5illustrates the predicted utilization of the DBaaS cluster resources (e.g., memory, CPU and input/output (I/O)) as well as the predicted service requests handled by the components (e.g., services209) of DBaaS cluster202based on the timeline of called components of DBaaS cluster202in accordance with an embodiment of the present disclosure.

Referring toFIG.5, in conjunction withFIGS.1-4,FIG.5illustrates the number of service requests, including a particular type, such as type 3 (replication), being processed by service A209A and service B209B in column501as well as the memory cost (utilization of memory), the CPU cost (utilization of CPU) and I/O cost (utilization of I/O) as shown in columns502-504, respectively. Such a prediction may be made at various future times, which may be user-designated. For example, table505A depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0. Table505B depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0+10 s (10 seconds in the future). Table505C depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0+20 s (20 seconds in the future). Table505D depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0+30 s (30 seconds in the future).

In one embodiment, the utilization of memory, CPU and I/O may be scaled. In one embodiment, the utilization of the CPU may correspond to the number of units (e.g., number of CPU units). In one embodiment, the utilization of the CPU may correspond to a fraction of complete utilization of the units. For example, an indication of 1.8 may indicate that 1 unit is completely utilized while a second unit is only 80% utilized. In one embodiment, the utilization of the memory may correspond to the number of gigabytes. In one embodiment, the utilization of I/O may correspond to the number of input/output operations.

Returning toFIG.2, control plane214further includes a tuning controller223that is configured to scale one or more pods, such as pods206of DBaaS cluster202, if the predicted utilization of the resources (discussed above) is above or below a threshold level, which may be user-designated. For example, referring toFIG.5, as shown in table505D, the memory cost corresponds to a total of 2.7 units (see element506) and the CPU cost corresponds to a total of 5.4 units (see element507).

In one embodiment, the utilization of such resources may be compared against a threshold level, which may be user-designated. For example, tuning controller223may determine if the memory cost at each of these time periods (e.g., time at 0+10 s, time at 0+20 s, etc.) exceeds a threshold level of 2.5 gigabytes. As shown in table505D ofFIG.5, the memory cost of 2.7 gigabytes exceeds the threshold level of 2.5 gigabytes at element506.

In another example, tuning controller223may determine if the CPU cost at each of these time periods (e.g., time at 0+10 s, time at 0+20 s, etc.) exceeds a threshold level of 5.0 units. As shown in table505D ofFIG.5, the CPU cost of 5.4 units exceeds the threshold level of 5.0 units at element507.

As a result of the predicted utilization of a resource exceeding or being below a threshold level, tuning controller223scales the pods, such as pods206of DBaaS cluster202, such as by increasing or decreasing the number of pods206in the node204of DBaaS cluster202identified as being a potential bottleneck. For instance, tuning controller223may increase the number of pods206of node204by a single pod, which is allocated 2 gigabytes of memory and 1 unit of CPU, which addresses the potential deficiency of memory and CPU utilization.

While the foregoing illustrates increasing the number of pods206of node204identified as being a potential bottleneck, it is noted that the number of pods206of node204may be decreased, such as when the predicted utilization of a resource is below a threshold level.

In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pod(s) in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring.

A further description of these and other features is provided below in connection with the discussion of the method for intelligently scaling DBaaS resources in a cloud platform.

Prior to the discussion of the method for intelligently scaling DBaaS resources in a cloud platform, a description of the hardware configuration of container orchestration system102(FIG.1) is provided below in connection withFIG.6.

Referring now toFIG.6, in conjunction withFIG.1,FIG.6illustrates an embodiment of the present disclosure of the hardware configuration of container orchestration system102which is representative of a hardware environment for practicing the present disclosure.

Computing environment600contains an example of an environment for the execution of at least some of the computer code601involved in performing the inventive methods, such as scaling a container resource of a DBaaS cluster in a cloud platform. In addition to block601, computing environment600includes, for example, container orchestration system102, network103, such as a wide area network (WAN), end user device (EUD)602, remote server603, public cloud604, and private cloud605. In this embodiment, container orchestration system102includes processor set606(including processing circuitry607and cache608), communication fabric609, volatile memory610, persistent storage611(including operating system612and block601, as identified above), peripheral device set613(including user interface (UI) device set614, storage615, and Internet of Things (IoT) sensor set616), and network module617. Remote server603includes remote database618. Public cloud604includes gateway619, cloud orchestration module620, host physical machine set621, virtual machine set622, and container set623.

Processor set606includes one, or more, computer processors of any type now known or to be developed in the future. Processing circuitry607may be distributed over multiple packages, for example, multiple, coordinated integrated circuit chips. Processing circuitry607may implement multiple processor threads and/or multiple processor cores. Cache608is memory that is located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on processor set606. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located “off chip.” In some computing environments, processor set606may be designed for working with qubits and performing quantum computing.

Volatile memory610is any type of volatile memory now known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, the volatile memory is characterized by random access, but this is not required unless affirmatively indicated. In container orchestration system102, the volatile memory610is located in a single package and is internal to container orchestration system102, but, alternatively or additionally, the volatile memory may be distributed over multiple packages and/or located externally with respect to container orchestration system102.

Peripheral device set613includes the set of peripheral devices of container orchestration system102. Data communication connections between the peripheral devices and the other components of container orchestration system102may be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cables (such as universal serial bus (USB) type cables), insertion type connections (for example, secure digital (SD) card), connections made though local area communication networks and even connections made through wide area networks such as the internet. In various embodiments, UI device set614may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smart watches), keyboard, mouse, printer, touchpad, game controllers, and haptic devices. Storage615is external storage, such as an external hard drive, or insertable storage, such as an SD card. Storage615may be persistent and/or volatile. In some embodiments, storage615may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where container orchestration system102is required to have a large amount of storage (for example, where container orchestration system102locally stores and manages a large database) then this storage may be provided by peripheral storage devices designed for storing very large amounts of data, such as a storage area network (SAN) that is shared by multiple, geographically distributed computers. IoT sensor set616is made up of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

End user device (EUD)602is any computer system that is used and controlled by an end user (for example, a customer of an enterprise that operates container orchestration system102), and may take any of the forms discussed above in connection with container orchestration system102. EUD602typically receives helpful and useful data from the operations of container orchestration system102. For example, in a hypothetical case where container orchestration system102is designed to provide a recommendation to an end user, this recommendation would typically be communicated from network module617of container orchestration system102through WAN103to EUD602. In this way, EUD602can display, or otherwise present, the recommendation to an end user. In some embodiments, EUD602may be a client device, such as thin client, heavy client, mainframe computer, desktop computer and so on.

Remote server603is any computer system that serves at least some data and/or functionality to container orchestration system102. Remote server603may be controlled and used by the same entity that operates container orchestration system102. Remote server603represents the machine(s) that collect and store helpful and useful data for use by other computers, such as container orchestration system102. For example, in a hypothetical case where container orchestration system102is designed and programmed to provide a recommendation based on historical data, then this historical data may be provided to container orchestration system102from remote database618of remote server603.

Public cloud604is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and/or other computer capabilities, especially data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages sharing of resources to achieve coherence and economies of scale. The direct and active management of the computing resources of public cloud604is performed by the computer hardware and/or software of cloud orchestration module620. The computing resources provided by public cloud604are typically implemented by virtual computing environments that run on various computers making up the computers of host physical machine set621, which is the universe of physical computers in and/or available to public cloud604. The virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set622and/or containers from container set623. It is understood that these VCEs may be stored as images and may be transferred among and between the various physical machine hosts, either as images or after instantiation of the VCE. Cloud orchestration module620manages the transfer and storage of images, deploys new instantiations of VCEs and manages active instantiations of VCE deployments. Gateway619is the collection of computer software, hardware, and firmware that allows public cloud604to communicate through WAN103.

Private cloud605is similar to public cloud604, except that the computing resources are only available for use by a single enterprise. While private cloud605is depicted as being in communication with WAN103in other embodiments a private cloud may be disconnected from the internet entirely and only accessible through a local/private network. A hybrid cloud is a composition of multiple clouds of different types (for example, private, community or public cloud types), often respectively implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technology that enables orchestration, management, and/or data/application portability between the multiple constituent clouds. In this embodiment, public cloud604and private cloud605are both part of a larger hybrid cloud.

Block601further includes the software components discussed above in connection withFIGS.2-5to intelligently scale DBaaS resources in a cloud platform. In one embodiment, such components may be implemented in hardware. The functions discussed above performed by such components are not generic computer functions. As a result, container orchestration system102is a particular machine that is the result of implementing specific, non-generic computer functions.

In one embodiment, the functionality of such software components of container orchestration system102, including the functionality for intelligently scaling DBaaS resources in a cloud platform may be embodied in an application specific integrated circuit.

As stated above, the DBaaS cluster may receive various types of requests to be processed. For example, the DBaaS cluster may receive a create, read, update or delete request to be processed. Such a request (create, read, update or delete requests are collectively referred to as a “CRUD” request) though may automatically generate numerous downstream requests, such as for indexing and replication. For example, the CRUD request may generate downstream requests for indexing and replication which are processed by the containers of the pods in the DBaaS cluster. For instance, the containers of a particular pod may be utilized to process the downstream requests for indexing and the containers of another particular pod may be utilized to process the downstream requests for replication. As a result of a request being converted into multiple requests with upstream and downstream relationships, the DBaaS cluster may not be able to service such requests in an efficient manner thereby resulting in a system bottleneck which negatively impacts system performance. A “system bottleneck,” as used herein, refers to an overloaded system in which components of the system, such as the DBaaS cluster, are unable to keep pace with the system thereby slowing overall performance. Unfortunately, the scaling mechanism discussed above, such as horizontal scaling by a horizontal pod autoscaler, is only able to address such a system bottleneck after the problem has impacted system performance. For example, such scaling occurs after the observed metrics, such as memory and CPU performance, has indicated that scaling is necessary. As a result, the scaling mechanism discussed above does not address such a system bottleneck since the DBaaS throughput cannot be changed in time in the cloud platform.

The embodiments of the present disclosure provide a means for addressing system bottlenecks at the DBaaS by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pods in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring as discussed below in connection withFIGS.7-8.FIG.7is a flowchart of a method for generating a dependency tree to find potential bottlenecks in the DBaaS nodes of the DBaaS cluster.FIG.8is a flowchart of a method for scaling the pods in the DBaaS nodes of the DBaaS cluster identified as being potential bottlenecks.

As stated above,FIG.7is a flowchart of a method700for generating a dependency tree to find potential bottlenecks in the DBaaS nodes of the DBaaS cluster in accordance with an embodiment of the present disclosure.

Referring toFIG.7, in conjunction withFIGS.1-6, in operation701, ingress gateway213receives user service requests to be processed by DBaaS cluster202. As discussed above, ingress gateway213receives service requests issued by a user, such as a user of computing device101. Such service requests may then be forwarded to a sidecar211which invokes a DBaaS service to handle such a service request.

In operation702, service mesh217generates tracing data216(first set of tracing data) from the user service requests.

As discussed above, in one embodiment, tracing data216may be obtained from a service mesh217that facilitates service-to-service communication between service cluster203and DBaaS cluster202. In one embodiment, service mesh217controls how different parts of an application share data with one another. In one embodiment, service mesh217corresponds to a dedicated infrastructure layer for facilitating service-to-service communications between services or microservices, using a proxy. In one embodiment, service mesh217consists of network proxies paired with each service in an application and a set of task management processed. The proxies are called the “data plane”201and the management processes are called the “control plane”214. In one embodiment, data plane201intercepts calls between different services and processes them; whereas, control plane214coordinates the behavior of proxies and provides APIs for operations and maintenance. In one embodiment, the service mesh architecture is implemented using various software tools including, but not limited to, Istio®, Linkerd®, Consul®, Traefik Mesh®, etc.

In one embodiment, service mesh217generates tracing data216by generating distributed traces spans for each service within it (see lines from sidecars208,211to tracing data216as shown inFIG.2). Such traces can be used to follow a single request (user service request received by ingress gateway213) through the mesh across multiple services and proxies.

In operation703, service mesh217stores the generated tracing data216(generated in operation702) in tracing server215.

In operation704, DBaaS cluster202generates tracing data216(second set of tracing data) from handling the user service requests.

In operation705, distributed tracing tool218stores the generated tracing data216(generated in operation704) in tracing server215.

In operation707, DBaaS component analyzer219generates a dependency tree, such as dependency tree300, to find potential bottlenecks in DBaaS nodes204of DBaaS cluster202based on analyzing tracing data216.

As discussed above, in one embodiment, DBaaS component analyzer219generates a dependency tree, such as dependency tree300, based on analyzing tracing data216using various software tools including, but not limited to, SolarWinds® Server and Application Monitor, Dynatrace®, Device42®, Retrace®, ManageEngine® Applications Manager, DataDog®, Extrahop®, AppDynamics®, Pinpoint, etc.

An illustration of such a dependency tree300is shown inFIG.3.

As shown inFIG.3, in conjunction withFIG.2, dependency tree300illustrates the chain of requests that are spawned from an initially received user request. For instance, a CRUD request may be received which automatically generates downstream requests for indexing and replication. Such downstream requests are chained together in dependency tree300showing the upstream and downstream relationship. For example, service request301of request type 1 may correspond to a create request, which generates a service request302of request type 2 (e.g., indexing) and a service request303of request type 3 (e.g., replication).

Additionally, as shown inFIG.3, in conjunction withFIG.2, dependency tree300illustrates the various services handling such request types, and the relationship between such services in terms of service pairs. For example, service X212A receives a service request301of request type 1, which is transferred to service A209A to handle. As a result, service X212A and service A209A form a service pair as identified as “X->A.” Furthermore, as shown inFIG.3, the service request302of request type 2 (generated from service request301) is transferred from service A209A to service B209B to handle thereby forming service pair “A->B.” Additionally, as shown inFIG.3, the service request303of request type 3 (generated from service request302) is transferred from service B209B to service C209C to handle thereby forming service pair “B->C.”

Furthermore, tracing data216includes the time duration (referred to herein as simply “time”) for handling such a service request type and the document count, which may be used for determining a potential bottleneck. For instance, as shown inFIG.4,FIG.4illustrates a table400representing the tracing data216that is used for generating dependency tree300ofFIG.3in accordance with an embodiment of the present disclosure. As shown inFIG.4, table400includes the various request types401(e.g., type 1, type 2, type 3), the service pairs402(e.g., X->A”), the time duration403for processing such a request and the document count404(“Doc Count”) corresponding to the number of documents processed during the processing of such a request. For example, as shown inFIG.4, the service pair (“X->A”) processes the request corresponding to request type 1 in 6 seconds involving 0.5 million (0.5M) documents. In another example, the service pair (“A->B”) processes the request corresponding to request type 2 in 8 seconds involving 2.5 million (2.5M) documents. In a further example, the service pair (“B->C”) processes the request corresponding to request type 3 in 6 seconds involving 15 million (15M) documents.

Such information may be used by DBaaS component analyzer219to identify potential bottlenecks in the DBaaS nodes (e.g., DBaaS node204) in DBaaS cluster202. In one embodiment, DBaaS component analyzer219identifies such potential bottlenecks based on information found in tracing data216, such as the time and document count. For example, DBaaS component analyzer219identifies a potential bottleneck304in processing service request303of type 3 by the service pair “B->C” based on time403and/or document count404being above or below a threshold level, which may be established by an expert. For example, a potential bottleneck may be established based on the document count exceeding 14 million over a time frame of 6 seconds. In one embodiment, such relationships between time403and/or document count404corresponding to a potential bottleneck based on exceeding or being less than a threshold level is established by an expert. Such relationships may be stored in a data structure which may be stored in a storage device (e.g., storage device611,615) of container orchestration system102. In one embodiment, DBaaS component analyzer219accesses the data structure to determine whether a potential bottleneck has been identified in dependency tree300using table400. In one embodiment, DBaaS component analyzer219utilizes a software tool for analyzing the data structure to determine whether a potential bottleneck has been identified in dependency tree300using the information found in tracing data216, such as, but not limited to, IBM® Cognos®, Microsoft® Power BI, Sisense®, Thoughtspot, etc.

Upon generating dependency tree300, pods206in DBaaS cluster202may be scaled (scaled up or down) in order to address potential bottlenecks as discussed below in connection withFIG.8.

FIG.8is a flowchart of a method800for scaling pods206in DBaaS nodes204of DBaaS cluster202identified as being potential bottlenecks in accordance with an embodiment of the present disclosure.

Referring toFIG.8, in conjunction withFIGS.1-7, in operation801, monitor server220monitors the service requests received by DBaaS cluster202.

In operation802, monitor server220identifies a chain of requests of different types generated from a monitored service request.

For example, a CRUD request may be received which automatically generates downstream requests for indexing and replication. Such downstream requests are chained together, such as shown in dependency tree300which illustrates the upstream and downstream relationship. For example, service request301of request type 1 may correspond to a create request, which generates a service request302of request type 2 (e.g., indexing) and a service request303of request type 3 (e.g., replication). In one embodiment, monitor server220identifies the chain of requests of different types generated from a monitored service request based on dependency tree300. For example, if monitor server220receives service request301of type 1, then monitor server220identifies a dependency tree300which is directed to such a service request, which includes the generated downstream requests that are chained together. In one embodiment, monitor server220identifies the appropriate dependency tree300based on matching the received service request with the service request at the root of dependency tree300. In one embodiment, such matching may be accomplished by matching the service request type of the monitored service request with the service request type at the root of dependency tree300.

In operation803, monitor server220identifies the services (e.g., services209) in nodes204of DBaaS cluster202to handle the chain of requests from dependency tree300.

For example, as shown inFIG.3, a service request301of request type 1 (e.g., read request) may generate a service request302of request type 2 (e.g., indexing) and a service request303of request type 3 (e.g., replication), where service pair X->A (services212A,209A) handle request type 1, service pair A->B (services209A,209B) handle request type 2 and service pair B->C (services209B,209C) handle request type 3 as shown in dependency tree300.

In operation804, DBaaS component analyzer219identifies a potential bottleneck in handling the identified services in a node204of DBaaS cluster202using dependency tree300and tracing data216.

As discussed above, in one embodiment, DBaaS component analyzer219identifies such potential bottlenecks based on information found in tracing data216, such as the time and document count, in connection with dependency tree300. For example, referring toFIGS.2-4, DBaaS component analyzer219identifies a potential bottleneck304in processing service request303of type 3 by the service pair “B->C” based on time403and/or document count404being above or below a threshold level, which may be established by an expert. For example, a potential bottleneck may be established based on the document count exceeding 14 million over a time frame of 6 seconds. In one embodiment, such relationships between time403and/or document count404corresponding to a potential bottleneck based on exceeding or being less than a threshold level is established by an expert. Such relationships may be stored in a data structure which may be stored in a storage device (e.g., storage device611,615) of container orchestration system102. In one embodiment, DBaaS component analyzer219accesses the data structure to determine whether a potential bottleneck has been identified in dependency tree300using table400. In one embodiment, DBaaS component analyzer219utilizes a software tool for analyzing the data structure to determine whether a potential bottleneck has been identified in dependency tree300using the information found in tracing data216, such as, but not limited to, IBM® Cognos®, Microsoft® Power BI, Sisense®, Thoughtspot, etc.

In operation805, metrics analyzer221analyzes consumption predictors for the components (e.g., services209) of DBaaS node204of DBaaS cluster202identified as being a potential bottleneck.

In operation806, metrics analyzer221determines the utilization of the resources for the components (e.g., services209) of DBaaS node204of DBaaS cluster202identified as being a potential bottleneck based on the analyzed consumption predictors.

As discussed above, “consumption predictors,” as used herein, refer to the metrics that are used to predict utilization of the resources for the components of node204of DBaaS cluster202identified as being a potential bottleneck. For example, such consumption predictors include CPU utilization, memory utilization, disk utilization, input/output utilization, timeline of called components of node204of DBaaS cluster202identified as being a potential bottleneck, a traffic generation model and the relationship of components of node204of DBaaS cluster202identified as being a potential bottleneck.

In one embodiment, metrics analyzer221analyzes the consumption predictors, such as CPU utilization, memory utilization, disk utilization, and input/output utilization, using various software tools, including, but not limited to, Paessler® PRTG, AIDA64 Extreme, Wise System Monitor, Rainmeter, SolarWind® Network Performance Monitor, etc. Based on such an analysis, the utilization of the resources for the components of node204of DBaaS cluster202identified as being a potential bottleneck is obtained.

In one embodiment, the timeline of called components of node204of DBaaS cluster202identified as being a potential bottleneck may be obtained and analyzed by metrics analyzer221based on analyzing tracing data216which includes the timeline of called components of nodes204. For example, such tracing data216includes the time of components (e.g., services209) calling each other, such as the time of service209A calling service209B, etc. In one embodiment, such information may be traced by tracing server215using various software tools, including, but not limited to, Datadog®, Dynatrace®, Retrace®, ContainIQ®, Jaeger, New Relic®, Honeycomb®, etc. In one embodiment, metrics analyzer221analyzes such information in tracing data216using various software tools, including, but not limited to, Dynatrace®, Device42®, Retrace®, ManageEngine® Applications Manager, Datadog®, Extrahop®, AppDynamics®, Pinpoint, etc. Based on identifying the timeline of the called components of node204of DBaaS cluster202identified as being a potential bottleneck, metrics analyzer221determines the extent of utilization of such components, such as whether such components are being utilized to a great extent within a short period of time.

A “traffic generation model,” as used herein, refers to a stochastic model of the packet flow or data sources, such as the traffic flow to DBaaS cluster202. In one embodiment, such a traffic generation model is created by monitor server220using a network traffic generator, such as iperf, bwping and Mausezahn. In one embodiment, such information in the traffic generation model is analyzed by metrics analyzer221via various software tools, including, but not limited to, SolarWinds® Network Traffic Analysis Tool, Auvik®, Wireshark®, Nagios®, etc. Based on analyzing the traffic generation model, metrics analyzer221determines the extent of utilization of such components, such as whether such components are being utilized to a great extent within a short period of time.

In one embodiment, the traffic generation model is created using the directed traffic of service mesh217, which controls the flow of traffic between services, into the mesh, and to outside services. In one embodiment, service mesh217maintains a service registry of all services in the mesh by name and by their respective endpoints. The registry is maintained to manage the flow of traffic (e.g., pod IP addresses). By using this service registry, and by running the proxies side-by-side with the services, service mesh217can direct traffic to the appropriate endpoint. Such directed traffic may be used to generate the traffic generation model.

In one embodiment, the relationship of components of node204of DBaaS cluster202identified as being a potential bottleneck may be identified based on dependency tree300, which illustrates how services (e.g., services209,212) can be paired. Such information may be obtained by metrics analyzer221based on analyzing dependency tree300, such as via various software tools, including, but not limited to, SAS® Visual Analytics, IBM® SPSS® Modeler, Tibco® Spotfire, etc. Such information may be used to determine the resources of which components need to be analyzed to determine their utilization.

In operation807, upon determining the utilization of the resources for the components of node204of DBaaS cluster202identified as being a potential bottleneck, consumption predictor222predicts the utilization of resources for the components (e.g., services209) of DBaaS node204of DBaaS cluster202identified as being a potential bottleneck based on the determined utilization of such resources obtained by metrics analyzer221in operation806and the timeline of called components of DBaaS cluster202.

As stated above, in one embodiment, such predicted utilization is based on a machine learning model that is trained to predict the utilization of such resources.

In one embodiment, consumption predictor222uses a machine learning algorithm (e.g., supervised learning) to train a machine learning model to predict utilization of a resource, such as a resource used by the components of node204of DBaaS cluster202identified as being a potential bottleneck, based on the current utilization of the resource and the timeline of called components of DBaaS cluster202. In one embodiment, such training is based on sample data consisting of past utilization data of the resources provided by metrics analyzer221along with tracing data216which identifies the timeline of the called out components as well as the number of service requests handled by such components (e.g., services209) according to such a timeline.

Such sample data is referred to herein as the “training data,” which is used by the machine learning algorithm to make predictions or decisions as to the utilization of the resources used by components based on past utilization of the resources in connection with the timeline of the called out components as well as the number of service requests handled by such components (e.g., services209) according to such a timeline. The algorithm iteratively makes predictions on the training data as to the predicted utilization of the resources until the predictions achieve the desired accuracy as determined by an expert. Examples of such learning algorithms include nearest neighbor, Naïve Bayes, decision trees, linear regression, support vector machines and neural networks.

As a result of training the machine learning model to predict the utilization of a resource, such as a resource used by the components of node204of DBaaS cluster202identified as being a potential bottleneck, based on the timeline of called components of DBaaS cluster202as well as the number of service requests handled by such components (e.g., services209) according to such a timeline, consumption predictor222predicts the future utilization of the resources used by the components of node204of DBaaS cluster202identified as being a potential bottleneck based on the information (the current utilization of resources used by the components of node204of DBaaS cluster202identified as being a potential bottleneck as well as the timeline of called components of DBaaS cluster202) provided by metrics analyzer221as illustrated inFIG.5.

FIG.5illustrates the predicted utilization of the DBaaS cluster resources (e.g., memory, CPU and input/output (I/O)) as well as the predicted service requests handled by the components (e.g., services209) of DBaaS cluster202based on the timeline of called components of DBaaS cluster202in accordance with an embodiment of the present disclosure.

Referring toFIG.5, in conjunction withFIGS.1-4,FIG.5illustrates the number of service requests, including a particular type, such as type 3 (replication), being processed by service A209A and service B209B in column501as well as the memory cost (utilization of memory), the CPU cost (utilization of CPU) and I/O cost (utilization of I/O) as shown in columns502-504, respectively. Such a prediction may be made at various future times, which may be user-designated. For example, table505A depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0. Table505B depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0+10 s (10 seconds in the future). Table505C depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0+20 s (20 seconds in the future). Table505D depicts such information (service requests being processed by services A and B and memory, CPU and I/O utilization) at time 0+30 s (30 seconds in the future).

In one embodiment, the utilization of memory, CPU and I/O may be scaled. In one embodiment, the utilization of the CPU may correspond to the number of units (e.g., number of CPU units). In one embodiment, the utilization of the CPU may correspond to a fraction of complete utilization of the units. For example, an indication of 1.8 may indicate that 1 unit is completely utilized while a second unit is only 80% utilized. In one embodiment, the utilization of the memory may correspond to the number of gigabytes. In one embodiment, the utilization of I/O may correspond to the number of input/output operations.

In operation808, tuning controller223determines if the predicted utilization of a resource for a component (e.g., service209) of DBaaS node204identified as being a potential bottleneck is above or below a threshold level, which may be user-designated.

If the predicted utilization of a resource exceeds a threshold level, then, in operation809, tuning controller223executes a scale operation to scale (increase or decrease) the number of pods206in DBaaS node204identified as being a potential bottleneck.

For example, referring toFIG.5, as shown in table505D, the memory cost corresponds to a total of 2.7 units (see element506) and the CPU cost corresponds to a total of 5.4 units (see element507).

In one embodiment, the utilization of such resources may be compared against a threshold level, which may be user-designated. For example, tuning controller223may determine if the memory cost at each of these time periods (e.g., time at 0+10 s, time at 0+20 s, etc.) exceeds a threshold level of 2.5 gigabytes. As shown in table505D ofFIG.5, the memory cost of 2.7 gigabytes exceeds the threshold level of 2.5 gigabytes at element506.

In another example, tuning controller223may determine if the CPU cost at each of these time periods (e.g., time at 0+10 s, time at 0+20 s, etc.) exceeds a threshold level of 5.0 units. As shown in table505D ofFIG.5, the CPU cost of 5.4 units exceeds the threshold level of 5.0 units at element507.

As a result of the predicted utilization of a resource exceeding or being below a threshold level, tuning controller223scales the pods, such as pods206of DBaaS cluster202, such as by increasing or decreasing the number of pods206in the node204of DBaaS cluster202identified as being a potential bottleneck. For instance, tuning controller223may increase the number of pods206of node204by a single pod, which is allocated 2 gigabytes of memory and 1 unit of CPU, which addresses the potential deficiency of memory and CPU utilization.

In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pod(s) in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring.

If, however, the predicted utilization of a resource is not above or below a threshold level, then, in operation810, tuning controller223does not execute a scale operation to scale (increase or decrease) the number of pods206in DBaaS node204identified as being a potential bottleneck.

As a result of the foregoing, embodiments of the present disclosure dynamically tune DBaaS performance in the cloud platform. Furthermore, embodiments of the present disclosure resolve the limitation of the DBaaS throughput not being able to be changed in time in the cloud platform when a bottleneck occurs in the DBaaS cluster.

Furthermore, the principles of the present disclosure improve the technology or technical field involving Database as a Service (DBaaS). As discussed above, the DBaaS cluster may receive various types of requests to be processed. For example, the DBaaS cluster may receive a create, read, update or delete request to be processed. Such a request (create, read, update or delete requests are collectively referred to as a “CRUD” request) though may automatically generate numerous downstream requests, such as for indexing and replication. For example, the CRUD request may generate downstream requests for indexing and replication which are processed by the containers of the pods in the DBaaS cluster. For instance, the containers of a particular pod may be utilized to process the downstream requests for indexing and the containers of another particular pod may be utilized to process the downstream requests for replication. As a result of a request being converted into multiple requests with upstream and downstream relationships, the DBaaS cluster may not be able to service such requests in an efficient manner thereby resulting in a system bottleneck which negatively impacts system performance. A “system bottleneck,” as used herein, refers to an overloaded system in which components of the system, such as the DBaaS cluster, are unable to keep pace with the system thereby slowing overall performance. Unfortunately, the scaling mechanism discussed above, such as horizontal scaling by a horizontal pod autoscaler, is only able to address such a system bottleneck after the problem has impacted system performance. For example, such scaling occurs after the observed metrics, such as memory and CPU performance, has indicated that scaling is necessary. As a result, the scaling mechanism discussed above does not address such a system bottleneck since the DBaaS throughput cannot be changed in time in the cloud platform.

Embodiments of the present disclosure improve such technology by receiving user service requests from a service cluster to be processed by the DBaaS cluster. A “service cluster,” as used herein, refers to a cluster of nodes for receiving and forwarding service requests to the DBaaS cluster. A “DBaaS cluster,” as used herein, refers to a cluster of nodes for handling such service requests. For example, an ingress gateway of the service cluster may receive and forward such requests to a sidecar which invokes a DBaaS service to handle such a service request. The DBaaS cluster and the service cluster each consists of a set of worker machines, called nodes, that run containerized applications (containerized applications package an application with its dependencies and necessary services). Each of the nodes may include one or more pods containing a group of one or more containers. A “container,” as used herein, refers to a standard unit of software that packages up code and all its dependencies so that the application runs quickly and reliably from one computing environment to another. A first set of tracing data from the user service requests is generated by a service mesh facilitating service-to-service communication between the service cluster and the DBaaS cluster. A second set of tracing data is generated by the DBaaS cluster from handling the user service requests. Such tracing data (both first and second sets) illustrates how the service components of a node of a DBaaS cluster operate, execute and perform in handling service requests. A dependency tree is then generated to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on the first and second sets of tracing data. A “dependency tree,” as used herein, refers to a graph illustrating the relationship between the services, such as the service pairs handling a particular type of request (e.g., create request, indexing, replication). One or more pods of a node of the DBaaS cluster are then scaled (scaled up or down) based on the dependency tree, which is used in part, to predict the utilization of the resources of the components of the DBaaS node identified as being a potential bottleneck. When the predicted utilization of the resources is above or below a threshold level, a scale operation is executed to scale the pod(s) of the DBaaS node identified as being a potential bottleneck. In this manner, system bottlenecks at the DBaaS are addressed by identifying potential bottlenecks involving nodes of the DBaaS cluster and intelligently scaling the pod(s) in a node of the DBaaS cluster identified as being a potential bottleneck prior to the bottleneck actually occurring. Furthermore, in this manner, there is an improvement in the technical field involving Database as a Service (DBaaS).

The technical solution provided by the present disclosure cannot be performed in the human mind or by a human using a pen and paper. That is, the technical solution provided by the present disclosure could not be accomplished in the human mind or by a human using a pen and paper in any reasonable amount of time and with any reasonable expectation of accuracy without the use of a computer.

In one embodiment of the present disclosure, a computer-implemented method for scaling a resource of a Database as a Service (DBaaS) cluster in a cloud platform comprises receiving user service requests from a service cluster to be processed by the DBaaS cluster, where the DBaaS cluster comprises one or more nodes, and where each of the one or more nodes comprises one or more pods containing a group of one or more containers. The method further comprises generating a first set of tracing data from the user service requests by a service mesh facilitating service-to-service communication between the service cluster and the DBaaS cluster. The method additionally comprises generating a second set of tracing data by the DBaaS cluster from handling the user service requests. Furthermore, the method comprises generating a dependency tree to discover application relationships to identify potential bottlenecks in nodes of the DBaaS cluster based on the first and second sets of tracing data. Additionally, the method comprises scaling one or more pods of a node of the DBaaS cluster based on the dependency tree.

Furthermore, in one embodiment of the present disclosure, the method additionally comprises analyzing the first and second sets of tracing data. The method further comprises generating the dependency tree based on the analyzing of the first and second sets of tracing data.

Additionally, in one embodiment of the present disclosure, the method further comprises monitoring service requests received by the DBaaS cluster. The method additionally comprises identifying a chain of requests of different types generated from a monitored service request. Furthermore, the method comprises identifying services in nodes of the DBaaS cluster to handle the chain of requests from the dependency tree.

Furthermore, in one embodiment of the present disclosure, the method additionally comprises identifying a potential bottleneck in handling the identified services in a node of the DBaaS cluster using the dependency tree and the first and second sets of tracing data.

Additionally, in one embodiment of the present disclosure, the method further comprises analyzing consumption predictors for components of the node of the DBaaS cluster identified as being the potential bottleneck. The method additionally comprises determining utilization of resources for the components of the node of the DBaaS cluster identified as being the potential bottleneck based on the analyzed consumption predictors.

Furthermore, in one embodiment of the present disclosure, the method additionally comprises predicting utilization of resources for the components of the node of the DBaaS cluster identified as being the potential bottleneck based on the determined utilization of resources for the components of the node of the DBaaS cluster identified as being the potential bottleneck and a timeline of called components of the DBaaS cluster. The method further comprises executing a scale operation to scale a number of the one or more pods in the node of the DBaaS cluster identified as being the potential bottleneck in response to the predicted utilization of resources being above or below a threshold level.

Additionally, in one embodiment of the present disclosure, the method further comprises having the consumption predictors comprise one or more of the following: CPU utilization, memory utilization, disk utilization, input/output utilization, timeline of called components of the node of the DBaaS cluster identified as being the potential bottleneck, a traffic generation model and relationship of components of the node of the DBaaS cluster identified as being the potential bottleneck.

Other forms of the embodiments of the computer-implemented method described above are in a system and in a computer program product.