Dynamically scaling out pods using a recursive way

In an approach for pod scheduling and recursion, a processor schedules a master pod and one or more worker pods for applications to be deployed on a cluster. A processor builds a topology between the master pod and the one or more worker pods. A processor monitors a workload in the one or more worker pods. A processor determines whether any of the one or more worker pods needs to scale out. In response to determining that one of the one or more worker pods needs to scale out, a processor schedules a next layer of the one or more worker pods according to the workload.

BACKGROUND

The present disclosure relates generally to the field of a container orchestration platform, and more particularly to dynamically scaling out pods.

A container orchestration platform is an open-source system for automating deployment, scaling, and management of containerized applications. The container orchestration platform may group containers that make up an application into logical units for easy management and discovery. Container orchestration is an automatic process of managing or scheduling the work of individual containers for applications based on microservices within multiple clusters. A pod may be a basic execution unit of a containerized application. A pod may represent a process running on a cluster. A pod may encapsulate an application's container (or, in some cases, multiple containers), storage resources, and options that govern how the container(s) should run. A pod may represent a unit of deployment: a single instance of an application in a container orchestration platform, which may include one or more containers that are tightly coupled and that share resources.

SUMMARY

Aspects of an embodiment of the present disclosure disclose an approach for pod scheduling and recursion. A processor schedules a master pod and one or more worker pods for applications to be deployed on a cluster. A processor builds a topology between the master pod and the one or more worker pods. A processor monitors a workload in the one or more worker pods. A processor determines whether any of the one or more worker pods needs to scale out. In response to determining that one of the one or more worker pods needs to scale out, a processor schedules a next layer of the one or more worker pods according to the workload.

DETAILED DESCRIPTION

The present disclosure is directed to systems and methods for dynamically scaling out master and worker pods using a recursive way in a container orchestration platform.

Auto-scaling may be used in a container orchestration platform. The container orchestration platform may scale out resources belonging to the same type of pods. Embodiments of the present disclosure recognize that scaling may not be just a single type of pods, but with different combination, such as a master-worker mode, master-master mode, or other star, net, or service mesh mode. Embodiments of the present disclosure disclose dynamically scaling out master-worker pods using a recursive way via an operator in a container orchestration platform. Embodiments of the present disclosure recognize not only scaling worker pods but also scaling a master pod. For example, in order to support performance analysis based on a distributed transaction database and performance benchmark tool, embodiments of the present disclosure disclose deployments of a master-worker mode in a container orchestration platform, e.g., in a Kubernetes orchestration platform.

Embodiments of the present disclosure disclose different types of service in a container orchestration platform, which may include a Deployment, a StatefulSet, a DaemonSet, a Job and a Cron Job. Embodiments of the present disclosure recognize a need of supporting running applications once or several times in a Deployment, a StatefulSet, and a DaemonSet. Embodiments of the present disclosure recognize a need of supporting a pre-defined logic process in a Job and a Cron Job. Embodiments of the present disclosure recognize a need of supporting a dynamic process with vertical or horizontal pod scaling.

Embodiments of the present disclosure disclose that a server may be a distributed runtime service, for example, including Deployment, StatefulSet, and DaemonSet. A client may also be distributed deployment. Client applications may be deployed on worker pods. Workloads may be scheduled by a master pod. Worker pods may communicate with the server. A Kubernetes Operator may reside alongside worker pods to monitor and take action to reclusively scale out with different topology trees. Embodiments of the present disclosure disclose building up a client distributed application deployment with a Kubernetes Job. A master pod can be triggered with a Job. Worker jobs may be triggered with the Job associated with the master pod. A master Job and worker Jobs may be created using kubectl with yaml files. A master pod and batch worker pods (1˜n) may be triggered with custom resource definition (CRD) operators. Embodiments of the present disclosure disclose building up communication between master pod and worker pods. A new layer of worker pods can be created by CRD operators on demand. Embodiments of the present disclosure disclose no need for a master to know how many pods to be scheduled in advance. The deployment topology may not need to be changed after scaled-out. Embodiments of the present disclosure disclose not only a master-worker deployment model, but also multiple deployment models, e.g., star, net, and mesh models.

Embodiments of the present disclosure disclose dynamically scaling out master-worker pods using a recursive way. Embodiments of the present disclosure disclose detecting a trigger condition, for example, exceeding a threshold, converting a worker pod to a second layer of a master pod, scheduling a second layer of worker pods, and building up relationship between new master pod and worker pods. Embodiments of the present disclosure disclose creating yet another layer of master-worker pods if the second layer of worker pods needs to be scaled-out, and so on. Embodiments of the present disclosure disclose that the master and slave pods can be built up with a hierarchy tree topology to satisfy distributed application requirements. The result may be returned from worker pods to master pod layer by layer, and be consolidated in a master node finally in the container orchestration platform.

The present disclosure will now be described in detail with reference to the Figures.FIG. 1is a functional block diagram illustrating a container orchestration platform scaling environment, generally designated100, in accordance with an embodiment of the present disclosure.

In the depicted embodiment, platform scaling environment100includes master node102, worker node104, and network120. In an embodiment, container orchestration platform scaling environment100may be a platform for scheduling and automating the deployment, management, and scaling of containerized applications. In an example, container orchestration platform scaling environment100may be a Kubernetes platform for automating application deployment. In some embodiments, container orchestration platform scaling environment100may be a cluster that may include multiple worker nodes104that deploy, run, and manage containerized applications and one master node102that controls and monitors the worker nodes. A cluster may be a Kubernetes cluster that is a set of node machines for running containerized applications. A cluster may include at least a worker node104and a master node102. A cluster may be a collection of cloud resources required for container running. A cluster may be associated with cloud resources such as cloud server nodes, load balancers, and virtual private clouds. A cluster may include one or more nodes. A node may be a virtual or physical machine that provides computing resources. A node may create workloads. A workload may control one or more pods. A workload may be an abstract model of a group of pods. A pod may represent a running process on a node in the cluster. A pod may include one or more containers. A cluster may have at least one worker node104. Worker node104may host pods that are the components of the application. Worker node104may deploy, run, and manage containerized applications. Master node102may manage, control, and monitor worker node104. Master node102may run a scheduler service that automates when and where the containers are deployed based on developer-set deployment requirements and available computing capacity. Worker node104may include a tool that is being used to manage the containers and a software agent that receives and executes orders from master node102. A container may be an executable unit of software in which application code is packaged together with libraries and dependencies.

In various embodiments of the present disclosure, master node102can be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a mobile phone, a smartphone, a smart watch, a wearable computing device, a personal digital assistant (PDA), or a server. In another embodiment, master node102represents a computing system utilizing clustered computers and components to act as a single pool of seamless resources. In other embodiments, master node102may represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In other embodiments, master node102may be of a standard compute engine machine type. In general, master node102can be any computing device or a combination of devices with access to worker node104and network120and is capable of processing program instructions, in accordance with an embodiment of the present disclosure. Master node102may include internal and external hardware components, as depicted and described in further detail with respect toFIG. 4.

In various embodiments of the present disclosure, worker node104can be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a mobile phone, a smartphone, a smart watch, a wearable computing device, a personal digital assistant (PDA), or a server. In another embodiment, worker node104represents a computing system utilizing clustered computers and components to act as a single pool of seamless resources. In other embodiments, worker node104may represent a server computing system utilizing multiple computers as a server system, such as in a cloud computing environment. In other embodiments, worker node104may be of a standard compute engine machine type. In general, worker node104can be any computing device or a combination of devices with access to master node102and network120and is capable of processing program instructions and executing operation module106, master pod108, and worker pods110A-N, in accordance with an embodiment of the present disclosure. Worker node104may include internal and external hardware components, as depicted and described in further detail with respect toFIG. 4.

Further, in the depicted embodiment, worker node104includes master pod108and worker pods110A-N. In the depicted embodiment, master pod108and worker pods110A-N are located on worker node104. However, in other embodiments, master pod108and worker pods110A-N may be located externally and accessed through a communication network such as network120. The communication network can be, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of the two, and may include wired, wireless, fiber optic or any other connection known in the art. In general, the communication network can be any combination of connections and protocols that will support communications between worker node104and master pod108and worker pods110A-N, in accordance with a desired embodiment of the disclosure.

In one or more embodiments, master pod108and worker pods110A-N may be pods that are groups of containers that share the same compute resources and the same network. Master pod108and worker pods110A-N may be a unit of scalability in container orchestration platform scaling environment100, for example, in a Kubernetes orchestration platform. Master pod108and worker pods110A-N may be pods that are a model of the pattern of multiple cooperating processes which form a cohesive unit of service. Master pod108and worker pods110A-N may simplify application deployment and management by providing a higher-level abstraction than the set of the constituent applications. Master pod108and worker pods110A-N may serve as unit of deployment, horizontal scaling, and replication. Master pod108and worker pods110A-N may automatically handle co-scheduling, termination, coordinated replication, resource sharing, and dependency management for containers in a pod. Master pod108and worker pods110A-N may enable data sharing and communication among the constituents of the pods. The applications in a pod may use a same network namespace and can thus find each other and communicate using localhost.

A workload may be an object that sets deployment rules for pods. Based on these rules, a deployment can be performed and the workload can be updated with the current state of an application. A workload may control one or more pods. A workload may be a group of pods which may be classified into Deployments, StatefulSets, DaemonSets, jobs, and cron jobs. A Deployment may provide declarative updates for pods and ReplicaSets. A Deployment may change the actual state to the desired state at a controlled rate. A Deployment may create new ReplicaSets, or to remove existing deployments and adopt all existing deployment resources with new deployments. A StatefulSet may be a workload application programming interface (API) object used to manage stateful applications. A StatefulSet may manage the deployment and scaling of a set of pods and provide guarantees about the ordering and uniqueness of the pods. A DaemonSet may ensure that all (or some) nodes run a copy of a pod. As nodes are added to the cluster, pods may be added to the nodes. As nodes are removed from the cluster, pods may be garbage collected. A Job may create one or more pods and may ensure that a specified number of pods successfully terminate. As pods successfully complete, the Job may track the successful completions. When a specified number of successful completions is reached, the task is complete. Deleting a Job may clean up the pods the Job created. A Cron Job may create Jobs on a time-based schedule. A Cron Job runs a job periodically on a given schedule, written in Cron format.

In one or more embodiments, master pod108may schedule workloads in work node104. Master pod108may be triggered by a Job. Master pod108may trigger worker pods110A-N. Worker pods110A-N may be trigged by a Job. In an example, master pod108may be created using a command-line client (e.g., kubectl) with a human-readable data-serialization language file (e.g, yaml). Kubectl is a command line tool for controlling Kubernetes clusters. The recursive yaml acronym stands for “yaml ain't markup language,” denoting it as flexible and data-oriented. Yaml can be used with an application that needs to store or transmit data. Yaml may be made up of bits and pieces of other languages. Worker pods110A-N may communicate with runtime services in worker node104. Worker pods110A-N can be triggered with the Job based on master pod108. Master pod108may have one or more worker pods110A-N. Master pod108may schedule workloads in worker pods110A-N. In an example, the communication and connection between master pod108and worker pods110A-N can be in a tree topology. In another example, the communication and connection between master pod108and worker pods110A-N can be in a star, net, mesh or other suitable topologies. In an example, master pod108and worker pods can be scaled out using a recursive way in a container orchestration platform, for example, a Kubernetes orchestration platform.

Further, in the depicted embodiment, worker node104includes operation module106. In the depicted embodiment, operation module106is located on worker node104. However, in other embodiments, operation module106is may be located externally and accessed through a communication network such as network120. The communication network can be, for example, a LAN, a WAN such as the Internet, or a combination of the two, and may include wired, wireless, fiber optic or any other connection known in the art. In general, the communication network can be any combination of connections and protocols that will support communications between worker node104and operation module106, in accordance with a desired embodiment of the disclosure. In some embodiments, operation module106can be located on master pod108and worker pods110A-N. Operation module106can be located on master pod108. Operation module106can be located on each of worker pods110A-N.

In one or more embodiments, operation module106may dynamically scale out master-worker pods using a recursive way for multiple layers of master and worker pods in a container orchestration platform, for example, a Kubernetes orchestration platform. In an example, operation module106may be software extensions to Kubernetes that make use of custom resources to manage applications and their components. Operation module106may follow Kubernetes principles, notably a control loop. Operation module106may be a method of packaging, deploying and managing a Kubernetes application. A Kubernetes application may be an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl tooling.

In one or more embodiments, operation module106is configured to schedule master pod108and worker pods110A-N to deploy applications to a cluster. Operation module106may dynamically scale out master pod108and worker pods110A-N using a recursive way for multiple layers of master pod108and worker pods110A-N. In an example, master pod108can be initially triggered with a Job. For example, a Job may create one or more pods and may ensure that a specified number of pods successfully terminate. As pods successfully complete, the Job may track the successful completions. When a specified number of successful completions is reached, the task is complete. Worker pods110A-N can be triggered with the Job based on master pod108. Master pod108may have one or more worker pods110A-N. Master pod108may schedule workloads in worker pods110A-N.

In one or more embodiments, operation module106is configured to build communication between master pod108and worker pods110A-N. The communication can be based on the deployments and the associated types of defined relationships. In an example, a deployment can be in a master-worker mode. In another example, the deployment mode can be in other topology, for example, star, net, and mesh topology. In an example, the communication between master pod108and worker pods110A-N can be in a tree topology. In another example, the communication between master pod108and worker pods110A-N can be in a star, net, mesh or other suitable topologies.

In one or more embodiments, operation module106is configured to monitor workloads in each of worker pods110A-N. Master pod108may schedule workloads in worker pods110A-N. In an example, operation module106may monitor workloads in each of worker pods110A-N such as monitoring the status of a Job in each of worker pods110A-N. In the depicted embodiment, operation module106can be located on worker node104and externally master pod108and worker pods110A-N. However, in some embodiments, operation module106can be located on master pod108and worker pods110A-N. In some embodiments, operation module106can be located on master pod108. Operation module106can be located on each of worker pods110A-N. Operation module106can monitor workloads in each of worker pods110A-N individually.

In one or more embodiments, operation module106is configured to determine whether any of worker pods100A-N needs to scale out. In an example, operation module106may determine whether any of worker pods100A-N needs to scale out based on a trigger condition such as a pre-defined threshold for each worker pods100A-N. If operation module106determines a workload in any worker pods100A-N (e.g., worker node100A) exceeds the pre-defined threshold, operation module106may schedule a next layer of worker pods according to the workload and convert a current worker pod (e.g., worker node100A) to a master pod. Operation module106may build up communication and relationships between the new master pod and new worker pods. Operation module106may monitor workloads in each of new worker pods. If operation module106determines that no workload in any worker pod100A-N exceeds the pre-defined threshold, operation module106may determine no scaling out is needed. Operation module106may process the workload. Operation module106may return a result to master pod108. Operation module106may consolidate a result to worker node104. Operation module106may dynamically scale out master-worker pods using the recursive way for multiple layers of master and worker pods.

FIG. 2is an example process200depicting example dynamical scaling out using an example recursive way for multiple layers of master pod108and worker pods110A-N, in accordance with an embodiment of the present disclosure.

In the example process200ofFIG. 2, master pod108may be triggered by a Job. In an example, master pod108may be created using a command-line client (e.g., kubectl) with a yaml file. Worker pods110A-N may communicate with runtime services in worker node104. Worker pods110A-N can be triggered with the Job based on master pod108. Master pod108may have one or more worker pods110A-N. Master pod108may schedule workloads in worker pods110A-N. In the depicted embodiment, the communication and connection between master pod108and worker pods110A-N can be in a tree topology. In other embodiments, the communication and connection between master pod108and worker pods110A-N can be in a star, net, mesh or other suitable topologies.

Operation module106may monitor workloads in master pod108and each of worker pods110A-N. In an example, operation module106may monitor workloads in each of worker pods110A-N such as monitoring the status of a Job in each of worker pods110A-N. In the depicted embodiment, operation module106can be located on worker node104and externally master pod108and worker pods110A-N. However, in some embodiments, operation module106can be located on master pod108and worker pods110A-N. In some embodiments, operation module106can be located on master pod108. Operation module106can be located on each of worker pods110A-N. Operation module106can monitor workloads in each of worker pods110A-N individually. Master pod108may schedule workloads in worker pods110A-N.

In one or more embodiments, operation module106may trigger a new worker pod, e.g., worker pod110X, based on the workloads. Master pod108keeps as the master pod for worker pod110X. In one or more embodiments, operation module106may trigger multiple new worker pods, e.g., worker pods210A-C, based on the workloads. In the example embodiments, operation module106may convert worker pod110A into a master pod for the new worker pods210A-C. In other embodiments, operation module106may trigger a single new worker pod, e.g., worker pod210D, based on the workloads. In the example embodiments, operation module106may convert worker pod110B into a master pod for the new worker pod210D. Operation module106may dynamically scale out master-worker pods using the recursive way for multiple layers of master pod108and worker pods110A-N.

FIG. 3is a flowchart300depicting operational steps of operation module106in accordance with an embodiment of the present disclosure.

Operation module106operates to schedule master pod108and worker pods110A-N to make applications be deployed on a cluster. Operation module106also operates to build up communication between master pod108and worker pods110A-N. Operation module106operate to monitor workloads in each of worker pods110A-N. Operation module106operates to determine whether any of worker pods100A-N needs to scale out. Operation module106also operates to schedule a next layer of worker pods100A-N according to the workload and convert a current worker pod (e.g., worker node100A) to a second master pod.

In step302, operation module106schedules master pod108and worker pods110A-N to make applications be deployed on a cluster. Operation module106may dynamically scale out master pod108and worker pods110A-N using a recursive way for multiple layers of master pod108and worker pods110A-N. In an example, master pod108can be initially triggered with a Job. For example, a Job may create one or more pods and may ensure that a specified number of pods successfully terminate. As pods successfully complete, the Job may track the successful completions. When a specified number of successful completions is reached, the task is complete. Worker pods110A-N can be triggered with the Job based on master pod108. Master pod108may have one or more worker pods110A-N. Master pod108may schedule workloads in worker pods110A-N.

In step304, operation module106builds up communication between master pod108and worker pods110A-N. Operation module106may build up the communication with a connection and topology between master pod108and worker pods110A-N. The connection and topology can be based on the deployments and the associated types of defined relationships. In an example, a deployment can be in a master-worker mode. In another example, the deployment mode can be in other topologies, for example, star, net, and mesh topology. In an example, the communication between master pod108and worker pods110A-N can be in a tree topology. In another example, the communication between master pod108and worker pods110A-N can be in a star, net, mesh or other suitable topologies.

In step306, operation module106monitors workloads in each of worker pods110A-N. Master pod108may schedule workloads in worker pods110A-N. In an example, operation module106may monitor workloads in each of worker pods110A-N such as monitoring the status of a Job in each of worker pods110A-N. In the depicted embodiment, operation module106can be located on worker node104and can be external to master pod108and worker pods110A-N. However, in some embodiments, operation module106can be located on master pod108and worker pods110A-N. In some embodiments, operation module106can be located on master pod108. Operation module106can be located on each of worker pods110A-N. Operation module106can monitor workloads in each of worker pods110A-N individually.

In step308, operation module106determines whether any of worker pods100A-N needs to scale out. In an example, operation module106may determine whether any of worker pods100A-N needs to scale out based on a trigger condition such as a pre-defined threshold for each worker pods100A-N. If operation module106determines a workload in any worker pods100A-N (e.g., worker node100A) exceeds the pre-defined threshold, the process moves to step314. In step314, operation module106may schedule a next layer of worker pods according to the workload and convert a current worker pod (e.g., worker node100A) to a master pod. Operation module106may build up communication and relationships between the new master pod and new worker pods. The process moves back to step306. Operation module106may monitor workloads in each of new worker pods. If operation module106determines that no workload in any worker pod100A-N exceeds the pre-defined threshold, operation module106may determine no scaling out is needed and the process moves to step310. In step310, operation module106may process the workload. In step312, operation module106may return a result to master pod108. Operation module106may consolidate a result to worker node104. Operation module106may dynamically scale out master-worker pods using the recursive way for multiple layers of master and worker pods.

FIG. 4depicts a block diagram400of components of master node102and worker node104in accordance with an illustrative embodiment of the present disclosure. It should be appreciated thatFIG. 4provides only an illustration of one implementation and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

Master node102and worker node104may include communications fabric402, which provides communications between cache416, memory406, persistent storage408, communications unit410, and input/output (I/O) interface(s)412. Communications fabric402can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, communications fabric402can be implemented with one or more buses or a crossbar switch.

Memory406and persistent storage408are computer readable storage media. In this embodiment, memory406includes random access memory (RAM). In general, memory406can include any suitable volatile or non-volatile computer readable storage media. Cache416is a fast memory that enhances the performance of computer processor(s)404by holding recently accessed data, and data near accessed data, from memory406.

Operation module106may be stored in persistent storage408and in memory406for execution by one or more of the respective computer processors404via cache416. In an embodiment, persistent storage408includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage408can include a solid state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer readable storage media that is capable of storing program instructions or digital information.

Communications unit410, in these examples, provides for communications with other data processing systems or devices. In these examples, communications unit410includes one or more network interface cards. Communications unit410may provide communications through the use of either or both physical and wireless communications links. Operation module106each may be downloaded to persistent storage408through communications unit410.

I/O interface(s)412allows for input and output of data with other devices that may be connected to master node102and worker node104. For example, I/O interface412may provide a connection to external devices418such as a keyboard, keypad, a touch screen, and/or some other suitable input device. External devices418can also include portable computer readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present invention, e.g., operation module106can be stored on such portable computer readable storage media and can be loaded onto persistent storage408via I/O interface(s)412. I/O interface(s)412also connect to display420.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows: