Patent Publication Number: US-10789091-B2

Title: Agile VM load balancing through micro-checkpointing and multi-architecture emulation

Description:
BACKGROUND 
     Technical Field 
     The present invention relates to system virtualization and, more particularly, to agile virtual machine load balancing. 
     Description of the Related Art 
     Virtualization is evolving as a key consolidation technology and is part of the foundation of cloud computing. The ability to create and manage virtual machines (VMs) has become a necessity for data center management. Cloud providers typically use these techniques to consolidate multiple applications or VMs onto a single node to minimize their costs and to take maximum advantage of virtualization. At the same time, cloud consumers expect VMs to obey desired service level agreements (SLAs) to satisfy their own clients&#39; needs. SLAs typically include performance SLAs in addition to others, such as availability, security, etc. This in turn means that providers provision resources to the VMs to handle varying loads, including peak loads. However, load variation on the application cannot generally be known beforehand. If resources are provisioned to handle a certain amount of load, then VM performance may degrade during peak loads, thus violating the promised SLAs. 
     One solution to this problem has been to over-provision resources to the VMs, by providing enough extra resources to the VM to handle peak loads. This approach has a drawback in that the resources in the system are under-utilized except when they are at peak load, which results in a higher cost to providers, which need to use more physical resources to provide the resources to the over-provisioned VMs. Another conventional solution is to migrate secondary VMs to a different machine when one VM on the same node reaches peak loads. However, migrating a VM may need significant resources and time and may even degrade the performance of the VM that is at peak load if resources such as processing and network resources are used for migration. 
     SUMMARY 
     A method for load balancing includes detecting an increased load for a first primary virtual machine (VM) on a first node that has a plurality of additional primary VMs running on a processor; deactivating one or more of the additional primary VMs, reducing said one or more deactivated VMs to a secondary state, to free resources at the first node for the first primary VM; and activating secondary VMs, located at one or more additional nodes, that correspond to the one or more deactivated VMs, raising said secondary VMs to a primary state. 
     A method for load balancing includes periodically micro-checkpointing a first primary VM and one or more additional primary VMs during execution of said primary VMs to maintain corresponding secondary VMs at one or more additional nodes; detecting an increased load for the first primary virtual machine (VM) on the first node that has a plurality of additional primary VMs running on a processor; deactivating one or more of the additional primary VMs, reducing said one or more deactivated VMs to a secondary state, to free resources at the first node for the first primary VM; and activating secondary VMs, located at one or more additional nodes having a hardware architecture incompatible with that of the first node, that correspond to the one or more deactivated VMs, raising said secondary VMs to a primary state on an emulation layer that emulated the hardware architecture of the first node. 
     A load balancing node includes a processor configured to run a plurality of primary virtual machines (VMs); and a load balancer configured to detect an increased load for a first primary VM, to deactivate one or more additional primary VMs, reducing said one or more deactivated VMs to a secondary state, to free resources for the first primary VM, and to communicate with one or more additional nodes to activate secondary VMs that correspond to the one or more deactivated VMs, raising said secondary VMs to a primary state. 
     These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
       The disclosure will provide details in the following description of preferred embodiments with reference to the following figures wherein: 
         FIG. 1  is a block diagram of a load balancing system in accordance with the present principles; 
         FIG. 2  is a block diagram of a load balancing system in accordance with the present principles; 
         FIG. 3  is a block/flow diagram of a method for agile load balancing in accordance with the present principles; 
         FIG. 4  is a block/flow diagram of a method for agile load balancing across disparate architectures in accordance with the present principles; and 
         FIG. 5  is a block diagram of a load balancing node in accordance with the present principles. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of the present invention provide agile migration of virtual machines (VMs) to handle peak-load situations and do load-balancing by using adaptive micro checkpointing. In the present embodiments, certain VMs (referred to as primary VMs) that are consolidated on a single node have a corresponding inactive secondary VM that is replicated onto another node and that is periodically updated by the primary VM. The secondary VM stays in a passive mode until it is activated by a load balancer. The load balancer component monitors the loads on the respective VMs on a node and dynamically switches processing from a primary VM to a secondary VM based on the load currently seen on the nodes. To make these changes in an adaptive manner, the load balancer uses metrics that include, e.g., application metrics such as transactions per second and operating system metrics such as processor utilization. 
     Referring now to the drawings in which like numerals represent the same or similar elements and initially to  FIG. 1 , a virtualization system is shown that includes a first physical node  102  and a second physical node  108 . The first node  102  has multiple VMs, including a first VM  104  and a second VM  106 . The first VM  104  has a primary VM  104  on the first node and a secondary VM  110  on the second node  108 . The second node  108  also has a third VM  112 . 
     During normal operation, the first VM  104 , second VM  106 , and third VM  112  run at the same time on their respective nodes. The first VM&#39;s primary iteration  104  runs on the first node  102  and periodically provides updated state information to the secondary iteration  110  on the second node  108  through micro-checkpointing. Micro-checkpointing may be understood as a live migration that never ends, as updated state information is sent periodically to keep the VMs synchronized. These updates can be sent at different rates for different types of application, where some applications will benefit from sending cold and hot portions of memory in advance, before the load peaks. 
     An appropriate checkpoint frequency can be determined by monitoring current performance over time, such as through a sustained, observed, transient peak in CPU utilization or by an unacceptable drop (according to the aforementioned SLA) in application throughput or latency, which might violate the application&#39;s SLA. Both high frequencies and low frequencies can harm an application: if the frequency is too low, the IO path from inside the VM to the outside (such as a network transmission) can “appear” to be too slow. If it is too high, the micro-checkpointing logic itself may take up too many resources (such as processing and network resources) away from the VM itself. Thus, an appropriate frequency would be adjusted over time according to the current performance monitored. 
     At some point in the operation of the system, the second VM  106  increases its processing needs. When the second VM  106  approaches peak load, such that the first node  102  cannot adequately serve both VMs at once. At this point, the secondary iteration of the first VM  110  activates using the most recent up-to-date (fault tolerant) micro-checkpoint that was successfully generated. No additional state needs to be synchronized after the peak in load is detected. This allows the first node  102  to quickly devote additional resources to the second VM  106  without compromising SLAs related to the first VM  104 . 
     Referring now to  FIG. 2 , the system is shown after the first VM is switched from the first node  102  to the second node  108 . When VMs are switched, the secondary iteration of the first VM  110  becomes the new primary VM  204 . The original primary VM  104  becomes the new secondary VM  202 , with the new primary VM  204  providing state updates to the new secondary VM  202 . Updating the secondary VM  202  on the first node  202  uses fewer resources than running the primary VM  104  did, which frees resources for the second VM  106  to run at peak load. The primary and secondary VMs can be switched again if the loads change. 
     It should be noted that, although only two nodes are shown, any number of nodes may be used and a given VM may have secondary VMs on multiple other nodes. In this case, a decision is made as to which node can best accommodate the additional burden of the transferred VM. Because the secondary VMs will not consume much in the way of processing resources, a given node may also accommodate multiple secondary VMs. 
     To keep load balancing agile, such that VM location can be changed quickly, the present embodiments may also optionally employ processor architecture emulation. For example, if the first node  102  runs on a first processor architecture, and the second node  108  runs on a second processor architecture, the second node  108  may provide an emulation layer that allows a VM from the first node  102  to be seamlessly transferred. This allows diverse processing nodes to be employed to aid in load balancing, which may be particularly advantageous in cloud computing environments that provide a wide variety of architectures. When loads on the native architecture nodes returns to normal, the emulated VMs may be switched back to nodes at which they will run more efficiently. 
     Referring now to  FIG. 3 , a method for agile load balancing is shown. Block  302  begins by checking the loads for each VM at a node. Block  304  makes a determination as to whether one or more VMs at the node have an increased load, such that there is a risk of failing to meet one or more SLAs. This determination may include a comparison of a load level to a threshold that characterizes the resources that a VM may use without starving the loads of other VMs. It should be noted that it may also be possible for multiple VMs to increase their loads, and so a combined load may also be compared to the threshold. If no VMs have a load that exceeds this threshold, block  306  updates the state information for secondary VMs at other nodes that correspond to the primary VMs running at the node. This update may be based on a specified update frequency for each VM, as different VMs may need more or less frequent state updates. 
     If one or more VMs at the node have increased their loads past the threshold, processing passes to block  308  which activates one or more secondary VMs on other nodes which correspond to primary VMs other than the high-load VM(s). Block  310  then deactivates the corresponding primary VMs to free up local resources at the node, allowing the high-load VMs to run unimpeded. Processing then returns to block  302  to continue checking loads. This process is performed at each node, with communication between different nodes to determine which nodes can best handle the increased load of activating VMs. It should be noted that the VMs need not be switched back when the loads return to normal levels. Instead, the load balancing process may simply wait until the load increases elsewhere, necessitating another switch. 
     Referring now to  FIG. 4 , a method for multi-architecture load balancing is shown. As above, block  402  checks the load level for all of the VMs at a node. Block  404  determines whether one or more VMs have increased their loads past a threshold. If so, block  408  activates one or more secondary VMs on nodes having different architectures. These nodes cannot run the VM natively, but may emulate the original architecture. Although this is less efficient than running a VM on its native architecture, it expands the options for load balancing. Block  410  deactivates VMs at the original node that corresponds to the activated VMs and returns processing to block  402 . 
     If no VM has a load past the threshold, block  414  determines whether the load has dropped for a VM that was previously experiencing increased load. If so, block  416  deactivates VMs at nodes with emulation. In contrast to the above examples, where VMs need not be switched back right away, multi-architecture load balancing incurs a penalty in the increased overhead needed to emulate the original architecture. Block  418  reactivates the original nodes and returns processing to block  402 . 
     If no load increases past the threshold and there are no emulated VMs to return to their original nodes, block  406  updates secondary VMs for all VMs at the node to maintain the state of the secondary VMs for potential future activation. It is assumed that the aggregate, emulated compute resources of the secondary destination host, towards which a load balancing switch will occur, are higher than the available non-emulated compute resources of the original host, on which the overloaded VM is currently running. This means that it may be necessary for the provider (and or client) to increase cost to provide a secondary machine that is more powerful than the primary such that the decision to perform a micro-checkpointing based switch can be realized on the destination host even while the secondary is being emulated. It is also assumed that, if this cost is prohibitively high, then this embodiment provide micro-checkpointing as described above, such that one load-balanced switch can be succeeded by another switch to a non-emulated host to restore the secondary VM to a new location, such that the original non-emulated cost basis for running the VM can be maintained in the long term. 
     As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon. 
     Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. 
     A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. 
     Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user&#39;s computer, partly on the user&#39;s computer, as a stand-alone software package, partly on the user&#39;s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user&#39;s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). 
     Aspects of the present invention are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. 
     Reference in the specification to “one embodiment” or “an embodiment” of the present principles, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment”, as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment. 
     It is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as readily apparent by one of ordinary skill in this and related arts, for as many items listed. 
     Referring now to  FIG. 5 , a load balancing node  500  is shown. A processor  502  and memory  504  serve to store and execute primary virtual machine(s)  506  and secondary virtual machine(s)  508 . The processor  502  and memory  504  thereby system resources which are consumed by the virtual machines in accordance with their respective loads. If one virtual machine demands, for example, more processing power than is available, the load balancing module  514  may trigger a reassignment of one or more virtual machine workloads to other nodes, activating the corresponding secondary virtual machines  508  on the target node and deactivating the local primary virtual machines  506  in question. It should be recognized that the load balancing module  514  need not be located at the nodes themselves but may be located at a dedicated central server or may be decentralized in only some of the nodes. 
     A micro-checkpoint engine  510  maintains connections between the primary virtual machine(s)  506  at the local node  500  with the corresponding secondary virtual machine(s)  508  at other nodes  500 . The micro-checkpoint engine  510  supplies updated state information to the secondary virtual machines  508 , in case those secondary virtual machines  508  need to be activated. In addition, the micro-checkpoint engine  510  accepts updated state information for local secondary virtual machines  508  from corresponding primary virtual machines  506  at other nodes. 
     As noted above, an emulation module  512  provides the ability to emulate other hardware architectures, such that virtual machines from other architectures may be temporarily run on the node  500 . It is worth noting that this gives the present embodiments the ability to make use of mobile devices for load sharing as well, as such devices typically have different hardware architectures from conventional cloud devices. 
     In such a scenario, a primary VM would be running on the mobile device. The secondary VM would be running on a server, with the server architecture likely being different from the mobile architecture. The micro-checkpointing engine  510  would thus link the device and the server together in the event that the VM providing the mobile service to the user needs more resources than are currently available on the device. Once the load balancer determines that a mobile SLA is being violated, a switch would occur where the server would immediately take over execution as described before through micro-checkpointing, with the output or display results of the VM returned back to the mobile device. Once the load has diminished, the VM could be returned in the same way back to the device. 
     Having described preferred embodiments of a system and method for agile VM load balancing through micro-checkpointing and multi-architecture emulation (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments disclosed which are within the scope of the invention as outlined by the appended claims. Having thus described aspects of the invention, with the details and particularity required by the patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.