Method and apparatus for managing IT infrastructure in cloud environments by migrating pairs of virtual machines

In example implementations, when a management program deploys new virtual machines, the management program may identify candidate virtual machines for replacement, score the possibilities of replacement and relate the new virtual machines to candidate virtual machines if it determines the probability of replacement is high. The management program may also migrate virtual machines and storage volumes used by the virtual machines to other physical servers and storage arrays by related pairs of virtual machines. The management program may also inherit management policies from existing virtual machines being replaced and leverage them to manage new virtual machines, which replace the existing virtual machines.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Stage entry of PCT Application No. PCT/US2014/050956, filed on Aug. 23, 2014, the contents of which are incorporated by reference.

BACKGROUND

Field

The example implementations described herein are related generally to computer systems and, more particularly, to an information technology (IT) management infrastructure in a cloud computing environment.

Related Art

Cloud computing is widely used in the related art. There are several related art solutions for managing cloud computing environments. One example related art solution for cloud environment management is as follows:an infrastructure administrator is assigned to manage IT resources and store the IT resources in resource pools; andan application administrator obtains IT resources from the resource pools using a self-service portal to execute his/her applications.

Generally, the infrastructure administrators will optimize configurations of IT resources to satisfying Service Level Agreements (SLAs) of the cloud services they offer. This is done, for example, by migrating virtual servers or virtual machines (VMs) from one physical server (or one hypervisor) to another different physical server (or another different hypervisor).

Further, some application administrators utilize computing environments (i.e., IT resources provided from the cloud and software, including operating systems (OS), middleware and certain applications on them) as immutable environments. In other words, the application administrator does not modify any configurations and settings of the environment once it is in use. Instead, the application administrator will create a new environment by obtaining additional IT resources (i.e., additional computing environments) from the resource pools, deploying software on the additional IT resources and replacing the existing environments with the new computing environment. This replacement may be done by switching workloads from the existing environment to the new environment by using network routers and/or load balancers. By replacing the computing environment, rather than reconfiguring the computing environment, the operation status of the environments may be simplified.

In related art cloud computing environments, the infrastructure administrators cannot know whether application administrators are treating their computing environments as immutable environments (i.e., replacing computing environments with new environments, rather than reconfiguring the existing environments).

Further, in related art cloud computing environments, an infrastructure administrator cannot distinguish a computing environment (e.g. a virtual machine) created by the application administrator as a replacement for an existing computing environment (e.g., a virtual machine), from a computing environment, created to be a new computing environment (i.e., a virtual machine not created to replace an existing virtual machine).

In related art cloud computing environments, the infrastructure administrator can make optimizations of IT infrastructures that become inefficiencies due to replacements of VMs done by application administrators. For example, an infrastructure administrator may assign greater IT resources to an existing VM and fewer resources to a newly created VM, with less performance requirements. However, as the newly created VM takes over workload from the existing VM, the new VM requires more resources, while the existing VM requires fewer resources, contrary to the optimizations previously completed by the infrastructure administrator.

Further, infrastructure administrators may be unable to apply existing know-how (or policies) for managing existing VMs after they are replaced by new VMs because infrastructure administrators cannot recognize that the new VM is a replacement for the existing VM. The infrastructure administrator may only recognize that some new VMs have been created and some existing VMs have been deleted, but cannot identify specific replacements. As a result, existing know-how and/or policies may be lost when existing VMs are replaced.

SUMMARY

Aspects of the present application may a management computer coupled to a plurality of servers, which manage a plurality of virtual machines. The management computer may include a memory configured to store application type information indicative of an application type of each of the plurality of virtual machines. The memory may also be configured to store virtual machine topology information indicative of a topology of the plurality of virtual machines. The management computer may also include a processor configured to make a relationship between a virtual machine of the plurality of virtual machines and another virtual machine of the plurality of virtual machines, which has a probability to replace the virtual machine. The relationship may be made based on an association between the virtual machine and the another virtual machine in the virtual machine topology information. The relationship may also be made based on an application type of the virtual machine and an application type of the another virtual machine in the application type information.

Aspects of the present application may also include a method. The method may include determining, by a computing device, application type information indicative of an application type of each of a plurality of virtual machines. The method may further include determining, by the computing device, virtual machine topology information indicative of a topology of the plurality of virtual machines. The method may also include making, by the computing device, a relationship between a virtual machine of the plurality of virtual machines and another virtual machine of the plurality of virtual machines, which has a probability to replace the virtual machine. The relationship may be made based on an association between the virtual machine and the another virtual machine in the virtual machine topology information. The relationship may also be made based on an application type of the virtual machine and an application type of the another virtual machine in the application type information.

Aspects of the present application may include a computer program containing instructions for executing a process. The instructions may include determining application type information indicative of an application type of each of a plurality of virtual machines. The instructions may further include determining virtual machine topology information indicative of a topology of the plurality of virtual machines. The instructions may also include making a relationship between a virtual machine of the plurality of virtual machines and another virtual machine of the plurality of virtual machines, which has a probability to replace the virtual machine. The relationship may be made based an association between the virtual machine and the another virtual machine in the virtual machine topology information. The relationship may also be made based an application type of the virtual machine and an application type of the another virtual machine in the application type information.

DETAILED DESCRIPTION

Some example implementations are described with reference to drawings. The example implementations that are described herein do not restrict the inventive concept, and one or more elements that are described in the example implementations may not be essential for implementing the inventive concept. Further, although certain elements may be referred to in the singular form, the elements are not intended to be limited to the singular and may also be implemented with one or more of the same element, depending on the desired implementation.

Some example implementations described herein are directed to methods and apparatuses which may optimize IT infrastructures by considering the possibility or probability of replacement of the existing VMs with new VMs. For example, in some example implementations, a management program that deploys new VMs may identify candidate VMs potentially being replaced. Further, the management program may score the possibility of replacement and pair the new VMs to the candidate VMs, if it judges the possibility of replacement to be high. Further, in some example implementations, the management program may migrate VMs and storage volumes used by the existing VMs and newly created VMs to other physical servers and storage arrays in pairs.

Further, some example implementations described herein may also provide methods and apparatuses which may allow the inheriting of management policies from existing VMs and the leveraging of the inherited management policies to manage new VMs, which may replace the existing VMs. For example, in some example implementations the management program may inherit management policies from existing VMs being replaced and leverage them to manage new VMs, which replace the existing VMs.

First Example Implementation

This first example implementation includes a management program, which may optimize IT infrastructures by considering the possibility that newly created VMs are replacements of existing VMs.

FIG. 1Aillustrates an example logical configuration of a cloud system1000in which one or more example implementations of the present application may be applied. The Cloud system1000may include a management program1200coupled to one or more applications and virtualized resources1300, an IT infrastructure1500, a self-service portal1600and an IT infrastructure management user interface1800.

As illustrated inFIG. 1A, an application administrator1010may interact with, or use, the cloud system1000via the self-service portal1600. Further, the IT infrastructure administrator1030may interact with, or use, the cloud system1000via the IT infrastructure management user interface1800.

FIG. 1Billustrates an example logical configuration of the IT infrastructure1500of the cloud system1000ofFIG. 1A. The IT infrastructure1500may include multiple storage systems such as storage system01(1510) and storage system02(1520) and multiple servers1540,1550. Storage system01(1510) may include two storage volumes1511,1512. Storage system02may include three Storage volumes1521,1522,1533.

As illustrated, server1540may be communicatively coupled to storage volume1511of storage system1(1510). A hypervisor (also referred to herein as a virtual machine monitor (VMM))1541may be running on the server1540. An application (APP)1543, operating system (OS)1542may be running on the hypervisor1541. Application1543may use storage volume1511of the storage system01(1510).

Further, server1550may be communicatively coupled to storage volumes1521and1522of storage system2(1520). Hypervisor1551may be running on the server1550. Applications1554,1555, and operating systems (OS)1552and1553may be running on the hypervisor1551. Application1554may use storage volume1521of the storage system02(1520). Further, Application1555may use storage volume1522of the storage system02(1520).

FIG. 2Aillustrates an example physical configuration of a cloud system1000according to one or more example implementations of the present application. The cloud system1000may include a management server2000, one or more servers3000, one or more storage arrays4000, a management network5000and a data network6000.

The servers3000and the storage arrays4000may be connected to each other via the data network6000. The data network6000is not particularly limited to a specific type of network and may include a LAN (Local Area Network), a WAN (Wide Area Network), or any other network that may be apparent to a person of ordinary skill in the art.

The management server2000, the servers3000and the storage arrays4000may be connected to each other via management network5000. The management network5000may also be include a LAN (Local Area Network), a WAN (Wide Area Network), or any other network that may be apparent to a person of ordinary skill in the art. Network is usually LAN.

The management network5000and data network6000are illustrated as separate networks in this example implementation. However, example implementations of the present application are not limited to this configuration and may be formed as a single combined network.

In this example implementation, the management server2000and the servers3000are illustrated as being separate servers. However, example implementations of the present application are not limited to this configuration and may other configurations. For example, a single server hosting a management program may be used. Management server2000may also be in the form of a management computer configured to manage multiple virtual machines.

In this example implementation, the servers3000and the storage arrays4000are illustrated as separate. However, example implementations of the present application are not limited to this configuration and may include other configurations. For example, the servers3000and the storages arrays4000can be combined into a single storage system.

FIG. 2Billustrates a first example implementation of a management server2000for the cloud system1000ofFIG. 2A. The management server2000may include a management interface (I/F)2100, a processor (such as a central processing unit (CPU))2200, an input and output device2300, a local disk2400and a memory2500. The management interface2100may be used to interface with the management network5000illustrated inFIG. 2A. The input and output device2300is not particularly limited and may be any type of user interface, which may be apparent to a person of ordinary skill in the art, such as a monitor, a keyboard, a mouse, etc.

The management program2410may be loaded into the memory2500and executed by the processor2200to perform management procedures. The procedures of the management program2410are discussed in greater detail below. In some example implementations, the management program2410may be analogous to the management program1200identified inFIG. 1.

The memory2500may contain a storage array table2510, a storage volume table2520, a physical server table2530, a virtual server table2540, a mapping table2550, a virtual server topology table2560, a virtual server pair table2570, a storage performance table2580and a server performance table2590.

The storage array table2510is discussed in greater detail below with respect toFIG. 5. The storage volume table2520is discussed in greater detail below with respect toFIG. 6. The physical server table2530is discussed in greater detail below with respect toFIG. 7. The virtual server table2540is discussed in greater detail below with respect toFIG. 8. The mapping table2550is discussed in greater detail below with respect toFIG. 9. The virtual server topology table2560is discussed in greater detail below with respect toFIG. 10. The virtual server pair table2570is discussed in greater detail below with respect toFIG. 11. The storage performance table2580is discussed in greater detail below with respect toFIG. 12. The server performance table2590is discussed in greater detail below with respect toFIG. 13.

FIG. 3illustrates an example image catalog table2420, in accordance with an example implementation of the present application. The image catalog table2420may be used when the application administrator1010deploys applications, or sends a deployment request, by using the self-service portal1600. When used, the image catalog table2420is loaded from the local disk2400into the memory2500of the management server2000.

As illustrated, the image catalog table2420includes columns providing information on the image catalogs. Column2421provides the identification information, or identifications, of the catalogs. Column2422provides the types of the applications. Further, column2423provides the names of operating systems on which applications run, and column2424provides the operating system versions on which applications run. Column2425provides the names of applications and column2426shows the versions of applications. Column2427provides the locations of storage volumes in which the applications are contained. These storage volumes are sometimes referred to as “golden images”.

Each row (242A-242E) of the image catalog table2420corresponds with one of the image catalogs. For example, row242A shows the catalog of a database type of application. This catalog includes the application MySQL, version 2.4 running on an Ubuntu operating system version 13.04. This image is located on volume01of storage system01.

FIG. 4illustrates an example VM template table2430, in accordance with an example implementation of the present application. The templates represented in the VM template table2430describe the resource configurations of several types of VMs. When used, this table is loaded from local disk2400to memory2500of the management server2000.

As illustrated, the VM template table2430includes a number of columns providing information on the VM templates. Column2431provides the identifications of the templates and column2432provides the VM types. Column2433provides the processor types. The values of column2433can be normal, high memory, high CPU and High I/O. Column2434provides the processor performance. The values of column2434are relative values on the basis of a normal CPU. Column2435shows the numbers of processors. Column2436provides the capacities of the memories and column2437provides the maximum Input/output Operations per Second (TOPS). Further, column2438shows the unit prices.

Each row (243A-243D) of the VM template table2430shows the resource configuration of each type of VM. For example, row243A shows the configuration of a normal type of VM. This type of VM may include two normal processors and 4 GB memory. The unit price of this type of VM is 10.

FIG. 5illustrates an example physical storage array table2510, in accordance with an example implementation of the present application. This physical storage array table2510may be created in the memory2500by the management program2410.

As illustrated, the physical storage array table2510includes a number of columns providing information on the physical storage arrays. Column2511provides the identification information, or identifications of the physical storage arrays. Column2512provides the port resources of each physical storage array. Column2513provides the cache resources of each physical storage array. Column2514provides the array group resources of each physical storage array.

Each row (251A-251C) of the physical storage array table2510shows the configuration of each physical storage array. For example, row251A shows the configuration of physical storage array01. The physical storage array has a data rate of 8 Gigabits per second (Gbps) at port A, B, C and D, 160 Gigabytes (GB) of C-01 cache and 128 GB of C-02 cache and 300 Terabytes (TB) of array group AG-002 and 300 TB of array group AG-102.

FIG. 6illustrates an example storage volume table2520, in accordance with an example implementation of the present application. The storage volume table2520may be created in the memory2500by the management program2410.

As illustrated, the storage volume table2520includes a number of columns providing information on the storage volumes. Column2521provides the identification information, or identifications of the virtual storage arrays owning the storage volumes. Column2522provides the identifications of the storage volumes. Column2523provides the cache resources assigned to each storage volume. Column2524provides the capacity of each storage volume.

Each row (252A-252E) of the storage volume table2520shows the configuration of each storage volume. For example, row252A shows the configuration of storage volume01of storage array01. This storage volume has 32 GB of cache resources and 80 TB of capacity.

FIG. 7illustrates an example physical server table2530, in accordance with an example implementation of the present application. This physical server table2530may be created in the memory2500by the management program2410.

As illustrated, the physical server table2530includes a number of columns providing information on the physical servers. Column2531provides the identification information, or identifications, of the physical servers. Column2532provides the numbers of cores and types of CPU of each physical server. Column2533provides the capacities of memories of each physical server. Column2534provides the port resources of each physical server.

Each row (253A-253D) of the physical server table2530shows the configuration of each physical server. For example, row253A shows the configuration of physical server01. The physical server has 12 cores of Normal CPUs, 32 GB of memory, a data rate of 4 Gbps at port A and B, and deduplication functionality installed.

FIG. 8illustrates an example virtual server table2540, in accordance with an example implementation of the present application. The virtual server table2540may be created in the memory2500by the management program2410.

As illustrated, the virtual server table2540includes a number of columns providing information on the virtual servers (virtual machines (VM)). Column2541provides the identifications of the virtual servers. Column2542provides the identifications of the physical servers on which the virtual servers are running. Column2543provides numbers of CPU cores assigned to each virtual server. Column2544provides capacities of memories assigned to each virtual server. Column2545provides the port assigned to each virtual server.

Each row (254A-254D) of the virtual server table2540shows the configuration of each virtual server01. For example, row254A shows the configuration of virtual server01. The virtual server is hosted on the physical server01and has 2 CPU cores, 4 GB of memory and a data rate of 4 Gbps at port A.

FIG. 9illustrates an example mapping table2550, in accordance with an example implementation of the present application. The mapping table2550may be created in the memory2500by the management program2410.

As illustrated, the mapping table2550includes a number of columns mapping relationship between the applications, virtual servers (VMs), and storage volumes. Column2551provides the identifications of the applications. Column2552provides the names of the applications. The names of the application may be specified in the application name field1620-A of the GUI1600-A (discussed below with respect toFIG. 14) of the self-service portal1600by the application administrator1010.

Column2553provides the identification of the image catalogs. An application type may be selected in the application type field1610-A of the GUI1600-A (discussed below with respect toFIG. 14) of the self-service portal1600by the application administrator1010. By matching application type information and the type column2422in the image catalog table2420, the identification of the image catalog may be determined.

Column2554provides the identifications of the virtual servers on which the applications are running.

Column2555provides the names of the virtual servers. In this example implementation, these names may be automatically created based on application name by the management program2410. But example implementations are not limited to this configuration, and may have other configurations. For example, the application administrator1010can specify the name of each virtual server.

Column2556provides the identifications of the ports of the virtual servers. Column2557provides the identifications of the storage arrays. Column2558provides the identifications of the ports of the storage arrays. Column2559provides the identifications of the storage volumes.

Each row (255A-255G) of the mapping table2550provides the end-to-end mapping between applications and storage volumes. For example, row255B shows that application2has the name of “Web-C”, may be created from image catalog4, and is running on the virtual server03whose name is “WebDB”. Further, virtual server03has two storage volumes052and055assigned for application2. The storage volume052of storage array02is assigned to the virtual server03through port B of the storage array and port A of the virtual server. The storage volume055of storage array01is assigned to the virtual server03through port A of the storage array and port A of the virtual server.

FIG. 10illustrates an example virtual server topology table2560, in accordance with an example implementation of the present application. The virtual server topology table2560may be created in the memory2500by the management program2410.

As illustrated, the mapping table2560includes a number of columns providing mapping information on the topology or interconnection of the virtual servers. Column2561provides the identifications of the virtual servers. Column2562provides the identifications of the parent virtual servers. Each row (256A-256D) shows the topology among the virtual servers. For example, row256B shows the virtual server02is connected to the virtual server01.

FIG. 11illustrates an example virtual server pair table2570, in accordance with an example implementation of the present application. The virtual server pair table2570may be created in the memory2500by the management program2410.

As illustrated, the virtual server pair table2570includes a number of columns providing pairing information indicating a pairing between two of the virtual servers. Column2571provides the identifications of the virtual servers. Column2572provides the identifications of the paired virtual servers.

Each row of the virtual server pair table2570shows the pair relationships between virtual servers. For example, row257A shows that the virtual server05is paired to the virtual server03. The procedure of pairing virtual servers is discussed in greater detail below.

FIG. 12illustrates an example storage performance table2580, in accordance with an example implementation of the present application. The storage performance table2580may be created in the memory2500by the management program2410.

As illustrated, the storage performance table2580includes a number of columns providing current and historical performance information for the storage volumes. Column2581provides the identifications of the storage arrays. Column2582provides the identifications of the storage volumes. Column2583provides the identifications of historical performance data of the storage volumes. Timestamps may be used as the history ID in column2583. Column2584provides the usage rate of the caches assigned to the storage volumes. Column2585provides the usage rate of the array group from which the storage volumes are curved. Column2586provides the usage rate of the ports assigned to the storage volumes.

Each row of the storage performance table2580shows the historical performance data of each storage volume. For example, row258A shows the performance data of storage volume01of storage array01which has at least three pieces of historical data (from 0 to 2).

FIG. 13illustrates an example server performance table2590, in accordance with an example implementation of the present application. The server performance table2590may be created in the memory2500by the management program2410.

As illustrated, the server performance table2590includes a number of columns providing current and historical performance information for the physical and virtual servers. Column2591provides the identifications of the physical and/or virtual servers. Column2592provides flags indicating whether the servers are physical servers or not. In this example implementation, if the value of “YES” is contained then the server is a physical server and if the value of “NO” is contained then the server is a virtual server. In other example implementations, this relation may be reversed with a value of “YES” indicating a virtual server and a value of “NO” indicating a physical server.

Column2593provides the identification of historical performance data of the servers. Timestamps may be used as the history ID2593. Column2594provides the usage rate of each of the CPUs of the servers. Column2595provides the usage rate of the memories of the servers. Column2596provides the usage rate of the disk of the servers. Column2597provides the usage rate of the ports of the servers.

Each row (259A-259B) of the server performance table2590shows the historical performance data of each server. For example, row259A shows the performance data of server01which is a physical server and has at least three pieces of historical data (from 0 to 2).

FIG. 14illustrates an example graphical user interface (GUI)1600-A of a self-service portal1600, in accordance with an example implementation of the present application. This GUI1600-A may be used when the application administrator1010deploys applications on the cloud system1000. In some example implementations, the application administrator may select an application type1610-A, for example “Web Server”. Candidate application types may be displayed based on the type information2422, OS name information2423, OS version information2424, application name information2425and application version information2426illustrated in the image catalog table2420.

The application administrator1010may then input an application name1620-A, such as “Web-A”, as illustrated inFIG. 15below. The application administrator1010may the select a number of the VMs1630-A to be created.

The management program2410may display the confirmation GUI1600-B illustrated inFIG. 15if the “Confirm” button1640-A is selected. Conversely, if the “Cancel” button1650-A ofFIG. 14is selected, the management program2410may cancel the deployment process.

FIG. 15illustrates an example confirmation GUI1600-B of the self-service portal1600, in accordance with an example implementation of the present application. This GUI1600-B may be displayed after the application administrator1010selects the “Confirm” button1640-A of the application deployment GUI1600-A of the self-service portal1600.

The confirmation GUI1600-B may include a number of fields providing information entered by the application administrator1010for confirmation. For example, field1610-B may provide the application type and field1620-B may provide the application name. Further, field1630-B may provide the number of VMs which run the application.

Additionally, field1640-B may provide a plurality of columns and rows of information about the VMs being provisioned. Column1641-B may provide the name of the VM. This name may be created from application name1620-B by the management program2410. Column1642-B may provide the number and type of the CPUs required. Column1643-B may provide the capacity of the memory and column1644-B may provide the capacity of storage volumes. Each row (164A-B-164D-B) may provide a configuration of each VM. For example, row164A-B shows the configuration of the VM named “Web-A-1”. This VM has 16 high speed CPUs, 8 GB memory and 2 TB of storage volume.

Field1650-B may provide the calculated unit cost of the application. According to the unit price information2438of the VM template table2430, a unit cost of one “High I/O” VM is 90. Further, the number of “High I/O” VM allocated for this application is 4. Therefore, the unit cost of this application is 360 (90×4). In some example implementations, the unit cost may also include the cost of storage volumes.

The management program2410may execute an application deployment process as may be known in the art, if the “Confirm” button1660-B is selected. Conversely, if the “Cancel” button1670-B is selected, the management program2410may cancel the deployment process. Further, if the “Back” button1680-B is selected, the management program2410redisplays the provisioning GUI1600-A of the self-service portal1600illustrated inFIG. 15.

FIG. 16illustrates an example VM network configuration GUI1600-C of the self-service portal1600, in accordance with an example implementation of the present application. This GUI1600-C may be used when the application administrator1010configures a network among the VMs which he/she is using. The application administrator1010may connect one VM to another VM by using the VM network editor1610-C. For example, VM05is being connected to VM02by an application administrator1010inFIG. 16.

If the “OK” button1620-C is selected, the management program2410may configure networks among VMs according to the topology information specified by application administrator1010. In this example implementation, the application administrator doesn't specify the concrete network parameters (such as IP addresses). However, in other implementations, the GUI1600-C may include text fields for specifying network parameters (such as IP addresses).

If the “Cancel” button1630-C is selected, the management program2410may cancel the network configuration process.

FIG. 17illustrates a flow diagram for a process10000for pairing VMs, in accordance with an example implementation of the present application.

At10010, the process10000for pairing VMs begins. At10020, the management program2410receives a request to deploying applications based on input provided by the application administrator1010using the self-service portal1600. The parameters illustrated in the confirmation GUI1600-B of self-service Portal1600are passed to the management program2410with the request.

After the request is received, the management program2410provisions storage volumes, creates virtual servers, assigns the storage volumes to the virtual servers and deploys the applications on the virtual servers based on the parameters received with the request at10030.

At10030, the management program2410receives a request to configure the virtual server's networks based on input provided by the application administrator1010using the self-service portal1600. The network topology information entered by the application administrator1010using the VM network configuration GUI1600-C of the self-service portal1600is passed to the management program2410with the request.

At10050, the management program2410configures virtual server's networks based on the network topology information received with the request. At10060, the management program2410identifies candidate VMs for replacement using the sub-sequence process illustrated inFIG. 18and discussed below. This sub-sequence process is used to identify one or more VMs that may be replaced by the VMs created in the step10030.

At10070, the management program2410determines whether any candidate VMs that may be replaced by the newly created VMs were identified in10060. If at least one candidate VMs were identified in10060(i.e. result is “YES”), the process10000proceeds to10080. If no candidate VMs were identified in10060(i.e. result is “NO”), then the process10000proceeds to10110.

At10080, the management program2410invokes a sub-sequence of scoring a possibility of replacement illustrated inFIG. 19and discussed below. Using the sub-sequence ofFIG. 19, a score indicating a possibility that the VMs created in10030were created to replace one or more VMs in the candidate VMs found in10060is calculated.

At10090, the management program2410judges whether the score calculated in10080is larger than a predefined threshold or not. The threshold may be specified by an IT infrastructure administrator1030. If calculated score is greater than the threshold (i.e. the result is “Yes”), then the process10000proceeds to10100. If calculated score is not greater than threshold (i.e. the result is “No”), then the process10000proceeds to10110.

At10100, the management program2410pairs the VM created in10030with the candidate VM based on the scores calculated in10080. In some example implementations, if multiple candidate VMs are identified as possibly being replaced with the VM created in10030, the VM created in10030is paired to the candidate VM having the highest score. The pairing information is stored to the Virtual Server Pair Table2570. The process then proceeds to10110.

At10110, the management program2410ends the process10000and the process of deploying the applications is completed.

FIG. 18illustrates a flow diagram of a process20000for identifying candidate VMs for replacement, in accordance with an example implementation of the present application. The process20000ofFIG. 18may be used by the management program2410as a candidate VM identifying sub-sequence in10060ofFIG. 17.

At20010, the process20000for identifying candidate VMs for replacement begins. At20020, the management program2410finds VMs associated the VM created in10030of the process10000ofFIG. 17by referring to the Virtual Server Topology Table2560. A VM may be considered associated with the VM created in10030if the VM shares a parent-child relationship with the VM created in10030. For example, a VM that is a child VM of the VM created in10030may be considered associated with the VM created in10030. Additionally, a VM that is a parent VM of the VM created in10030may be considered associated with the VM created in10030.

At20030, the management program2410gets the type of the application running on each of the VMs identified as associated with the VM created in10030based on the mapping table2550and image catalog table2420.

At20040, the management program2410determines whether the application type of each of the VMs identified as associated with the VM created in10030is either a “Load Balancer” or “Router”. If any of the VMs identified as associated with the VM created in10030is either a “Load Balancer” or “Router” (i.e. the result is “Yes”), then the process20000proceeds to20050. If none of the VMs identified as associated with the VM created in10030is either a “Load Balancer” or “Router” (i.e. the result is “No”), then the process20000proceeds to20080. As a replacement VM may likely be used as a load balancer or a router, VMs identified as either a “load balancer” or “router” may be considered as having a probability of being created as a replacement in some implementations. In other implementation, other technologies used for switching input/output (I/O) among VMs may be used as an indication of a probability of being a replacement VM.

In20050, the management program2410finds all associated VMs of each of the VMs identified in20040as is either a “Load Balancer” or “Router” (except the VM created in the step10030) by referring Virtual Server Topology Table2560. A VM may be considered associated with the VM identified in20040if the VM shares a parent-child relationship with the VM identified in20040. For example, a VM that is a child VM of the VM identified in20040may be considered associated with the VM identified in20040. Additionally, a VM that is a parent VM of the VM identified in20040may be considered associated with the VM identified in20040.

In20060, the management program2410filters out VMs whose application types are not same as the one of the VM created in the step10030. In20070, the management program2410identifies any VMs not filtered out in20060as a “candidate VMs for replacement” by the VM created in10030. In20080, the management program2410ends the sub-sequence process20000and returns to process10000at10070and proceeds from there.

FIG. 19illustrates a flow diagram for a process30000for scoring possibility of replacement, in accordance with an example implementation of the present application. The process30000ofFIG. 19may be used by the management program2410as a scoring sub-sequence in10080ofFIG. 17.

At30010, the process30000for scoring a possibility of replacement begins.

At30020, the management program2410gets performance data for each of the candidate VMs identified in10060ofFIG. 17at the time the application administrator1010requested creation of VMs in10020. At30030, the management program2410determines whether the performance of each of the candidate VMs is worse than a predefined threshold. If the performance of any of the candidate VMs is worse than the threshold (i.e. the result is “Yes”), then the process30000proceeds to30040. If the performance of any of the candidate VMs is not worse than the threshold (i.e. the result is “No”), then the process30000proceeds to30050. Performance worse than the threshold may be considered an indication that the newly created VM was created for load balancing rather than replacement of the candidate VMs.

At30040, the management program2410adds 1 point to (e.g., increments) the score.

At30050, the management program2410determines whether the number of VMs created in10030equals the number of candidate VMs identified in10060ofFIG. 17. If the result is “Yes” then the process30000proceeds to30060. If the result is “No” then the process30000proceeds to30070. The number of VMs created equaling the number of candidate VMs identified may be considered an indicator of replacement.

At30060, the management program2410adds 1 point to (e.g., increments) the score.

At30070, the management program2410determines whether the version of the application and the version of the OS deployed in10030is the same as the version of the application and the version of the OS of the candidate VMs. If the result is “Yes” then the process30000proceeds to30080. If the result is “No” then the process30000proceeds to30090. The same version of application and the same version of the OS may be considered an indicator of load balancing rather than replacement.

At30080, the management program2410subtracts 1 point from (e.g., decrements) the score. At30090, the management program2410ends the sub-sequence process30000and returns to process10000at10090and proceeds from there.

FIG. 20illustrates a flow diagram for a process40000for migrating VMs to another physical server and/or storage volumes to another storage array, in accordance with an example implementation of the present application.

At40010, the process40000for migrating VMs to another physical server and/or storage volumes to another storage array begins. At40020, the management program2410monitors performances of the storage arrays, the physical servers and the virtual servers.

At40030, the management program2410determines whether migration of one or more VMs is necessary based on performance of the one or more VMs. If migration of one or more VMs is determined necessary (i.e. the result is “Yes”), then the process40000proceeds to the step40040. If migration of one or more VMs is determined as not necessary (i.e. the result is “No”), then the process40000proceeds to the step40050.

At40050, the management program2410determines whether migration of one or more storage volumes is necessary based on the performance of the one or more storage volumes. If migration of one or more storage volumes is determined necessary (i.e., the result is “Yes”), then the process40000proceeds to the step40060. If migration of one or more storage volumes is determined as not necessary (i.e., the result is “No”), then the process40000proceeds to the step40070.

At40060, the management program2410invokes a sub-sequence process60000of migrating storage volumes illustrated inFIG. 22described in detail below.

At40070, the management program2410waits an amount of time before returning to the step40020and repeating the process40000. The amount of time is not particularly limited and may be a second, a minute, an hour, a day, a week, or any other time period that may be apparent to a person of ordinary skill in the art.

FIG. 21illustrates a flow diagram for a process50000for migrating one or more VMs, in accordance with an example implementation of the present application. The process50000ofFIG. 21may be used by management program2410as sub-sequence for migrating one or more VMs in40040ofFIG. 20.

At50010, the process50000for migrating one or more VMs begins. At50020, the management program2410finds a paired VM of the VM being migrated based on the pairing information stored in the Virtual Server Pair Table2570. At50030, the management program2410attempts to identify a physical server, which has performance capacity to spare for running both the VM being migrated and VM paired to the VM being migrated based on the server performance table2590.

At50040, the management program2410determines a physical server having performance capacity to spare for running both the VM being migrated and the VM paired to the VM being migrated was identified. If a physical server was identified (i.e. the result is “Yes”), then the process50000proceeds to50050. If no physical server was identified (i.e., the result is “No”), then the process50000proceeds to50060.

At50050, the management program2410migrates the VM and the VM paired to the VM being migrated to the physical server identified in50030.

At50060, the management program2410identifies a physical server, which has performance to spare for running the VM being migrated based on the server performance table2590.

At50070, the management program2410identifies a physical server, which has performance to spare for running the VM paired to the VM being migrated even if the load of the VM paired to the VM being migrated increased to be the same load as of the VM being migrated based on the server performance table2590.

At50080, the management program2410migrates the VM and the VM paired to the VM being migrated to the physical servers found in50060and50070, respectively. At50090, the management program2410ends the sub-sequence process50000and returns to process40000at40050and proceeds from there.

FIG. 22illustrates a flow diagram for a process60000for migrating one or more storage volumes, in accordance with an example implementation of the present application. The process60000ofFIG. 22may be used by management program2410as sub-sequence for migrating one or more storage volumes in40060ofFIG. 20.

At60010, the process60000for migrating one or more storage volumes begins. At60020, the management program2410finds a VM using the volume being migrated. At60030, the management program2410finds a VM paired to the VM found in60020based on the pairing information stored in the virtual server pair table2570. At60040, the management program2410finds a storage volume assigned to the paired VM.

At60050, the management program2410attempts to identify a storage array, which has performance to spare for processing I/O to both the volume being migrated and the paired volume based on the storage performance table2580.

At60060, the management program2410determines whether a storage array was identified in60050. If a storage array was identified (i.e. the result is “Yes”), then the process6000proceeds to60070. If no storage array was identified (i.e. the result is “No”) then the process proceeds to60080.

At60070, the management program2410migrates the volume and the paired volume to the storage array found in60050.

At60080, the management program2410finds a storage array, which has performance capacity to spare for processing I/O to the volume being migrated. At60090, the management program2410finds a storage array, which has performance to spare for processing I/O to the paired volume even the load of the paired volume increased to be the same load as the volume being migrated.

At60100, the management program2410migrates the volume and the paired volume to the storage arrays found in60080and60090respectively. At60110, the management program2410ends the sub-sequence process60000and returns to process40000at40070and proceeds from there.

In this example implementation, when a management program deploys new VMs, it may identify candidate VMs for replacement, score a possibility of replacement and pairs the new VMs to candidate VMs, if the possibility for replacement is determined to be high. Further in this example implementation, the management program may also migrate VMs and storage volumes used by the VMs to other physical servers and storage arrays in unit. By doing this, an existing VM and a new VM, which may be considered to be a replacement for the existing VM can be migrated together. This may avoid situations where migration of the existing VM for load balancing of physical servers may become an inefficient use of time and labor of infrastructure administrators due to replacement of VMs by application administrators.

Second Example Implementation

The second example implementation may illustrate a management program that may inherit management policies from an existing VMs being replaced and leverage them to manage new VMs, which replace the existing VMs.

FIG. 23illustrates a second example implementation of a management server2000′ for the cloud system ofFIG. 2A. Most parts of management server2000′ are similar to the management server2000inFIG. 2B. However, in the example ofFIG. 23, management server2000′ has storage performance table2580′ and server performance table2590′, which are different from the storage performance table2580and server performance table2590of the management server2000shown inFIG. 2B. Additionally, the management server2000includes two additional tables, a storage thresholds table25A0and a server thresholds table25B0.

FIG. 24illustrates an example storage performance table2580′, in accordance with the second example implementation of the present application. Most parts of the storage performance table2580′ similar to the storage performance table2580inFIG. 12. However, the storage performance table2580′ has the original storage ID column2587′ and the original volume ID column2588′. These columns may be used to show that performance data has been inherited from another storage volume. For example, the row258A′ shows that the performance data of storage volume02of storage array02. This performance data was inherited from storage volume01of storage array01. The method to inherit the performance data is discussed later with respect toFIG. 27.

FIG. 25illustrates an example server performance table2590′, in accordance with the second example implementation of the present application. Most parts of the server performance table2590′ are similar to the server performance table2590inFIG. 13. However, the server performance table2590′ also has the original server ID column2598′, which creates different rows259A′ and259B′ fromFIG. 13. This column may be used to show that the performance data is inherited from another server. For example, the row259B′ shows the performance data of virtual server03. The performance data was inherited from virtual server02. The method to inherit the performance data is discussed later with respect toFIG. 28.

FIG. 26illustrates an example storage thresholds table25A0, in accordance with an example implementation of the present application.

As illustrated, the storage thresholds table25A0includes a number of columns providing information on the storage thresholds. Column25A1provides the identification information, or identifications, of the storage arrays. Column25A2provides the identifications of the storage volumes. Column25A3provides the identifications of the storage arrays originally having the thresholds. Column25A4provides the identifications of the storage volumes originally having the thresholds. Column25A5provides the time periods of thresholds being activated. Column25A6provides the thresholds of usage rate of the caches assigned to the storage volumes. Column25A7provides the thresholds of usage rate of the array group from which the storage volumes are curved. Column25A8provides the thresholds of usage rate of the ports assigned to the storage volumes.

Each row of the storage thresholds table25A0shows the thresholds of performance data of each storage volume. For example, row25AA shows the thresholds of performance data of storage volume02of storage array02. The thresholds are defined for at least three time periods (e.g., from 0:00 AM to 7:00 AM, from 7:00 AM to 0:00 PM and from 0:00 PM to 5:00 PM). The management program2410provides alerts to infrastructure administrators1030if one or more performance data exceeds the thresholds in the time period. The row25AA also shows that the thresholds assigned to the storage volume02of the storage array02were inherited from the storage volume01of the storage volume01. The method to inherit the thresholds is disclosed later.

FIG. 27illustrates an example server thresholds table25B0, in accordance with an example implementation of the present application.

As illustrated, the server thresholds table25B0includes a number of columns providing information on the server thresholds. Column25B1provides the identifications of the physical or virtual servers. Column25B2provides the identifications of the physical or virtual servers originally having the thresholds. Column25B3provides flags indicating whether the servers are a physical server or not. In this example implementation, if the value of “YES” is contained then the server is a physical server and if the value of “NO” is contained then the server is a virtual server. In other example implementations, this relation may be reversed with a value of “YES” indicating a virtual server and a value of “NO” indicating a physical server.

Column25B4provides the time periods of thresholds being activated. Column25B5provides the thresholds of usage rate of the CPUs of the servers. Column25B6provides the thresholds of usage rate of the memories of the servers. Column25B7provides the thresholds of usage rate of the disk of the servers. Column25B8provides the thresholds of usage rate of the ports of the servers.

Each row (25BA-25BB) of the server thresholds table25B0shows the thresholds of performance data of each server. For example, row25BB shows the thresholds of performance data of virtual server03. The thresholds are defined for at least three time periods (e.g., from 0:00 AM to 9:00 AM, from 9:00 AM to 3:00 PM and from 3:00 PM to 5:00 PM). The management program2410provides alerts to infrastructure administrators1030if one or more performance data exceeds the thresholds in the time period. The row25BB also shows that the thresholds assigned to the virtual server03were inherited from the virtual server02. The method to inherit the thresholds is disclosed later.

FIG. 28illustrates a flow diagram for a process70000for inheriting policies of management from a VM to be deleted, in accordance with the second example implementation of the present application. In this example implementation, the term of “policies of management” may mean one or more sets of performance data and thresholds of performance data assigned to a VM and the storage volumes associated with the VM. However, example implementations of the “policies of management” used are not limited to this example and may include any additional or alternative information that may be apparent to a person of ordinary skill in the art.

At70010, the process70000for inheriting policies of management from a VM to be deleted begins. At70020, the management program2410may receive a request to delete on or more VMs from an application administrator1010through the self-service Portal1600. At70030, the management program2410attempts to identify any VMs paired with the one or more VMs, for which deletion is requested based on the virtual server pair table. The management program2410may also identify any storage volumes associated with the paired VMs based on the mapping table2550.

At70040, the management program2410determines whether any VMs paired with the one or more VMs to be deleted were identified, and whether any storage volumes were identified. If a paired VM was identified, or a storage volume was identified (i.e. the result is “Yes”), then the process70000proceeds to70060. If no paired VM was identified, and no storage volume was identified (i.e., the result is “No”), then the process70000proceeds to70050.

At70050, the management program2410deletes the one or more VMs requested in70020and any storage volumes associated with the deleted VMs. The management program2410also deletes the configuration data, performance data and threshold data from any tables in the memory2500.

At70060, the management program2410identifies rows of Server Performance Table2590and Server Thresholds Table258B0having Server IDs equal to (i.e. matching) the IDs of the one or more VMs, which are to be deleted. At70070, the management program2410copies the Server IDs from the server ID columns2591′,25B1of the rows identified in70060to the Original Server IDs columns2598′,25B2of the rows identified in70060. At70080, the management program2410changes Server IDs of the server ID columns2591′,25B1of the rows found in70060to the server IDs of the VMs paired with the one or more VMs, which are to be deleted, identified in70030.

At70090, the management program2410finds rows whose Storage IDs and Volume IDs equal to the IDs of storage arrays and the IDS of the storage volumes used by the VMs being deleted in the Storage Performance Table2580′ and the Storage Thresholds Table25A0, respectively. At70100, the management program2410copies the Storage IDs of the rows found in70090from the storage ID columns2581′,25A1to Original Storage ID columns2587′,25A3of the rows found in70090. At70100, the management program2410also copies the Volume IDs of the rows found in70090from the volume ID columns2582′,25A2to Original volume ID columns2588′,25A4of the rows found in70090.

At70110, the management program2410changes the Storage IDs in the storage ID columns2581′,25A1of the of the rows found in70090to the IDs of the storage arrays used by VMs paired with the one or more VMs, which are to be deleted, identified in70030. At70110, the management program2410also changes the volume IDs in the volume ID columns2582′,25A2of the of the rows found70090to the IDs of the storage volumes use by VMs paired with the one or more VMs, which are to be deleted, identified in70030. Once70110has been completed, the process precedes to70050.

At70120, the management program2410ends the management policy inheriting process70000.

FIG. 29illustrates an example performance monitoring GUI2410-A of the management program2140, in accordance with the second example implementation of the present application.

Field2411-A of the GUI2410-A illustrates performance data and thresholds of a virtual server in a graph. It illustrates CPU usage, memory usage, disk usage and port usage of the virtual server03. The solid lines represent the performance and the dotted lines represent the thresholds. Field2411-A also shows that the virtual server03is considered as a replacement of the virtual server02. The timing of replacement (i.e., deletion of the virtual server02) is also shown in the graph.

Field2412-A of the GUI2410-A illustrates performance data and thresholds of a storage volume in a graph. It illustrates cache usage, array group usage and port usage of the storage volume02of the storage array02. The solid lines represent the performance and the dotted lines represent the thresholds. Field2412-A also shows that the storage volume02of the storage array02is considered as a replacement of the storage volume01of the storage array01. The timing of replacement (i.e., deletion of the storage volume01of storage array01) is also shown in the graph.

If “Close” button2413-A is selected, the management program2410may close the GUI2410-A.

In this example implementation, a management program may inherit management policies from existing VMs being replaced and may leverages them to manage new VMs, which replace the existing VMs. In this example implementation, the inheritance of management policies is done at the time of deletion of existing VMs but it is not limited to this. For example, inheritance can be done at the time of configuration changes of routers and/or load balancers to replace VMs. This may allow infrastructure administrators1030to manage VMs and storage volumes used by the VMs easily by using the inherited management policies.

In some of the example implementations discussed above, a management program may identify VMs created to replace existing VMs, and pair the VMs so that they can be managed together. Further in some of the example implementations discussed above, a management program may use a process to allow a VM replacing an existing VM to inherit management policies. These example implementations may allow a reduction in operational costs to manage underlying IT infrastructures of cloud environments. However, example implementations need not achieve these or any other benefit, which may be apparent to a person of ordinary skill in the art.

Some example implementations may be represented in form of a computer program containing instructions. The instructions for the program may be stored in a computer readable storage medium, which includes tangible and non-transitory media such as flash memory, random access memory (RAM), Hard Disk Drive (HDD) and the like. Alternatively, instructions may be stored in the form of a computer readable signal medium, which includes other media such as carrier waves.

Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the example implementations disclosed herein. Various aspects and/or components of the described example implementations may be used singly or in any combination. It is intended that the specification and examples be considered as examples, with a true scope and spirit of the application being indicated by the following claims.