Patent ID: 12242362

DETAILED DESCRIPTION

Embodiments of the present invention disclose a system and method for autonomous data sharding and data store topology alterations for disaster recovery preparation. In the embodiments of the present invention, the disclosed system monitors resources across a network connecting multiple data centers, estimates the time to create a backup of data from a first data center to a second data center in an event of a disaster that takes one or more data centers offline, creates data shards if the estimated time does not meet the recovery time objective (RTO), and copies the data shards to other data centers to ensure that a backup of data is always available such that RTO will be met in the event of the disaster. Copying the data shards to other data centers changes a topology of data stores across multiple data centers in the network. By constantly altering the data store topology, the disclosed system and method guarantee that both the RTO and the recovery point objective (RPO) are met in the event of the disaster. In the event of a disaster, the present invention ensures that all the data can be transferred between the remaining data centers within the RTO and ensure that at least two copies if the data are present.

FIG.1is diagram of system100which includes components performing autonomous data sharding to meet the recovery time objective (RTO) in an event of a disaster that takes a data center offline, in accordance with one embodiment of the present invention. System100includes multiple data centers, for example, data center 1110, data center 2120, and data center 3130. The number of data centers shown inFIG.1is for a purpose of illustration; it should be appreciated that the number of data centers may be any number more than three. Typically, the multiple data centers are located at different sites. System100further includes management server140. As shown inFIG.1, data center 1110, data center 2120, data center 3130, and management server140are interconnected over network150. Network150may be a wired and/or wireless network. Network150may be any communication protocol that allows data to be transferred between components of system100.

Each of the multiple data centers includes one or more data stores. Each of the one or more data stores has multiple copies in at least two of the data centers. Each of the one or more data stores may be separate databases, separate systems, separate shards, or any other subset of data stored within the multiple data centers. As an example shown inFIG.1, data center 1110includes data store A160and data store B170; data center 2120includes data store B170and data store C180; data center 3130includes data store A160and data store C180. Data store A160has two copies, one copy in data center 1110and another copy in data center 3130; data store B170has two copies, one copy in data center 1110and another copy in data center 2120; data store C180has two copies, one copy in data center 2120and another copy in data center 3130.

Each of the multiple data centers includes a data sharding engine. As an example shown inFIG.1, data center 1110includes data sharding engine 1115, data center 2120includes data sharding engine 2125, and data center 3130includes data sharding engine 3135. From autonomous replication engine142on management server140, each of the data sharding engines receives information on which data store sharding is needed and information about sharding sizes. Data sharding engine115performs the data sharding in data center 1110; data sharding engine125performs the data sharding in data center 2120; data sharding engine135performs the data sharding in data center 3130.

Management server140includes multiple modules. To ensure the RTO is met and at least two copies of the data exists across two or more data centers in an event of a disaster that takes a data center offline, management server140performs analysis of all the data stores in respective ones of the data centers, determines when to create one or more data shards and shard sizes, and instructs the data sharding engines in respective ones of the data centers to copy the one or more shards to other data centers.

In the embodiment shown inFIG.1, management server140is located in a computer system separated from the data centers. In another embodiment, management server140may be located in anyone of the data centers. In yet another embodiment, management server140may be located in at least two data centers to ensure that proper data backup occurs in an event of a disaster that takes one data center offline.

Management server140includes network monitoring engine141, autonomous replication engine142, data sharding calculation engine143, and monitoring database144. Network monitoring engine141extracts information about the current network speed for communication between data centers. For example, the rate of data transfer between a first data center and a second data center may be 2 Tb/hr. Autonomous replication engine142calls data sharding calculation engine143to dynamically calculate the shard sizes to meet the RTO. The calculation of the shard sizes uses information about the network speed and a growth cap. Autonomous replication engine142instructs the data sharding engines to execute sharding based on the calculated shard sizes and copy data shards to destination data centers. Autonomous replication engine142alters a topology of data storage across multiple data centers such that the RTO can be met in an event of a disaster that takes one or more of the multiple data centers offline.

Monitoring database144stores current and historical information about data center topologies, network speeds at different times (e.g., speeds at different times of the day, speeds for different days of the week, speeds during holidays, etc.), and system settings including the RTO, the RPO, the growth cap, and the storage migration constant.

FIG.2is a flowchart showing operational steps for altering a topology of data storage across multiple data centers such that the recovery time objective (RTO) can be met in the event of a disaster that takes a data center offline, in accordance with one embodiment of the present invention. The operational steps are implemented by one or more computers or servers. For example, the operational steps are implemented by management server140and/or data sharding engines (including data sharding engine 1115, data sharding engine 2125, and data sharding engine 3135) on one or more computers or servers.

In step201, the one or more computers or servers receive, from an administer, system settings that are defined for RTO and RPO for multiple data centers in a network. The system settings include the RPO, the RTO, a growth cap (γ), and a storage migration constant (μ). The growth cap (γ) ensures that there is sufficient free space after a data shard migration; for example, a growth cap of 80% will ensure that the current data storage at a given data center plus the size of a data shard to be copied into the same data center will sum to less than 80% of a total storage capacity of the data center. The relationship between the storage capacity (C) of the data center, the current used storage(S) of the data center, the growth cap (γ), and the size (σ) of a data shard can be written as

C≥S+σγ.

The storage migration constant (μ) ensures that the data shard to be copied is sized to be larger than a minimum amount in order to meet the RTO such that any additional data storage will not immediately require sharding and data backup between data centers to occur again. For example, if x is the size of a data shard to be copied from a first data center to a second data center, a storage migration constant of 2 will ensure that a data shard with a size of 2× is copied to the second data center; if this is not done and only a data shard with a size of x is copied, any incoming data to the first data center will immediately require the sharding process to occur again.

For the purpose of illustrating the system and method, the following system settings are used in an example included in later paragraphs of this document. The RTO is 4 hours, the RPO is 0 hours (no data loss allowed), the growth cap (γ) is 80%, and the storage migration constant (μ) is 2.

In step202, the one or more computers or servers back up data of the multiple data centers based on a current topology.FIG.3illustrates initial state300of a data store topology across data centers, in accordance with one embodiment of the present invention. Data center 1310, data center 2320, and data center 3330are interconnected by network340. At initial state300shown inFIG.3, data center 1310, data center 2320, and data center 3330include no data shard. A respective one of the data centers includes data for the respective one of the data centers and one or more backups of data for other data centers. For example, data center 1310includes data store A350and data store B360, in which data store A350is data for data center 1310while data store B360is a backup of data for data center 2320; data center 2320includes data store B360and data store C370, in which data store B360is data for data center 2320while data store C370is a backup of data for data center 3330; data center 3330includes data store A350and data store C370, in which data store C370is data for data center 3330while data store A370is a backup of data for data center 1310.

In step203, the one or more computers or servers monitor computing/network resources and update a database that stores information of the computing/network resources. The database also stores the current topology, historical topologies, and the system settings. The computing/network resources include, but are not limited to, network speeds, CPU utilization, memory utilization, source and destination resources, physical storage capacity, write rate (i.e., used to predict database growth rate), type of facility (e.g., active-active, active-passive, hot site, warm site, and cold site). In an example included in this document, it is assumed (for the purpose of simplification) that the network speed (v) is 2 TB per hour for copying data for both transmit and receive lanes.

In step204, the one or more computers or servers estimate time needed to copy a respective one of data stores of a data center that is assumed offline. The data center that is assumed offline is a respective one of the multiple data centers. The time needed to copy the data is estimated based on information obtained in step203. The time needed to copy the data can be calculated by dividing the size of the respective one of data stores by the network speed (v).

FIG.4illustrates state400of a data store topology across data centers where an assumed disaster takes a data center offline, in accordance with one embodiment of the present invention. InFIG.4, an estimation of time needed to copy data is shown. The example of executing step204(estimating time needed to copy data) is shown inFIG.4. As shown by numeral405, dada center 1310is assumed offline. As shown by numeral410, the time needed to copy data store A350from data center 3330to data center 2320is estimated in step204.

In step205, the one or more computers or servers compare the estimated time in step204with the RTO and determine whether the time needed to copy the data is greater than the RTO. In response to determining that the time needed to copy the data is greater than the RTO (YES branch of step205), in step206, the one or more computers or servers identify that sharding is needed for the respective one of the data stores in order to meet the RTO when the data center is offline.

In the example shown inFIG.4, with the exemplary system settings given in a previous paragraph, the one or more computers or servers determine that the time needed to copy data store A350with 9 TB of data from data center 3330to data center 2320is greater than the RTO (4 hours); therefore, in step205, the one or more computers or servers identify that data sharding is needed for data store A350in order to meet the RTO when dada center 1310is offline.

After step206or in response to determining that the time needed to copy data is not greater than the RTO (NO branch of step205), the one or more computers or servers execute step207, to reiterate steps204-206until the time needed to copy data is estimated for all the data stores of the data center that is assumed offline.

The example of executing step207(reiterating steps204-206) is shown inFIG.4. As shown by numeral420, the one or more computers or servers determine the time needed to copy data store B360from data center 2320to data center 3330. With the exemplary system settings given in a previous paragraph, the one or more computers or servers determine that the time needed to copy data store B360with 2 TB of data from data center 2320to data center 3330is not greater than the RTO (4 hours). Thus, data sharding is not needed for data store B360, and copying data store B360from data center 2320to data center 3330has no issue for meeting the RTO when dada center 1310is offline. In the example shown inFIG.4, the time is estimated so far for all the data stores (data store A350and data store B360); therefore, step207(reiterating steps204-206) is completed.

After completing step207, the one or more computers or servers execute step208. In step208, the one or more computers or servers reiterate steps204-207for another data center set as the data center that is assumed offline, until all the multiple data centers are in turn assumed offline one at a time. In the example shown inFIG.4, after data center 1310is assumed offline, data center 2320and data center 3330will be in turn assumed offline. When either of the data centers is assumed offline, the one or more computers or servers reiterate steps204-207. If either of data center 2320and data center 3330is assumed offline, the time needed to copy data store C370with 5 TB of data is estimated. The one or more computers or servers determine that the time needed to copy data store C370with 5 TB of data is not greater than the RTO (4 hours). Thus, data sharding is not needed for data store C370, and copying data store C370to another data center has no issue for meeting the RTO.

After completing the iterations of204-207for all the multiple data centers, the time needed to copy data has been estimated for each data store across the multiple data centers. From all the data stores in the multiple data centers, the one or more computers or servers identify data stores that need sharding, by comparing the time needed with the RTO. In step209, the one or more computers or servers determine whether at least one data store needs sharding. In the example shown inFIG.4, among three data stores (data store A350, data store B360, data store C370), only data store A350needs sharding.

In response to determining that at least one data store needs sharding (YES branch of step209), in step210, the one or more computers or servers calculate shard sizes for one or more data stores that need sharding. Steps501-503inFIG.5(A)are operational steps for calculating the shard sizes and will be described in later paragraphs with reference toFIG.5(A).

In step211, the one or more computers or servers alter the current topology by creating one or more data shards and copying the one or more data shards to one or more destination data centers, prior to an event of a disaster that takes one or more of the multiple data centers offline. The one or more data shards are copied prior to the disaster occurring. During the disaster occurring, the remaining data (which has not been copied to other data centers prior to the disaster) in the one or more data stores that need sharding can be copied within a time period required by the RTO. Therefore, altering the current topology in step211ensures the RTO is met in the event of the disaster that takes the one or more of the multiple data centers offline. A topology of the data stores in the multiple data centers in the network is altered to prepare data recovery in the event of the one or more of the multiple data centers being offline.

Copying the one or more data shards in step211is based on assignments of the one or more data shards to the one or more destination data centers. The assignments are determined by the one or more computers or servers, and steps504-515inFIG.5(A)andFIG.5(B)are operational steps of determining the assignments. Steps504-515will be described in latter paragraphs with reference toFIG.5(A)andFIG.5(B).

The one or more computers or servers obtain a new topology after implementing sharding for the one or more data stores that need sharding. The new topology of the data stores across the multiple data centers will be discussed in a later paragraph with reference to the illustration inFIG.6.

After step211or in response to determining that no data store needs sharding (NO branch of step209), in step212, the one or more computers or servers determine whether at least one data center is offline. In other words, the one or more computers or servers determine whether there is an event of a disaster that takes one or more of the multiple data centers offline. In response to determining that no data center is offline (NO branch of step211), the one or more computers or servers reiterate step202to back up data of the multiple data centers based on the current topology (when no data store needs sharding) or the new topology (when at least one data store needs sharding).

In response to determining that at least one data center is offline (YES branch of step211), in step213, the one or more computers or servers implement a data recovery plan. By implementing the data recovery plan, the one or more computers or servers ensure that backups of all the data stores exist in data centers that are still online after the event of the disaster taking one or more of the data centers offline. Because dynamic sharding has been performed in step211for preparation for the disaster, the recovery plan can be completed in a manner that the RTO requirement is met. Implementing the data recovery plan and a new topology after implementing the data recovery plan will be discussed in a later paragraph with reference toFIG.7.

FIG.5(A)andFIG.5(B)present a flowchart showing operational steps for calculating shard sizes and determining assignments of the one or more data shards to the one or more destination data centers, in accordance with one embodiment of the present invention. The operational steps are implemented by one or more computers or servers. For example, the operational steps are implemented by management server140on one or more computers or servers.

Referring toFIG.5(A), in step501, the one or more computers or servers select a respective one of the one or more data stores that need sharding. The one or more data stores that need sharding are identified in step206shown inFIG.2. In the example shown inFIG.4, only data store A350needs sharding; therefore, data store A350is selected in step501and is run through the operational steps inFIG.5(A)andFIG.5(B). However, in a general case, the one or more data stores that need sharding are in turn selected and are run through the operational steps inFIG.5(A)andFIG.5(B).

In step502, the one or more computers or servers calculate a size of a data shard for the respective one of the one or more data stores that need sharding. Copying the size of the data shard to another data center allows to update a topology of data storage across multiple data centers such that an updated topology meets the RTO in the event of a disaster that unexpectedly takes the data center offline.

In the example shown inFIG.4, for data store A350with 9 TB, calculation of the size of the data shard that is required to be copied prior to the disaster is performed as follows:

σ=μ·(S-RTO·v)=2·(9⁢TB-4⁢hr·2⁢T⁢Bhr)=2⁢TB.
In the above calculation of the size (σ) of the data shard that is required to be copied prior to the disaster, the exemplary system settings given in a previous paragraph and the capacity value of data store A350are used. The storage migration constant (μ) is 2, the storage(S) of data store A350is 9 TB, the RTO is 4 hours, the network speed (v) is 2 TB per hour. The size (σ) of the data shard that is required to be copied prior to the disaster is calculated as 2 TB. For data store A350, a size of a remaining data shard is calculated as (9 TB−2 TB)=7 TB.

In step503, the one or more computers or servers determine whether calculation of the size is completed for all the one or more data stores that need sharding. In response to determining that the calculation is not completed (NO branch of step503), the one or more computers or servers reiterate steps501and502, until the calculation is done for all the one or more data stores that need sharding. In the example shown inFIG.4, only data store A350needs sharding; therefore, no reiteration of steps501and502is needed.

After the calculation is done for all the one or more data stores that need sharding, the one or more computers or servers obtain sizes of the one or more data shards that are required to be copied to destination data centers prior to the disaster. In response to determining that the calculation is completed (YES branch of step503), in step504, in an ordered list, the one or more computers or servers order the one or more data shards, according to an order of the shard sizes from largest to smallest.

In step505, the one or more computers or servers determine whether there is sufficient space in data centers to which the one or more data shards may potentially be copied. The one or more computers or server determine whether the sufficient space for the calculated shard sizes is available across these data centers with consideration for the growth cap. This step is only a quick check to determine, based on the one or more data shards, whether there is enough capacity across all the data centers to which the one or more data shards may potentially be copied; these data centers do not already contain the one or more data stores that need sharding. In this step, a specific data center where a specific data shard is to be copied has not yet determined. In the example, data store A350needs sharding and the size of the data shard is calculated as 2 TB. The data shard is to be copied to data center 2320and data center 2320is only one that does not already contain data store A350. Determining whether there is sufficient space on data center 2320is as follows:

C≥S+σY→20⁢TB≥(2⁢TB+5⁢TB)+2⁢TB0.8→20⁢TB≥11.25TB
In the above calculation, the storage capacity (C) of data center 2320is 20 TB, the current used storage(S) of data center 2320is (2 TB+5 TB), the size (σ) of the data shard is 2 TB, and the growth cap (γ) is 0.8. The above calculation indicates that there is sufficient space in data center 2320for the calculated shard size.

In response to determining that there is no sufficient space (NO branch of step505), in step506, the one or more computers or servers notify the administrator to adjust network/storage parameters and/or procure additional storage capacity. The administrator is notified that it is not possible to meet the RTO with the current settings and topology in an event of a disaster. The administer will adjust the network/storage parameters and/or procure the additional storage capacity. In some instances, the growth cap (γ) may be increased to temporarily resolve the issue; however, this situation means that data centers are close to their maximum capacities and additional storage should be procured soon. In one embodiment, the one or more computers or servers notify the administrator when current data capacities reach a threshold level (e.g., 70% full).

In step507, the one or more computers or servers receive from the administer new system settings and/or the additional storage capacity. Upon receiving the new system settings and/or the additional storage capacity, the one or more computers or servers reiterate step505.

In response to determining that there is sufficient space (YES branch of step505), in step508, the one or more computers or servers select a respective one of the one or more data shards in the ordered list, according to the order of the shard sizes from largest to smallest.

In step509, the one or more computers or servers determine a destination data center for the respective one of the one or more data shards. The determined destination data center must be a data center that does not already contain data of the respective one of the one or more data shards. The one or more computers or servers may obtain a list of potential data centers if more than one data center does not contain the data of the respective one of the one or more data shards; under this situation, the potential data centers may be ordered based on available free space, and a data center with the most free space available will be determined as the destination data center. In another embodiment, policies may be put in place by the administrator to split the respective one of the one or more data shards across the greatest number of data centers for the highest availability and reliability. In yet another embodiment, policies may be put in place by the administrator to split the respective one of the one or more data shards across the fewest number of data centers to optimize energy consumption. In the example shown inFIG.4, only data center 2320does not contain data store A350so data center 2320is selected as the destination data center.

Referring toFIG.5(B), in step510, the one or more computers or servers determine whether the respective one of the one or more data shards fits the destination data center. In other words, the one or more computers or servers determine whether the size of the respective one of the one or more data shards is less than or equal to the remaining available space of the destination data center.

In response to determining that the respective one of the one or more data shards does not fit the destination data center or the size of the respective one of the one or more data shards is greater than the remaining available space of the destination data center (NO branch of step510), in step513, the one or more computers or servers split the respective one of the one or more data shards into smaller data shards. A size of each of the smaller data shards is smaller than the original size of the respective one of the one or more data shards; therefore, one of the smaller data shards can fit the destination data center. In step514, the one or more computers or servers assign a first smaller data shard to the destination data center. In step515, the one or more computers or servers reiterate step509to determine a destination data center for a second smaller data shard. The one or more computers or servers determine whether the second smaller data shard fits this destination data center. If the second smaller data shard fits this destination data center, the one or more computers or servers assign the second smaller data shard to this destination data center; otherwise, the one or more computers or servers split the second smaller data shard and repeat a splitting process of steps513-515.

In response to determining that the respective one of the one or more data shards fits the destination data center or the size of the respective one of the one or more data shards is not greater than the remaining available space of the destination data center (YES branch of step510), in step511, the one or more computers or servers assign the respective one of the one or more data shards to the destination data center.

In step512, the one or more computers or servers determine whether all the one or more data shards are assigned to one or more destination data centers. In response to determining that not all the one or more data shards are assigned to the one or more destination data centers (NO branch of step512), the one or more computers or servers select another one of the one or more data shards and reiterate step508shown inFIG.5(A). One or more reiterations of step508are executed, until all the one or more data shards are assigned to the one or more destination data centers.

In response to determining that all the one or more data shards are assigned to the one or more destination data centers (YES branch of step512), the one or more computers or servers finish assignments for all the one or more data shards. Once the assignments are completed, the one or more computers or servers execute step211inFIG.2. When executing step211, the one or more computers or servers alter the current topology by creating the one or more data shards and copying the one or more data shards to the one or more destination data centers.

FIG.6illustrates final state600of a data store topology across data centers after data sharding is performed to prepare for an event of a disaster that takes a data center offline, in accordance with one embodiment of the present invention. As an example, in step211shown inFIG.2, final state600of the data store topology is created. As shown inFIG.6, data store A350is sharded into data shard A1610and data shard A2620. Data shard A1610is copied to data center 2320prior to an event of a disaster that takes data center 1310offline.

As discussed in previous paragraphs with reference toFIG.2, after the current topology of the data stores across data centers is altered in step211, the one or more computers or servers determine whether the event of the disaster occurs. In response to determining that the event of the disaster occurs (one of the multiple data centers is offline), in step213shown inFIG.2, the one or more computers or servers implement the data recovery plan.

As an example,FIG.7illustrates implementation of the data recovery plan and state700of a data store topology across data centers after data recovery from the disaster that takes data center 1310offline. In the example, during the event of the disaster, data center 1310is offline (as shown by numeral705). Prior to the event of the disaster, data shard A1610(with 2 TB) has been copied to data center 2320. In implementing the data recovery plan, as shown by numeral715, the one or more computers or servers copy a remaining data shard (i.e., data shard A2620with 7 TB) from data center 3330to data center 2320; therefore, data center 2320has all data of data store A350(including data shard A1610and data shard A2620) after the data recovery. Because data shard A1610in data store A360has been copied to data center 2320prior to the disaster, copying data in the remaining data shard (i.e., data shard A2620) from data center 3330to data center 2320can be done within the time required by the RTO. In implementing the data recovery plan, as shown by numeral725, the one or more computers or servers also copy data store B360from data center 2320to data center 3330; therefore, data center 3330has data store B360after the data recovery. As discussed in a previous paragraph with reference toFIG.4, the time needed to copy data store B360with 2 TB from data center 2320to data center 3330is not greater than the RTO; therefore, copying data store B360from data center 2320to data center 3330has no issue for meeting the RTO. In implementing the data recovery plan, the requirement of RTO is met, due to data sharding and topology alterations prior to the event of the disaster. After implementing the data recovery plan, both data center 2320and data center 3330have data of all data stores.

Various aspects of the present disclosure are described by narrative text, flowcharts, block diagrams of computer systems and/or block diagrams of the machine logic included in computer program product (CPP) embodiments. With respect to any flowcharts, depending upon the technology involved, the operations can be performed in a different order than what is shown in a given flowchart. For example, again depending upon the technology involved, two operations shown in successive flowchart blocks may be performed in reverse order, as a single integrated step, concurrently, or in a manner at least partially overlapping in time.

A computer program product embodiment (CPP embodiment or CPP) is a term used in the present disclosure to describe any set of one, or more, storage media (also called mediums) collectively included in a set of one, or more, storage devices that collectively include machine readable code corresponding to instructions and/or data for performing computer operations specified in a given CPP claim. A storage device is any tangible device that can retain and store instructions for use by a computer processor. Without limitation, the computer readable storage medium may be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these mediums include: diskette, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded device (such as punch cards or pits/lands formed in a major surface of a disc) or any suitable combination of the foregoing. A computer readable storage medium, as that term is used in the present disclosure, is not to be construed as storage in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through a fiber optic cable, electrical signals communicated through a wire, and/or other transmission media. As will be understood by those of skill in the art, data is typically moved at some occasional points in time during normal operations of a storage device, such as during access, de-fragmentation or garbage collection, but this does not render the storage device as transitory because the data is not transitory while it is stored.

InFIG.8, computing environment800contains an example of an environment for the execution of at least some of the computer code involved in performing the inventive methods, such as program(s)826for autonomous data sharding and topology alterations for disaster recovery preparation. In addition to block826, computing environment800includes, for example, computer801, wide area network (WAN)802, end user device (EUD)803, remote server804, public cloud805, and private cloud806. In this embodiment, computer801includes processor set810(including processing circuitry820and cache821), communication fabric811, volatile memory812, persistent storage813(including operating system822and block826, as identified above), peripheral device set814(including user interface (UI) device set823, storage824, and Internet of Things (IoT) sensor set825), and network module815. Remote server804includes remote database830. Public cloud805includes gateway840, cloud orchestration module841, host physical machine set842, virtual machine set843, and container set844.

Computer801may take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer or any other form of computer or mobile device now known or to be developed in the future that is capable of running a program, accessing a network or querying a database, such as remote database830. As is well understood in the art of computer technology, and depending upon the technology, performance of a computer-implemented method may be distributed among multiple computers and/or between multiple locations. On the other hand, in this presentation of computing environment800, detailed discussion is focused on a single computer, specifically computer801, to keep the presentation as simple as possible. Computer801may be located in a cloud, even though it is not shown in a cloud inFIG.8. On the other hand, computer801is not required to be in a cloud except to any extent as may be affirmatively indicated.

Processor set810includes one, or more, computer processors of any type now known or to be developed in the future. Processing circuitry820may be distributed over multiple packages, for example, multiple, coordinated integrated circuit chips. Processing circuitry820may implement multiple processor threads and/or multiple processor cores. Cache821is memory that is located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on processor set810. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located off chip. In some computing environments, processor set810may be designed for working with qubits and performing quantum computing.

Computer readable program instructions are typically loaded onto computer801to cause a series of operational steps to be performed by processor set810of computer801and thereby effect a computer-implemented method, such that the instructions thus executed will instantiate the methods specified in flowcharts and/or narrative descriptions of computer-implemented methods included in this document (collectively referred to as “the inventive methods”). These computer readable program instructions are stored in various types of computer readable storage media, such as cache821and the other storage media discussed below. The program instructions, and associated data, are accessed by processor set810to control and direct performance of the inventive methods. In computing environment800, at least some of the instructions for performing the inventive methods may be stored in block826in persistent storage813.

Communication fabric811is the signal conduction path that allows the various components of computer801to communicate with each other. Typically, this fabric is made of switches and electrically conductive paths, such as the switches and electrically conductive paths that make up buses, bridges, physical input/output ports and the like. Other types of signal communication paths may be used, such as fiber optic communication paths and/or wireless communication paths.

Volatile memory812is any type of volatile memory now known or to be developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, the volatile memory is characterized by random access, but this is not required unless affirmatively indicated. In computer801, the volatile memory812is located in a single package and is internal to computer801, but, alternatively or additionally, the volatile memory may be distributed over multiple packages and/or located externally with respect to computer801.

Persistent storage813is any form of non-volatile storage for computers that is now known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is being supplied to computer801and/or directly to persistent storage813. Persistent storage813may be a read only memory (ROM), but typically at least a portion of the persistent storage allows writing of data, deletion of data and re-writing of data. Some familiar forms of persistent storage include magnetic disks and solid state storage devices. Operating system822may take several forms, such as various known proprietary operating systems or open source Portable Operating System Interface type operating systems that employ a kernel. The code included in block826typically includes at least some of the computer code involved in performing the inventive methods.

Peripheral device set814includes the set of peripheral devices of computer801. Data communication connections between the peripheral devices and the other components of computer801may be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cables (such as universal serial bus (USB) type cables), insertion-type connections (for example, secure digital (SD) card), connections made through local area communication networks and even connections made through wide area networks such as the internet. In various embodiments, UI device set823may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smart watches), keyboard, mouse, printer, touchpad, game controllers, and haptic devices. Storage824is external storage, such as an external hard drive, or insertable storage, such as an SD card. Storage824may be persistent and/or volatile. In some embodiments, storage824may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer801is required to have a large amount of storage (for example, where computer801locally stores and manages a large database) then this storage may be provided by peripheral storage devices designed for storing very large amounts of data, such as a storage area network (SAN) that is shared by multiple, geographically distributed computers. IoT sensor set825is made up of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

Network module815is the collection of computer software, hardware, and firmware that allows computer801to communicate with other computers through WAN802. Network module815may include hardware, such as modems or Wi-Fi signal transceivers, software for packetizing and/or de-packetizing data for communication network transmission, and/or web browser software for communicating data over the internet. In some embodiments, network control functions and network forwarding functions of network module815are performed on the same physical hardware device. In other embodiments (for example, embodiments that utilize software-defined networking (SDN)), the control functions and the forwarding functions of network module815are performed on physically separate devices, such that the control functions manage several different network hardware devices. Computer readable program instructions for performing the inventive methods can typically be downloaded to computer801from an external computer or external storage device through a network adapter card or network interface included in network module815.

WAN802is any wide area network (for example, the internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, now known or to be developed in the future. In some embodiments, WAN802may be replaced and/or supplemented by local area networks (LANs) designed to communicate data between devices located in a local area, such as a Wi-Fi network. The WAN and/or LANs typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and edge servers.

End user device (EUD)803is any computer system that is used and controlled by an end user (for example, a customer of an enterprise that operates computer801), and may take any of the forms discussed above in connection with computer801. EUD803typically receives helpful and useful data from the operations of computer801. For example, in a hypothetical case where computer801is designed to provide a recommendation to an end user, this recommendation would typically be communicated from network module815of computer801through WAN802to EUD803. In this way, EUD803can display, or otherwise present, the recommendation to an end user. In some embodiments, EUD803may be a client device, such as thin client, heavy client, mainframe computer, desktop computer and so on.

Remote server804is any computer system that serves at least some data and/or functionality to computer801. Remote server804may be controlled and used by the same entity that operates computer801. Remote server804represents the machine(s) that collect and store helpful and useful data for use by other computers, such as computer801. For example, in a hypothetical case where computer801is designed and programmed to provide a recommendation based on historical data, then this historical data may be provided to computer801from remote database830of remote server804.

Public cloud805is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and/or other computer capabilities, especially data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages sharing of resources to achieve coherence and economies of scale. The direct and active management of the computing resources of public cloud805is performed by the computer hardware and/or software of cloud orchestration module841. The computing resources provided by public cloud805are typically implemented by virtual computing environments that run on various computers making up the computers of host physical machine set842, which is the universe of physical computers in and/or available to public cloud805. The virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set843and/or containers from container set844. It is understood that these VCEs may be stored as images and may be transferred among and between the various physical machine hosts, either as images or after instantiation of the VCE. Cloud orchestration module841manages the transfer and storage of images, deploys new instantiations of VCEs and manages active instantiations of VCE deployments. Gateway840is the collection of computer software, hardware, and firmware that allows public cloud805to communicate through WAN802.

Some further explanation of virtualized computing environments (VCEs) will now be provided. VCEs can be stored as images. A new active instance of the VCE can be instantiated from the image. Two familiar types of VCEs are virtual machines and containers. A container is a VCE that uses operating-system-level virtualization. This refers to an operating system feature in which the kernel allows the existence of multiple isolated user-space instances, called containers. These isolated user-space instances typically behave as real computers from the point of view of programs running in them. A computer program running on an ordinary operating system can utilize all resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and devices assigned to the container, a feature which is known as containerization.

Private cloud806is similar to public cloud805, except that the computing resources are only available for use by a single enterprise. While private cloud806is depicted as being in communication with WAN802, in other embodiments a private cloud may be disconnected from the internet entirely and only accessible through a local/private network. A hybrid cloud is a composition of multiple clouds of different types (for example, private, community or public cloud types), often respectively implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technology that enables orchestration, management, and/or data/application portability between the multiple constituent clouds. In this embodiment, public cloud805and private cloud806are both part of a larger hybrid cloud.