# Patent

## Title
System and method for reduced cloud IP address utilization

## Abstract

System and method for providing cloud computing services is described. In one embodiment, the system includes a cloud computing environment, the cloud computing environment comprising resources for supporting cloud workloads, each cloud workload having associated therewith an internal cloud address; and a routing system disposed between external workloads of an external computing environment and the cloud workloads, the routing system for directing traffic from an external address to the internal addresses of the cloud workloads.

## Background
CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims the benefit under Title 35, United States Code §119(e) of U.S. Provisional Patent Application No. 61/160,030 filed on Mar. 13, 2009, the disclosure of which is incorporated herein by reference in its entirety. This application is related to the following commonly-assigned, co-pending applications, each of which is also incorporated herein by reference in its entirety:
1. U.S. patent application Ser. No. 12/612,807;
2. U.S. patent application Ser. No. 12/612,818;
3. U.S. patent application Ser. No. 12/612,841;
4. U.S. patent application Ser. No. 12/612,882;
5. U.S. patent application Ser. No. 12/612,895;
6. U.S. patent application Ser. No. 12/612,903;
7. U.S. patent application Ser. No. 12/612,925;
8. U.S. patent application Ser. No. 12/613,077;
9. U.S. patent application Ser. No. 12/613,098;
10. U.S. patent application Ser. No. 12/613,112; and
11. U.S. patent application Ser. No. 12/197,833.


BACKGROUND
Cloud computing is a type of computing in which dynamically scalable and typically virtualized resources are provided as services via the Internet. As a result, users need not, and typically do not, possess knowledge of, expertise in, or control over the technology and/or infrastructure implemented in the cloud. Cloud computing generally incorporates infrastructure as a service (“IaaS”), platform as a service (“PaaS”), and/or software as a service (“SaaS”). In a typical embodiment, cloud computing services provide common applications online, which applications are accessed using a web browser and the software and data for which are stored on servers comprising the cloud.
Cloud computing customers typically do not own or possess the physical infrastructure that hosts their software platform; rather, the infrastructure is leased in some manner from a third-party provider. Cloud computing customers can avoid capital expenditures by paying a provider for only what they use on a utility, or resources consumed, basis or a subscription, or time-based, basis, for example. Sharing computing power and/or storage capacity among multiple lessees has many advantages, including improved utilization rates and an increase in overall computer usage.
With the advent of cloud computing and cloud storage, a challenge exists with respect to IP address usage and the sharing of cryptographic keys between a known and trusted location and the computing cloud.
SUMMARY
One embodiment is a system for providing cloud computing services to an enterprise. The system comprises a cloud computing environment, the cloud computing environment comprising resources for supporting cloud workloads, each cloud workload having associated therewith an internal cloud address; and a routing system disposed between external workloads of an external computing environment and the cloud workloads, the routing system for directing traffic from an external address to the internal addresses of the cloud workloads.



BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates an exemplary IaaS cloud structure such as may be implemented in one embodiment.
FIG. 2 is a flow diagram of one embodiment of a process for implementing reduced cloud IP address utilization.



DETAILED DESCRIPTION
To better illustrate the advantages and features of the embodiments, a particular description of several embodiments will be provided with reference to the attached drawings. These drawings, and other embodiments described herein, only illustrate selected aspects of the embodiments and are not intended to limit the scope thereof. Further, despite reference to specific features illustrated in the example embodiments, it will nevertheless be understood that these features are not essential to all embodiments and no limitation of the scope thereof is thereby intended. Any alterations and further modifications in the described embodiments, and any further applications of the principles of the embodiments as described herein are contemplated as would normally occur to one skilled in the art. Furthermore, some items are shown in a simplified form, and inherently include components that are well known in the art. Further still, some items are illustrated as being in direct connection for the sake of simplicity and clarity. Despite the apparent direct connection, it is understood that such illustration does not preclude the existence of intermediate components not otherwise illustrated.
The embodiments described herein provide a mechanism for providing reduced cloud IP address utilization. To this end, one or more embodiments described herein provide a method and mechanism to enable the use of a single IP address to address many instances within a cloud infrastructure. Additionally, one or more such embodiments provide a method and mechanism to provide the keys necessary to secure traffic and storage in the cloud.
FIG. 1 illustrates an exemplary IaaS cloud structure. As shown in FIG. 1, the cloud structure includes a hardware layer 100 comprising storage assets 102, processing assets 104, and network assets 106. To facilitate usefulness of the cloud to a variety of enterprises, workloads are sponsored in the cloud as virtual machines possibly accessing virtualized storage and/or virtualized networks. This is accomplished via a virtualization layer 108. Thus, the hardware layer 100 is insulated from the actual workloads to be sponsored in the cloud at a layer 110 by the virtualization layer 108 hardware, storage, and networking so that the operating system selected by the enterprise can be sponsored on whatever hardware the cloud services provider makes available. Having established the hardware and virtualization layers 100, 108, the assets 102, 104, and 106 are available in a standardized way to workloads hosted in the workload layer 110, which is the layer the customer typically views as the “cloud”. It will be recognized that some of the workloads sponsored in the cloud, specifically, workloads 111, are workloads that are germane to the operation of the cloud and may consist of monitoring processes for enabling the cloud services provider to monitor the health of the cloud, management processes to enable the cloud services provider to ensure that service-level agreements are enforced, and so on.
Enterprises using the cloud are represented by virtualization processes and storage shown as workloads 112. These processes are typically started by an enterprise via a cloud portal or API utilized by administrative personnel or processes running at the enterprise or in the cloud. A typical cloud services provider may be using standard ITIL practices and may utilize a configuration management database (“CMDB”) 114, which affects the entire cloud infrastructure and which describes the practice and policies used for instantiating virtualized workloads and storage.
FIG. 2 is a flow diagram of one embodiment of a system 200 for providing reduced cloud IP address utilization. As shown in FIG. 2, one or more applications of an enterprise, represented in FIG. 2 by applications 202A-202C, hosted in a known and trusted environment 204 under the control of the enterprise. As will be described, the applications 202A-202C require access to storage and/or processes in a cloud 206; as a result, such storage and/or processes may be disposed in an unknown location and under an unknown trust model. In accordance with one embodiment, as will be described in detail hereinbelow, it is possible to annex portions of the cloud 206 so that the applications 202A-202C can utilize resources disposed within the cloud 206 in the same manner as if they were disposed in a trusted and known location, such as a data center (not shown) of the enterprise.
In the embodiment illustrated in FIG. 2, a secure bridge 208 provides a secure connection between applications 202A-202C and a virtual router 210, which secure connection can be created based upon keys that were previously traded between the provider of the cloud 206 (“the cloud services provider”) and the enterprise as is known in the art. The virtual router 210 lies within the infrastructure and under the control of the cloud services provider (IaaS) and as such, the public and/or private keys that the IaaS holds are secured by the IaaS infrastructure, data center security, physical security, and others.
As a result, because the secure bridge 208 and virtual router 210 possess mutually shared keys or mutually shared parts of key pairs, a secured connection can be created between the applications 202A-202C and the virtual router 210 via the secure bridge 208 as described in detail in U.S. patent application Ser. No. 12/612,841, which has been incorporated by reference in its entirety. In the embodiment shown in FIG. 2, each application 202A-202C has access to the keys mentioned hereinabove in connection with the secure bridge 208 such that endpoint security from the applications 202A-202C to an external address 212, shown in FIG. 2 as “174.65.x.x”, via the router 210 can exist without the need of the secure bridge 208.
As previously noted, the virtual router 210 lies within the infrastructure of the cloud services provider and acts as a Network Address Translator (“NAT”) to workloads within a specific domain. In a representative embodiment, the IP address is a private addresses, such as a 10-net, so that only the single external address 212 is necessary to access the cloud processes, as will be described. In such a case, as shown in FIG. 2, a distributor 214 forwards traffic from the external address 212 to internal addresses 216 (e.g., “10.1.1.x”) so that only one external address needs to be associated with the processes within the cloud 206. In one embodiment, the foregoing may be accomplished by matching port ranges to specific workloads, represented by workloads 218A-218D, so that if auto load balancing of each of the workloads is required, the port ranges can enable a distinction to be made between the workloads operating within the cloud 206 via a given 10-net.
In one embodiment, the distributor 214 maintains all port mappings in a distributor registry 226 so that requests from applications 202A-202C can be sent to the appropriate one of the workloads 218A-218D. Information may be split between the distributor registry 226 and the secure bridge 208 and further information, such as tokens from a Representational State Transfer (“REST”) call can be maintained with the port mapping such that better information can be provided during the mapping process.
In the embodiment shown in FIG. 2, the internal addresses 216 may be partitioned such that the “x” therein represents some kind of subnet IP address grouping (e.g., 10.1.1.10-20). Accordingly, instead of using a port, IP addresses may be used within the address space. In this embodiment, the distributor 214 may map a single external address to within the 10-net address space so that it is multiplexed into various types of workloads and or storage. In the illustrated embodiment, storage is also protected this way by allowing one of the workloads, such as the workload 218D, to hold keys and to process traffic for storage in the cloud 206. As a result, the storage can be encrypted and decrypted on-the-fly as needed by either an internal-to-the-cloud or external-to-the-cloud process.
Once the secure connection between the applications 202A-202C and the distributor 214 is established based on attestation by the virtual router 210 to the fact that the distributor has not changed and therefore may be trusted in its process model, cryptographic keys may be transmitted from the applications across to the distributor so that a process can receive keys, as represented in FIG. 2 by a dashed line 222. This process ensures that storage or other content and traffic can be protected via keys received external to the cloud 206. This is significant because it allows for the keys to be stored in a location other than the cloud 206, thereby providing a layer of protection generally not afforded in a public location such as a cloud computing environment.
It will be recognized that, as illustrated in FIG. 2, an additional distributor 224 may be provided. In such cases, the distributor registry 226 enables distribution address spaces to be properly managed. Each of the workloads 218A-218D communicates to with the distributor 214 via an address, such as, for example, a 10-net address 228, that is communicated to the workloads when they are started up so that any new processes that are initiated due to load balancing issues, load expansion issues, load contraction issues and the like can communicate their local address 216 to the distributor 214. It should be noted that what is communicated to the workloads may be the address of the distributor 214 or a token, such as a DNS name, that may be used by the workload to lookup the address of the distributor 214. In any event, it will be noted that the communicated address is one that is not hard-wired in the system; it is either provided as described or discovered via token dereferencing in a manner well known in the art. As a result, through implementation of some predetermined policy, the communicated local address and its associated port range 230 can be logged in to the distributor registry 226 so that the addresses can be correctly maintained according to address, port, and workload type.
The distributor 214 also provides configuration information to the workloads 218A-218D so that configuration information can be normalized and provided to new workloads that are being brought up as a result of load expansion or contraction. Additionally, the configuration described hereinabove is provided external to the cloud 206 via the same mechanism as was discussed with key distribution. This mechanism provides for a simpler method of keeping the workloads 218A-218D up to date and the startup or boot configuration normalized.
One embodiment is implemented using one or more embodiments described in U.S. patent application Ser. No. 12/197,833 (hereinafter the “'833 application), incorporated by reference hereinabove to find, analyze, and qualify clouds, such as the cloud 206, so that the required infrastructure for allowing the deployment of everything within the cloud 206 will function correctly. The embodiment described in the '833 application also keeps track of reputation of all clouds so that the best price, performance, stability, etc., can be selected for a deployment within the cloud.
In an alternative embodiment, the distributor 214 can reside external to the cloud 206 or in a different cloud. In this case, the virtual router 210 must attest to the validity of the workloads 218A-218D so that it is attested that no changes have been made. In the illustrated embodiment, each of the workloads 218A-218D has a secure connection through the virtual router 210 to the applications 202A-202C so that the annexation is still valid. It will be recognized, however, that this embodiment requires more traffic over the network.
In one embodiment, additional virtual routers, such as a virtual router 232, may be added. In this embodiment, an L4 switch or some other load-balancing mechanism (not shown) may be provided in a known manner to distribute traffic between the external addresses connected to the virtual routers 210, 232. In another embodiment, the information from the distributor registry 226 is available to the workloads 218A-218D such that the information provided by the distributor registry provides the workloads with information necessary to perform clustering, remedial clustering, and/or other types of cooperative computing and storage management.
In one embodiment, the secure bridge 208 may be replaced by a secure connection implemented using a pair of cryptographic keys held by both the trusted environment/enterprise 204 and the cloud 206.
It will be recognized that various ones of the elements and/or modules described herein may be implemented using one or more general purpose computers or portions thereof executing software applications designed to perform the functions described or using one or more special purpose computers or portions thereof configured to perform the functions described. The software applications may comprise computer-executable instructions stored on computer-readable media. Additionally, repositories described herein may be implemented using databases or other appropriate storage media.
While the preceding description shows and describes one or more embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure. For example, various steps of the described methods may be executed in a different order or executed sequentially, combined, further divided, replaced with alternate steps, or removed entirely. In addition, various functions illustrated in the methods or described elsewhere in the disclosure may be combined to provide additional and/or alternate functions. Therefore, the claims should be interpreted in a broad manner, consistent with the present disclosure.

## Claims

1. A system for proving cloud computing service:
a cloud structure;
a cloud disposed in the cloud structure, the cloud structure including hardware layer computer assets for supporting cloud workloads, each cloud workload being disposed in the cloud structure and having associated therewith a cloud address; and
a routing system including a virtual router and a distributor connected to the virtual router, the routing system disposed in the cloud structure and between external workloads and the cloud workloads, each external workload being disposed in an external computing environment and having associated therewith an external address, wherein the routing system uses a secure connection established through the virtual router to route traffic from one of the external addresses to one of the cloud addresses, wherein the virtual router includes a network address translator (“NAT”), and wherein the distributor maps a NAT address provided by the virtual router to one of the cloud addresses.

2. The system of claim 1 wherein the secure connection includes using a pair of cryptographic keys.

3. The system of claim 1 wherein the cloud structure is provided by a cloud services provider utilizing a configuration management database (CMDB).

4. The system of claim 3 wherein the cloud workloads is a germane cloud workload used by the cloud services provider to monitor the health of the cloud and provide management processes.

5. The system of claim 1 wherein the distributor is configured to perform load balancing subsequent to the distributor mapping the NAT address.

6. The system of claim 1 wherein the cloud addresses include an IP address either alone or in combination with at least one port designation, and
wherein the distributor maps the at least one port designation to the cloud workloads via the cloud addresses.

7. The system of claim 6 wherein each of the IP addresses includes a 10-net address.

8. The system of claim 6 wherein the distributor maintains the port designation mapping in a distributor registry.

9. A system for providing cloud computing services, the system comprising:
a cloud structure including hardware layer computer assets, wherein the computer assets are configured to support cloud workloads disposed in a cloud of the cloud structure, each cloud workload having associated therewith a cloud address; and
a routing system disposed in the cloud structure, wherein the routing system is configured to route traffic between external workloads and the cloud workloads, each external workload being disposed in an external computing environment and having associated therewith an external address, wherein the routing system includes a virtual router and a distributor connected to the virtual router, wherein the routing system uses a secure connection established through the virtual router to route traffic from the external addresses to the cloud addresses,
wherein the virtual router is configured to operate as a network address translator (“NAT”), and wherein the distributor maps a NAT address provided by the virtual router to one of the cloud addresses.

10. The system of claim 9 wherein the computer assets include storage assets, processing assets, and network assets.

11. The system of claim 9 wherein the distributor includes a load balancer configured to perform load balancing subsequent to the distributor mapping the NAT address.

12. The system of claim 9 wherein each of the cloud addresses includes an IP address either alone or in combination with at least one port designation, and
wherein the distributor maps the at least one port designation to the cloud workloads via the cloud addresses.

13. The system of claim 9 wherein the secure connection includes using a pair of cryptographic keys.

14. A method for providing cloud computing services comprising:
providing a cloud disposed in a cloud structure including hardware layer computer assets, the computer assets configured to support cloud workloads disposed in the cloud, wherein each cloud workload includes a cloud address; and
routing traffic between external workloads and the cloud workloads, the external workloads being disposed in an external computing environment, wherein each external workload includes an external address, wherein routing traffic includes using a secure connection established through a virtual router disposed in the cloud structure and connected to a distributor disposed in the cloud structure to thereby route traffic from the external addresses to the cloud addresses, wherein the virtual router is configured to operate as a network address translator (“NAT”), and wherein the distributor maps a NAT address provided by the virtual router to one of the cloud addresses.

15. The method of claim 14 wherein the secure connection includes using a pair of cryptographic keys.

16. The method of claim 15 wherein the cryptographic keys are held by both the cloud and the external computing environment.

17. The method of claim 14 further comprising:
performing load balancing, subsequent to the distributor mapping a NAT address.

18. The method of claim 14 wherein each of the cloud address includes an IP address either alone or in combination with at least one port designation.

19. The method of claim 18 further comprising:
mapping the at least one port designation to the cloud workloads via the cloud addresses.

20. The method of claim 18 wherein each of the IP addresses comprises a 10-net address.