Cloud computing nodes for aggregating cloud computing resources from multiple sources

A disclosed example method to aggregate resources in a cloud involves receiving a request for a resource at a first tenant application programming interface provider of an internal cloud service. The example method involves using a virtualization platform application programming interface client of the internal cloud service to access the resource when the resource is an internal resource. When the resource is an external resource, a tenant application programming interface client of the internal cloud service is used to send a second request for the resource to a second tenant application programming interface provider of an external cloud service.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to cloud computing and, more particularly, to cloud computing nodes.

BACKGROUND

Internal cloud architectures and external cloud architectures are used in cloud computing and cloud storage systems for offering infrastructure as a service (IaaS) cloud services. Examples of internal and external cloud architectures include the VMware vCloud® Director cloud architecture software, Amazon EC2® web service, and OpenStack® open source cloud computing service. IaaS cloud service is a type of cloud service that provides access to physical and/or virtual resources in a cloud environment. These services provide a tenant application programming interface (API) that supports operations for manipulating IaaS constructs such as virtual machines (VMs) and logical networks. In internal and external cloud architectures, tenants and physical infrastructure are kept separate from one another. Some tenant APIs include the notion of users and permissions on objects to control who has access to different tenant-facing resources (e.g., virtual data centers (VDCs), VMs, and networks) inside the IaaS.

DETAILED DESCRIPTION

Example cloud computing nodes (CCNs) disclosed herein facilitate implementing and managing different types of cloud architectures. Disclosed example CCNs are configured as building block interfaces that enable interconnecting different cloud computing systems to provide end users (e.g., tenants) with cloud services that use resources aggregated from the different cloud systems while appearing as part of a seamless or unified cloud to the end users. To provide such unified cloud services, an example CCN includes a tenant application programming interface (API) provider in communication with a virtual data center (VDC) of the CCN. The example CCN also includes a cloud resource pool of aggregated resources delegatable to the VDC for use by the tenant API provider. In addition, the example CCN includes a tenant API client in communication with the cloud resource pool. The tenant API client is configured to communicate with an external CCN to aggregate at least some of the resources from another cloud service associated with the external CCN. Disclosed example CCNs aggregate capacity from both the physical infrastructure of a corresponding cloud service, and from external CCNs associated with external cloud services. Disclosed example CCNs delegate those resources into VDCs, in which virtual machines (VMs) can be deployed. Disclosed example CCNs may be organized into flexible topologies that cover multiple different types of hybrid cloud architectures. Example cloud architectures in which example CCNs disclosed herein may be used include internal cloud architectures, external cloud architectures, hybrid cloud architectures, and cloud reseller architectures. Although specific types of cloud architectures are discussed herein, example CCNs disclosed herein may also be used to implement other types of cloud architectures not discussed herein.

In examples disclosed herein, an internal cloud is a cloud in which a tenant and a cloud service provider are part of the same organization. In examples disclosed herein, an external cloud is a cloud that is provided by an organization that is separate from a tenant that accesses the external cloud. For example, the tenant may be part of an enterprise, and the external cloud may be part of a cloud service provider that is separate from the enterprise of the tenant and that provides cloud services to different enterprises and/or individuals. In examples disclosed herein, a hybrid could is a cloud architecture in which a tenant is provided with seamless access to both internal cloud resources and external cloud resources. For example, a VDC corresponding to a particular tenant may access resources spanning across multiple cloud providers to provide the tenant with access to such resources from the different ones of the multiple cloud providers. Example CCNs disclosed herein facilitate establishing such hybrid clouds so that a hybrid cloud tenant need not know which cloud service provider(s) are providing its resources and so that the hybrid cloud tenant need not handle different interfaces (e.g., different APIs) to access resources from different service providers forming the hybrid cloud. Example CCNs disclosed herein provide a tenant with a single tenant API through which the tenant can access all resources provided by a hybrid cloud in a seamless manner without needing to know whether the resources are internal resource or external resources. In disclosed examples, the CCNs handle aggregation and management of resources from different cloud service providers that form the hybrid cloud without requiring the tenant to know of such aggregation and management of hybrid resources from the different providers. In some examples, tenants access the tenant APIs of CCNs through different client interfaces including dedicated cloud service client applications, web browsers, mobile apps, file system explorers/navigators, etc.

Examples disclosed herein are described in connection with infrastructure as a service (IaaS) cloud services. However, disclosed example CCNs may be adapted for use with other types of cloud services including, for example, software as a service (SaaS) and platform as a service (PaaS) cloud services. IaaS cloud services are cloud offerings that allow a tenant to deploy one or more VMs useable by the tenant to run applications or other software. The VMs run according to corresponding service level agreements (SLAs) made between corresponding tenants and corresponding IaaS providers. IaaS services enable tenants to dynamically increase or decrease the number of resources needed from IaaS providers on demand. In IaaS services, tenants are unaware of the physical infrastructure implementing the IaaS services. The entity that provides an IaaS service is an IaaS provider. An IaaS provider can be an internal information technology (IT) organization or an external public cloud provider (e.g., Amazon, Terremark, etc.). Disclosed example CCNs facilitate providing a tenant with an IaaS service formed of resources aggregated from two or more separate service providers without requiring the overhead of specialized, tenant-specific or service-specific specialty configurations of prior cloud services that join multiple service providers to provide a single IaaS service to a tenant.

In examples disclosed herein, a tenant (e.g., an IaaS tenant) is a user of an IaaS. The tenant is responsible for deploying workloads in the form of VMs or multi-VM configurations in the IaaS. A tenant of an IaaS is isolated from other tenants of the same IaaS. In some examples, a tenant is an organization (e.g., an enterprise) including multiple people or users. In other examples, a tenant may be a single user.

To provide tenants with access to cloud resources, disclosed example CCNs are provided with VDCs. In examples disclosed herein, a tenant accesses the IaaS through a VDC object. Disclosed example CCNs make cloud resources available via VMs that are deployed on VDCs. In examples disclosed herein, a VDC is uniquely accessible by a tenant in the same way that an account in an IaaS service uniquely corresponds to a particular tenant. In disclosed examples, VDCs are logically isolated from each other in that two tenants with access to two different VDCs of the same IaaS service provided by a CCN are unaware of each other, and unable to access and modify each other's VMs (e.g., cloud resources). In some examples, a VDC provides tenants with access to the same internet (e.g., a same inter-network network). In this manner, VMs deployed in one VDC can communicate to VMs in another VDC (e.g., based on modulo restrictions enforced by, for example, a firewall rule) using internet communications (e.g., internet protocol (IP)/hypertext transfer protocol (HTTP) communications). In examples disclosed herein, VMs are deployed on an underlying physical infrastructure (e.g., physical resources) shared by multiple tenants. Thus, although the tenants of a single CCN share physical resources, the logical resources (e.g., the VMs) are separate and not shared between the tenants.

FIG. 1depicts an example cloud service102having a CCN104, and a cloud service106having a CCN108. In the illustrated example, the CCN104provides tenants110and112with access to resources114of the cloud service102and/or to resources116of the cloud service106. In the illustrated example, the cloud service102may be an internal cloud provided by an enterprise, and the tenants110and112may be different business divisions of the enterprise that access logically separated resources aggregated by the CCN104from the resources114of the cloud service102and/or from the resources116of the cloud service106. The cloud service106of the illustrated example may be an external cloud provided by a cloud service provider (e.g., a business such as Amazon or Terremark that specializes in providing cloud-based solutions to different enterprises and/or individuals). In the illustrated example, the CCN104of the cloud service102communicates with the CCN108of the cloud service106so that the CCN104can aggregate one or more of the resources114and/or one or more of the resources116to provide the tenant110and/or the tenant112with seamless access to the aggregated resources114and/or116.

The CCN104of the illustrated example is useful to provide service offering flexibilities to different users. For example, the same CCN104is useable to provide some tenants with internal-only cloud services that provide access only to internal cloud resources, and to provide some tenants with hybrid cloud services that provide access to internal and external cloud resources. In some examples, the CCN104may provide only internal cloud access to the tenant110and hybrid cloud access to the tenant112. In such examples, the CCN104provides the tenant110with access to only the resources114of the cloud service102in accordance with an internal cloud access SLA of the tenant110. Also in such examples, the tenant112can access any combination of the resources114of the cloud service102and the resources116of the cloud service106in accordance with a hybrid cloud access SLA of the tenant112. Under such hybrid cloud access services, the tenant112need only interface with a tenant API (e.g., one of the tenant API providers206a-cofFIG. 2) of the CCN104, and the CCN104manages communications with the CCN108to aggregate and manage one or more of the resources116for use by the tenant112.

In the illustrated example, the cloud service106may provide IaaS cloud services to a tenant120. In the illustrated example, the CCN108of the cloud service106is able to provide hybrid cloud services to the tenant112in connection with the CCN104and to provide single-provider cloud services to the tenant120. That is, the same CCN108is re-useable, or simultaneously useable to provide hybrid cloud services and single-provider cloud services to different tenants or different CCNs according to different SLAs.

FIG. 2depicts an example CCN200, which may be used to implement the CCN104and/or the CCN108ofFIG. 1. The example CCN200is a universal building block for cloud infrastructures that can be used alone or in combination with other CCNs to implement internal, external, and/or hybrid cloud architectures. The CCN200of the illustrated example aggregates resources from multiple sources. Example sources can be one or more virtualization platforms (e.g., VMware ESX, VMware vCenter, Xen hypervisor, etc.) and/or one or more other CCNs. For example, the CCN104ofFIG. 1may use the resources114of the cloud service102or may access the resources116of the cloud service106via the CCN108. In the illustrated example, the CCN200aggregates resources for use by one or more tenants (e.g., one or more of the tenants110,112, and/or120ofFIG. 1).

The CCN200of the illustrated example includes a delegation layer202and an aggregation layer204. The delegation layer202of the illustrated example provides an interface for tenants (e.g., the tenants110,112, and/or120ofFIG. 1) to access virtualized resources based on resources aggregated by the aggregation layer204. The delegation layer202of the illustrated example also provides other CCNs with access to virtualized resources based on resources aggregated by the aggregation layer204. For example, a delegation layer (e.g., similar to the delegation layer202ofFIG. 2) of the CCN108ofFIG. 1provides the tenant120and the CCN104with access to virtualized resources (e.g., VMs) that run on one or more of the resources116ofFIG. 1.

To provide tenants (e.g., the tenants110,112, and/or120ofFIG. 1) and/or other CCNs with access to and/or management of cloud services and/or resources, the delegation layer202of the illustrated example is provided with tenant API providers206a-c. Although three tenant API providers206a-care shown at the delegation layer202, fewer or more tenant API providers206a-cmay be provided at the delegation layer202. In the illustrated example, the tenant API providers206a-care exposed as end-user API interfaces to enable tenants to interact with an IaaS cloud service (e.g., the cloud service102and/or the cloud service106ofFIG. 1). The tenant API providers206a-cof the illustrated example are also used by the CCN200to deploy VMs (e.g., virtualized resources) in response to tenant requests. The example tenant API providers206a-ccontrol lifecycles of VMs based on different VM management procedures such as “create” VMs, “configure” VMs, VM “power-ops” (e.g., power operations such as power on, power off, reset, suspend, shutdown, etc.), “destroy” VMs, etc.

In the illustrated example, to enable inter-CCN communications and/or inter-CCN cloud service management, the example CCN is provided with one or more tenant API clients208at the aggregation layer204. In the illustrated example, the tenant API clients208communicate with tenant API providers210(e.g., similar to the tenant API providers206a-c) of other CCNs to enable hybrid cloud services.

Referring briefly toFIG. 9, the tenant API provider206aand the tenant API client208are shown as part of a same example API902implemented in the example CCN200ofFIG. 2. The example API902is a protocol between the tenant API provider206aand the tenant API client208. The API902defines how a client (e.g., a tenant or user) can implement a method or methods on the CCN200, which implements the API902. In the illustrated example ofFIG. 9, the tenant API provider206aoperates as a server that implements the API902for clients (e.g., tenants or users) that access the API902via the tenant API provider206a. Also in the illustrated example, the tenant API client208of the CCN200operates as a tenant or user that invokes methods of other APIs implemented in other CCNs by accessing those other APIs via corresponding tenant API providers (e.g., tenant API providers210ofFIG. 2) of those other APIs in the other CCNs.

In the illustrated example, the API902includes an example API definition904that specifies routines, functions, protocols, data formats, etc. for the API902, the tenant API provider206a, and the tenant API client208. For example, the API definition904may define how information is to be exchanged between the tenant API provider206aand the tenant API client208and how information is to be handled (e.g., changed, provided, etc.) by the API902. In some examples, multiple separate APIs902are implemented in the CCN200for different ones of the tenant API clients206a-cand/or different ones of the tenant API providers208. Although the tenant API clients208are shown separate from the tenant API providers206a-cin the CCN200, in some examples, the same tenant API for a corresponding tenant is configured to implement tenant API provider procedures and tenant API client procedures. In such examples, the same tenant API has interfaces at both the delegation layer202to interact with tenants and at the aggregation layer204to interact with other CCNs.

Referring briefly toFIG. 10, an example cloud service1002(e.g., substantially similar or identical to the cloud service102ofFIG. 1) is provided with the CCN200having the example API902ofFIG. 9, and an example cloud service1004(e.g., substantially similar or identical to the cloud service106ofFIG. 1) is provided with an example CCN1006having an example API1008(e.g., similar or identical to the API902). In the illustrated example, the cloud service1002is an internal cloud service, and the cloud service1004is an external cloud service. The API1008of the illustrated example is provided with a tenant API provider1010(e.g., similar or identical to the tenant API provider206a) and with a tenant API client1012(e.g., similar or identical to the tenant API client208). In the illustrated example ofFIG. 10, the tenant API provider206aof the API902interfaces with one or more tenants1014. Also in the illustrated example, the tenant API client208of the API902is in communication with the tenant API provider1010of the example API1008of the example CCN1006of the cloud service1004. In this manner, the example CCN200can provide the one or more tenants1014with access to internal resources of the cloud service1002and/or external resources from the cloud service1004without informing the one or more tenants1014whether the resources are internal resources or external resources.

Referring again toFIG. 2, in some examples, the tenant API clients208are configured to migrate VMs between different IaaS providers (e.g., VMs between the internal cloud service102and the external cloud service106ofFIG. 1). In some such examples, the tenant API clients208may perform VM migrations as cold migrations such as when VMs are powered off, or as live migrations such as when VMs are powered on. By configuring the tenant API clients208with the ability to move VMs around during runtime, the CCN200is provided with more opportunities to improve placements of VMs at runtime. In this manner, if resources requested by a tenant dynamically change during runtime, the CCN200can substantially reduce or eliminate inefficiencies in VM placements (e.g., the CCN200can perform VM placement optimization) by using the tenant API clients208to power down, destroy, move, etc. VMs.

The tenant API clients208of the illustrated example are also configured to connect networks together across different IaaS providers (e.g., networks across the cloud services102and106ofFIG. 1). In this manner, VMs deployed across different IaaS providers can communicate with one another on private and secure virtual networks (e.g., inter-CCN virtual networks). In some examples, the tenant API providers206a-cand the tenant API clients208are configured to create secure, private virtual networks across VMs of the CCN200(e.g., intra-CCN virtual networks) by, for example, deploying virtual private network (VPN) gateways at different VMs separated by a public internet, and connecting the VMs over a secure, private virtual network via the VPN gateways.

By providing the tenant API providers206a-cat the delegation layer202and the tenant API clients208at the aggregation layer204, the example CCN200ofFIG. 2is able to implement cloud architectures of different configurations and different topologies. That is, examples disclosed herein enable implementing different types of cloud architectures for different uses or types of services by arranging CCNs in different desired topologies based on the tenant API providers206a-coperating as egress information/service/resource providers and the tenant API clients208operating as ingress information/service/resource receivers. Different example configurations and topologies such as internal cloud architectures, external cloud architectures, hybrid cloud architectures, and reseller cloud architectures are discussed in detail below in connection withFIGS. 3-6.

To aggregate resources for use by tenants (e.g., the tenants110,112, and120ofFIG. 1), the CCN200of the illustrated example is provided with an example cloud resource pool212in the aggregation layer204. The cloud resource pool212stores aggregated resources virtualized through a virtualization platform214(e.g., VMware ESX, VMware vCenter, Xen hypervisor, etc.). In the illustrated example, the virtualization platform214aggregates virtual resources that run on physical resources216. In the illustrated example, the physical resources216may be computers, servers, printers, scanners, storage devices, etc. In the illustrated example, the virtual resources of the cloud resource pool212may be reserved logical partitions or logical portions (e.g., time slices or amounts of processing, storage, printing, scanning, etc.) apportioned by the virtualization platform214on the physical resources216.

Referring briefly to the illustrated example ofFIG. 1, the resources114and116are physical resources such as the physical resources216ofFIG. 2. In addition, the CCNs104and108are provided with corresponding cloud resource pools (e.g., similar to the cloud resource pool212ofFIG. 2) that aggregate virtual resources that run on the physical resources114and116through a virtualization platform (e.g., the virtualization platform214ofFIG. 2). In this manner, the CCN104ofFIG. 1can access virtual resources that run on the physical resources114of the cloud service102, and the CCN104can also access, through the CCN108, virtual resources that run on the physical resources116of the cloud service106.

In the illustrated example ofFIG. 2, to access virtual resources from the virtualization platform214, the cloud resource pool212is in communication with a placement and routing engine218in the aggregation layer204. In the illustrated example, the placement and routing engine218includes the tenant API clients208. Also in the illustrated example, the placement and routing engine218includes a virtualization platform API client220to access virtual resources through the virtualization platform214. The placement and routing engine218of the illustrated example performs placement and routing processes to place and route virtual resources accessed through the virtualization platform214for use by tenants through different ones of the tenant API providers206a-c.

In the illustrated example ofFIG. 2, to provide tenants (e.g., the tenants110,112, and120ofFIG. 1) access to virtual resources aggregated in the cloud resource pool212, the CCN200is provided with virtual data centers (VDCs)222a-cin the delegation layer202. In the illustrated example, each of the VDCs222a-cis accessible independently by a corresponding tenant via a corresponding one of the tenant API providers206a-c. The tenant API providers206a-cof the illustrated example delegate the virtual resources in the cloud resource pool212to corresponding ones of the VDCs222a-cfor access by corresponding tenants via the VDCs222a-c. In the illustrated example, virtual resources are delegated in the VDCs222a-cas VMs instantiated and managed by, for example, the virtualization platform214.

In the illustrated example, aggregated virtual resources in the cloud resource pool212are used to provision VMs based on requests from tenants of the CCN200. As such, the CCN200of the illustrated example places (e.g., creates) the VMs in aggregated virtual resource capacity provided by the virtualization platform(s)214and/or provided by other CCNs in communication with the CCN200. Over time, dynamic placement of VMs (e.g., creating and destroying VMs) may create challenges in keeping well-optimized or efficient placements of VMs as VMs are constantly evaluated for the best possible (e.g., cheapest) placements in the aggregated virtual resources of the cloud resource pool212. To substantially reduce or eliminate such VM placement challenges, the placement and routing engine218of the illustrated example is configured to migrate VMs to different virtual resources as discussed above (e.g., as cold migrations or live migrations) by invoking commands through the tenant API clients208when the tenant API clients208find better placements for different VMs.

In the illustrated example, the tenant API client208can expand and/or restrict features of the physical resources216from which it aggregates virtual resources in the cloud resource pool212. Examples of such expandable or restrictable value-add features include advanced VM placement policies (e.g., a placement policy that allows distributing a set of VMs across two of the VDCs222a-c) and/or the ability to explicitly choose between high-available zones of resources for VM creation and not-so-high-available zones of resources for VM creation.

To authenticate and authorize users and permissions, the CCN200of the illustrated example is provided with an authentication, authorization, accounts (AAA) service224in the delegation layer202. In the illustrated example, permission checking is performed by the AAA service224and is isolated in the delegation layer202of the CCN200. In this manner, users and accounts are maintained only in the delegation layer202, and the user and account information is never propagated to underlying resources (e.g., resource in the cloud resource pool212or aggregated from other CCNs) in the aggregation layer204. In the illustrated example, for a tenant authenticated and authorized by the AAA service224, the aggregation layer204of the CCN200is configured to connect to that tenant via a corresponding tenant API (e.g., one of the tenant API providers206a-c). In this manner, exchanges with that tenant are isolated to the CCN200so that the tenant's exchanges with the CCN200remain separate from other CCNs. In this manner, examples disclosed herein enable decoupling or isolating tenants/users in the delegation layer202of the CCN200from tenants/users in remote CCNs. Using the AAA service224to manage access to virtual resources in this manner is useful, for example, to an enterprise that prefers not to upload all its employees' information and passwords to third-party cloud providers. In this manner, sensitive information is maintained at the enterprise (e.g., in the internal cloud) using security standards enforced by the enterprise. Using the AAA service224to manage access to virtual resources is also useful to simplify or substantially reduce the amount of information that needs to be propagated through a CCN topology. For example, the AAA service224can be configured to manage authentications and authorizations of tenants based on different polices to access resources, rather than needing to propagate user identifiers, passwords, and/or other credentials of tenants to external CCNs (e.g., the CCN106ofFIG. 1, or to the virtualization platform214or the physical resources216. Managing access to virtual resources using the AAA service224also enables arbitrary nesting of CCNs while being able to keep secure partitions and restricted access privileges between tenants and resources.

The example CCN200ofFIG. 2does not increase performance overhead or adversely affect performance of VMs. In the illustrated examples, disclosed example CCNs create VMs to run directly on a virtualization host such as the virtualization platform214, and the VMs offer services directly on aggregated network resources (e.g., aggregated network resources in the cloud resource pool212). As such, disclosed example CCNs can be used in highly nested configurations without significantly reducing VM performance.

The example CCN200ofFIG. 2may be implemented using different types of alternate path configurations to, for example, manage load balancing, resource availability, and connection failures. Example alternate path configurations that are useable with the example CCN200ofFIG. 2include active/passive configurations and active/active configurations across two sites (e.g., across the cloud services102and106ofFIG. 1). To facilitate the alternate path configurations, the example CCN200is provided with two network interface card (NIC) adapters228and230in the illustrated example. In active/passive configurations, the NIC adapter228is a primary NIC adapter and the NIC adapter230is a standby NIC adapter. In active/standby configurations, the standby NIC adapter230is inactive until it is needed to assume the role of the primary NIC adapter228. That is, the standby NIC adapter230can takeover in response to failure, congestion, or unavailability of the primary NIC adapter228. In active/active configurations, both of the NIC adapters228and230are active at the same time to concurrently manage load balancing. In active/active configurations, either of the NIC adapters228or230can assume full management responsibilities in response to the failure, congestion, or unavailability of the other NIC adapter228or230.

While an example manner of implementing the CCN200has been illustrated inFIG. 2, one or more of the elements, processes and/or devices illustrated inFIG. 2may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example tenant API providers206a-c, the example API clients208, the example cloud resource pool212, the placement and routing engine218, the virtualization platform API client220, the example VDCs222a-c, the example AAA service224and/or, more generally, the example CCN200ofFIG. 2may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example tenant API providers206a-c, the example tenant API clients208, the example cloud resource pool212, the placement and routing engine218, the virtualization platform API client220, the example VDCs222a-c, the example AAA service224and/or, more generally, the example CCN200could be implemented using one or more circuit(s), programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)), etc. When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example tenant API providers206a-c, the example tenant API clients208, the example cloud resource pool212, the placement and routing engine218, the virtualization platform API client220, the example VDCs222a-c, and/or the example AAA service224are hereby expressly defined to include a tangible computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. storing the software and/or firmware. Further still, the example CCN200ofFIG. 2may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated inFIG. 2, and/or may include more than one of any or all of the illustrated elements, processes and devices.

FIG. 3depicts an example internal cloud architecture300that may be implemented using the CCN200ofFIG. 2. In the illustrated example, the internal cloud300is implemented by an IT organization of an enterprise302to provide access to IaaS resources to a tenant304(e.g., users or employees of the enterprise). In the illustrated example ofFIG. 3, a single CCN200is used to implement the internal cloud architecture300to provide the tenant304access to virtualized internal resources provided by the virtualization platform214. Although a single CCN200is shown in the example internal cloud architecture300ofFIG. 3, the internal cloud architecture300may be implemented using any number of nested CCNs internal to the enterprise302to provide the tenant304access to resources of the enterprise302.

In the illustrated example ofFIG. 3, a tenant API client208(FIG. 2) is used to aggregate virtual resources into the cloud resource pool212(FIG. 2) from the virtualization platform214run by the enterprise302, and a corresponding one of the tenant API providers206a-c(FIG. 2) is used to delegate the virtual resources (e.g., VMs) to the corresponding tenant304. In the illustrated example, user authentications of the tenant304and permissions are enforced by the AAA service224(FIG. 2) of the CCN200at the enterprise302based on user and account information maintained at the enterprise302and based on policies of the enterprise302.

FIG. 4depicts an example external cloud architecture400that may be implemented using the CCN200ofFIG. 2. In the illustrated example ofFIG. 4, the tenant304of the enterprise302is provided with third-party IaaS cloud service from a third-party cloud provider402that implements the external cloud architecture400. In the illustrated example ofFIG. 4, a single CCN200is used to implement the external cloud architecture400to provide the tenant304with access to virtualized external resources provided by the virtualization platform214. Although a single CCN200is shown in the external cloud architecture400ofFIG. 4, the external cloud architecture400may be implemented using any number of nested CCNs in the third-party cloud provider402external to the enterprise302to provide the tenant304access to external resources of the third-party cloud provider402.

In the illustrated example ofFIG. 4, a tenant API client208(FIG. 2) is used to aggregate virtual resources into the cloud resource pool212(FIG. 2) from the virtualization platform214run by the third-party cloud provider402, and a corresponding one of the tenant API providers206a-c(FIG. 2) is used to delegate the virtual resources (e.g., VMs) to the corresponding tenant304across separate logical networks of the enterprise302and the third-party cloud provider402. In the illustrated example, user authentications of the tenant304and user permissions are enforced by the AAA service224(FIG. 2) of the CCN200at the third-party cloud provider402based on user and account information from the enterprise302and/or the third-party cloud provider402and based on policies of the enterprise302and/or the third-party cloud provider402.

FIG. 5depicts an example hybrid cloud architecture500that may be implemented using the CCN200ofFIG. 2. In the illustrated example ofFIG. 5, the tenant304of the enterprise302is provided with a hybrid IaaS cloud service from the hybrid cloud architecture500implemented by the example enterprise302ofFIG. 3and by the example third-party cloud provider402ofFIG. 4. In the illustrated example, the hybrid cloud architecture500includes an internal cloud service502implemented by the enterprise302and an external cloud service504implemented by the third-party cloud provider402. The internal cloud service502includes the CCN200and the virtualization platform214, and the external cloud service504includes a CCN506(e.g., substantially similar or identical to the CCN200) and a virtualization platform508(e.g., substantially similar or identical to the virtualization platform508). In some examples, the virtualization platform214may be omitted from the internal cloud service502, and the enterprise302may instead use only the virtualization platform508of the third-party cloud provider402via the CCN200and the CCN506.

In the illustrated example ofFIG. 5, the two CCNs200and506are in communication with one another to implement the hybrid cloud architecture500to provide the tenant304with seamless access to virtualized internal resources provided by the virtualization platform214of the enterprise302and seamless access to virtualized external resources provided by the virtualization platform508of the third-party cloud provider402. In the illustrated example ofFIG. 5, a tenant API client208(FIG. 2) of the CCN200is used to aggregate virtual internal resources into the cloud resource pool212from the virtualization platform214run by the enterprise302, and to aggregate virtual external resources from the virtualization platform508run by the third-party cloud provider402. For example, to aggregate resources from the third-party cloud provider402, the tenant API client208of the CCN200communicates with a tenant API provider (e.g., substantially similar or identical to the tenant API providers206a-cofFIG. 2) of the CCN506. In such examples, the tenant API provider of the CCN506acknowledges or treats the tenant API client208of the CCN200as a tenant that requests resources from the CCN506.

Inter-cloud interactions between the CCN200and the CCN506to aggregate resources requested by the tenant304are transparent to the tenant304. That is, the CCN200and the CCN506perform such inter-cloud interactions without the tenant API providers206a-cand/or the tenant API clients208exposing to the tenant304whether resources are provisioned through the CCN200as internal resources or through the CCN506as external resources. Thus, the tenant need not be aware of, and need not manage, any of the inter-cloud interactions between the CCNs200and506. In this manner, the tenant304is unaware of whether resources delegated to it are provisioned at the internal cloud service502or at the external cloud service504.

In the illustrated example, the tenant304can log into the CCN200and seamlessly deploy VMs across multiple cloud providers regardless of whether the cloud providers are internal cloud providers or external cloud providers. For example, one of the tenant API providers206a-ccorresponding to the tenant304is used to delegate to the corresponding tenant304aggregated virtual resources (e.g., VMs) aggregated across the enterprise302and the third-party cloud provider402. In the illustrated example, user authentications of the tenant304and user permissions are enforced by the AAA service224(FIG. 2) of the CCN200at the internal cloud service502based on user and account information of the enterprise302. In the illustrated example ofFIG. 5, by using the AAA service224of the CCN200, the enterprise302need not expose its sensitive user and account information to the third-party cloud provider402. Instead, the AAA service224provides authentication, authorization, and accounts services for user and account information of the enterprise302in the CCN200without exposing such user and account information to the third-party cloud provider402.

In some examples, the CCN506of the external cloud service504implements an AAA service (e.g., substantially similar or identical to the AAA service224) to perform user authentications and/or enforce user permissions on communications from the CCN200. In such examples, the third-party cloud provider402treats the CCN200as a user or tenant of its services and enforces its policies on the CCN200using its AAA services.

FIG. 6depicts an example cloud reseller architecture600that may be implemented using the CCN200ofFIG. 2. In the illustrated example ofFIG. 6, the tenant304of the enterprise302is provided with an IaaS cloud service from cloud reseller602that resells use of external resources provided by one or more third-party cloud providers604. In the illustrated example, the reseller cloud architecture600includes the CCN200in communication with one or more third-party cloud services606a-bof the third-party cloud providers604. Although two third-party cloud services606a-bare shown, the third-party cloud providers604may include any number of third-party cloud services. In the illustrated example, the third-party cloud services606a-binclude corresponding CCNs608a-b(e.g., substantially similar or identical to the CCN200) to perform inter-cloud communications with the CCN200of the cloud reseller602. In this manner, the CCN200of the cloud reseller602can aggregate external resources from the third-party cloud services606a-bto resell use of those resources to the enterprise302. In the illustrated example, the cloud reseller602does not maintain or provide its own resources (e.g., the physical resources216ofFIG. 2). As such, the cloud reseller602of the illustrated example does not include a virtualization platform (e.g., the virtualization platform214ofFIG. 2). In the illustrated example, the third-party cloud services606a-bare provided with corresponding virtualization platforms610a-b(e.g., substantially similar or identical to the virtualization platform508). The cloud reseller602of the illustrated example relies on the virtualization platforms610a-bof the third-party cloud services606a-bto aggregate resources requested by the tenant304.

In the illustrated example ofFIG. 6, the CCN200is in communication with one or more of the CCNs608a-bto provide the tenant304with seamless access to virtualized resources provided by one or more of the virtualization platforms610a-b. In the illustrated example ofFIG. 6, a tenant API client208(FIG. 2) of the CCN200is used to aggregate virtual resources into the cloud resource pool212(FIG. 2) from one or more of the virtualization platforms606a-brun by the third-party cloud provider(s)604. For example, to aggregate resources from the third-party cloud provider(s)604, the tenant API client208of the CCN200communicates with one or more tenant API provider(s) (e.g., substantially similar or identical to the tenant API providers206a-cofFIG. 2) of corresponding ones of the CCNs608a-b. In such examples, the tenant API provider(s) of the CCN(s)608a-backnowledge or treat the tenant API client208of the CCN200as a tenant that requests resources from the CCN(s)608a-b. Inter-cloud interactions between the CCN200and the CCN(s)608a-bto aggregate resources requested by the tenant304are transparent to the tenant304. In the illustrated example, the CCN200and the CCN(s)608a-bperform such inter-cloud interactions without the tenant304being aware or needing to manage any of the inter-cloud interactions. In this manner, the tenant304is unaware of whether resources delegated to it are from the cloud reseller602or one or more of the third-party cloud providers604. In some examples, the tenant304is not even aware of the identities of the third-party cloud providers604and/or that the cloud reseller602is using resources of the third-party cloud providers604. Under a reseller business model, the cloud reseller602may change backend cloud resource providers (e.g., the third-party cloud provider(s)604ofFIG. 6) based on changes in price or quality. In some examples, the reseller602may also offer additional services (e.g., web portals or graphical user interfaces for accessing cloud resources, failover processes, authentication processes, etc.) to tenants.

In the illustrated example, the cloud reseller602uses one of the tenant API providers206a-c(FIG. 2) to delegate to the tenant304virtual resources (e.g., VMs) aggregated from one or more of the third-party cloud services606a-b. In the illustrated example, user authentications of the tenant304and permissions are enforced by the AAA service224(FIG. 2) of the CCN200at the cloud reseller602based on user and account information of the enterprise302and/or of the cloud reseller602. In some examples, the CCNs608a-bof the third-party cloud services606a-bimplement corresponding AAA services (e.g., substantially similar or identical to the AAA service224) to perform user authentications and/or enforce permissions on communications from the CCN200. In such examples, the third-party cloud providers604treat the CCN200as a user or tenant of their services and enforce their policies on the CCN200using their AAA services.

As shown inFIGS. 3-6, example CCNs disclosed herein can be connected to virtualization platforms (e.g., the virtualization platform214ofFIGS. 2-5, the virtualization platform508ofFIG. 5, and/or the virtualization platforms610a-bofFIG. 6) and/or connected to tenant API providers (e.g., the tenant API providers210ofFIG. 2) of other CCNs (e.g., the CCNs108ofFIG. 1, the CCN506ofFIG. 5, and/or the CCNs608a-bofFIG. 6). As such, disclosed example CCNs enable multi-level nesting of cloud services to aggregate resources from a plurality of cloud services for seamless access by tenants. For example, as discussed above, some entities may run their own virtualization platforms to aggregate their own resources. Some entities may communicate with external third-party cloud providers to aggregate external third-party resources with their own resources aggregated using their own virtualization platforms. Yet other entities may build business models on reselling access to resources aggregated from other third-party cloud providers without expending any capital on acquiring and maintaining their own resources. In many architectures, disclosed example CCNs enable seamless access to resources aggregated across multiple cloud services while providing the ability to dynamically increase or decrease needed resources in real time and while protecting sensitive user and account information of tenants.

FIG. 7is a flow diagram representative of example machine readable instructions that may be used to implement the CCN200ofFIGS. 2-6to implement one or more of the cloud architectures ofFIGS. 3-6. In this example, the machine readable instructions comprise a program for execution by a processor such as the processor812shown in the example processor platform800discussed below in connection withFIG. 8. The program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-ray disc, or a memory associated with the processor812, but the entire program and/or parts thereof could alternatively be executed by a device other than the processor812and/or embodied in firmware or dedicated hardware. Further, although the example program is described with reference to the flowchart illustrated inFIG. 7, many other methods of implementing the example CCN200may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.

Turning to the example process ofFIG. 7, initially a tenant API provider receives a request for a resource (block702). For example, the tenant API provider206a(FIG. 2) of the CCN200may receive the request from the tenant304(FIGS. 3-6). The AAA service224(FIG. 2) determines whether the user of the tenant304making the request is authenticated (block704). For example, the AAA service224performs an authentication procedure to determine whether the requesting user is known (e.g., whether the user is registered with the CCN200). In some examples, the AAA service224performs the authentication based on an account of a VDC (e.g., the VDC222a) through which the user is requesting the resource. If the AAA service224determines at block704that the requesting user is not authenticated, the AAA service224denies the user access (block706) to the requested resource, and the example process ofFIG. 7ends.

If the requesting user is authenticated at block704, the AAA service224determines whether the user is authorized (block708). For example, the AAA service224performs an authorization procedure to determine whether the requesting user has permissions authorizing the user to access the requested resource. If the AAA service224determines at block708that the requesting user is not authorized to access the requested resource, the AAA service224denies the user access (block706) to the requested resource, and the example process ofFIG. 7ends.

If the requesting user is authorized at block708, the AAA service224grants the user access (block710) to the requested resource. In the illustrated example, the CCN200grants the requesting user access to the resource without exposing to the user whether the resource is an internal resource or an external resource. In this manner, the requesting user can access the resource via the CCN200in a seamless manner regardless of whether the resource is an internal resource aggregated internally by the CCN200or an external resource aggregated externally via another CCN. The tenant API provider206adetermines the placement/routing of the requested resource (block712). For example, the tenant API provider206adetermines the location of a VM corresponding to the requested resource. In some examples, the location of the requested resource is in the same cloud service as the CCN200that received the tenant request such as in, for example, the internal cloud architecture300ofFIG. 3, the external cloud architecture400ofFIG. 4, and/or the internal cloud service502ofFIG. 5. In other examples, the requested resource is an external resource located in an external cloud service separate from the CCN200that received the tenant request such as in, for example, the external cloud service504ofFIG. 5and the third-party cloud services606a-bofFIG. 6.

If the requested resource is an external resource (block714), a tenant API client208(FIG. 2) calls into an external CCN (block716) to request the resource. For example, the tenant API client208of the CCN200may request the resource from the CCN506in the hybrid cloud architecture500ofFIG. 5and/or may request the resource from one of the CCNs608a-bin the cloud reseller architecture600ofFIG. 6. In the illustrated example, the resource accessed through another CCN external to the CCN200via the tenant API client208is an external resource.

If the requested resource is not an external resource (block714) (e.g., the requested resource is an internal resource), the tenant API provider206arequests use of the resource through the virtualization platform API client220(FIG. 2) (block718). For example, the tenant API provider206acalls into the VDC222a(FIG. 2) to request use of the resource through the virtualization platform API client220. In the illustrated example, the resource accessed through the virtualization platform API client220is an internal resource available directly through the CCN200without needing to be accessed through another CCN (e.g., the CCN108ofFIG. 1, the CCN506ofFIG. 5, or the CCNs608a-bofFIG. 6). After requesting the resource at block716or block718, the CCN200assigns the resource (block720) for use by the requesting user. The example process ofFIG. 7then ends.

FIG. 8is a block diagram of an example processor platform800capable of executing the instructions ofFIG. 7to implement the CCN200ofFIGS. 2-6to implement one or more of the cloud architectures ofFIGS. 3-6. The processor platform800can be, for example, a server, a personal computer, and/or any other suitable type of computing device.

The processor platform800of the instant example includes a processor812. The processor1012of the illustrated example is hardware. For example, the processor812can be implemented by one or more integrated circuits, logic circuits, microprocessors or controllers from any desired family or manufacturer.

The processor platform800of the illustrated example also includes an interface circuit820. The interface circuit820may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), and/or a PCI express interface.

In the illustrated example, one or more input devices822are connected to the interface circuit820. The input device(s)822permit(s) a user to enter data and commands into the processor812. The input device(s) can be implemented by, for example, a keyboard, a mouse, a touchscreen, a track-pad, a trackball, isopoint and/or a voice recognition system.

One or more output devices824are also connected to the interface circuit820of the illustrated example. The output devices824can be implemented, for example, by display devices (e.g., a liquid crystal display, a cathode ray tube display (CRT), a printer and/or speakers). The interface circuit820of the illustrated example, thus, typically includes a graphics driver card, graphics driver chip or graphics driver processor.

The processor platform800also includes one or more mass storage devices828for storing software and data. Examples of such mass storage devices828include floppy disk drives, hard drive disks, compact disk drives, compact disks, Blu-ray disk drives, RAID systems, and digital versatile disk (DVD) drives.

Coded instructions832(e.g., to implement the example process ofFIG. 7) may be stored in the mass storage device828, in the volatile memory814, in the non-volatile memory816, and/or on a removable tangible computer readable storage medium such as a CD or DVD.