Virtual domains within a shared device

In general, this disclosure describes techniques for using virtual domains. In one example, a method comprises receiving, by a computing device, configuration data defining: an external virtual domain for a network function, the external virtual domain connected to a public network and managed by a provider for the computing device; a virtual domain for the network function, the virtual domain separate from the external virtual domain, configured with a secure tunnel interface, connected to a customer network, and managed by a customer of the provider for the computing device; forwarding, by the external virtual domain implementing a route-based virtual private network, encrypted network traffic, received from the public network via a secure tunnel, to the secure tunnel interface configured in the virtual domain; decrypting, by the virtual domain, the encrypted network traffic to generate network traffic; and forwarding, by the virtual domain, the network traffic to the customer network.

TECHNICAL FIELD

The disclosure relates to data centers and, more specifically, to using virtual domains within a shared device that applies a network function.

BACKGROUND

A data center provider may employ a facility, such as a data center or warehouse, in which multiple customers of the provider offer connections and/or locate network, server, and storage gear and interconnect to a variety of telecommunications, cloud, and other network service provider(s) with a minimum of cost and complexity. Such data centers may be shared by the multiple customers. By interconnecting at such facilities, the customers of the provider including telecommunications providers, Internet Service Providers (ISPs), application service providers, service providers, content providers, and other providers, as well as enterprises, enjoy less latency and the freedom to focus on their core business.

A network operator, such as the data center provider, may provide network functions that can be applied to packets traversing a switching fabric of the data center that is managed by the data center provider. The network functions may be implemented using specialized hardware appliances, such as firewalls or routers and sometimes referred to as Physical Network Functions (PNFs), or implemented using a Network Functions Virtualization (NFV) architecture or infrastructure (NFVI) that may include Virtualized Network Functions (VNFs). A network function (PNF or VNF) may provide firewall, routing, carrier grade network address translation (CG-NAT), performance enhancement proxies for video, transport control protocol (TCP) optimization and header enrichment, caching, load balancing, or other network functions, for example. A VNF may be executed by one or more virtual machines, containers, or other execution environment of the NFV Infrastructure. In this way, virtualized network functions may be executed by servers, switches, storage devices, and cloud computing infrastructure.

SUMMARY

In general, this disclosure describes techniques for using virtual domains within a device shared among multiple customers of a data center provider to provide secure virtual private networking, inter-customer network forwarding, and application of network functions. A device that offers network functions, such as PNFs and VNFs, may provide for domain isolation of customer networks coupled to the device. Customer networks may be associated with customers of the data center provider, which may include enterprises, network service providers, cloud service providers, and other customers. Customer equipment associated with customer networks may be co-located within a data center of the data center provider and connected to the data center switching fabric or may be connected to the data center via a network service provider or other connection to the data center switching fabric.

In some examples, the data center provider may deploy the device and configure the device with multiple virtual domains for respective customers and an external virtual domain that operates as a gateway for the virtual domains to reach external networks, such as networks of internet service providers or cloud service providers that are not linked to one of the virtual domains of the device. A co-located customer network in the data center may reach branch offices for the customer via the external virtual domain. In addition, the data center provider may configure the device with preconfigured connections between pairs of virtual domains to enable routing of packets among virtual domains between customer networks linked to the virtual domains. Such preconfigured connections may include a connection between a virtual domain for a customer and the external virtual domain to enable the customer network for the customer to reach external networks.

In some examples, the device integrates secure virtual private networking (VPN) functionality by performing route-based VPN techniques and terminating secure VPN tunnels within the virtual domains. Contrary to deployments in which a secure VPN gateway terminates secure VPN tunnels (e.g., decrypts encrypted traffic among other operations) and forwards the decrypted traffic to the destination network, the external virtual domain configured in the device applies a route-based VPN to the encrypted traffic received at the external virtual domain to direct the encrypted traffic to a virtual domain of the device that is linked to the destination network for the encrypted traffic. This virtual domain is configured with a secure tunnel interface and decrypts the traffic before forwarding the traffic to the destination network. In effect, the device enables secure VPN across multiple virtual domains having secure VPN tunnel endpoints.

The aspects described above, and further aspects described herein may provide one or more technical advantages that present at least one practical application. For example, isolating the virtual domains for customers and terminating secure VPN tunnels within the customer-specific virtual domains may improve security of customer traffic because unencrypted traffic for multiple customers does not traverse common links. Moreover, this technical advantage accrues with only a single device shared by the multiple customers. As another example, the techniques may facilitate multi-tenancy on a single device while promoting secure traffic handling, as described above. As a result, the data center provider may deploy a single device (or lease a single device to a reseller) that can be securely shared among multiple different customers of the data center provider, thereby increasing utilization of the device and at least in some cases leading to efficiencies.

In one example, a computing device comprises processing circuitry coupled to a memory, the processing circuitry and memory configured to implement: a network function; an external virtual domain for the network function, the external virtual domain connected to a public network and managed by a provider for the computing device; and a virtual domain for the network function, the virtual domain separate from the external virtual domain, configured with a secure tunnel interface, connected to a customer network, and managed by a customer of the provider for the computing device, wherein the external virtual domain implements a route-based virtual private network to forward encrypted network traffic, received from the public network via a secure tunnel, to the secure tunnel interface configured in the virtual domain, and wherein the virtual domain is configured to decrypt the encrypted network traffic to generate network traffic and forward the network traffic to the customer network.

In one example, a computing device comprises processing circuitry coupled to a memory, the processing circuitry and memory configured to implement: a network function; a first virtual domain for the network function, the first virtual domain connected to a first customer network, and managed by a first customer of a provider for the computing device; a second virtual domain for the network function, the second virtual domain separate from the first virtual domain, connected to a second customer network, and managed by a second customer of a provider for the computing device; and a virtual domain connection enabling network traffic forwarding from the first virtual domain to the second virtual domain, the virtual domain connection configured by the provider for the computing device.

In one example, a method comprises receiving, by a computing device, configuration data defining: an external virtual domain for a network function, the external virtual domain connected to a public network and managed by a provider for the computing device; a virtual domain for the network function, the virtual domain separate from the external virtual domain, configured with a secure tunnel interface, connected to a customer network, and managed by a customer of the provider for the computing device; forwarding, by the external virtual domain implementing a route-based virtual private network, encrypted network traffic, received from the public network via a secure tunnel, to the secure tunnel interface configured in the virtual domain; decrypting, by the virtual domain, the encrypted network traffic to generate network traffic; and forwarding, by the virtual domain, the network traffic to the customer network.

Like reference characters denote like elements throughout the figures and text.

DETAILED DESCRIPTION

FIG.1is a block diagram that illustrates a system100including an example device104configured and operating according to techniques described herein. In the example ofFIG.1, system100includes a cloud exchange102that includes a data center switching fabric connected to device104. The data center switching fabric, device104, and other devices of cloud exchange102(not shown) may be configured by programmable network platform103.

In some instances, cloud exchange102may represent an Equinix Cloud Exchange Fabric provided by Equinix, Inc. of Redwood City, Calif. Further example details of a cloud-based services exchange can be found in U.S. patent application Ser. No. 15/099,407, filed Apr. 14, 2016 and entitled “CLOUD-BASED SERVICES EXCHANGE;” U.S. patent application Ser. No. 14/927,451, filed Oct. 29, 2015 and entitled “INTERCONNECTION PLATFORM FOR REAL-TIME CONFIGURATION AND MANAGEMENT OF A CLOUD-BASED SERVICES EXCHANGE;” and U.S. patent application Ser. No. 14/927,306, filed Oct. 29, 2015 and entitled “ORCHESTRATION ENGINE FOR REAL-TIME CONFIGURATION AND MANAGEMENT OF INTERCONNECTIONS WITHIN A CLOUD-BASED SERVICES EXCHANGE;” each of which are incorporated herein by reference in their respective entireties.

System100also includes a public network120connected to a customer branch office118, a customer network130, and a cloud service provider network128that provides one or more cloud services114A-114N (collectively, “cloud services114”). Public network120may be a network that is publicly available with few or no restrictions. For example, public network120may be a network that is part of or otherwise connected to the Internet, a network service provider network, an autonomous system, an Internet service provider network, or a combination or one or more of such networks.

Customer network130is a network deployed and/or managed by a customer of a provider that deploys and/or manages cloud exchange102. Cloud exchange102may be implemented at least in part using a data center switching fabric located within one or more data centers (not shown) in which a customer also co-locates networking, computing, and other equipment for customer network130. This data center switching fabric may be alternatively referred to as a data center fabric, exchange fabric, cloud exchange fabric, switching network, data center network, or other network that operates as data center infrastructure and can be deployed and configured to interconnect multiple customers networks of the data center provider having access to ports within the data center switching fabric. The customer for customer network130may be an enterprise, a reseller, a managed service provider, a network service provider, an internet service provider, a cloud service provider, or other entity. Customer branch office118may be a remote branch office of the customer for customer network130, connected to public network120.

Cloud service provider network128is a network deployed and/or managed by a cloud service provider that is a customer of the provider that deploys and/or manages cloud exchange102. Cloud service provider network128may represent a public, private cloud, hybrid cloud, a virtual private cloud within a public cloud, or other network that offers cloud services. Cloud services114source and sink cloud service traffic exchanged with other networks, such as customer network130. Cloud provider office123may be a remote branch office of the cloud provider entity for cloud service provider network128, connected to public network120.

Device104may be a network appliance, router, firewall, switch, or other physical network device that provides a network function105. Device104may be a Physical Network Function (PNF) as defined by the European Telecommunications Standards Institute (ETSI). Device104may provide forwarding and network address translation services for network traffic to and from the VNF204.

In accordance with techniques described in this disclosure, device104stores configuration data109that defines a configuration of device104. Programmable network platform103or an operator may invoke an interface of device104to add, delete, or modify configuration data109. Configuration data109defines external virtual domain106and multiple virtual domains110A-110K (collectively, “virtual domains110”). Each of external virtual domain106and virtual domains110is a virtual instance of device104that is connected to at least one network via a different port of device104(not shown) and that is isolated from the other virtual domains unless configuration data109defines a virtual connection between the virtual domain and one or more other virtual domains.

A provider may deploy a device104that can be shared among multiple different customers of the provider, e.g., a data center provider. The provider configures device104with configuration data109to define virtual domains110for respective customers of the provider or other entity, such as a reseller. As such, each customer has a separate, isolated virtual domain that forwards traffic associated with a network for the customer and received at the virtual domain. For example, virtual domain110A is associated with a first port of device104connected to customer network130and forwards traffic associated customer network130, virtual domain110K is associated with a second port of device104connected to cloud service provider network128and forwards traffic associated with a cloud service provider for cloud service provider network128.

Device104may apply network function105to any network traffic traversing a domain, according to policies configured by the customer associated with the domain. For example, a customer for customer network130may configure virtual domain110A with particular routing policies, firewall rules, NAT rules, or other network function rules, policies, or configuration, in order to realize the customer's network function preferences with respect to network traffic traversing virtual domain110A of device104. Configuration data109may limit customers to only configuring their respective domains110and prevent customers from configuring or modifying, e.g., the existence of virtual domains106,110; virtual domain connections between any of106,110; or connections of virtual domains106,110to networks. The device104may limit configuration of virtual domains to authorized parties, e.g., resellers or customers, using Terminal Access Controller Access-Control System (TACACS), TACACS+, or Remote Authentication Dial-In User Service (RADIUS), or Diameter, for virtual domain authorization, for instance. In some examples, only the provider of the network function105for device104may be authorized to configure external virtual domain106and virtual domain connections121. Additional details for limiting authorization are found in U.S. patent application Ser. No. 16/836,77, filed Mar. 31, 2020, entitled “Virtual Network Function Virtual Domain Isolation,” which is incorporated by reference herein in its entirety.

In some examples, virtual domains may include a virtual routing and forwarding (VRF) instance having an interface to the corresponding network. For example, external virtual domain106has an interface having a public IP address111on the public network120. Virtual domain110A has an interface with IP address112A on customer network130. Virtual domain110K has an interface with an IP address on cloud service provider network128. In some examples, the VRFs of the virtual domains106,110are configured with routing information to facilitate inter-VRF forwarding (e.g., route leaking) of traffic based on properties of the traffic. The provider for device104may preconfigure, by setting configuration data109, these virtual domain connections to “stitch” two virtual domains110together by configuring import and export route targets in the corresponding VRFs to enable route leaking between the virtual domains. Optional virtual domain connections121A-121C (collectively, “virtual domain connections121”) between virtual domains110are illustrated inFIG.1. In some examples, the customers are prevented by a configuration of device104to configure or otherwise modify virtual domain connections121A-121C. The virtual domain connections121are configured by the provider for device104, in some cases based on subscriptions by a customer to cloud services and/or data center services or requests for connectivity to other customers. In some cases, the virtual domain connections121are preconfigured before the customers have access to modify the respective configurations for their respective virtual domains. For customer network130to access the Internet and/or customer branch office118, and vice-versa, the virtual domain110may in some cases have virtual domain connection121with external virtual domain106, which operates as a VPN gateway for device104. IP addresses112A-112K for respective virtual domains110A-110K may be private or public.

In some examples, virtual domains may represent firewall domains. For example, external virtual domain106may be a firewall domains that applies network function105(a firewall, in such examples) to network traffic received from public network120or from any of virtual domains110. The firewall domains may implement firewall rules to determine whether and how network traffic received passes through external virtual domain106or other virtual domains110. The other virtual domains110may also be configured to implement firewall preferences for the respective customers for the virtual domains110.

Device104may avoid labeled VPN traffic because the network traffic is inside the device104traversing virtual domains106,110. For example, virtual domain110A may be a virtual firewall domain or a VRF instance. The reseller or customer/tenant have the ability to configure virtual domain110A with configuration (e.g. command-line interface) restrictions to virtual domain110A. Device104has configuration data indicating ports for the customer belong to virtual domain110A and, upon receiving traffic from the customer network coupled to such ports, device104executing virtual domain110A applies the routing table or firewall rules for the domain and then forwards as normal.

Each of IP addresses112for respective virtual domains110may be a public IP for a site-to-site secure tunnel119, or virtual domains can share a public IP111using virtual domain connections121. Secure tunnel endpoints resolve to particular virtual domains110. If virtual domains share a public IP111, VPN107demultiplexes based on the destination public IP, source IP for the external site, or a combination thereof.

Virtual domains110may advertise routes using BGP over secure tunnels119. Customer branch office118or other site may advertise its gateway router. Cloud service provider network128may advertise its gateway router. External virtual domain106may perform forwarding for routes received from external gateway routers. Received routes may be stored in routes125for route-based virtual private networking. Received routes may also be stored in one or more virtual domains110for forwarding based on such routes (or firewall rules). These routes may be used for inter-virtual domain forwarding via virtual domain connections121.

Accordingly, virtual domain110K may forward cloud service traffic from cloud service114A received via cloud service provider network128to external virtual domain106via virtual domain connection121C and vice-versa. In some examples, cloud service traffic received at virtual domain110K may be processed using the network function105prior to forwarding to external virtual domain106for forwarding to a destination. External virtual domain106may also apply network function105, according to policies for the network function105for external virtual domain106. A similar process may be applied in the reverse direction from a cloud server114to cloud client118.

Device104implements a route-based virtual private network (VPN)107using routes125to determine whether to route network traffic into one of secure tunnels119A-119K (collectively, “secure tunnels119”) and to determine into which virtual domain110to route network traffic received from any of secure tunnels119. The provider of device104sets configuration data109to create secure endpoints115A-115K (collectively, “secure endpoints115”) that are secure virtual tunnel interfaces for device104. Secure tunnels119may represent IPSec tunnels or other encrypted tunnels that may be used to implement VPN107.

Secure endpoints115send and receive encrypted, tunneled network traffic within the respective virtual domains110to facilitate traffic isolation among the various customers for respective virtual domains110. When external virtual domain106receives encrypted network traffic via one of secure tunnels119, external virtual domain106applies routes125to route the encrypted network traffic, based on the source IP/port and/or destination IP/port information of the encrypted network traffic, into the correct one of virtual domains110. For example, the device104provider may configure a route of routes125to cause external virtual domain106forward traffic that is sourced by customer branch office118, destined to customer network130, or both sourced by customer branch office118and destined to customer network130, to the secure endpoint115A in virtual domain110A for the customer associated with customer branch office118and customer network130. The matching prefix for this route may be based on information provided by that customer, and the destination for this route may be the secure tunnel interface represented by secure endpoint115A. Device104decrypts the encrypted network traffic. Virtual domain110may then apply network function105, in accordance with virtual domain110A-specific preferences for network105, to the network traffic, and virtual domain110A may forward the network traffic to customer network130. In some cases, the underlying network traffic is destined for another network, such as cloud service provider network128. Virtual domain110A may consequently forward the network traffic to the virtual domain110that is on a path to the destination network, via one of virtual domain connections121.

In this way, device104creates a secure VPN that terminates within the customer's virtual domain110A, the secure VPN having endpoints within the customer branch office118. The customer branch office118may have a dedicated VPN gateway (not shown) to operate as the other secure endpoint for secure tunnel119A. The secure VPN thus stretches between virtual domain110A and the customer branch office118and, although encrypted network traffic for multiple customers may intermingle at external virtual domain106, it is encrypted. The unencrypted network traffic for a customer is only present in device104within the virtual domain110configured for that customer. Device104may operate similarly with respect to virtual domain110K and secure tunnel119K for a VPN for the cloud service provider. In some cases, the VPN further protects the internal routing information by encrypting the IP headers of the customer network traffic. Customer network traffic is encapsulated by another set of IP headers for the secure tunnel119to which the customer network traffic is directed.

In some cases, the customer entry point into device104may be considered an A-side of a customer-provider interconnection while the exit-side connection may be considered a Z-side. The Z-side can also be configured as a hub where multiple spokes (A-side customers) connect to services. With a subscription, a customer receives Z-side services information. However, the Z-side receives connectivity information from all A-side subscribers to such services while maintaining privacy among A-side customers.

In effect, each customer manages a single “layer” of device104, i.e., a corresponding one of virtual domains110. The customer is only aware of that layer, its own network, and any networks to which the customer's layer has been stitched using one of virtual domain connections121. The techniques may avoid L3VPNs that require additional labels or extra protocol tags to packets in order to separate customer traffic. Rather, the techniques facilitate isolation using virtual domains110and secure tunnels119and route-based VPNs. Once network traffic leaves device104to public network120, it is encrypted network traffic with appropriate headers for reaching a destination. By contrast, L3VPN is labeled traffic and may not be encrypted. The external locations, such as customer branch office118, terminate the corresponding secure tunnel119. Network traffic sourced by customer branch office118is encrypted once it leaves domain towards external virtual domain106(public IP111). IN some cases, the remote end, Z-side, or intermediate hops are unaware of there being traffic for multiple customers, i.e., they may consider it all single customer traffic coming from a single device104. However, in reality, each flow belongs to a different customer, and customers control only their corresponding virtual domain110in order that the same platform (device104) can be shared among multiple customers.

By integrating the VPN and the VPN gateway in a single device104using route-based VPNs applied at external virtual domain106, which effectively functions as a VPN gateway within device104for networks connected to device104, device104may enable multiple customer VPNs across respective virtual domains110s, all sharing the same uplinks to public network106. In addition, because secure tunnels119terminate in respective virtual domains110, the data is encrypted until it reaches the corresponding virtual domain110for a customer. This may reduce security vulnerabilities and facilitate privacy. Even the provider that manages external virtual domain106does not have access to decrypted network traffic for customers in most cases. As a result, the techniques may allow device104, its network function105, license, and other services and resources to be shared across multiple customers of the provider in a secure manner, which facilitating connectivity and reachability with external branch offices and among customer networks connected to cloud exchange102. This may reduce the time for on-boarding or delay for new customers, which may be added by simple configuration changes to device104and a one-time hookup to a port. This may also improve utilization of device104for a reseller or the provider.

FIG.2is a block diagram that illustrates a system200including an example device204that executes a virtualized network function, configured and operating according to techniques described herein. In the example of system200, a VNF204is configured similarly to device104ofFIG.1, which may be a PNF for the same network function, i.e., network function105.

The cloud exchange202ofFIG.2is similar to that of cloud exchange102and includes Network Functions Virtualization infrastructure (NFVI)202, which may be provided in a data center environment that also includes the cloud exchange102switching fabric (not shown).

NFVI202may deploy one or more services, such as Virtualized Network Functions. NFVI202includes computing hardware, storage hardware, and network hardware for executing VNFs. In some examples, NFVI202further includes a virtualization layer over the hardware to offer virtual computing, virtual storage, and virtual network for executing VNFs. NFVI202may be executed by one or more computing devices in a centralized or distributed manner. In the example ofFIG.1, NFVI202platform includes servers, e.g., server234, running virtualization software (e.g., hypervisors) that enable virtual execution environments on which VNF images (including the network infrastructure software) are deployed. Server234may provide one or more virtual machines236A-236N (collectively, “VMs136”). Each of VMs236emulates hardware. In other words, each of VMs236provides a virtualized operating system and application suite (e.g., to deploy VNFs) for customer access. Alternatively, or additionally, each of the servers may provide containers (e.g., such as those provided by the open source Docker Container application), or other virtual execution environments in which VNFs are implemented. VNF204is deployed as one of these virtual execution environments, here VM236A.

VNF204may provide similar functionality to hardware-based network devices such as dedicated network appliances, but VNFs provide such functionality in software. A VNF is primarily a software construct and thus may be decoupled from the underlying hardware. For example, a VNF204can provide the same routing, switching firewall, intrusion detection or other services that have traditionally been provided by specialized hardware, but in software. VNF204can provide forwarding and network address translation services for network traffic to and from the VNF204.

VNF204stores configuration data109that configures virtual domains106,110similar to corresponding virtual domains in device104ofFIG.1. Accordingly, VNF204operates similarly to device104to implement route-based VPN to terminate secure tunnel119within virtual domains110such that VNF204can be shared among multiple customers (with corresponding virtual domains110) with secure VPN traffic forwarding for customer traffic. Such operations and configurations of VNF204are described in detail with respect to device104ofFIG.1operating in system100. As withFIG.1, a user may use programmable network platform103to configure VNF204with configuration data109.

Other virtual domains may be present in addition to, or instead of, the virtual domains of the example system200illustrated inFIG.1. For example, VNF204may include an Internet exchange virtual domain that couples the VNF204to an Internet exchange network, cloud exchange network, or a switching fabric.

In the example illustrated inFIG.1, one server134and one VNF204have been shown. A typical NFVI infrastructure may include more than one server, and the servers may include more than one VNF.

FIG.3illustrates a conceptual view of a network system having a metro-based cloud exchange that provides multiple cloud exchange points that include a device, according to techniques described herein. The multiple cloud exchange points may be used to implement, at least in part, a NFVI202and/or connectivity with physical network functions for application to network traffic. For example, NFVI202may be connected to a cloud exchange fabric of cloud exchange328A and include VNF204. As illustrated, however, device104is connected to a cloud exchange fabric of cloud exchange328A. These systems may be deployed within a single data center, in some cases.

Each of cloud-based services exchange points328A-328C (described hereinafter as “cloud exchange points” and collectively referred to as “cloud exchange points328”) of cloud-based services exchange300(“cloud exchange300”) may represent a different data center geographically located within the same metropolitan area (“metro-based,” e.g., in New York City, N.Y.; Silicon Valley, Calif.; Seattle-Tacoma, Wash.; Minneapolis-St. Paul, Minn.; London, UK; etc.) to provide resilient and independent cloud-based services exchange by which cloud-based services customers (“cloud customers”) and cloud-based service providers (“cloud providers”) connect to receive and provide, respectively, cloud services. In various examples, cloud exchange300may include more or fewer cloud exchange points328. In some instances, a cloud exchange300includes just one cloud exchange point328. As used herein, reference to a “cloud exchange” or “cloud-based services exchange” may refer to a cloud exchange point. A cloud exchange provider may deploy instances of cloud exchanges300in multiple different metropolitan areas, each instance of cloud exchange300having one or more cloud exchange points328.

Each of cloud exchange points328includes network infrastructure and an operating environment by which cloud customers308A-308C (collectively, “cloud customers308”) receive cloud services from multiple cloud service providers310A-310N (collectively, “cloud service providers310”). The cloud service provider310may host one of more cloud services114. As noted above, the cloud service providers310may be public or private cloud service providers.

Cloud exchange300provides customers of the exchange, e.g., enterprises, network carriers, network service providers, and SaaS customers, with secure, private, virtual connections to multiple cloud service providers (CSPs) globally. The multiple CSPs participate in the cloud exchange by virtue of their having at least one accessible port in the cloud exchange by which a customer may connect to the one or more cloud services offered by the CSPs, respectively. Cloud exchange300allows private networks of any customer to be directly cross-connected to any other customer at a common point, thereby allowing direct exchange of network traffic between the networks of the customers.

Cloud customers308may receive cloud-based services directly via a layer 3 peering and physical connection to one of cloud exchange points328or indirectly via one of network service providers306A-306B (collectively, “NSPs306,” or alternatively, “carriers306”). Cloud customers308may include customers associated with a VNF204as described above. For example, cloud customers308may include systems used by any or all of customer devices used by cloud client118to access cloud services114via VNF204. NSPs306provide “cloud transit” by maintaining a physical presence within one or more of cloud exchange points328and aggregating layer 3 access from one or customers308. NSPs306may peer, at layer 3, directly with one or more cloud exchange points328and in so doing offer indirect layer 3 connectivity and peering to one or more customers308by which customers308may obtain cloud services from the cloud exchange300. Each of cloud exchange points328, in the example ofFIG.3, is assigned a different autonomous system number (ASN). For example, cloud exchange point328A is assigned ASN1, cloud exchange point328B is assigned ASN2, and so forth. Each cloud exchange point328is thus a next hop in a path vector routing protocol (e.g., BGP) path from cloud service providers310to customers308. As a result, each cloud exchange point328may, despite not being a transit network having one or more wide area network links and concomitant Internet access and transit policies, peer with multiple different autonomous systems via external BGP (eBGP) or other exterior gateway routing protocol in order to exchange, aggregate, and route service traffic from one or more cloud service providers310to customers. In other words, cloud exchange points328may internalize the eBGP peering relationships that cloud service providers310and customers308would maintain on a pair-wise basis. Instead, a customer308may configure a single eBGP peering relationship with a cloud exchange point328and receive, via the cloud exchange, multiple cloud services from one or more cloud service providers310. While described herein primarily with respect to eBGP or other layer 3 routing protocol peering between cloud exchange points and customer, NSP, or cloud service provider networks, the cloud exchange points may learn routes from these networks in other way, such as by static configuration, or via Routing Information Protocol (RIP), Open Shortest Path First (OSPF), Intermediate System-to-Intermediate System (IS-IS), or other route distribution protocol.

As examples of the above, customer308C is illustrated as having contracted with a cloud exchange provider for cloud exchange300to directly access layer 3 cloud services via cloud exchange points328C. In this way, customer308C receives redundant layer 3 connectivity to cloud service provider310A, for instance. Customer308C, in contrast, is illustrated as having contracted with the cloud exchange provider for cloud exchange300to directly access layer 3 cloud services via cloud exchange point328C and also to have contracted with NSP306B to access layer 3 cloud services via a transit network of the NSP306B. Customer308B is illustrated as having contracted with multiple NSPs306A,306B to have redundant cloud access to cloud exchange points328A,328B via respective transit networks of the NSPs306A,306B. The contracts described above are instantiated in network infrastructure of the cloud exchange points328by L3 peering configurations within switching devices of NSPs306and cloud exchange points328and L3 connections, e.g., layer 3 virtual circuits, established within cloud exchange points328to interconnect cloud service provider310networks to NSPs306networks and customer308networks, all having at least one port offering connectivity within one or more of the cloud exchange points328.

In some examples, cloud exchange300allows a corresponding one of customer customers308A,308B of any network service providers (NSPs) or “carriers”306A-306B (collectively, “carriers306”) or other cloud customers including customers308C to be directly connected, via a virtual layer 2 (L2) or layer 3 (L3) connection to any other customer network and/or to any of CSPs310, thereby allowing direct exchange of network traffic among the customer networks and CSPs310. The virtual L2 or L3 connection may be referred to as a “virtual circuit.”

Carriers306may each represent a network service provider that is associated with a transit network by which network subscribers of the carrier306may access cloud services offered by CSPs310via the cloud exchange300. In general, customers of CSPs310may include network carriers, large enterprises, managed service providers (MSPs), as well as Software-as-a-Service (SaaS), Platform-aaS (PaaS), Infrastructure-aaS (IaaS), Virtualization-aaS (VaaS), and data Storage-aaS (dSaaS) customers for such cloud-based services as are offered by the CSPs310via the cloud exchange300.

In this way, cloud exchange300streamlines and simplifies the process of partnering CSPs310and customers (via carriers306or directly) in a transparent and neutral manner. One example application of cloud exchange300is a co-location and interconnection data center in which CSPs310and carriers306and/or customers308may already have network presence, such as by having one or more accessible ports available for interconnection within the data center, which may represent any of cloud exchange points328. This allows the participating carriers, customers, and CSPs to have a wide range of interconnectivity options within the same facility. A carrier/customer may in this way have options to create many-to-many interconnections with only a one-time hook up to one or more cloud exchange points328. In other words, instead of having to establish separate connections across transit networks to access different cloud service providers or different cloud services of one or more cloud service providers, cloud exchange300allows customers to interconnect to multiple CSPs and cloud services.

Cloud exchange300includes a programmable network platform320for dynamically programming cloud exchange300to responsively and assuredly fulfill service requests that encapsulate business requirements for services provided by cloud exchange300and/or cloud service providers310coupled to the cloud exchange300. Programmable network platform320may include a network service orchestrator332that handles tenant (e.g., cloud client) requests for deployment of VNFs. For example, network service orchestrator332may organize, direct and integrate underlying services through VMs136(or containers), as well as other software and network sub-systems, for managing various services (e.g., deployment of VNFs). The programmable network platform320may, as a result, orchestrate a business-level service across heterogeneous cloud service providers310according to well-defined service policies, quality of service policies, service level agreements, and costs, and further according to a service topology for the business-level service.

The programmable network platform320enables the cloud service provider that administers the cloud exchange300to dynamically configure and manage the cloud exchange300to, for instance, facilitate virtual connections for cloud-based services delivery from multiple cloud service providers310to one or more cloud customers308. The cloud exchange300may enable cloud customers308to bypass the public Internet to directly connect to cloud services providers310so as to improve performance, reduce costs, increase the security and privacy of the connections, and leverage cloud computing for additional applications. In this way, enterprises, network carriers, and SaaS customers, for instance, can at least in some aspects integrate cloud services with their internal applications as if such services are part of or otherwise directly coupled to their own data center network.

In other examples, programmable network platform320enables the cloud service provider to configure cloud exchange300with a L3 instance requested by a cloud customer308, as described herein. A customer308may request an L3 instance to link multiple cloud service providers by the L3 instance, for example (e.g., for transferring the customer's data between two cloud service providers, or for obtaining a mesh of services from multiple cloud service providers).

Programmable network platform320may represent an application executing within one or more data centers of the cloud exchange300or alternatively, off-site at a back office or branch of the cloud provider (for instance). Programmable network platform320may be distributed in whole or in part among the data centers, each data center associated with a different cloud exchange point328to make up the cloud exchange300. Although shown as administering a single cloud exchange300, programmable network platform320may control service provisioning for multiple different cloud exchanges. Alternatively or additionally, multiple separate instances of the programmable network platform320may control service provisioning for respective multiple different cloud exchanges.

In the illustrated example, programmable network platform320includes a service interface (or “service API”)314that defines the methods, fields, and/or other software primitives by which applications330, such as a customer portal, may invoke the programmable network platform320. The service interface314may allow carriers306, customers308, cloud service providers310, and/or the cloud exchange provider programmable access to capabilities and assets of the cloud exchange300according to techniques described herein.

For example, the service interface314may facilitate machine-to-machine communication to enable dynamic provisioning of virtual circuits in the cloud exchange for interconnecting customer and/or cloud service provider networks. In this way, the programmable network platform320enables the automation of aspects of cloud services provisioning. For example, the service interface314may provide an automated and seamless way for customers to establish, de-install and manage interconnections among multiple, different cloud providers participating in the cloud exchange.

Cloud service provider network310A may represent an example instance of cloud service provider network128ofFIGS.1-2, customer308A may represent an example instance of customer network130ofFIGS.1-2. Programmable network platform320may provision one or more virtual circuits in cloud exchange point328A to interconnect customer network308A to cloud service provider network310A and to forward the traffic via device104for processing with virtual domains110associated with the corresponding cloud service provider and customer, respectively. For example, a first virtual circuit may connect customer network308A to virtual domain110A of device104, and a second virtual circuit may connect cloud service provider network310A to virtual domain110K. Device104includes a virtual domain connection121B that stitches the first virtual circuit and second virtual circuit together to enable cloud service traffic to flow between the customer network308A and the cloud service provider network310A, via the virtual domains110A and110K that apply network function105in accordance with each customer's (the customer for virtual domain110A and the customer/cloud service provider for virtual domain110K in this example). policies for network function105.

FIG.4is a flowchart illustrating an example mode of operation for a device, according to techniques of this disclosure. This mode of operation includes a setup phase and an operation phase. Although shown for purposes of illustration with the operation phase following the setup phase, this order is not required, for in some instances setup may occur while the device is in an operating mode. During a setup phase, a computing device, such as device104or VNF204(executing on a server, for instance), receives first configuration data109defining an external virtual domain106for a network function105(400). The external virtual domain106may be managed and only configurable by a provider of the computing device, and not configurable by a customer of the provider. The computing device may operate as a gateway for virtual domains110to public network120. The computing device receives second configuration data defining a virtual domain110A for the network function105and a secure tunnel interface for the virtual domain110A (402). The computing device receives third configuration data defining a virtual domain connection121A between the external virtual domain106and the virtual domain110A (404). The computing device obtains and installs, to a route-based virtual private network107, a route for a prefix corresponding to a customer network130attached to a first port of the computing device that is a port of the virtual domain110A, the route mapped to the secure tunnel interface of the virtual domain110A (406). The secure tunnel interface may be a secure tunnel endpoint.

The computing device subsequently receives encrypted network traffic at a second port of the computing device that is a port of the external virtual domain106. This port may be connected to (or is otherwise the port by which the computing device reaches) a public network120(408). The external virtual domain106does not decrypt the encrypted network traffic. The computing device forwards, based on the route109, using the route-based virtual private network107, the encrypted network traffic to the secure tunnel interface of the virtual domain110A to which the prefix for route109is mapped (410).

The virtual domain110A decrypts the encrypted network traffic (412) and forwards the decrypted network traffic to the customer network130based on the route109(414). In this way, virtual domains110for respective customers with secure VPN tunnels terminating within the customer-specific virtual domains110may improve security of customer traffic because unencrypted traffic for multiple customers that traverses public network120does not traverse common links or a common virtual domains, such as external virtual domain106.

FIG.5is a block diagram illustrating further details of one example of a computing device that operates in accordance with one or more techniques of the present disclosure.FIG.5may illustrate a particular example of a server or other computing device500that includes one or more processor(s)502for executing any one or more of any system, application, or module described herein. Although the techniques of this disclosure are described inFIG.5with respect to a server executing a VNF, a physical appliance such as a dedicated router or firewall appliance may include similar components that operate similarly as described with respect to those of computing device500. However, rather than executing a VNF524, such a physical appliance would typically natively execute the network function and implement virtual domains, as described elsewhere herein.

One or more processor(s)502may execute VMs526and VNF524. Other examples of computing device500may be used in other instances. Although shown inFIG.5as a stand-alone computing device500for purposes of example, a computing device may be any component or system that includes one or more processors or other suitable computing environment for executing software instructions and, for example, need not necessarily include one or more elements shown inFIG.5(e.g., communication units506; and in some examples components such as storage device(s)508may not be co-located or in the same chassis as other components).

As shown in the specific example ofFIG.5, computing device500includes one or more processors502, one or more input devices504, one or more communication units506, one or more output devices512, one or more storage devices508, and user interface (UI) device510, and communication units506. Computing device500, in one example, further includes one or more applications522, hypervisor517, and operating system516that are executable by computing device500. Each of components502,504,506,508,510, and512are coupled (physically, communicatively, and/or operatively) for inter-component communications. In some examples, communication channels514may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data. As one example, components502,504,506,508,510, and512may be coupled by one or more communication channels514.

Processors502, in one example, are configured to implement functionality and/or process instructions for execution within computing device500. For example, processors502may be capable of processing instructions stored in storage device508. Examples of processors502may include, any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or equivalent discrete or integrated logic circuitry.

One or more storage devices508may be configured to store information within computing device500during operation. Storage device508, in some examples, is described as a computer-readable storage medium. In some examples, storage device508is a temporary memory, meaning that a primary purpose of storage device508is not long-term storage. Storage device508, in some examples, is described as a volatile memory, meaning that storage device508does not maintain stored contents when the computer is turned off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art. In some examples, storage device508is used to store program instructions for execution by processors502. Storage device508, in one example, is used by software or applications running on computing device500to temporarily store information during program execution.

Storage devices508, in some examples, also include one or more computer-readable storage media. Storage devices508may be configured to store larger amounts of information than volatile memory. Storage devices508may further be configured for long-term storage of information. In some examples, storage devices508include non-volatile storage elements. Examples of such non-volatile storage elements include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories.

Computing device500, in some examples, also includes one or more communication units506. Computing device500, in one example, utilizes communication units506to communicate with external devices via one or more networks, such as one or more wired/wireless/mobile networks. Communication units506may include a network interface card, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device that can send and receive information. In some examples, computing device500uses communication unit506to communicate with an external device. Communication units506may include or be connected to one or more ports of computing device500. Ports may be associated to different virtual domains in the configuration data109for VNF524or for computing device500as a whole.

Computing device500, in one example, also includes one or more user interface devices510. User interface devices510, in some examples, are configured to receive input from a user through tactile, audio, or video feedback. Examples of user interface devices(s)510include a presence-sensitive display, a mouse, a keyboard, a voice responsive system, video camera, microphone or any other type of device for detecting a command from a user. In some examples, a presence-sensitive display includes a touch-sensitive screen.

One or more output devices512may also be included in computing device500. Output device512, in some examples, is configured to provide output to a user using tactile, audio, or video stimuli. Output device512, in one example, includes a presence-sensitive display, a sound card, a video graphics adapter card, or any other type of device for converting a signal into an appropriate form understandable to humans or machines. Additional examples of output device512include a speaker, a cathode ray tube (CRT) monitor, a liquid crystal display (LCD), or any other type of device that can generate intelligible output to a user.

Computing device500may include operating system516. Operating system516, in some examples, controls the operation of components of computing device500. For example, operating system516, in one example, facilitates the communication of one or more applications522, hypervisor517, VMs526and VNF524with processors502, communication unit506, storage device508, input device504, user interface devices510, and output device512. Application522, hypervisor517, VMs526and VNF524may also include program instructions and/or data that are executable by computing device500. Hypervisor517that enables process-based or system-based VMs526to execute as isolated device instances for executing VNF524. In some cases, however, computing device500may execute VNF524as one or more deployed containers using container virtualization.

VNF524may represent an example instance of VNF204and stores configuration data109and routes125that define, at least in part, the configuration and operation of VNF204to establish and implement virtual domains, in accordance with techniques described in this disclosure and as described in detail above with respect to VNF204and device104. The virtual domains may be associated with different customers of a provider or reseller for VNF524.

FIG.6is a flowchart illustrating an example mode of operation for a device, in accordance with techniques described in this disclosure. This mode of operation includes a setup phase and an operation phase. Although shown for purposes of illustration with the operation phase following the setup phase, this order is not required, for in some instances setup may occur while the device is in an operating mode.

During a setup phase, a computing device, such as device104or VNF204(executing on a server, for instance), receives first configuration data109defining virtual domain110A for network function105(600). The network function to be applied by virtual domain110A may be managed and only configurable by a first customer of a provider of the computing device, and not configurable by the provider. First configuration data109may associate a first port of the computing device with virtual domain110A. The computing device receives second configuration data109defining a virtual domain110K for the network function105(602). The virtual domains110A,110K are separate from one another. Second configuration data109may associate a second port of the computing device with virtual domain110K.

The computing device receives third configuration data109defining a virtual domain connection121C between the virtual domain110A and virtual domain110K (604). Virtual domain connection121C enables network traffic forwarding from virtual domain110A to virtual domain110K and, typically, vice-versa.

The computing device subsequently receives network traffic at the first port of the computing device that is a port of virtual domain110A. This port may be connected to (or is otherwise the port by which the computing device reaches) a first customer network (e.g., customer network130) (606). Virtual domain110A applies network function105to the network traffic based on policies or other configuration set by the first customer for virtual domain110A (608).

Virtual domain110A forwards the network traffic via virtual domain connection112C to virtual domain110B (610). Virtual domain110B applies network function105to the network traffic based on policies or other configuration set by the second customer for virtual domain110B (612). Virtual domain110B may subsequently forward the network traffic via the second port to a second customer network (e.g., cloud service provider network128).