Patent Description:
A partner network may obtain services from the WAN. For example, a partner network may be an enterprise network and the WAN may host services for users of the enterprise network. The partner network may be connected to the WAN via the Internet. A path between the partner network and the WAN may include one or more Internet Service Providers (ISPs). In general, traffic traversing the Internet may be routed on a lowest cost basis. Accordingly, neither the partner network nor the WAN may have significant control of routing decisions between the partner network and the WAN.

One attempt to give a network greater control over routing decisions is software defined wide area networking (SD-WAN). SD-WAN may control routing decisions within an enterprise network. An SD-WAN appliance, which may be referred to as an SD-WAN edge is a physical or virtual network function that is placed at an organization's branch/regional/central office site, data center, and in public or private cloud platforms. The SD-WAN edge may perform classification of traffic and forwarding based on availability of a route to another SD-WAN edge. In the context of cloud services hosted by the WAN operator, however, an SD-WAN may not have information or ability to select routes over the Internet to the WAN hosting a cloud service.

<NPL> describes modern multi-domain networks now span over datacenter networks, enterprise networks, customer sites and mobile entities. Such networks are critical and, thus, must be resilient, scalable and easily extensible. The emergence of Software-Defined Networking (SDN) protocols, which enables to decouple the data plane from the control plane and dynamically program the network, opens up new ways to architect such networks. The authors propose DISCO, an open and extensible DIstributed SDN COntrol plane able to cope with the distributed and heterogeneous nature of modern overlay networks and wide area networks.

DISCO controllers manage their own network domain and communicate with each others to provide end-to-end network services. This communication is based on a unique lightweight and highly manageable control channel used by agents to self-adaptively share aggregated network-wide information. The authors implemented DISCO on top of the Floodlight OpenFlow controller and the AMQP protocol. They demonstrated how DISCO's control plane dynamically adapts to heterogeneous network topologies while being resilient enough to survive to disruptions and attacks and providing classic functionalities such as end-point migration and network-wide traffic engineering. The experimentation results presented are organized around three use cases: interdomain topology disruption, end-to-end priority service request and virtual machine migration.

<CIT> describes systems and methods provide for end-to-end identity-aware routing across multiple administrative domains. A first ingress edge device of a second overlay network can receive a first encapsulated packet from a first egress edge device of a first overlay network. The first ingress edge device can de-encapsulate the first encapsulated packet to obtain an original packet and a user or group identifier. The first ingress edge device can apply a user or group policy matching the user or group identifier to determine a next hop for the original packet. The first ingress edge device can encapsulate the original packet and the user or group identifier to generate a second encapsulated packet. The first ingress edge device can forward the second encapsulated packet to the next hop.

In some instances, well-known components are shown in block diagram form in order to avoid obscuring such concepts.

This disclosure describes various examples related to selection of a path between a partner network and a wide area network (WAN). In an aspect, the WAN includes a plurality of front-end devices that are connected to internet service providers (ISPs), for example, at peering locations such as points of presence (POPs). Generally, traffic for services hosted in the WAN may ingress the WAN at any of the front-end devices. Conventionally, the front-end devices advertise their presence to the ISPs using border gateway protocol (BGP) anycast announcements. Such announcements associate the same internet protocol (IP) address and prefix with each of the front-end devices. Accordingly, traffic for the services hosted in the WAN may be forwarded by the ISPs to any of the front-end devices.

In an aspect, communication properties (e.g., quality of service (QoS) or quality of experience (QoE)) between the partner network and a service hosted in the WAN may be improved by selecting at least a portion of the path between the partner network and the service. For example, traffic carried by different ISPs may have different characteristics in terms of latency, jitter, etc. Additionally, paths within the WAN from the front-end devices to the service may have different characteristics. However, because ingress traffic may conventionally arrive at any of the front-end devices, the WAN may have limited options for routing traffic to the service.

In an aspect, the present disclosure provides for selection of a path between the partner network and the WAN by advertising different BGP address prefixes for different front-end devices of the WAN. For instance, the WAN may advertise unicast BGP address prefixes for a plurality of front-end devices. Further, the WAN may implement an agent within the partner network (e.g., at an SD-WAN appliance), that measures paths including different front-end devices of the WAN. Accordingly, the WAN may identify paths from the partner network via the Internet to services hosted in the WAN that have better characteristics. The WAN may then control the agent to route traffic to the service along a selected path. For instance, the agent may tunnel traffic to the unicast BGP address prefix associated with selected path. Thus, the WAN may improve a connection between the partner network and the WAN via the Internet by selecting a specific path for traffic associated with a service.

Turning now to <FIG>, examples are depicted with reference to one or more components and one or more methods that may perform the actions or operations described herein, where components and/or actions/operations in dashed line may be optional. Although the operations described below in <FIG> are presented in a particular order and/or as being performed by an example component, the ordering of the actions and the components performing the actions may be varied, in some examples, depending on the implementation. Moreover, in some examples, one or more of the actions, functions, and/or described components may be performed by a specially-programmed processor, a processor executing specially-programmed software or computer-readable media, or by any other combination of a hardware component and/or a software component capable of performing the described actions or functions.

<FIG> is a conceptual diagram <NUM> of an example of an architecture for connectivity between a partner network <NUM> and a wide area network (WAN) <NUM> via the internet <NUM>. The WAN <NUM> may host a service <NUM> for the partner network <NUM>.

The partner network <NUM> may include any computer network that may connect to the WAN <NUM> via the Internet <NUM>. For example, the partner network may include an enterprise network, which may itself be a WAN connecting multiple locations of the enterprise. As another example, the partner network <NUM> may include a radio access network (RAN). For example, the partner network <NUM> may include a user device <NUM> that wirelessly connects to a base station <NUM>. The partner network <NUM> may include a user plane function (UPF) <NUM> that handles user traffic to a from a core network. In some implementations, the service <NUM> may include a core network function.

In some cases, the partner network may utilize a software defined WAN (SD-WAN) appliance <NUM>. For example, the SD-WAN appliance <NUM> may be an SD-WAN edge device that controls routing to other SD-WAN edge devices and to the Internet <NUM> and/or WAN <NUM>. In some implementations, an SD-WAN appliance may include software executed on a network device such as a router or server. For instance, an SD-WAN appliance <NUM> may be provided by a third party and executed on a device of the partner network <NUM>.

The Internet <NUM> may be a network of internet service providers (ISPs) <NUM> connected according to the Internet protocol. For simplicity, a first ISP 122a and a second ISP 122b are illustrated, but it should be understood that the Internet <NUM> may include numerous ISPs connected directly or indirectly. The ISPs may also be referred to as autonomous systems (AS) and be associated with an autonomous system number (ASN), which may be used for routing according to a border gateway protocol (BGP).

The WAN <NUM> may include computing resources spread across a geographic region and connected via communication links such as fiber optic cables. For example, the WAN <NUM> may include front-end devices <NUM> and data centers <NUM>. The front-end devices <NUM> may be referred to as edge devices and may include routers and/or servers, for example. The front-end devices <NUM> may be located at a point of presence (POP) and have a peering connection to one or more ISPs <NUM>. A peering connection may be associated with a BGP IP address prefix <NUM>. For example, each of the front-end devices <NUM> may use BGP anycast announcements to establish the same IP address prefix (<NUM>. <NUM>/<NUM>). Accordingly, the ISPs <NUM> may forward traffic to the WAN <NUM> via the peering connections. The data centers <NUM> may include computing resources (e.g., servers) within the WAN <NUM>. For example, in a cloud computing scenario, the WAN <NUM> may host a service <NUM> (e.g., an application) at one or more of the data centers <NUM>. The data centers <NUM> may also include routers configured to forward traffic to front-end devices <NUM> and/or other data centers <NUM>. In some implementations, the service <NUM> may be a transport service associated with an egress point of the WAN <NUM>, which may be another front-end device <NUM>.

As illustrated in <FIG>, traffic for the service <NUM> may take different paths from the partner network <NUM> to the WAN <NUM>. For example, a first path <NUM> may travel from the SD-WAN appliance <NUM> to the first ISP 122a, to the front-end device 142a, and data centers 144a, 144b, 144c. As another example, a second path <NUM> may travel from the SD-WAN appliance <NUM> to the second ISP 122b, to the front-end device 142b, and to the data center 144c. In some cases, the SD-WAN appliance <NUM> may provide limited selection of between path <NUM> and <NUM>, but such selection may only apply to selection of an initial ISP <NUM> because the initial ISP <NUM> may choose to forward traffic to another ISP rather than directly to the WAN <NUM>. Additionally, as illustrated, path selection by the WAN <NUM> is limited because traffic may arrive at any front-end device <NUM>. <FIG> is a conceptual diagram <NUM> of an example of an architecture for connectivity between a partner network <NUM> and a WAN <NUM> using a routing agent <NUM> within the partner network <NUM> to direct traffic to a selected front-end device of the WAN <NUM> via the Internet <NUM>. The WAN <NUM> includes a routing controller <NUM> that communicates with the routing agent <NUM> and provides routing policies to the routing controller <NUM>.

The partner network <NUM> may be similar to the partner network <NUM> and include user device <NUM>. In the case of a RAN, the partner network <NUM> may include the base station <NUM> and UPF <NUM>. The partner network <NUM> may include the routing agent <NUM>. In some implementations, the routing agent <NUM> may be implemented on a SD-WAN appliance <NUM>. In some implementations, the routing agent <NUM> may be implemented on other devices in the partner network <NUM> including, for example, a user device <NUM>.

The WAN <NUM> may be similar to the WAN <NUM> and include front-end devices <NUM> and data centers <NUM>. In an aspect, one or more of the front-end devices <NUM> may advertise a unicast BGP address prefix for the front-end device <NUM>. For example, the front-end device 142a may send an advertisement <NUM> to the ISP 122a. The advertisement <NUM> may advertise the BGP address prefix 243a (e.g., <NUM>. <NUM>/<NUM>) for the front-end device 142a. Because the advertisement <NUM> is a BGP unicast announcement, the ISP 122a may forward traffic for the specified address prefix 243a to the specific front-end device 142a. The ISP 122a may propagate the advertisement <NUM> to establish paths to the front-end device 142a through other ISPs <NUM>. Similarly, the front-end device 142b may send an advertisement <NUM> to the ISP 122b advertising a BGP address prefix 243b (e.g., <NUM>. <NUM>/<NUM>) for the front-end device 142b. In an aspect, the routing controller <NUM> may control the front-end devices <NUM> to advertise a plurality of unicast BGP address prefixes for a plurality of front-end devices <NUM> of the WAN <NUM>. Accordingly, the routing controller <NUM> may establish paths through the Internet <NUM> to specific front-end devices <NUM>.

In an aspect, the front-end devices <NUM> may also advertise an anycast BGP prefix <NUM> that is common to the front-end devices <NUM>. The anycast BGP prefix <NUM> may be less specific (e.g., <NUM>/<NUM>) than the unicast BGP prefixes <NUM>. According to BGP, the ISPs <NUM> may select the more specific routes for the unicast BGP prefixes <NUM> if available. The anycast BGP prefix <NUM> may gracefully handle certain types of failure. For example, if multiple front-end devices are associated with the same anycast BGP prefix <NUM> and one of the front-end devices associated with a unicast BGP prefix <NUM> becomes unavailable, an ISP <NUM> may select a different route to the anycast BGP prefix <NUM>. In an aspect, unicast BGP address prefixes <NUM> may be limited by availability and/or cost. The routing controller <NUM> may select which front-end devices <NUM> to associate with a unicast BGP address prefix. For example, the routing controller <NUM> may analyze a network topology of the WAN <NUM> to determine front-end devices <NUM> that may be associated with desirable (e.g., relatively lower latency or greater bandwidth) paths to the service <NUM> within the WAN <NUM>. In some implementations, a unicast BGP address prefix may be re-used by front-end devices in geographically isolated regions of the WAN <NUM>. For instance, if a front-end device 142c is located in a different region or country such that traffic from the ISP 122b is unlikely to be routed to the front-end device 142c, the front-end device 142c may also advertise the unicast BGP address prefix of <NUM>. <NUM>/<NUM> to establish a path in its respective region.

In an aspect, the advertisement of unicast BGP address prefixes may establish paths <NUM>, <NUM>. Unlike the case of the paths <NUM>, <NUM> in <FIG>, the routing controller <NUM> and/or the routing agent <NUM> may have greater control over the paths <NUM>, <NUM>. For example, the routing agent <NUM> may tunnel traffic for the service <NUM> to a respective front-end device <NUM> associated with a selected path. Although the ISPs <NUM> may be autonomous systems and select different routes over the Internet <NUM>, the use of a unicast BGP address prefix may ensure traffic reaches a specific ingress point on the WAN <NUM>. Moreover, the differentiation between the paths <NUM>, <NUM> allows the routing agent <NUM> to measure the paths <NUM>, <NUM>. For example, the routing agent <NUM> may send ping packets to the service along each available path to determine a latency associated with each path. Once again, although the ISPs <NUM> may make different routing decisions, such decisions are likely to be relatively static and produce consistent path measurements.

The routing controller <NUM> may select a path (e.g., path <NUM>, <NUM>) within the WAN <NUM> for the service <NUM> for users of the partner network <NUM>. The path <NUM>, <NUM> includes a selected device of the plurality of front-end devices <NUM>. For example, the routing controller <NUM> may select a path based on internal routing preferences of the WAN <NUM>. For instance, the routing controller <NUM> may select a path that has a lowest latency or a greatest available bandwidth. In some implementations, the routing controller <NUM> may select the path based on the measurements <NUM> provided by the routing agent <NUM>. In some implementations, the routing agent <NUM> may provide measurements <NUM> of the paths <NUM>, <NUM> to the respective front-end device <NUM>. Such measurements <NUM> may represent a portion of the path where the WAN <NUM> lacks direct control of routing. The routing controller <NUM> may perform measurements of various paths within the WAN <NUM>. In some cases, the routing controller <NUM> may add a measurement within the WAN <NUM> from the front-end device <NUM> to the service <NUM> to the measurement <NUM> of the portion of the path outside of the WAN <NUM>. In other implementations, the routing agent <NUM> may measure the total path <NUM>, <NUM>. Accordingly, the routing controller <NUM> may dynamically select a path between the partner network <NUM> and the service <NUM>.

The routing controller <NUM> may control the routing agent <NUM> to use the selected path for traffic for the service <NUM>. For example, the routing controller <NUM> may export a routing rule <NUM> to the routing agent <NUM>. The routing rule <NUM> may specify the unicast address prefix <NUM> for the selected front-end device <NUM>. The routing agent <NUM> may forward traffic for the service to the unicast address prefix based on the routing rule. For example, the routing agent <NUM> may encapsulate packets for the service within a packet for the unicast address prefix for the selected front-end device <NUM> based on the routing rule. The front-end device <NUM> may then decapsulate packets for the service from packets addressed to the unicast address prefix for the selected device based on the routing rule. As another example, the routing agent <NUM> may rewrite headers of packets for the service with the unicast address prefix for the selected device based on the routing rule. In some implementations, the header may include a port corresponding to an original address for the service. The front-end device <NUM> may then forward the data packets addressed to the unicast address prefix for the selected device142 to the service <NUM> based on an original address within the WAN <NUM> according to the routing rule. The routing controller <NUM> may configure routing tables within the WAN <NUM> such that the front-end device <NUM> and other devices forward the packets to the service <NUM> along the selected path <NUM>, <NUM>. Accordingly, the routing controller <NUM> may have increased control over the path of packets for a service and thereby improve operation of the system. For instance, the routing controller <NUM> may select a path <NUM>, <NUM> to meet a SLA for the service <NUM>.

<FIG> is a diagram <NUM> of example components of the routing agent <NUM> and the routing controller <NUM> and communications there between. The routing agent <NUM> may include a measurement component <NUM>, a control component <NUM>, a forwarding component <NUM>, and a monitoring component <NUM>. The routing controller <NUM> may include an advertisement component <NUM>, a path selection component <NUM>, an external rule component <NUM> and an internal routing component <NUM>.

The measurement component <NUM> may be configured to measure a plurality of paths to a service within the WAN <NUM>. Each of the plurality of paths may be associated with one of a plurality of front-end devices <NUM> of the WAN <NUM> that are associated with respective unicast border gateway protocol address prefixes <NUM>. In an aspect, the measurement component <NUM> may receive a list of the unicast border gateway protocol address prefixes 243from the routing controller <NUM>. In some implementations, where the routing agent <NUM> is connected to two or more ISPs <NUM>, there may be multiple available paths to each of the front-end devices <NUM>. For example, a path to edge device 142a using the unicast address prefix 243a and the first ISP 122a may be different than a path to edge device 142a using the unicast address prefix 243a and the second ISP 122b. The measurement component <NUM> may generate a measurement <NUM> for each potential path. The measurements <NUM> may include latency, delay, throughput, and/or devices traversed. In some implementations, the measurements <NUM> may include a trace route indicating one or more networks (e.g., ISPs <NUM>) between the partner network <NUM> and the WAN <NUM>.

The control component <NUM> may be configured to communicate with the routing controller <NUM>. For example, the control component <NUM> may establish a session with the routing controller <NUM>. In an aspect, the session may utilize an anycast BGP address prefix for reliability. The control component <NUM> may provide the measurements <NUM> to the routing controller <NUM> via the session. The control component <NUM> may receive the routing rule <NUM> from the routing controller <NUM> via the session.

The forwarding component <NUM> may be configured to forward data packets for the service <NUM> to the respective border gateway protocol address prefix <NUM> of the selected device <NUM> via the Internet <NUM>. For example, the forwarding component <NUM> may be a router or a software routing stack. The forwarding component <NUM> may identify packets for the service <NUM>. For instance, the forwarding component <NUM> may identify the packets based on a <NUM>-tuple for each packet. The forwarding component <NUM> may tunnel the packets to the border gateway protocol address prefix <NUM>. For instance, the forwarding component <NUM> may use encapsulation and/or header rewriting. In some implementations, the forwarding component <NUM> may select an ISP <NUM> to forward the packets. The selected ISP may be based on the routing rule <NUM>. For instance, the routing controller <NUM> may determine the ISP <NUM> on the selected path <NUM>, <NUM> based on the measurements <NUM>.

The monitoring component <NUM> may monitor operation of the routing agent <NUM>. For example, the monitoring component may collect statistics regarding traffic forwarded according to the routing rules. In some implementations, the monitoring component <NUM> may generate an operator dashboard that is viewable by an operator of the partner network <NUM>. For example, the monitoring component <NUM> may include a user interface that allows a network operator to view the statistics. In some implementations, the network operator may configure the routing agent <NUM>. For instance, the monitoring component <NUM> may allow the network operator to select which services <NUM> receive routing rules.

The advertisement component <NUM> may be configured to advertise a plurality of unicast border gateway protocol address prefixes <NUM> for a plurality of front-end devices <NUM> of the WAN <NUM>. For example, the advertisement component <NUM> may control the plurality of front-end devices <NUM> to send advertisements <NUM> to connected ISPs <NUM>. In some implementations, the advertisement component <NUM> may select the plurality of front-end devices <NUM> based on a network topology of the WAN <NUM>. For example, the advertisement component <NUM> may allocate a limited number of unique unicast border gateway protocol address prefixes <NUM> among a larger number of front-end devices <NUM>. In some implementations, the advertisement component <NUM> may select geographically distributed front-end devices. In some implementations, the advertisement component <NUM> may select front-end devices that are associated with different paths to a data center <NUM> or a service <NUM>.

The path selection component <NUM> may be configured to select a path <NUM>, <NUM> within the WAN <NUM> for a service <NUM> for users of the partner network <NUM>. The path <NUM>, <NUM> may include a selected device of the plurality of front-end devices <NUM>. For example, in an implementation, the path selection component <NUM> may select a best path from any front-end device <NUM> associated with a unicast BGP address prefix <NUM> to the data center <NUM> hosting a service <NUM>. For instance, the best path may be a lowest cost path, a path having the least load, a path having the most bandwidth, a path having the best performance (e.g., latency and jitter), or a best combination of such metrics. In other implementations, the path selection component <NUM> may receive the measurements <NUM> from the routing agent <NUM> and select a path based on the measurements <NUM>. The use of the measurements <NUM> may advantageously account for the portion of the path <NUM>, <NUM> over the Internet <NUM>, which may be responsible for substantial latency. In an aspect, the path selection component <NUM> may be configured to select a best combination of an external portion of the path from the routing agent <NUM> to a front-end device <NUM> and an internal portion of the path from the front-end device <NUM> to the data center <NUM> or front-end device <NUM> hosting the service <NUM>.

The external rule component <NUM> may be configured to export a routing rule <NUM> to the routing agent <NUM> within the partner network <NUM>. The routing rule <NUM> may specify at least a unicast address prefix <NUM> for the selected device. Accordingly, the routing agent <NUM> may route traffic for the service <NUM> to the selected device. In some implementations, the routing rule <NUM> may specify particular tunneling settings or ISPs. For instance, the routing rule <NUM> may specify settings for encapsulation, header rewriting, or port forwarding.

The internal routing component <NUM> may be configured to control the selected front-end device <NUM> to receive data packets for the service <NUM> of the partner network <NUM> at the selected device. For instance, the internal routing component <NUM> may configure any packet processing (e.g., decapsulation or header rewriting) to be performed on the arriving data packets. The internal routing component <NUM> may configure routing tables at the front-end device <NUM> and each other device on the selected path to forward the data packets toward the service <NUM> along the selected path.

<FIG> is a schematic diagram of an example of a device <NUM> (e.g., a computing device) for communication between a partner network and a service hosted in a WAN connected to the partner network via an Internet. The device <NUM> may be implemented as one or more computing devices in the partner network <NUM>. For instance, the device <NUM> may be implemented as an SD-WAN appliance <NUM>. A plurality of user devices <NUM> may be configured to forward data packets to the device <NUM>. For example, user devices <NUM> may be connected to the device <NUM> via a local area network (LAN) or virtual private network (VPN). In some implementations where the partner network <NUM> is a RAN, the user devices <NUM> may be connected via a UPF <NUM> according to a radio network specification. In some implementations, the UPF <NUM> may execute a software stack under the control of the WAN <NUM> (e.g., according to the radio network specification). In some implementations, the device <NUM> may be a user device <NUM>.

In an example, device <NUM> can include a processor <NUM> and/or memory <NUM> configured to execute or store instructions or other parameters related to providing an operating system <NUM>, which can execute one or more applications or processes, such as, but not limited to, the routing agent <NUM> for forwarding data packets to the WAN along a selected path. For example, processor <NUM> and memory <NUM> may be separate components communicatively coupled by a bus (e.g., on a motherboard or other portion of a computing device, on an integrated circuit, such as a system on a chip (SoC), etc.), components integrated within one another (e.g., processor <NUM> can include the memory <NUM> as an on-board component), and/or the like. Memory <NUM> may store instructions, parameters, data structures, etc. for use/execution by processor <NUM> to perform functions described herein.

In an example, the routing agent <NUM> may include the measurement component <NUM>, the control component <NUM>, and the forwarding component <NUM>. The routing agent <NUM> may optionally include the monitoring component <NUM>.

<FIG> is a flow diagram of an example of a method <NUM> for forwarding packets to a WAN <NUM> according to a path selected by the WAN <NUM>. For example, the method <NUM> can be performed by a device <NUM> and/or one or more components thereof to measure potential paths and route the data packets along the selected path to a selected front-end device <NUM> of the WAN <NUM>.

At block <NUM>, the method <NUM> includes measuring a plurality of paths to the service within the WAN, each of the plurality of paths associated with one of a plurality of front-end devices of the WAN that are associated with respective unicast BGP address prefixes. In an example, the routing agent <NUM> and/or the measurement component <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can measure the plurality of paths <NUM>, <NUM> to the service <NUM> within the WAN <NUM>. Each of the plurality of paths <NUM>, <NUM> is associated with one of a plurality of front-end devices 142a, 142b of the WAN <NUM> that are associated with respective unicast BGP address prefixes 143a, 143b. In some implementations, the measurements include latency, delay, throughput, or devices traversed. In some implementations, the measurements include a trace route indicating one or more networks between the partner network and the WAN. In some implementations, at sub-block <NUM>, the block <NUM> may measuring a portion of each path <NUM>, <NUM> to a respective one of the plurality of front-end devices 142a, 142b, of the WAN via the respective unicast border gateway protocol address prefixes 143a, 143b. That is, the measurement component <NUM> may measure a portion of the paths <NUM>, <NUM> that is external to the WAN <NUM>. In such implementations, the measurements may not depend on internal routing within the WAN <NUM>. The WAN <NUM> may determine the portion of the path within the WAN <NUM> based on internal measurements, a network topology, and/or costs. In some other implementations, the block <NUM> may include measuring a full path between the routing agent <NUM> and the service <NUM>. Additionally, the measurement component <NUM> may continue to perform measurements after a path is selected, for example, to determine whether to update the path.

At block <NUM>, the method <NUM> includes providing measurements of the plurality of paths to the service to the WAN. In an example, the control component <NUM> and/or the routing agent <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can provide the measurements <NUM> of the plurality of paths <NUM>, <NUM> to the service <NUM> to the WAN <NUM>. For instance, the control component <NUM> may communicate the measurements <NUM> to the routing controller <NUM>. In some implementations, the control component <NUM> may provide the measurements in response to a request from the routing controller <NUM>.

At block <NUM>, the method <NUM> includes receiving a routing rule specifying a unicast address prefix for a selected device of the plurality of front-end devices of the WAN. In an example, the control component <NUM> and/or the routing agent <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can receive the routing rule <NUM> specifying the unicast address prefix <NUM> (e.g.,. prefix 243a) for a selected device (e.g., front-end device 142a) of the plurality of front-end devices <NUM> of the WAN <NUM>.

At block <NUM>, the method <NUM> includes forwarding data packets for the service to the respective border gateway protocol address prefix of the selected device via the Internet. In an example, the forwarding component <NUM> and/or the routing agent <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can forward the data packets <NUM> for the service <NUM> to the respective border gateway protocol address prefix 243a of the selected device 142a via the Internet <NUM>. For instance, the forwarding component <NUM> may establish a tunnel to the selected device 142a via the Internet <NUM>. In some implementations, at sub-block <NUM>, the block <NUM> may include encapsulating packets for the service within a packet for the unicast address prefix for the selected device based on the routing rule. In some implementations, at sub-block <NUM>, the block <NUM> may include rewriting headers of packets for the service with the unicast address prefix for the selected device based on the routing rule and a port corresponding to an original address for the service. For example, the tunneling technique may depend on the capabilities and/or configuration of the selected front-end device <NUM>.

<FIG> illustrates an example of a device <NUM> including additional optional component details as those shown in <FIG>. In one aspect, device <NUM> may include processor <NUM>, which may be similar to processor <NUM> for carrying out processing functions associated with one or more of components and functions described herein. Processor <NUM> can include a single or multiple set of processors or multi-core processors. Moreover, processor <NUM> can be implemented as an integrated processing system and/or a distributed processing system.

Device <NUM> may further include memory <NUM>, which may be similar to memory <NUM> such as for storing local versions of operating systems (or components thereof) and/or applications being executed by processor <NUM>, such as the routing agent <NUM>, the measurement component <NUM>, the control component <NUM>, the forwarding component <NUM>, the monitoring component <NUM>, etc. Memory <NUM> can include a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof.

Further, device <NUM> may include a communications component <NUM> that provides for establishing and maintaining communications with one or more other devices, parties, entities, etc. utilizing hardware, software, and services as described herein. Communications component <NUM> may carry communications between components on device <NUM>, as well as between device <NUM> and external devices, such as devices located across a communications network and/or devices serially or locally connected to device <NUM>. For example, communications component <NUM> may include one or more buses, and may further include transmit chain components and receive chain components associated with a wireless or wired transmitter and receiver, respectively, operable for interfacing with external devices.

Additionally, device <NUM> may include a data store <NUM>, which can be any suitable combination of hardware and/or software, that provides for mass storage of information, databases, and programs employed in connection with aspects described herein. For example, data store <NUM> may be or may include a data repository for operating systems (or components thereof), applications, related parameters, etc. not currently being executed by processor <NUM>. In addition, data store <NUM> may be a data repository for the routing agent <NUM>.

Device <NUM> may optionally include a user interface component <NUM> operable to receive inputs from a user of device <NUM> and further operable to generate outputs for presentation to the user. User interface component <NUM> may include one or more input devices, including but not limited to a keyboard, a number pad, a mouse, a touch-sensitive display, a navigation key, a function key, a microphone, a voice recognition component, a gesture recognition component, a depth sensor, a gaze tracking sensor, a switch/button, any other mechanism capable of receiving an input from a user, or any combination thereof. Further, user interface component <NUM> may include one or more output devices, including but not limited to a display, a speaker, a haptic feedback mechanism, a printer, any other mechanism capable of presenting an output to a user, or any combination thereof.

Device <NUM> may additionally include a routing agent <NUM> for communication between a partner network and a service hosted in a WAN connected to the partner network via an Internet, measurement component <NUM> for measuring a plurality of paths to the service within the WAN, control component <NUM> for providing measurements of the plurality of paths to the service to the WAN and receiving a routing rule, and a forwarding component <NUM> for forwarding data packets for the service to the respective border gateway protocol address prefix of the selected device via the Internet, etc..

<FIG> is a schematic diagram of an example of a device <NUM> (e.g., a computing device) for managing connectivity between a partner network and a WAN. The device <NUM> may be implemented as one or more computing devices in the WAN <NUM>. For example, the device <NUM> may be implemented as a server at a data center <NUM>.

In an example, device <NUM> can include a processor <NUM> and/or memory <NUM> configured to execute or store instructions or other parameters related to providing an operating system <NUM>, which can execute one or more applications or processes, such as, but not limited to, the routing controller <NUM> for configuring routing between the partner network <NUM> and a data center <NUM> hosting the service <NUM>. For example, processor <NUM> and memory <NUM> may be separate components communicatively coupled by a bus (e.g., on a motherboard or other portion of a computing device, on an integrated circuit, such as a system on a chip (SoC), etc.), components integrated within one another (e.g., processor <NUM> can include the memory <NUM> as an on-board component), and/or the like. Memory <NUM> may store instructions, parameters, data structures, etc. for use/execution by processor <NUM> to perform functions described herein.

In an example, the routing controller <NUM> includes the advertisement component <NUM>, the path selection component <NUM>, the external rule component <NUM>, and the internal routing component <NUM>.

<FIG> is a flow diagram of an example of a method <NUM> for managing connectivity between a partner network and a WAN. For example, the method <NUM> can be performed by a device <NUM> and/or one or more components thereof to establish paths between the partner network and the WAN, and select a path for traffic for a service hosted in the WAN for the partner network.

At block <NUM>, the method <NUM> includes advertising a plurality of unicast border gateway protocol address prefixes for a plurality of front-end devices of the WAN. In an example, the routing controller <NUM> and/or the advertisement component <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can advertise a plurality of unicast border gateway protocol address prefixes <NUM> for a plurality of front-end devices <NUM> of the WAN <NUM>. In some implementations, each of the plurality of unicast border gateway protocol address prefixes <NUM> identify a respective front-end device <NUM> at a point of presence (POP) or peering connection between the WAN <NUM> and an ISP <NUM>. In some implementations, at sub-block <NUM>, the block <NUM> may optionally include selecting the plurality of front-end devices of the WAN based on a network topology of the WAN.

At block <NUM>, the method <NUM> includes selecting a path within the WAN for a service for users of the partner network, the path including a selected device of the plurality of front-end devices. In an example, the path selection component <NUM> and/or the routing controller <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can select a path <NUM>, <NUM> within the WAN <NUM> for a service <NUM> for users of the partner network <NUM>. The path <NUM>, <NUM> includes a selected device (e.g., front-end device 142a) of the plurality of front-end devices <NUM>.

In some implementations, at sub-block <NUM>, the block <NUM> optionally includes determining measurements of a plurality of paths <NUM>, <NUM> to the service <NUM> within the WAN <NUM> from the agent <NUM>, each of the plurality of paths <NUM>, <NUM> associated with one of the plurality of front-end devices <NUM> of the WAN <NUM>. For example, the path selection component <NUM> may receive the measurements <NUM> of the plurality of paths from the agent <NUM> (e.g., from measurement component <NUM>). As another example, the path selection component <NUM> may receive the measurements of the plurality of paths from an external service such as an application monitoring service. In some implementations, at sub-block <NUM>, the block <NUM> optionally includes receiving a measurement <NUM> for a portion of each of the plurality of paths <NUM>, <NUM> between the partner network <NUM> and the respective front-end device <NUM> and adding a measurement within the WAN from the front-end device <NUM> to the service <NUM>. For example, the path selection component <NUM> may receive the measurements <NUM> of portion of each of the plurality of paths from the agent <NUM> or the external service. In some implementations, selecting the path within the WAN <NUM> is based on a load on the path <NUM>, <NUM>.

At block <NUM>, the method <NUM> includes exporting a routing rule to an agent within the partner network, the routing rule specifying a unicast address prefix for the selected device. In an example, the external rule component <NUM> and/or the routing controller <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can export a routing rule <NUM> to the agent <NUM> within the partner network <NUM>, the routing rule <NUM> specifying a unicast address prefix <NUM> for the selected device. In some implementations, the agent <NUM> within the partner network <NUM> is located at a software defined wide area networking (SD-WAN) appliance <NUM>. In some implementations, the agent <NUM> within the partner network <NUM> is an application controlled by the WAN <NUM>. For example, the application may be executed by a user device <NUM> or a UPF <NUM> under control of the WAN <NUM>.

At block <NUM>, the method <NUM> includes receiving data packets for the service of the partner network at the selected device. In an example, the internal routing component <NUM> and/or the routing controller <NUM>, e.g., in conjunction with processor <NUM>, memory <NUM>, and operating system <NUM>, can configure the front-end device 142a to receive the data packets <NUM> for the service <NUM> of the partner network <NUM> at the selected device 142a. The front-end device 142a may receive the data packets <NUM> for the service <NUM> of the partner network <NUM>. In some implementations, at sub-block <NUM>, the block <NUM> may optionally include decapsulating, at the selected device, packets for the service from packets addressed to the unicast address prefix for the selected device based on the routing rule. In some implementations, at sub-block <NUM>, the block <NUM> may optionally include forwarding the data packets addressed to the unicast address prefix for the selected device to the service based on an original address within the WAN according to the routing rule.

Device <NUM> may further include memory <NUM>, which may be similar to memory <NUM> such as for storing local versions of operating systems (or components thereof) and/or applications being executed by processor <NUM>, such as the routing controller <NUM>, the advertisement component <NUM>, the path selection component <NUM>, the external rule component <NUM>, and the internal routing component <NUM>, etc. Memory <NUM> can include a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof.

Additionally, device <NUM> may include a data store <NUM>, which can be any suitable combination of hardware and/or software, that provides for mass storage of information, databases, and programs employed in connection with aspects described herein. For example, data store <NUM> may be or may include a data repository for operating systems (or components thereof), applications, related parameters, etc. not currently being executed by processor <NUM>. In addition, data store <NUM> may be a data repository for the routing controller <NUM>.

Device <NUM> may additionally include a routing controller <NUM> for managing connectivity between a partner network and a WAN, an advertisement component <NUM> for advertising a plurality of unicast border gateway protocol address prefixes for a plurality of front-end devices of the WAN, a path selection component <NUM> for selecting a path within the WAN for a service for users of the partner network, an external rule component for exporting a routing rule to an agent within the partner network, an internal routing component <NUM> for receiving data packets for the service of the partner network at the selected device, etc..

Accordingly, in one or more aspects, one or more of the functions described may be implemented in hardware, software, firmware, or any combination thereof. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), and floppy disk where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

Claim 1:
An apparatus for communication between a partner network (<NUM>) and a service (<NUM>) hosted in a wide-area network, WAN, (<NUM>) connected to the partner network via an Internet (<NUM>), wherein the apparatus is a software defined wide area networking, SD-WAN, appliance within the partner network, and the partner network is a computer network that connects to the WAN (<NUM>) via the Internet (<NUM>), the apparatus comprising:
a memory (<NUM>) storing one or more instructions for managing routing for the partner network; and
at least one processor (<NUM>) coupled to the memory and configured to execute the instructions, wherein the at least one processor is configured to:
measure (<NUM>) a plurality of paths from the partner network (<NUM>) to the service within the WAN to generate a measurement for each of the plurality of paths, wherein each of the plurality of paths (<NUM>, <NUM>) is associated with one of a plurality of front-end devices (<NUM>) of the WAN that are associated with respective unicast border gateway protocol address prefixes (<NUM>), and wherein the measurements include at least one of latency, delay, throughput, devices traversed, and a trace route indicating one or more networks between the partner network and the WAN (<NUM>);
provide (<NUM>), to the WAN (<NUM>), the measurements (<NUM>) of the plurality of paths to the service;
receive (<NUM>), from the WAN (<NUM>), a routing rule (<NUM>) specifying a unicast address prefix (243a) for a selected front-end device (142a) of the plurality of front-end devices of the WAN (<NUM>), wherein the selected front-end device is associated with a path of the plurality of paths that the WAN has identified as having better characteristics based on the measurements (<NUM>) provided by the SD-WAN appliance; and
forward (<NUM>) data packets for the service to the respective border gateway protocol address prefix of the selected device via the Internet (<NUM>).