Patent ID: 12238004

DETAILED DESCRIPTION

It will be readily understood that the components of the present invention, as generally described and illustrated in the Figures herein, could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of the embodiments of the invention, as represented in the Figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of certain examples of presently contemplated embodiments in accordance with the invention. The presently described embodiments will be best understood by reference to the drawings, wherein like parts are designated by like numerals throughout.

The invention has been developed in response to the present state of the art and, in particular, in response to the problems and needs in the art that have not yet been fully solved by currently available apparatus and methods.

Embodiments in accordance with the present invention may be embodied as an apparatus, method, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “module” or “system.” Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer-usable program code embodied in the medium.

Any combination of one or more computer-usable or computer-readable media may be utilized. For example, a computer-readable medium may include one or more of a portable computer diskette, a hard disk, a random access memory (RAM) device, a read-only memory (ROM) device, an erasable programmable read-only memory (EPROM or Flash memory) device, a portable compact disc read-only memory (CDROM), an optical storage device, and a magnetic storage device. In selected embodiments, a computer-readable medium may comprise any non-transitory medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

Embodiments may also be implemented in cloud computing environments. In this description and the following claims, “cloud computing” may be defined as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned via virtualization and released with minimal management effort or service provider interaction and then scaled accordingly. A cloud model can be composed of various characteristics (e.g., on-demand self-service, broad network access, resource pooling, rapid elasticity, and measured service), service models (e.g., Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“IaaS”)), and deployment models (e.g., private cloud, community cloud, public cloud, and hybrid cloud).

Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++, or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on a computer system as a stand-alone software package, on a stand-alone hardware unit, partly on a remote computer spaced some distance from the computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

The present invention is described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions or code. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

Referring toFIG.1, a network environment may include one or more cloud computing platforms102, such as AMAZON WEB SERVICES (AWS), MICROSOFT AZURE, GOOGLE CLOUD PLATFORM, or the like. As will be discussed below, multiple cloud computing platforms102from multiple providers may be used simultaneously. As known in the art, a cloud computing platform102may be embodied as a set of computing devices coupled to networking hardware and providing virtualized computing and storage resources such that a user may instantiate and execute applications, implement virtual networks, and allocate and access storage without awareness of the underling computing devices and network hardware. Each cloud computing platform102may implement some or all aspects of the cloud computing model described above. One or more of the cloud computing platforms102may be a public cloud providing cloud computing services to multiple entities for a fee. One or more of the cloud computing platforms102may also be a private cloud computing platform built and maintained on a premise of the entity utilizing the private cloud computing platform102. In some implementations, systems and methods described herein may be implemented by a combination of one or more public private cloud computing platforms102and one or more private cloud computing platforms102.

A cloud computing platform102from the same provider may be divided into different regional clouds, each regional cloud including a set of computing devices in or associated with a geographic region and connected by a regional network. These regional clouds may be connected to one another by a cloud backbone network104. The cloud backbone network104may provide high throughput and low latency network connections for traffic among a plurality of regional clouds104a-104c. The cloud backbone network104may include routers, switches, servers and/or other networking components connected by high capacity fiber optic networks, such as transoceanic fiber optic cables, the Internet backbone, or other high-speed network. Each regional cloud104a-104cmay include cloud computing devices and networking hardware located in and/or processing traffic from a particular geographic region, such as a country, state, continent, or other arbitrarily defined geographic region.

A regional cloud104a-104cmay include one or more points of presence (POPs)106a-106c. For example, each regional cloud104a-104cmay include at least one POP106a-106c. A cloud POP106a-106cmay be a physical location hosting physical network hardware that implements an interface with an external network, such as a wide area network (WAN) that is external to the cloud computing platform102. The WAN may, for example, be the Internet108. A WAN may further include a 5G Cellular Network and/or a LONG TERM EVOLUTION (LTE) cellular network.

For example, a high-speed, high-capacity network connection of an Internet service provider (ISP) may connect to the POP106a-106c. For example, the network connection may be a T1 line, leased line, fiber optic cable, Fat Pipe, or other type of network connection. The POP106a-106cmay have a large amount of servers and networking equipment physically at the POP106a-106cenabled to handle network traffic to and from the network connection and possibly providing computing and storage at the POP106a-106c.

The POP106a-106ctherefore enables users to communicate with the cloud computing platform102very efficiently and with low latency. A cloud computing platform102may implement other entrance points from the Internet108in a particular regional cloud104a-104c. However, a POP106a-106cmay be characterized as providing particularly low latency as compared to other entrance points.

Edge clusters110a-110cmay execute throughout a cloud computing platform102. Edge clusters110a-110cmay operate as a cooperative fabric for providing authenticated access to applications and performing other functions as described herein below. Edge clusters110a,110c,110dmay be advantageously hosted at a cloud POP106a-106c. Edge clusters110bmay also be implemented at another location within a cloud computing platform102other than a cloud POP106a-106c. In some instances, one or more edge cluster108emay also execute on customer premise equipment (CPE)112. One or more edge cluster108eon CPE112may be part of a fabric including one or more edge clusters110a-110dexecuting in a cloud computing platform102. Edge clusters110a-110don cloud computing platforms102of different providers may also form a single fabric functioning according to the functions described herein below.

Each edge cluster110a-110emay be implemented as a cluster of cooperating instances of an application. For example, each edge cluster110a-110emay be implemented as a KUBERNETES cluster managed by a KUBERNETES master, such that the cluster includes one or pods, each pod managing one or more containers each executing an application instance implementing an edge cluster110a-110eas described herein below. As known in the art, a KUBERNETES provide a platform for instantiating, recovering, load balancing, scaling up, and scaling down, an application including multiple application instances. Accordingly, the functions of an edge cluster110a-110cas described herein may be implemented by multiple application instances with management and scaling up and scaling down of the number of application instances being managed by a KUBERNETES master or other orchestration platform.

Users of a fabric implemented for an enterprise may connect to the edge clusters110a-110efrom endpoints112a-112d, each endpoint being any of a smartphone, tablet computer, laptop computer, desktop computer, or other computing device. Devices110a-110amay connect to the edge clusters110a-110eby way of the Internet or a local area network (LAN) in the case of an edge cluster hosted on CPE112.

Coordination of the functions of the edge clusters110a-110eto operate as a fabric may be managed by a dashboard114. The dashboard114may provide an interface for configuring the edge clusters110a-110eand monitoring functioning of the edge clusters110a-110e. Edge clusters110a-110emay also communicate directly to one another in order to exchange configuration information and to route traffic through the fabric implemented by the edge clusters110a-110e.

In the following description, the following conventions may be understood: reference to a specific entity (POP106a, edge cluster110a, endpoint112a) shall be understood to be applicable to any other instances of that entity (POPs106b-106c, edge clusters110b-110e, endpoints112b-112d). Likewise, examples referring to interaction between an entity and another entity (e.g., an edge cluster110aand an endpoint112a, an edge cluster110aand another edge cluster110b, etc.) shall be understood to be applicable to any other pair of entities having the same type or types. Unless specifically ascribed to an edge cluster110a-110eor other entity, the entity implementing the systems and methods described herein shall be understood to be the dashboard114and the computing device or cloud computing platform102hosting the dashboard114.

Although a single cloud computing platform102is shown, there may be multiple cloud computing platforms102, each with a cloud backbone network104and one or more regional clouds104a-104c. Edge clusters110a-110emay be instantiated across these multiple cloud computing platforms and communicate with one another to perform cross-platform routing of access requests and implementation of a unified security policy across multiple cloud computing platforms102.

Where multiple cloud computing platforms102are used, a multi-cloud backbone104may be understood to be defined as routing across the cloud backbone networks104of multiple cloud computing platforms102with hops between cloud computing platforms being performed over the Internet108or other WAN that is not part of the cloud computing platforms102. Hops may be made short, e.g., no more than 50 km, in order to reduce latency. As used herein, reference to routing traffic over a cloud backbone network104may be understood to be implementable in the same manner over a multi-cloud backbone as described above.

FIG.2illustrates an example approach for managing application instances200a-200bof an enterprise that are managed by a fabric implemented by a fabric including edge clusters110a,110b. Each edge cluster110a,110bmay be in a different regional cloud104a,104bof a cloud computing platform102, each regional cloud104a,104bbeing connected to the cloud backbone network104of that cloud computing platform102. In the illustrated example, each edge cluster110a,110bexecutes within a cloud POP106a,106bof each regional cloud104a,104b, respectively. In other implementations, one or both of the edge clusters110a,110bis not executing within the cloud POP106a,106bof the regional cloud104a,104bby which it is executed.

Each application instance200a,200bmay have a corresponding presentation layer204a,204b. The presentation layer204a,204bmay, for example, be a web interface by which an interface to the application instance200a,200bis generated and transmitted to a user endpoint112aand by which interactions received from the user endpoint112aare received and processed by the application instance200a,200b. The presentation layer204a,204bmay also be a graphical user interface (GUI) native to a computing platform simulated b the cloud computing platform102, the GUI being transmitted to the endpoint112aand interactions with the GUI being received from the user endpoint112aand submitted to the application instance200a,200bby the GUI. In yet another alternative, the presentation layer204a,204bis a module programmed to communicate with a client executing on the user endpoint112ain order to transmit information to the endpoint112aand receive user interactions from the user endpoint112a.

The edge clusters110a,110bmay act as a gateway to the presentation layers204a,204band provide access to the presentation layers204a,204bonly to authenticated users. An example approach implemented by the edge clusters110a,110bis described below with respect toFIG.3. The edge clusters110a,110bmay be configured by the dashboard114. The dashboard114may incorporate or cooperate with an identity provider (IDP)206such as OKTA, ONELOGIN, CENTRIFY, EMPOWERID, OPTIMAL IDM, BITIUM, LAST PASS, and PINGIDENTITY. Alternatively, the IDP206may be a cloud provider or a vendor providing virtual machines (e.g., VMWARE) within which the edge clusters110a,110band application instances200a,200bare executing.

As shown inFIG.2, application instances200a,200bmay communicate with one another as part of their functionality. In some instances, this communication may be routed by way of the edge clusters110a,110bof the fabric managing the application instances200a,200b.

Referring toFIG.3, the instances of an application200a,200b(hereinafter only application instance200ais discussed) may be instantiated and access thereto controlled using the illustrated method300.

The method300may include receiving302, such as by the dashboard114, an application definition. The application definition may be received from an administrator, a script, or from another source. The application definition may specify an executable of which the application instance200awill be an instance, configuration parameters for an instance of the executable, or other configuration information. The dashboard114may further receive304one or both of a name and a domain in a like manner. The name and/or domain may be according to a DNS (domain name service). As discussed in greater detail below, the dashboard114and edge clusters110a-110emay implement DNS internal to the fabric managed by the edge clusters110a-110e. Accordingly, the DNS may manage mapping of names and domains to actual addresses, e.g. IP addresses, of application instances in one or more cloud computing platforms102and one or more regional clouds104a,104bof one or more cloud computing platforms102.

The method300may include receiving306selection of a cloud computing platform102and possibly selection of a particular regional cloud104a,104b, e.g. California, USA regional cloud for the AWS cloud computing platform102. The selection of step306may be received from an administrator, read from a configuration script, or received from another source. In some implementations, this step is omitted and the dashboard114automatically selects a cloud computing platform102and possibly a regional cloud104a,104b. In yet another alternative, only a cloud computing platform102is selected and the cloud computing platform102automatically selects a regional cloud104a,104b.

The method300may include receiving308a definition of some or all of an IDP206to use for controlling access to the application instance200a, an authentication certificate associated with the application instance200afor use in authenticating users with the application instance200a, and an authentication policy governing access to the application instance200a(e.g., user accounts, groups, or the like that may access the application instance200a). The information of step308may be received from an administrator, read from a configuration script, or received from another source.

The method300may include receiving310access controls. The access controls may be received from an administrator, read from a configuration script, or received from another source. The access controls may include some or all of time-based limitations (times of day, days of the week, etc. during which the application instance200amay be accessed), location-based limitations (locations from which endpoints110a-110dmay access the application instance), a requirement for two-factor authentication, etc., or other type of access control.

The method300may further include invoking312instantiating an instance of the executable specified at step302as the application instance200ain the cloud computing platform102, and possibly regional cloud, specified at step306. For example, the cloud computing platform102may provide an interface for instantiating application instances on virtualized computing resources. Accordingly, step312may include invoking this functionality to cause the cloud computing platform102or other tool to instantiate an application instance200a. In some embodiments, the application instance200aalready exists such that step312is omitted and one or more edge clusters110a-110eare configured to manage access to the application instance200a.

The method300may include discovering314an internet protocol (IP) address of the application instance200a. For example, in response to an instruction to create the application instance200a, the cloud computing platform102may create the application instance200aand assign an IP address to the application instance200a. The cloud computing platform102may then return the IP address to the entity that requested instantiation, which may be the dashboard114in the illustrated example.

The method300may further include the dashboard114configuring316one or more edge clusters110a-110eof the fabric to manage access to the application instance200a. This may include storing a link between the name and/or domain from step304with the IP address from step314by the DNS. In some embodiments, the name and/or domain from step304may be distributed to endpoints110a-110ewhereas the IP address is not. Accordingly, the edge clusters110a-110emay function as DNS servers and further control access to the application instance200aby refraining from forwarding traffic to the IP address until a source of the traffic has been properly authenticated.

Step316may further include configuring one or more edge clusters110a-110eof the fabric according to the authentication requirements of step308and the access controls of step310. For example, one or more edge clusters110a-110emay be programmed to condition allowance of a request to access the application instance200aon some or all of (a) receiving confirmation from the specified IDP206that a source of the request is authenticated, (b) verifying a certificate submitted with the request according to a certificate received at step308, (c) verifying that the request was received according to the access controls of step310(from a permitted location, at a permitted time, etc.).

An edge cluster110aconfigured as described above with respect to step316may receive318a request to access the application instance200afrom an endpoint112a, the request including the name and/or domain of the application instance200a.

The edge cluster110amay perform320authentication of the request and/or the endpoint112a. This may include instructing the endpoint112ato authenticate with the IDP206and receiving verification from the IDP206that the endpoint112ais authenticated, such as authenticated for a particular user identifier. Step320may include authentication by another approach such as verification of a user identifier and password, verification of a certificate, or other authentication means.

If authentication is not successful at step320, the remainder of the steps of the method300may be omitted and the request may be ignored, recorded, flagged as potentially malicious, or subject to other remedial action.

In response to successful authentication at step320, the edge cluster110amay resolve322the name and/or domain of the request to the IP address mapped to it at step316and connect324the user endpoint112ato the application instance. Connection may include establishing a network connection to the application instance200a. Th edge cluster110amay implement network address translation (NAT) such that the IP address is not disclosed to the user endpoint112a. Accordingly, a different IP address, such as the address of the edge cluster110a, may be used as the destination of traffic sent by the user endpoint112aand the edge cluster110amay route the traffic to the IP address of the application instance200ausing NAT and forward the traffic to the application instance200a.

In some embodiments, the edge cluster110amay monitor326activities of the user endpoint112awith respect to the application instance200aand block further access in response to suspicious activity. Examples of suspicious activity may include access patterns that are different from past access by the endpoint112a: access from a different country, a different time of day, an unusually high volume of traffic, or the like. The edge cluster110amay therefore compile information of typical access patterns for the edge cluster110ain order to detect anomalous access patterns.

FIG.4illustrates a system400that may be implemented by a fabric of edge clusters110a-110cin a network environment, such as the network environment100. In the illustrated system, an intelligent routing module402programs cloud DNS404of a cloud computing platform102. The intelligent routing module402may be a component within the dashboard114or managed in response to user instructions input to the dashboard114. The cloud DNS404may control the routing of traffic received from the user endpoint112aamong various ingress points408a-408cof the cloud computing platform102. The ingress points408a-408cmay include ingress points to different regional clouds and/or different ingress points to the same regional cloud.

A user endpoint112amay transmit a request to a cloud computing platform102over the Internet108. The request may be a request to access a resource name, such as in the form of a URL including a domain name and possibly one or more other items of information, such as a sub-domain, computer name, and possibly one or more other items of identifying information of a requested resource. The resource name may reference an application instance200aand may include a name and domain configured for the application instance200aas described above with respect to step304of the method300.

The cloud DNS404may receive the request and resolve the resource name to an address, such as an IP address, assigned to one or more of the edge clusters110a-110cimplementing a fabric. The resolution by the cloud DNS404may be according to programming of the cloud DNS404by the intelligent routing module402. Accordingly, a resource name may be associated by the intelligent routing module402to any edge cluster110a-110eof a fabric. The cloud DNS404may implement Anycast DNS whereby the routing of a request is conditioned on a location of the user endpoint112athat issued the request.

In some implementations, an edge cluster110a-110cof a fabric may implement alternative routing logic406. A request received by an edge cluster110amay be evaluated according to the alternative routing logic406, which may instruct the edge cluster110ato instruct the endpoint112athat generated the request to resubmit the request to a different edge cluster110b. For example, the alternative routing logic406may transmit alternative service (“Alt-Svc”) messages according to hypertext transport protocol (HTTP). In some implementations, the cloud DNS404may be incapable of fine-grained routing of requests. For example, there may be edge clusters110a-110cat various geographic locations in a regional cloud whereas the cloud DNS404only enables a user to perform geographic name resolution to a single address within each regional cloud. Accordingly, the intelligent routing module402may program the cloud DNS to route requests to an edge cluster110ain a regional cloud. The intelligent routing module402may further configure the alternative routing logic406of that edge cluster110ato evaluate the location of user endpoints112aand route requests from that user endpoint112ato another edge cluster110bin that regional cloud. For example, edge cluster110bmay be closer to the user endpoint112athen the edge cluster110a.

The system400may be used to perform arbitrary routing of traffic between a user endpoint112aand any of the edge clusters110a-110c. Various applications of the system400are described herein below.

For example, an edge cluster110a-110emay be associated with the name and/or domain assigned to the application instance200ain the cloud DNS404and/or alternative routing logic406such that requests addressed to the name and/or domain of the application instance200awill be routed according to the static IP address or Anycast IP address of associated with the edge cluster110a-110efor the application instance200a. In another example, a request to resolve the name and/or domain of an application instance200amay be resolved by the cloud DNS404and or alternative routing logic406to an IP address that may be a static IP address of a particular edge cluster110a-110eor an Anycast IP address that could be resolved to one of multiple edge clusters110a-110e.

The source of the resolution request may then transmit a request to the IP address returned to it, with the request being routed according to functionality associated with that IP address (static routing or Anycast routing).

FIG.5illustrates a method500of performing routing using the system400. The method500may be performed by the intelligent routing module402and/or dashboard114.

The method500may include monitoring502ingress locations. This may include tracking ingress locations408a-408cof a cloud computing platform102at which requests from user endpoints112a-112dare received. Monitoring502may include compiling statistics such as a frequency of requests for a given ingress points408a-408c(requests per hour, minute, or other time interval) over time. The ingress point408a-408cof requests may be detected due to reporting by the cloud computing platform102, by the edge cluster110a-110dthat received a request recording an ingress point408a-408cthrough which the request was received, or by some other means.

The method500may further include monitoring504the locations of user endpoints112a-112dfrom which requests were received. The location of an endpoint112a-112dat a time of generation of a request may be obtained by: inferring a location from a source IP address of the request, reading the location from a header included in the request, reading the location from an explicitly provided location value provided by the endpoint112a-112dwithin the request. Monitoring504the locations may include some or all of compiling statistics for each location represented in received requests at varying degrees of specificity: requests from a country, from each state or province within the country, from each metropolitan area within the country, within each city within the country, etc. Statistics may be in the form of a frequency of requests (requests per day, hour, minute, or other time window) over time.

The method500may include configuring506the cloud DNS404and/or configuring508alternative routing logic406according to the data obtained from the monitoring steps502,504. Example approaches for configuring routing of requests for a fabric of edge cluster110a-110eaccording to usage data are described below with respect toFIGS.6through10B.

The method500may include receiving510an original request from user endpoint112ato resolve a name and/or domain of the application instance200a. The original request may be a domain resolution request or a request to access the application instance200aincluding the name and/or domain. The original request may be received by the cloud DNS404. In response to the programming of step506, the cloud DNS404resolves512the name and/or domain to an IP address of an edge cluster, e.g., edge cluster110a. The user endpoint112amay receive this IP address from the cloud DNS404and transmit a second request to access the application instance200ato the IP address of the edge cluster110a. Alternatively, the cloud DNS404may forward the original request to the IP address.

Resolving512the domain name to an IP address may include using any of the approaches described above with respect toFIG.4. These may include resolving the IP address to an Anycast IP address, resolving the IP address using geographic domain name service (GeoDNS) to a static or Anycast IP address, or resolving of the IP address to an Anycast IP address or static IP address followed by using alternative routing logic to redirect a request to an alternative edge cluster110a-110e.

The edge cluster110areceives514the request to access the application instance200a(the original request forwarded by cloud DNS404or the second request from the user endpoint112a). The edge cluster110amay evaluated516whether there is alternative routing logic406applicable to the request. For example, the alternative routing logic may map a routing rule to one or both of the application instance200aand one or more locations of user end points. Accordingly, step516may include determining whether the location of the user endpoint112aand/or application instance200aare referenced by a routing rule and if not, facilitates application access518through the edge cluster110a. This may include routing traffic through an ingress point408a-408cof the cloud computing platform associated with the edge cluster110a, e.g. an ingress point408a-408cdetermined according to programming of the cloud DNS404. If so, the method500may include the edge cluster110aforwarding520the request to a second IP address, e.g. the IP address of a second edge cluster110bhaving a different ingress location to the cloud computing platform in the same or different regional cloud. Redirecting may include one or both of the edge cluster110aforwarding the request to the second edge cluster110band the edge cluster110atransmitting the second IP address of the second edge cluster110bto the user endpoint112awith an instruction to access the application instance200aat the second IP address. The user endpoint112amay thereafter perform522application access (e.g., send access requests to and receive responses from the application instance200a) through an ingress408a-408ccorresponding to the second IP address, such has an ingress location408a-408cthat is physically closest to a computing device executing the second edge cluster112b. Selection of the ingress location408a-408cfor a given IP address may be performed by the cloud DNS404or by other routing logic. For example, traffic addressed to the IP address may be routed by the Internet108to the ingress location408a-408caccording to DNS information provided to routing devices of the Internet108by the cloud computing platform102.

Referring toFIGS.6and7, routing of requests, such as using the DNS system400, may be performed to take into the account latency and cost. Referring specifically toFIG.6, routing options may be grouped into “lanes,” including a cost effective lane, fast lane, and performance lane. The cost effective lane avoids ingress locations at cloud POPs106a-106cand routing of traffic over the cloud backbone104inasmuch as there may be additional charges for such usage. The cost effective lane may reduce at the expense of higher latency. The fast lane may include an ingress location at a cloud POP106a-106c(e.g., the cloud POP106a-106cclosest to the user endpoint112agenerating a request) with intra-cloud traffic being routed over the cloud backbone104. The fast lane may provide reduced latency at increased cost from utilization of the POPs106a-106cand cloud backbone104. The performance lane may provide an intermediate level of latency and cost by using an ingress location other than a cloud POP106a-106cwhile still routing intra cloud traffic over the cloud backbone104.

The lane used may be a user-configurable parameter. For example, a particular application instance200amay be assigned to a lane such that the intelligent routing module402will program the cloud DNS404and/or alternative routing logic406to route requests to that application instance200aaccording to that lane. Application instances may be assigned to lanes individually, as a group (e.g., all instances of the same executable). Lanes may be additionally or alternatively be assigned to users or groups of users. For example, all requests from a user or group may be routed according to a particular lane or a combination. In another example, a particular combination of user and application instance may be assigned to a particular lane.

Referring toFIG.7, the illustrated method700may be used by the intelligent routing module402to implement the three lanes, or other number of lanes. If the lane for a user and/or application instance is found702to be the cost effective lane, the intelligent routing module402configures704the fabric to bypass cloud POPs106a-106cand the cloud backbone104. For example, for an application instance200ain a first regional cloud, requests to access the application instance200amay be routed over the Internet108to an ingress point that is not a POP106a-106c, including requests that are closer to a second regional cloud than to the first regional cloud. This configuration may include assigning an edge cluster110ain the first regional cloud a static IP address that is not an Anycast IP address. In this manner, traffic addressed to the application instance will be routed to the static IP address over the Internet108rather than through a cloud POP106a-106cor the cloud backbone104. For example, step704may include programming GeoDNS of a cloud computing platform102to resolve a domain name to a static IP address for a given location of a user endpoint112a-112dthat results in bypass of POPs106a-106cof the cloud computing platform102.

If the lane for a user and/or application instance200ais found706to be the fast lane, the intelligent routing module402may configure708the fabric such that ingress is performed at a cloud POP106a-106cwith use of the cloud backbone for intra-cloud traffic. This may include associating the name and/or domain of the application instance with an edge cluster110alocated within a cloud POP106a-106c. The edge cluster110amay be assigned an Anycast IP address in the cloud DNS404. In this manner, traffic from user endpoints112a-112dlocated nearer to a different regional cloud than that hosting the edge cluster110awould be routed to a nearest POP106a-106cand then over the cloud backbone104to the POP106a-106chosting the edge cluster110a. User endpoints112a-112dlocated nearer to the same regional cloud hosting the edge cluster110athan other regional clouds of the cloud computing platform, may be routed over the Internet108to the POP106a-106chosting the edge cluster110a.

If the lane for a user and/or application instance200ais the performance lane, the intelligent routing module402may configure710the fabric such that ingress is performed at a cloud POP106a-106cwithout use of the cloud backbone for intra-cloud traffic. This may include associating the name and/or domain of the application instance with an edge cluster110alocated within a cloud POP106a-106c. The edge cluster110amay be assigned a static IP address (not Anycast) in the cloud DNS404. The static IP address may be resolved from a domain name of a request according to the location of a user endpoint112a-112dthat generated the request. The resolution to the static IP address according to user endpoint112a-112dlocation may be programmed into the GeoDNS of the cloud computing platform102.

In this manner, traffic from user endpoints112a-112dlocated nearer to a different regional cloud than that hosting the edge cluster110awould be routed over the Internet108to the POP106a-106chosting the edge cluster110arather than over the cloud backbone104. User endpoints112a-112dlocated nearer to the same regional cloud hosting the edge cluster110athan other regional clouds of the cloud computing platform, may be routed over the Internet108to the POP106a-106chosting the edge cluster110a.

Note that in some instances, the benefit of one of the three lanes relative to another may be small. Accordingly, in some embodiments, a user preference may be overridden and substituted for a lower cost option when this occurs. For example, if a measured or estimated (see estimation techniques described below with respect toFIGS.12and13) latency of a user endpoint112a-112ewith respect to an application instance200for one lane is within a threshold difference (e.g., a predefined number of milliseconds) of the latency for a second lane and the second lane has lower cost, the second lane may be substituted for routing traffic between the user endpoint112a-112e.

FIGS.8,9A, and9Billustrate an approach for routing traffic for an application instance200ato edge clusters110a-110eof a fabric while taking into account cacheability of content provided by that application instance200a.

For example, a method800may include monitoring802application access locations of user endpoints112a-112eaccessing the application instance200a. The location of an endpoint112a-112dat a time of generation of a request may be obtained by: inferring a location from a source IP address of the request, reading the location from a header included in the request, reading the location from an explicitly provided location value provided by the endpoint112a-112dwithin the request. Monitoring802the locations may include some or all of compiling statistics for each location represented in received requests at varying degrees of specificity: requests from a country, from each state or province within the country, from each metropolitan area within the country, within each city within the country, etc. Statistics may be in the form of a frequency of requests (requests per day, hour, minute, or other time window) over time.

The method800may further include monitoring804data read and write patterns804. This may include monitoring a cache for the application instance. Monitoring read and write patterns804may include monitoring a rate at which entries in a cache are overwritten or marked as invalid by the application200a. Monitoring read and write patterns804may include inspecting requests and compiling statistics regarding the number of read requests and write requests, e.g. a number of write requests within a time window (e.g., every day, hour, minute, etc.) and a number of read requests within the time window sampled periodically over time. Step804may include calculating a ratio of these values over time, e.g., a ratio of reads per writes over time or within a time window preceding a time of calculation of the ratio.

The method800may include characterizing806the cacheability of the application. This may include evaluating such factors as the ratio of reads per writes (a higher ratio of reads means higher cacheability) and labeling of data provided by the application in response to requests (e.g., whether the data is flagged as cacheable, a time to live (TTL) of the data). A cacheability score may be calculated as a function of these factors (a sum, weighted sum, etc.) and compared to one or more thresholds. For example, if the cacheability score is found808to be above a first threshold (highly cacheable), the intelligent routing module402may program the cloud DNS404and intelligent routing module to route access to the application200athrough a plurality of edge clusters110a-110e. For example, the name and/or domain of the application200amay be mapped to an Anycast IP address associated with the plurality of edge clusters110a-110e. Accordingly, requests from each user endpoint112a-112dwill be routed to the edge cluster110a-110eclosest to it, which will have a high likelihood of having requested data to the cacheability of the application instance200a.

In some embodiments, if the cacheability is found812to be below the first threshold but above a second threshold, step814is performed, which may be the same as step810but for a reduced number of edge clusters110a-110e. For example, the set of edge clusters110a-110eassociated with the Anycast IP address may be limited to those closest to the application instance200arelative to those edge clusters110a-110ethat are excluded. In some embodiments, a single threshold is used such that steps812and814are not performed.

If the cacheability is not found to meet a threshold condition (below the first threshold or below multiple thresholds), then the method800may include the intelligent routing module402configuring816the cloud DNS404and/or alternative routing logic406such that traffic from each user endpoint112a-112eand addressed to the application instance200awill be routed to a single edge cluster110a, e.g. the edge cluster110aclosest to the application instance200aor at least in the same regional cloud or the same POP106a-106cas the application instance200a. This routing may be according to any of the three lanes described above (cost effective, fast lane, performance) such that traffic may be routed through POPs106a-106cand the cloud back bone104(fast lane), through a POP106aclosest to the edge cluster110abut not the cloud backbone104(performance), or through an ingress location without using a POP106aor the cloud backbone104(cost effective).

For example, to achieve the fast lane, the cloud DNS404may be configured such that edge cluster110ais the only edge cluster associated with an Anycast IP address. Accordingly, all requests addressed to that IP address will be routed by the cloud DNS404through a POP106a-106cclosest to the source of the request and through the cloud backbone104to the edge cluster110a.

FIG.9Aillustrates the case900of a highly cacheable application instance200athat is located close to edge cluster110d(e.g., same POP106a-106cor same regional cloud). A plurality of edge clusters110a-110dmay include caches902a-902d. Data from responses to requests transmitted from the application instance904may be cached in the caches902a-902d. The manner in which data is cached, cache hits are identified, and the caches902a-902dare maintained may be according to any approach known in the art for implementing a cache, such as approaches for caching responses to HTTP content.

A fabric DNS906may be a combination of the cloud DNS404, the intelligent routing module402, and any alternate routing logic relating to access of the application instance200aas described above. As is apparent the fabric DNS906in the highly cacheable case is configured to route requests to a plurality of edge clusters110a-110e, such as to the edge cluster110a-110enearest to the endpoint112a-112dthat originated the request. Accordingly, if a response to the request is cached, that nearest edge cluster110a-110emay provide the response without waiting for the application instance200a.

FIG.9Billustrates a non-cacheable case in which the fabric DNS906is configured to route requests directly to the edge cluster110e, such as the edge cluster110enearest to the application instance200a.FIG.9Billustrates the case where traffic is routed to edge cluster110eover the Internet108, i.e., the cost effective lane. In other instances, the traffic could be routed over the cloud backbone104to implement the fast lane.

FIG.10illustrates an approach for implementing the backup and restoration of a application instance200aand an application instance200b. The application instances200a,200bmay be instances of the same executable image or otherwise be capable of performing the same functions. The application instance200bmay be a backup (i.e., secondary application instance) to the application instance200a(i.e., primary application instance). In the following description, a single secondary application instance is described with the understanding that there may be multiple secondary application instances that may be monitored and used for restoration in the same manner.

To reduce the risk of failure of a regional cloud104a,104bor access to a regional cloud104b, the application instance200amay be located within a regional cloud104aof a CSP102awhile the application instance200bmay be located within a different regional cloud104bof either the same CSP102aor a different CPS102b.

Each application instance200a,200bmay be accessible by a single edge cluster110aor different edge clusters110a,110b, respectively, such as edge clusters110a,110bhosted by the same regional clouds104a,104bas the application instances200a,200b. Each regional cloud104a,104bmay implement a cloud DNS404a,404bperforming some or all of the functions of a cloud DNS404as described above.

A health monitor1000may be executed with respect to the application instances200a,200b. The health monitor1000may be used to detect unavailability of the application instances200a,200bdue to failure of the application instances200a,200bthemselves or failure of a regional cloud104a,104b, could service provider102a,102b, or a network used to access the regional clouds104a,104b. Accordingly, the health monitor1000may advantageously execute on a computer system that is separate from any regional cloud104a,104band/or cloud service provider102a,102bhosting the application instances200a,200b, such as on-premise computing equipment of an entity controlling the application instances200a,200bor a different cloud service provider. The health monitor1000, CSP102a, and CSP102bmay be connected to one another by a wide area network (WAN), such as the Internet or other network.

The health monitor1000may detect failure of application instance200athat is currently the primary application instance and manage failover to the current secondary application instance200baccording to the methods described below. To that end, the health monitor1000may interact with the intelligent routing module402in order to redirect traffic from the application instance200ato the application instance200bas the new primary application instance as described below. A user may view the current status of the application instances200a,200bas well as health monitoring statistics (e.g., latency, results of health checks, etc.) using the dashboard114. In particular, each time the status of the application instances200a,200bchanges (unavailable, switched to primary, switched to secondary), this information may be presented on the dashboard114.

In some instances, the health monitor1000may test accessibility of the application instances200a,200bfrom outside of the regional clouds104a,104band/or the CSPs102a,102b. Accordingly, the health monitor1000may interact with a component, such as another edge cluster110cexecuting in a third CSP102cconnected to the WAN and/or third regional cloud104c. The health monitor1000may performance of tests with respect to the application instances200a,200bby the edge cluster110c, such as pings, measuring latency, performing health checks, or other checks.

Referring toFIG.11, the illustrated method1100may be executed in order to configure a backup and restore relationship between the application instance200aand the application instance200b. The method1100may be executed under the control of the dashboard114or some other component executing in any of the cloud computing platforms102a,102b,102cor a different computing device.

The method1100includes instantiating1102the application instance200ain the regional cloud104aof the CSP102aand instantiating1104the application instance200bin the regional cloud104bof the CSP102b. As noted above, each application instance200a,200bmay be an instance of the same executable.

The method1100may include configuring1106the health monitor1000with a backup relationship between the application instances200a,200b. For example, the backup relationship may be represented by a data structure including some or all of identifiers of the application instances200a,200b, IP addresses of the application instances200a,200b, and an indication that the application instance200ais currently primary and that the application instance200bis secondary. The data structure may further identify the CSP102a,102band regional clouds104a,104bhosting each application instance200a,200b, respectively.

The method1100may further include associating1108the application instance200awith the intelligent routing module402as the DNS server for the application instance200. Likewise, step1108includes associating the application instance200bwith the intelligent routing module402as the DNS server for the primary application instance200. For example, step1108may include associating a domain name assigned to each application instance200a,200bwith the intelligent routing module402. Accordingly, requests to resolve the domain names of the application instances200a,200bwill be resolved by the intelligent routing module402. For example, the intelligent routing module402may resolve the requests to resolve the domain names of the application instances200a,200busing the approach described above with respect toFIGS.3to5with further modification as described in greater detail below with respect toFIG.13.

FIG.12illustrates a method1200for monitoring the application instances200a,200b. The method1200may be performed under control of the health monitor1000in response to backup relationship being created between the application instances200a,200baccording to the method1100.

The method1200may include monitoring1202reachability of one or more edge clusters110a,110bthrough which the application instances200a,200bmay be accessed. Monitoring1202reachability may include sending a ping or other message to the edge clusters from an edge cluster110cor other component that is external to one or both of (a) any of the regional clouds104a,104bhosting the application instances200a,200band (b) any of the CSPs102a,102bhosting the application instances200a,200b. Step1202may include monitoring accessibility between multiple locations and the regional clouds104a,104band/or CSPs102a,102b.

The method1200may further include deriving1204health data from control and/or data traffic at any edge clusters110a,110bthrough which the current primary application instance200ais accessed. The current secondary application instance200bmay remain unused and inaccessible while the application instance200ais the primary application instance. Accordingly, step1204may be performed without involvement of the current secondary application instance200b. Alternatively, simulated data traffic and/or control traffic may be generated by the health monitor1000or other component and transmitted to the current secondary application instance200bin order to assess functionality of the application instance200band any edge cluster110bthrough which the application200bwould be accessible if made primary.

Health data may be derived from the data and/or control traffic by detecting signs of failure, such as error messages or timing out of requests. The health data may be derived in the form of statistics for the data and/or control traffic, e.g., latency between forwarding traffic from the edge cluster110ato the current primary application instance200aand the time that responses to the traffic are received, a number of packets per unit time that are sent and received over each network connection to the current primary application instance200aby way of the edge cluster110a, or other metric.

The method1200may include monitoring1206reachability of each application instance200a,200b. Inasmuch as reachability of the edge clusters110a,110bthrough which the application instances200a,200bare accessed may be independently assessed at step1204, step1206may include monitoring accessibility between an edge cluster110a,110bthrough which the application instance200a,200baccessed by way of the edge cluster110a,110b. Accordingly, step1206may include sending a ping or other request from the edge cluster110a,110bto the application instance200a,200b, respectively, receiving a response to the ping or other request, and forwarding the response (or data derived therefrom) to the health monitor1000. In some embodiments, the health monitor1000may generate the ping or other request and send the ping or other request to the application instance200a,200bby way of the edge cluster110a,110bthrough which the application instance200a,200bis accessed and receive any response through this same path.

The method1200may include directly monitoring1208health and latency of the application instances200a,200b. For example, step1208may include receiving, by the health monitor1000, the result of tests of one or both of the health and latency of the application instances200a,200bas performed by the application instances200a,200b. For example, step1208may include receiving log data generated by the application instances200a,200bthemselves. In some embodiments, the health monitor1000may transmit instructions to the application instances200a,200bto perform tests of one or both of health and latency and return the result to the health monitor1000. In some embodiments, the edge cluster110a,110bthrough an application instance200a,200bis accessed may transmit instructions to the application instances200a,200bto perform tests of one or both of health and latency and return the result to the health monitor1000directly or by way of the edge cluster110a,110bforwarding the result of the health and/or latency check to the health monitor100.

FIG.13illustrates a method1300that may be executed by the health monitor1000. The method1300may include configuring1302the CSP102ahosting the current primary application instance200a, e.g., the cloud DNS404a, according to the backup relationship received according to the method1100. Step1302may be performed by way of the intelligent routing module402.

Step1302may include configuring a geo DNS of the cloud DNS404ato resolve requests to resolve the domain name of the current primary application instance200ato an IP address of the edge cluster110athrough which the current primary application instance200ais accessed as described above with respect toFIG.5.

Step1302may additionally or alternatively include configuring an Anycast IP address, which may be the IP address to which the domain name of the current primary application instance200a(which may be the same as that of the current secondary application instance200a) is resolved. For example, a same Anycast IP address may be associated with both of the edge clusters110a,110bor only whichever of the edge clusters110a,110bis used to access the current primary application instance.

The traffic addressed to the Anycast IP may be routed by a given CSP102a,102bto other edge clusters based on the location of a source of traffic without regard to the functionality described below with respect to the edge clusters110a,110b. For example, for a given geographic region, traffic from the region and addressed to the Anycast IP address may be routed to one of the edge clusters110a,110bas described below. Traffic from other geographic regions addressed to the same Anycast IP address may be routed to different edge clusters without regard to the current routing for the edge clusters110a,110b.

The edge clusters110amay be configured to route traffic addressed to the Anycast IP address to the application instances200aas described above with respect toFIG.5, which may include forwarding the traffic to a second IP address assigned to the current primary application instance200a. The cloud DNS404amay be configured to route traffic addressed to the Anycast IP address to the edge cluster110a. The cloud DNS404bof the CSP102bmay also be configured to route traffic addressed to the Anycast IP address to the edge cluster110a. Alternatively, the cloud DNS404bof the CSP102bmay be configured to route traffic addressed to the Anycast IP address to the edge cluster110b, which then forwards the traffic to the edge cluster110a, such as using alternative routing logic as described above.

Note that if the edge clusters110a,110bare in the same CSP102a, the cloud DNS404bmay be configured to route traffic addressed to the Anycast IP address to the edge cluster110arather than to the edge cluster110b. In some instances, step1302may include configuring the cloud DNS404bof multiple regional clouds104aof the same CSP102amay be configured to route traffic addressed to the Anycast IP address to the edge cluster110arather than to the edge cluster110b.

The method1300may include receiving, by the cloud DNS404aand/or cloud DNS404b, a request to resolve the domain name of the primary application instance200a. The cloud DNS404aand/or cloud DNS404bmay resolve1304the domain name to the Anycast IP address described above. Thereafter, the source of the request to resolve the domain name may transmit traffic addressed to the Anycast IP address, which is then routed by the CSP102ato the edge cluster110aand by the edge cluster110ato the primary application instance200a. The application200awill then process the traffic and transmit any return traffic to the source of the traffic, such as by way of the edge cluster110a.

As the application instance200aprocesses the traffic, the health monitor1000may monitor the health and accessibility of the application instance200aand possibly the application instance200b, such as according to the method1200described above.

The method1300may include evaluating1306whether failure is detected. Failure may be detected in any of the monitoring steps of the method1200. For example, step1306may include detecting that the edge cluster110ais unreachable at step1202, determining that the control and/or data traffic at the edge cluster110aindicates an error at step1204, determining that the current primary application instance200ais not reachable at step1206, determining that the health checks and/or latency from step1208indicate an error, or some other indication of failure of unreachability of the application instance200a.

Step1306may include both determining that the current primary application instance200ahas failed or is inaccessible and that the current secondary application instance200bremains functional and accessible. Where there are multiple secondary application instances200b, step1306may include determining that at least one of the secondary application instances200bis available and functioning and selecting one of the accessible and functioning secondary application instances200b.

If the condition of step1306is met, the method1300may include configuring1308routing logic one or both of the CSPs102a,102bsuch that traffic addressed to the Anycast IP address (see step1304) will be routed to the edge cluster110b, which will then forward the traffic to the new primary application instance200b. Step1308may include configuring the cloud DNS404a,404bof one or both CSPs102a,102bor otherwise configuring the routing logic of the CSPs102a,102b. Step1308may include configuring the routing logic of one or both of the CSPs102a,102bto disable or otherwise prevent routing of traffic addressed to the Anycast IP address to the edge cluster110a. Alternatively, alternative routing logic of the edge cluster110amay be configured to route traffic addressed to the Anycast IP address to the edge cluster110bfor forwarding to the new primary application instance200b.

The method1300may further include updating1310the backup relationship recorded by the health monitor1000. For example, step1310may include updating the backup relationship to indicate that the application instance200bis now the primary instance and the status of application instance200a, i.e., how the application instance is or is not accessible (e.g., CSP not accessible, application instance200afailed, etc.) and the current health and/or latency of the application instance200a.

Following steps1308and1310, the application instance200bis the primary application instance. The application instance200bwill then process the traffic addressed to the Anycast IP and transmit any return traffic to the source of the traffic, such as by way of the edge cluster110b. Subsequent to step1308, the methods described herein, including the methods1100,1200, and1300may be performed with the application instance200band edge cluster110btaking the place of the application instance200aand the edge cluster110a. When the application instance200abecomes accessible and functional, the application instance200amay function as a secondary application instance accessible through edge cluster110bas described above.

In some embodiments, when the original primary application instance200abecomes accessible and functional, the health monitor1000may invoke transitioning back to the application instance200afunctioning as the primary instance and the application instance200bfunctioning as the secondary instance. Transitioning may include configuring the routing logic of one or more CSPs as described with respect to step1302as well as updating the backup relationship of the health monitor1000to indicate the current roles of the application instances200a,200b. Where there are multiple secondary application instances200b, those secondary application instances200bthat are not selected to become the primary instance may continue to function as secondary application instances following step1308.

FIG.14illustrates an example computing device1400that may be used to implement a cloud computing platform or any other computing devices described above. In particular, components described above as being a computer or a computing device may have some or all of the attributes of the computing device1400ofFIG.14.FIG.14is a block diagram illustrating an example computing device1400which can be used to implement the systems and methods disclosed herein.

Computing device1400includes one or more processor(s)1402, one or more memory device(s)1404, one or more interface(s)1406, one or more mass storage device(s)1408, one or more Input/Output (I/O) device(s)1410, and a display device1430all of which are coupled to a bus1412. Processor(s)1402include one or more processors or controllers that execute instructions stored in memory device(s)1404and/or mass storage device(s)1408. Processor(s)1402may also include various types of computer-readable media, such as cache memory.

Memory device(s)1404include various computer-readable media, such as volatile memory (e.g., random access memory (RAM)1414) and/or nonvolatile memory (e.g., read-only memory (ROM)1416). Memory device(s)1404may also include rewritable ROM, such as Flash memory.

Mass storage device(s)1408include various computer readable media, such as magnetic tapes, magnetic disks, optical disks, solid-state memory (e.g., Flash memory), and so forth. As shown inFIG.14, a particular mass storage device is a hard disk drive1424. Various drives may also be included in mass storage device(s)1408to enable reading from and/or writing to the various computer readable media. Mass storage device(s)1408include removable media1426and/or non-removable media.

I/O device(s)1410include various devices that allow data and/or other information to be input to or retrieved from computing device1400. Example I/O device(s)1410include cursor control devices, keyboards, keypads, microphones, monitors or other display devices, speakers, printers, network interface cards, modems, lenses, CCDs or other image capture devices, and the like.

Display device1430includes any type of device capable of displaying information to one or more users of computing device1400. Examples of display device1430include a monitor, display terminal, video projection device, and the like.

Interface(s)1406include various interfaces that allow computing device1400to interact with other systems, devices, or computing environments. Example interface(s)1406include any number of different network interfaces1420, such as interfaces to local area networks (LANs), wide area networks (WANs), wireless networks, and the Internet. Other interface(s) include user interface1418and peripheral device interface1422. The interface(s)1406may also include one or more user interface elements1418. The interface(s)1406may also include one or more peripheral interfaces such as interfaces for printers, pointing devices (mice, track pad, etc.), keyboards, and the like.

Bus1412allows processor(s)1402, memory device(s)1404, interface(s)1406, mass storage device(s)1408, and I/O device(s)1410to communicate with one another, as well as other devices or components coupled to bus1412. Bus1412represents one or more of several types of bus structures, such as a system bus, PCI bus, IEEE 1394 bus, USB bus, and so forth.

For purposes of illustration, programs and other executable program components are shown herein as discrete blocks, although it is understood that such programs and components may reside at various times in different storage components of computing device1400, and are executed by processor(s)1402. Alternatively, the systems and procedures described herein can be implemented in hardware, or a combination of hardware, software, and/or firmware. For example, one or more application specific integrated circuits (ASICs) can be programmed to carry out one or more of the systems and procedures described herein.

In the above disclosure, reference has been made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific implementations in which the disclosure may be practiced. It is understood that other implementations may be utilized and structural changes may be made without departing from the scope of the present disclosure. References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

Implementations of the systems, devices, and methods disclosed herein may comprise or utilize a special purpose or general-purpose computer including computer hardware, such as, for example, one or more processors and system memory, as discussed herein. Implementations within the scope of the present disclosure may also include physical and other computer-readable media for carrying or storing computer-executable instructions and/or data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. Computer-readable media that store computer-executable instructions are computer storage media (devices). Computer-readable media that carry computer-executable instructions are transmission media. Thus, by way of example, and not limitation, implementations of the disclosure can comprise at least two distinctly different kinds of computer-readable media: computer storage media (devices) and transmission media.

Computer storage media (devices) includes RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM), Flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer.

An implementation of the devices, systems, and methods disclosed herein may communicate over a computer network. A “network” is defined as one or more data links that enable the transport of electronic data between computer systems and/or modules and/or other electronic devices. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a computer, the computer properly views the connection as a transmission medium. Transmissions media can include a network and/or data links, which can be used to carry desired program code means in the form of computer-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer. Combinations of the above should also be included within the scope of computer-readable media.

Computer-executable instructions comprise, for example, instructions and data which, when executed at a processor, cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. The computer executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts are disclosed as example forms of implementing the claims.

Those skilled in the art will appreciate that the disclosure may be practiced in network computing environments with many types of computer system configurations, including, an in-dash vehicle computer, personal computers, desktop computers, laptop computers, message processors, hand-held devices, multi-processor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile telephones, PDAs, tablets, pagers, routers, switches, various storage devices, and the like. The disclosure may also be practiced in distributed system environments where local and remote computer systems, which are linked (either by hardwired data links, wireless data links, or by a combination of hardwired and wireless data links) through a network, both perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices.

Further, where appropriate, functions described herein can be performed in one or more of: hardware, software, firmware, digital components, or analog components. For example, one or more application specific integrated circuits (ASICs) can be programmed to carry out one or more of the systems and procedures described herein. Certain terms are used throughout the description and claims to refer to particular system components. As one skilled in the art will appreciate, components may be referred to by different names. This document does not intend to distinguish between components that differ in name, but not function.

It should be noted that the sensor embodiments discussed above may comprise computer hardware, software, firmware, or any combination thereof to perform at least a portion of their functions. For example, a sensor may include computer code configured to be executed in one or more processors, and may include hardware logic/electrical circuitry controlled by the computer code. These example devices are provided herein purposes of illustration, and are not intended to be limiting. Embodiments of the present disclosure may be implemented in further types of devices, as would be known to persons skilled in the relevant art(s).

At least some embodiments of the disclosure have been directed to computer program products comprising such logic (e.g., in the form of software) stored on any computer useable medium. Such software, when executed in one or more data processing devices, causes a device to operate as described herein.

While various embodiments of the present disclosure have been described above, it should be understood that they have been presented by way of example only, and not limitation. It will be apparent to persons skilled in the relevant art that various changes in form and detail can be made therein without departing from the spirit and scope of the disclosure. Thus, the breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

The foregoing description has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. Further, it should be noted that any or all of the aforementioned alternate implementations may be used in any combination desired to form additional hybrid implementations of the disclosure.