Patent ID: 12224897

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth to provide a thorough understanding of embodiments. However, it will be understood by those of ordinary skill in the art that the embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the embodiments.

One or more specific embodiments of the present invention will be described below. To provide a concise description of these embodiments, all features of an actual implementation may not be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure.

An enterprise may use distributed cloud services to perform business functions. For example, consuming applications may access a provider application via network nodes to execute a fraud detection process. For various reasons, however, the provider application might not be available to perform the function.

FIG.1illustrates100some sources of technical challenges110for a cloud service including, scalability, failure handling, concurrency, transparence, quality of service, heterogeneity, openness, security, etc. Some common misconceptions about distributed systems include: that the network is reliable; that latency is zero; that bandwidth is infinite; that the network is secure; that topology does not change; that there is one common, consistent administrator; that transport costs are zero; that the network is homogeneous; etc. Instead, a distributed system is one in which the failure of a computer that a user did not even know existed can render a service unavailable. In fact, failures in today's complex, distributed and interconnected systems are not the exception—they are the normal case and are not predictable.

To address such situations, consuming applications may implement “resilience patterns” to handle unexpected problems. The resilience may result in a user not even being aware of the problem or implement a graceful degradation of service.FIG.2illustrates200cloud service resilience. Here, a client accesses a cloud210service220that utilizes containers A through B. Moreover, as illustrated200one container is currently unavailable (container B as illustrated by the “X” inFIG.2). As a result, a resilience pattern may be implemented such that the other containers (A, C, and D) are utilized instead.FIG.3illustrates300resilience pattern software design. A software development (“Dev”) team310may implement a resilience measure to improve production availability and provide a build to an Information Technology (“IT”) operations (“Ops”) team320. The Ops team320may measure the effect of the resilience measure so the enterprise may learn and improve resilience. In this way, generalizable solutions to recurring problems that occurs within a well-defined context may be created. The solutions may provide designers with a template showing how to solve a problem (and that can be used in many different situations). Although examples are described herein in connection with particular types of resilience patterns, note that embodiments might be associated with any other type of resilience pattern (e.g., a retry pattern, a timeout pattern, a fallback pattern, etc.).

FIG.4illustrates a typical cloud service network400. The network400includes a provider application410that is accessed by consumer applications430(applications1through4inFIG.4) via network nodes420. Note that cloud services typically promise high reliability to the customers. Many service providers, for example, have a Service Level Agreement (“SLA”) of over 99% availability with customers. There are many factors, however, that could adversely impact the reliability of cloud-based services. This is because in a distributed environment failures can occur for many reasons (unreliable networks, low bandwidth, high latency, topology changes, transport costs, etc.)—all of which can reduce the reliability of cloud services. To address this, cloud consumer applications430currently implement resilience patterns or implementations440to handle unexpected situations in the productive environment. The objective of these resilience implementations440is to identify reliability issues as early as possible and solve problems before a customer notices it or at least provide a graceful degradation of service. Some well-known resilience implementations440are circuit breakers, bounded queues, rate limits, timeout, and bulk heads.

Note that the resilience implementations440are implemented close to the consumer applications430. That is, currently resilience implementations440are implemented in the client environment and tested against the availability of the provider application410. As a result, each consumer application430needs to handle resilience explicitly to provide dedicated implementation for each consumer. Each consumer of the provider application410needs to implement the resilience implementations440which leads to duplicate efforts and/or each implementation could be different.

Moreover, the resilience implementations440may have a dependency on the programming language, Application Programming Interface (“API”), and/or protocols of the consumer applications430. Foer example, consumer applications430might have been developed in different programming languages, be executed on different execution environment, use different protocols, etc. As a result, there may be a dependency on language, execution environment, protocols, etc. For example, there are many open-source resilience libraries available for a spring-boot java consumer application430but far fewer options for an Advanced Business Application Programming (“ABAP”)-based consumer application430. Similarly, each consumer application430may have different scope and development teams which can also cause resilience implementations440to vary (e.g., some teams might maintain a high quality for a resilience implementation440while another team does not spend enough).

In addition, each client needs to maintain the resilience implementations440which explicitly leads to maintenance overhead. Further, the provider application410by default cannot offer reliability. That is, with the current resilience implementations440it is not possible to have a central instance of a resilience implementation440that all clients can access. Overall, the network nodes420used only for routing (and there is no execution at the network nodes420) because they are passive nodes that do not have intelligence and an execution environment.

Some embodiments described herein improve the reliability of cloud services with the effective utilization of “active” network nodes. Traditional network nodes are passive in nature and are used to route packets from a source to a destination. Traditional network nodes do not have execution environments and therefore are not capable of runtime decisions or executions. In contrast, “active” networks are a new generation of networks that implement active network nodes. Active network nodes have execution environments and can execute programs dynamically (making the network more effective and intelligent). Moreover, resilience patterns are the design patterns used by developers to handle unexpected situations that occur in a productive environment. These are solutions for well-known problems that occur in distributed systems and as well exponentially avoid duplication and maintenance of efforts. Traditionally, resilience patterns are implemented at the consumer applications. According to some embodiments described herein, resilience patterns are instead included into active network nodes (instead of consumer applications) and make use of their execution environment to handle the problems associated with distributed systems. That is, embodiments may provide a method to move resilience handling from consumer environment to the network—improving service reliability and associated efforts for maintainability.

To provide improved and efficient implementation of application resilience patterns for cloud services in a fast, automatic, and accurate manner,FIG.5is a high-level system500architecture in accordance with some embodiments. The system500includes an active network node550between a provider application and a consumer application. As used herein, devices, including those associated with the system500and any other device described herein, may exchange information via any communication network which may be one or more of a Local Area Network (“LAN”), a Metropolitan Area Network (“MAN”), a Wide Area Network (“WAN”), a proprietary network, a Public Switched Telephone Network (“PSTN”), a Wireless Application Protocol (“WAP”) network, a Bluetooth network, a wireless LAN network, and/or an Internet Protocol (“IP”) network such as the Internet, an intranet, or an extranet. Note that any devices described herein may communicate via one or more such communication networks.

The active network node550may store information into and/or retrieve information from various data stores, which may be locally stored or reside remote from the active network node550. Although a single active network node550is shown inFIG.5, any number of such devices may be included. Moreover, various devices described herein might be combined according to embodiments of the present invention. For example, in some embodiments, the active network node550and a resilience implementation560might comprise a single apparatus. The system500functions may be automated and/or performed by a constellation of networked apparatuses, such as in a distributed processing or cloud-based architecture. As used herein, the term “automated” may refer to any process or method that may be performed with little or no human intervention.

An operator, administrator, or enterprise application may access the system500via a remote device (e.g., a Personal Computer (“PC”), tablet, or smartphone) to view information about and/or manage operational information in accordance with any of the embodiments described herein. In some cases, an interactive graphical user interface display may let an operator or administrator define and/or adjust certain parameters (e.g., to implement various mappings or resilience parameters) and/or provide or receive automatically generated prediction results (e.g., reports and alerts) from the system500.

The active network node550may include: an execution environment (e.g., Unix shell); and operating system capable of supporting one or more execution environments; and active hardware (e.g., a network processor). The resilience pattern or implementation560may provide an ability of the system500to manage and graciously recover from failures. The resilience implementation560may ensure that applications are available whenever users need them. Note that a resilience implementation560may have a mechanism to identify a failure event quickly and automatically. Moreover, the resilience implementation560may help prevent failures and/or preserve business continuity during a failure. Resilience implementations560might be associated with, for example, loose coupling (e.g., self-containment, asynchronous communication, relaxed temporal constraints, idempotency), isolation (e.g., bulkheads, complete parameter checking, shed load), latency control (e.g., bounded queues, circuit breakers, timeouts, fail fast), supervision (e.g., monitor, error handler), etc.

FIG.6is a method that might be performed by some, or all, of the elements of any embodiment described herein. The flow charts described herein do not imply a fixed order to the steps, and embodiments of the present invention may be practiced in any order that is practicable. Note that any of the methods described herein may be performed by hardware, software, an automated script of commands, or any combination of these approaches. For example, a computer-readable storage medium may store thereon instructions that when executed by a machine result in performance according to any of the embodiments described herein.

At S610, a computer processor of an active network node within a network environment between a provider application and a consumer application may automatically detect, via a platform and language independent centralized resilience process, a failure event in an active network. That is, the centralized resilience process is independent of various platforms and/or languages used by one or more consumer applications. The active network may, for example, route packets to support a distributed cloud service. The active network node may have, for example, an execution environment, an operating system to support the execution environment, and active hardware. Responsive to the detection of the failure event, an application resilience pattern may be dynamically implemented by the centralized resilience process to facilitate recovery from the detected failure event at S620without participation of the consumer application.

FIG.7is a proposed solution for a system700to improve cloud service reliability with the effective utilization of active network nodes in accordance with some embodiments. As before, the system700includes a provider application710that is accessed by consumer applications730(applications1through4inFIG.7) via network nodes720. In this case, however, cloud consumer applications do not implement resilience patterns—instead, a resilience implementation760is implemented in an active network node750to handle unexpected situations in the productive environment (e.g., circuit breaker, bounded queues, rate limits, timeout, bulk heads, etc.).

Note that the resilience implementations760are implemented close to the provider application710and are moved from the consumer environment to the network. As a result, the capabilities of the active network node750may be utilized effectively to execute resilience patterns dynamically. Moreover, the duplicate implementation of the resilience implementation760is eliminated from the consumer applications730by creating a centralized implementation in the network. This may help ensure that standardized reliability handling is done for all the consumer applications730that are involved (by the provider application710). No specific skills or resilience implementations760are required by the consumer (who still receives all the reliability benefits) and dependency on programming language, execution environment, protocol, etc. may be eliminated (because execution happens in the active network node750which is triggered by data packets).

Instead of client-centric reliability handling, embodiments may provide network-centric reliability handling. Moreover, while typical passive networks are used only for packet routing active networks in accordance with some embodiments are capable of runtime executions and decision making (in addition to packet routing). In addition, embodiments may reduce platform, language, and protocol dependency for resilience implementation (and may be independent of language, platform, and protocols). Further, in the typical approach each consumer needs to handle reliability explicitly—but embodiments provided herein use centralized reliability handling (and every client can benefit without additional cost or efforts). For example, rate limits, timeouts, etc. no longer need to be controlled by the client, instead the network is capable of handling rate limits and timeouts (reducing server load). Similarly, each consumer application no longer needs resources with specific skill sets to handling reliability scenarios because reliability scenarios are handled in the network (no specific handling or skills are required by the consumers).

In one widely used scenario, a target can be another cloud service (including an external service). In this case, reliability patterns and handling can be moved out of consumer and into the network active network to handle cloud reliability.FIG.8is another use case800according to some embodiments. Here, a consumer application830accesses an active network node850via one or more network nodes820. Moreover, the active network node850incorporates a resilience implementation860in support of a web dispatcher870, component872(e.g., a load balancer), and/or a database874. That is, it is also possible that a target can be the web dispatcher870, the component872, and/or the database874. In this case, the proposed architecture can handle the load, etc. in the network and prevent the web dispatcher870, the component872, and/or the database874from getting overloaded. In this way, the unexpected crashing of such components can be prevented (without specific handling by the client) helping ensure the reliability of cloud. Although a load balancer is one example of a component872, note that it might comprise any component872that receives and processes input from multiple sources (e.g., an application server, API gateway, etc.). Embodiments may protect any of such components872from becoming overloaded and crashing.

FIG.9is a circuit breaker pattern900. Here, a client910attempts to access an external service930via another service920. The circuit breaker pattern900is an application resilience pattern that can be used to limit several requests to a service based on configured thresholds—helping to prevent the service from being overloaded. The circuit breaker pattern900may continuously monitor for failures. Once failure reaches a defined threshold, the circuit breaker trips, and all further calls to the circuit breaker return with an error. The circuit breaker pattern900may move between the following states: from closed940to open942, from half-open944to closed940, and between open942and half-open944. When closed940, the service can be accessed and if failures exceed a threshold value, the circuit will trip and will move to open942. When open942, the service is not available and the failures are recorded. After a wait time duration has elapsed, the breaker transitions from open942to half-open944and allows only a configurable number of calls. Further calls are rejected until all permitted calls have been completed. If the failure rate or slow call rate is greater than or equal to the configured threshold, the state changes back to open942. If the failure rate and slow call rate is below the threshold, the state changes back to closed940.

FIG.10is a circuit breaker example1000. Here, a payment service1010sends a request to a fraud check service1030(e.g., through a closed1020circuit breaker). After checking that it is closed at1040, a call is executed1042and transaction metrics are recorded1044. If the state was not closed at1040and not half-open at1042(therefore “open”) the call is rejected at1050. If the state was half-open at1046and not a “test” call at1048, the call is also rejected at1050. In the case of a “test” call, it may be executed at1042and transaction metrics may be recorded at1044.

The circuit breaker is a widely used resilience pattern that is typically implemented at the consumer application and is used to limit a number of requests to a service based on configured thresholds (to help prevent the service from being overloaded). The pattern can help to minimize failure impact and user experience by achieving one of the following:handle and support to resolve the service unavailability issue automatically without being noticed by the end user; orinform the end user about the temporary service unavailability with a graceful degradation.
FIG.11is a typical circuit breaker implementation1100where a “tax service”1110is an external service (provider service) that can be used to retrieve tax rates for transactions. Many consumer applications1130consume the tax service1110via network nodes1120to retrieve the tax payable information. These consumer applications1130may be developed in various languages and deployed on various platforms (and APIs may be used for consumer-provider communications). Each consumer application1130implements the circuit breaker1140pattern to gracefully handle service load issues at production. Since the consumer applications1130are developed in different languages, running on different platforms, the implementation1100may need to use multiple native resilience libraries.

FIG.12is a circuit breaker implementation1200according to some embodiments. As before, many consumer applications1230consume a tax service1210via network nodes1220to retrieve the tax payable information. In this case, however, the implementation of the circuit breaker1260pattern is moved to the network. The circuit breaker1260pattern is at an active network node1250and gets triggered based on when requests from consumer applications1230are received. Such an approach makes it centralized and language/platform independent. Moreover, duplicate implementation in each consumer application1230is avoided.

A “bounded queue” pattern may be used in the asynchronous processing paradigm to slow down a busy service by initially inserting requests into a queue.FIG.13is a bounded queue pattern1300where service A1310sends a request to a queue1320. The number of requests that an application can process at one point in time can be decided by the size of queue1320. If the queue1320is not full, the request is then sent to service B1330. If the queue1310becomes full, it creates a back pressure by rejecting additional messages. This ensures that that application (e.g., service B1330) will not get overloaded and crash. Another advantage is that, if network latency causes a database to be not available momentarily, the data can remain in the queue. When the database is again available, the worker can pick the data from the queue and write it to the database.

FIG.14is a typical bounded queue implementation1400. Many consumer applications1430consume a database1410via network nodes1420. These consumer applications1430may be developed in various languages and deployed on various platforms. Each consumer application1430implements the bounded queue1440pattern to gracefully handle failure events.

FIG.15is a bounded queue implementation1500in accordance with some embodiments. As before, many consumer applications1530access a database1510via network nodes1520. In this case, however, the implementation of the bounded queue1560pattern is moved to the network. The bounded queue1560pattern is at an active network node1550and gets triggered based on when requests from consumer applications1530are received. Such an approach makes it centralized and language/platform independent. Moreover, duplicate implementation in each consumer application1530is avoided.

FIG.16shows architecture details for a system1600according to some embodiments. In such a system1600, a client1630sends a smart packet1620to an active network node1650associated with a web service1670, database1674, etc. Note that a resilience implementation in the active network node1650might take one of two approaches:a discrete approach in which programs are injected into active network node1650separately from passive data; andan integrated approach in which programs are integrated into every packet along with passive data.
The system1600ofFIG.16incorporates the discrete approach, where resilience patterns will be injected to the active network node1650as an executable program. In this way, resilience patterns may be available in active network nodes and any consumer can benefit from it without explicitly handling the scenario. This may be referred to as a control plane1660because the patterns control reliability, etc.

Embodiments may integrate data along with smart packets1620and send the same to the active network node1650(or data plane). The data plane may act as a trigger for the control plane1660. When the smart packet1620is received, the integrated data triggers the control plane1660execution. For example, the smart packet1620payload may contain code and data including:Perform packet authorizationIf (authorized==false)Return “ERROR”ElseTrigger Failure Handling and Resilience patterns in the AN

According to some embodiments, the Active Network (“AN”) lets an individual user, or groups of users, inject customized programs into the nodes of the network. The active network uses smart packets (with code and data) and includes:active network nodes that can execute code on the data; andactive packets that carry code to active network nodes.
FIG.17is an example of a smart packet1700in accordance with some embodiments. The smart packet1700includes an IP portion1710with an IP header and a router alert option. The smart packet1700may also include an Active Network Encapsulation Protocol (“ANEP”) header portion1720with a version, flag, type identifier, header length, packet length, source identifier, destination identifier, integrity checksum, etc. In addition, the smart packet1700may include a smart packet payload portion1730with a version, type, content, sequence number etc.

FIG.18is a human machine interface display1800in accordance with some embodiments. The display1800includes a graphical representation1810of elements of cloud-based computing environment (e.g., to efficiently implement resilience patterns). Selection of an element (e.g., via a touchscreen or computer pointer1820) may result in display of a pop-up window containing various options (e.g., to adjust rules or logic, assign various devices, etc.). The display1800may also include a user-selectable “Setup” icon1830(e.g., to configure parameters such as thresholds for resilience patterns as described with respect any of the embodiments described herein).

Note that the embodiments described herein may be implemented using any number of different hardware configurations. For example,FIG.19is a block diagram of an apparatus or platform1900that may be, for example, associated with the system500ofFIG.5(and/or any other system described herein). The platform1900comprises a processor1910, such as one or more commercially available Central Processing Units (“CPUs”) in the form of one-chip microprocessors, coupled to a communication device1920configured to communicate via a communication network (not shown inFIG.19). The communication device1920may be used to communicate, for example, with one or more remote user platforms, cloud resource providers, etc. The platform1900further includes an input device1940(e.g., a computer mouse and/or keyboard to input rules or logic) and/an output device1950(e.g., a computer monitor to render a display, transmit predictions or alerts, and/or create recommendations). According to some embodiments, a mobile device and/or PC may be used to exchange information with the platform1900.

The processor1910also communicates with a storage device1930. The storage device1930can be implemented as a single database or the different components of the storage device1930can be distributed using multiple databases (that is, different deployment information storage options are possible). The storage device1930may comprise any appropriate information storage device, including combinations of magnetic storage devices (e.g., a hard disk drive), optical storage devices, mobile telephones, and/or semiconductor memory devices. The storage device1930stores a program1912and/or a resilience platform1914for controlling the processor1910. The processor1910performs instructions of the programs1912,1914, and thereby operates in accordance with any of the embodiments described herein. For example, the processor1910may automatically detect, via a platform and language independent centralized resilience process, a failure event in an active network that routes packets to support a distributed cloud service. The failure event might be associated with, for example, an unreliable network, a low bandwidth, a high latency, a topology change, transport costs, etc. Responsive to the detection of the failure event, the processor1910may dynamically implement an application resilience pattern (e.g., a circuit breaker or bounded queue) by the centralized resilience process to facilitate recovery from the detected failure event without participation of the consumer application.

The programs1912,1914may be stored in a compressed, uncompiled and/or encrypted format. The programs1912,1914may furthermore include other program elements, such as an operating system, clipboard application, a database management system, and/or device drivers used by the processor1910to interface with peripheral devices.

As used herein, information may be “received” by or “transmitted” to, for example: (i) the platform1900from another device; or (ii) a software application or module within the platform1900from another software application, module, or any other source.

In some embodiments (such as the one shown inFIG.19), the storage device1930further stores a resilience pattern1960and an active network node database2000. An example of a database that may be used in connection with the platform1900will now be described in detail with respect toFIG.20. Note that the database described herein is only one example, and additional and/or different information may be stored therein. Moreover, various databases might be split or combined in accordance with any of the embodiments described herein.

Referring toFIG.20, a table is shown that represents the active network node database2000that may be stored at the platform1900according to some embodiments. The table may include, for example, entries identifying active network nodes (e.g., with an execution environment, operating system, and active hardware). The table may also define fields2002,2004,2006,2008, for each of the entries. The fields2002,2004,2006,2008may, according to some embodiments, specify: an active network node identifier2002, a resilience pattern2004, a provider application2006, and consumer application2008. The active network node database2000may be created and updated, for example, when new nodes are added, resilience patterns are adjusted, etc.

The active network node identifier2002might be a unique alphanumeric label or link, associated with an active network, that includes an execution environment, an operating system to support the execution environment, and active hardware. The resilience pattern2004may contain or describe an application resilience pattern that facilitates recovery from a detected failure event. The provider application2006may, according to some embodiments, execute a cloud service for several consumer applications2008in the active network.

Thus, embodiments may provide a system and method to improve cloud service reliability with the help of active network nodes.

The following illustrates various additional embodiments of the invention. These do not constitute a definition of all possible embodiments, and those skilled in the art will understand that the present invention is applicable to many other embodiments. Further, although the following embodiments are briefly described for clarity, those skilled in the art will understand how to make any changes, if necessary, to the above-described apparatus and methods to accommodate these and other embodiments and applications.

In some embodiments, specific resilience patterns are described. Note, however, that embodiments may be associated with any type of resilience pattern. Although specific hardware and data configurations have been described herein, note that any number of other configurations may be provided in accordance with some embodiments of the present invention (e.g., some of the information associated with the databases described herein may be combined or stored in external systems). Moreover, although some embodiments are focused on particular types of applications and cloud services, any of the embodiments described herein could be applied to other types of applications and cloud services. In addition, the displays shown herein are provided only as examples, and any other type of user interface could be implemented. For example,FIG.21shows a tablet computer2100rendering a cloud services system display2110. The display2110may, according to some embodiments, be used to view more detailed elements about components of the system (e.g., when a graphical element is selected via a touchscreen) and/or to configure operation of the system (e.g., to establish new rules or logic for the system via a “Setup” icon2120).

The present invention has been described in terms of several embodiments solely for the purpose of illustration. Persons skilled in the art will recognize from this description that the invention is not limited to the embodiments described but may be practiced with modifications and alterations limited only by the spirit and scope of the appended claims.