Cloud service validation for multiple programming stacks

Techniques for validating a cloud service for multiple programming stacks are disclosed. A computer system may configure a corresponding set of requests for each programming stack client in a plurality of programming stack clients based on usage metrics of a cloud service. Each programming stack client in the plurality of programming stack clients may be configured to be used to build a software application for the cloud service. The usage metrics may indicate historical requests executed on the cloud service via the plurality of programming stack clients. The computer system may then, for each programming stack client in the plurality of programming stack clients, trigger execution of the corresponding set of requests for the programming stack client on the cloud service via the programming stack client.

BACKGROUND

Cloud services comprise software, platforms, or infrastructure that are hosted by cloud service providers and made available for use by users through the Internet. Some cloud services enable users to build software applications through various programming stacks. A programming stack is a bundle of components, such as tools, frameworks, technologies, and languages, that go into composing a system and enable users to develop software applications.

DETAILED DESCRIPTION

Example methods and systems of validating a cloud service for multiple programming stacks are disclosed. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of example embodiments. It will be evident, however, to one skilled in the art that the present embodiments can be practiced without these specific details.

Cloud services may receive workload requests from various kinds of programming stack clients. For example, a cloud service may receive transactional workload request from an Advanced Business Application Programming (ABAP)® client and analytical workload requests from a Node.js® client. Current validation systems do not reflect this variation in workload patterns, but rather validate a cloud service with a single programming stack client. As a result, current validation systems fail to detect errors that are specific to certain programming stack clients, such as certain syntax errors or data errors that are specific to a particular client, thereby leaving the cloud service vulnerable to functional problems related to these client-specific errors. Other technical challenges may arise as well.

The implementation of the features disclosed herein involves a non-generic, unconventional, and non-routine operation or combination of operations. By applying one or more of the solutions disclosed herein, some technical effects of the system and method of the present disclosure are to provide cloud service validation for multiple programming stacks. In some example embodiments, a computer system configures a corresponding set of requests for each programming stack client in a plurality of programming stack clients based on usage metrics of a cloud service. Each programming stack client in the plurality of programming stack clients may be configured to be used to build a software application for the cloud service. The usage metrics may indicate historical requests executed on the cloud service via the plurality of programming stack clients. Next, the computer system may, for each programming stack client in the plurality of programming stack clients, trigger execution of the corresponding set of requests for the programming stack client on the cloud service via the programming stack client.

By orchestrating the execution of workload requests on a cloud service via multiple programming stack clients based on usage metrics that indicate historical requests executed on the cloud service via the multiple programming stack clients, the features of the present disclosure improve the validation of the cloud service, as the workload requests that are used to validate the cloud service are configured to reflect the actual workload requests handled by the cloud service. Other technical effects will be apparent from this disclosure as well.

The methods or embodiments disclosed herein may be implemented as a computer system having one or more modules (e.g., hardware modules or software modules). Such modules may be executed by one or more hardware processors of the computer system. In some example embodiments, a non-transitory machine-readable storage device can store a set of instructions that, when executed by at least one processor, causes the at least one processor to perform the operations and method steps discussed within the present disclosure.

The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and benefits of the subject matter described herein will be apparent from the description and drawings, and from the claims.

FIG.1is an example network diagram illustrating a system100. A platform (e.g., machines and software), in the example form of an enterprise application platform112, provides server-side functionality, via a network114(e.g., the Internet) to one or more clients.FIG.1illustrates, for example, a client machine116with programmatic client118(e.g., a browser), a small device client machine122with a small device web client120(e.g., a browser without a script engine), and a client/server machine117with a programmatic client119.

Turning specifically to the enterprise application platform112, web servers124and Application Program Interface (API) servers125can be coupled to, and provide web and programmatic interfaces to, application servers126. The application servers126can be, in turn, coupled to one or more database servers128that facilitate access to one or more databases130. The web servers124, API servers125, application servers126, and database servers128can host cross-functional services132. The cross-functional services132can include relational database modules to provide support services for access to the database(s)130, which includes a user interface library136. The application servers126can further host domain applications134. The web servers124and the API servers125may be combined.

The cross-functional services132provide services to users and processes that utilize the enterprise application platform112. For instance, the cross-functional services132can provide portal services (e.g., web services), database services, and connectivity to the domain applications134for users that operate the client machine116, the client/server machine117, and the small device client machine122. In addition, the cross-functional services132can provide an environment for delivering enhancements to existing applications and for integrating third-party and legacy applications with existing cross-functional services132and domain applications134. In some example embodiments, the system100comprises a client-server system that employs a client-server architecture, as shown inFIG.1. However, the embodiments of the present disclosure are, of course, not limited to a client-server architecture, and could equally well find application in a distributed, or peer-to-peer, architecture system.

FIG.2is a block diagram illustrating an example validation system200. The validation system200may be configured to perform a validation process for a cloud service260. In some example embodiments, the cloud service260comprises a database-as-a-service (DBaaS). A DBaaS is a cloud database offering that provides customers with access to a database without having to deploy and manage the underlying infrastructure. The database may run on a cloud computing platform and access to the database may be provided as-a-service, where users pay fees to a cloud provider for services and computing resources. The cloud service260may provide tools to users enabling them to create and manage database instances. Other types and configurations of cloud services260are also within the scope of the present disclosure.

In some example embodiments, the validation system200may comprise a gateway service210, an execution engine220, an intelligence component230, a script service240, and one or more databases250. The gateway service210, the execution engine220, the intelligence component230, the script service240, and the database(s)250, as well as the cloud service260, may be implemented as cloud-based components by the enterprise application platform112ofFIG.1, such as by being incorporated into the application server(s)126, the database server(s)128, or the database(s)130of the enterprise application platform112. However, the components of the validation system200, as well as the cloud service260, may be implemented in other ways as well. The components of the validation system200may communicate with one another, as well as with the cloud service260, via a network connection.

The validation system200may be configured to orchestrate the execution of requests on the cloud service260from different programming stack clients. The requests may comprise requests to perform one or more actions on the cloud service260, such as on one or more database tables being hosted by the cloud service260. Other types of requests are also within the scope of the present disclosure.

FIG.3illustrates an example of sets of requests320being executed on the cloud service260via a plurality of programming stack clients310(e.g., programming stack clients310-1to310-N). Each programming stack client310may be configured to be used to build a software application for the cloud service260. Examples of programming stack clients310include, but are not limited to, ABAP® clients, Node.js® clients, JAVA® clients, and Go (also referred to as Golang) clients. Other types of programming stack client310are also within the scope of the present disclosure.

Each programming stack client310may be used to execute its own corresponding set of requests320on the cloud service260(e.g., set of requests320-1for programming stack client310-1, . . . , set of requests320-N for programming stack client310-N). The validation system200may be configured to configure the corresponding set of requests320for each programming stack client310. Referring back toFIG.2, the gateway service210may be configured to access a list of requests320stored in the database250using the script service240. The database250may store a comprehensive set of requests320for use by the validation system200. In some example embodiments, the comprehensive set of requests stored in the database250is formed by receiving request feature definitions (e.g., component selection, syntax, etc.) submitted by a user via a computing device, multiplexing the request feature definitions to generate requests based on different combinations of the different request feature definitions, and storing the generated requests in the database250. The multiplexing results in the generation of an exponential combination of requests, such as millions of queries available to be run against the cloud service260for validation. However, executing all of the available requests stored in the database250for each programming stack client310in the plurality of programming stack clients310overloads the validation system200and the cloud service260, which negatively affects the functioning of the validation system200and the cloud service260.

In order to improve the efficiency of the validation process, the validation system200may be configured to configure the corresponding set of requests320for each programming stack client310based on usage metrics of the cloud service260. The usage metrics may indicate historical requests executed on the cloud service260via the plurality of programming stack clients310. The usage metrics may be restricted to only those usage metrics that correspond to historical requests executed within a most recent period of time, such as restricting the usage metrics to those usage metrics that correspond to historical requests that were executed within the last 30 days. The intelligence component230may be configured to collect the usage metrics from the cloud service260. The usage metrics may include, but are not limited to, what kind of workload requests come from which programming stack clients310(e.g., transactional queries, analytical queries, graph queries, spatial queries), how much of volume of data is processed from specific types of queries, what data pattern is used for queries (e.g., max and min values used for a specific feature), a number of partitions being queried or otherwise targeted by requests, a volume of requests for each programming stack client310, combination of features used for requests executed using each programming stack client310, and configuration of the cloud service260(e.g., scaleout node count). Other types of usage metrics are also within the scope of the present disclosure.

The gateway service210may provide the list of requests obtained from the database250and the usage metrics obtained from the intelligence component230to the execution engine220. The execution engine220may be configured to determine how to orchestrate the validation of the cloud service260based on an orchestration logic that configures the corresponding set of requests320to be used for each programming stack client310. The orchestration logic may comprise mapping types of requests from the list of requests obtained from the database250to programming stack clients310based on indications in the usage metrics of what type of requests were previously executed using the programming stack clients. Certain types of requests may be included in the corresponding sets of requests320across multiple programming stack clients310, as these types of requests may be basic and more commonly used, whereas other types of requests may only be included in the corresponding set of requests320for only a subset of the plurality of programming stack clients310. The execution engine220may further

In some example embodiments, the execution engine220may map one or more request types to each programming stack client310in the plurality of programming stack clients310based on corresponding request types of the historical requests executed on the cloud service260via the programming stack client310, as indicated by the usage metrics. The one or more request types may comprise one or more types of queries. For example, the one or more request types may comprise a combination of one or more of collection SQL queries, graph SQL queries, transactional SQL queries, or analytical SQL queries. However, other types of request types and other types of queries are also within the scope of the present disclosure.

The execution engine220may configure the corresponding set of requests320for the programming stack client310to be of the one or more request types mapped to the programming stack client310. For example, the validation system200may configure the corresponding set of requests320for a first programming stack client310to include collection SQL queries and transactional SQL queries based on collection SQL queries and transaction SQL queries being mapped to the first programming stack client310, while the validation system200may configure the corresponding set of requests320for a second programming stack client310to include collection SQL queries and analytical SQL queries based on collection SQL queries and analytical SQL queries being mapped to the second programming stack client310.

The execution engine220may also be configured to specify test data on which to execute a set of queries based on the usage metrics. For example, in a scenario where the usage metrics indicate that the historical requests executed on the cloud service260via a particular programming stack client310comprised queries that were performed on a particular portion of a database (e.g., specific tables, rows, columns, partitions, etc.), the validation system200may configure the set of requests320for that particular programming stack client310to comprise queries to be performed on the same particular portion of the database.

Additionally or alternatively, the execution engine220may be configured to configure a corresponding volume of the set of requests320for each programming stack client310based on a corresponding volume of the historical requests for the programming stack client310indicated by the usage metrics. For example, in a scenario where the usage metrics indicate that the total number of times that a particular type of query was executed on the cloud service260via a particular programming stack client310within the last 30 days is 1,328, the validation system200may configure the set of requests320for that particular programming stack client310to comprise 1,328 queries of that particular type of query.

In some example embodiments, the execution engine220may, for each programming stack client310in the plurality of programming stack clients310, trigger execution of the corresponding set of requests320for the programming stack client310on the cloud service260via the programming stack client310, at operation520. For example, the execution engine220may send one or more instructions to each programming stack client310to execute the corresponding set of requests320for the programming stack client310on the cloud service260. The one or more instructions may include the set of requests320.

The gateway service210may obtain results of the execution of the sets of requests320for the plurality of programming stack clients310on the cloud service260. For example, the sets of requests320may be executed on the cloud service260, and the results of the execution of the sets of requests320may be stored in the database250, from which the gateway service210may obtain the stored results, such as for use in further validation operations. The gateway service210may be configured to cause an indication of the results to be displayed on a computing device. For example, the gateway service210may perform one or more validation operations to compute a summary of the results for the different requests320for each programming stack client310, such as corresponding indications of whether each request320passed or failed a validation standard (e.g., one or more criteria).

FIG.4illustrates an example graphical user interface400in which results of execution of sets of requests320on a cloud service are displayed. InFIG.4, the results are displayed in a table that indicates for each request320that was executed as part of the validation process, a corresponding indication of whether the corresponding result of the execution either passed the validation process for the corresponding programming stack client310, failed the validation process for the corresponding programming stack client310, or skipped the validation process for the corresponding programming stack client310. The passing of the validation process may be based on a determination that data that was output as a result of the execution of the request320satisfied one or more validation criteria. The failing of the validation process may be based on a determination that data that was output as a result of the execution of the request320failed to satisfy one or more validation criteria. The skipping of the validation process may be based on the request320not being executed via the programming stack client310.

FIG.5is a flowchart illustrating an example method of validating a cloud service for multiple programming stacks. The method500can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device), or a combination thereof. In one example embodiment, one or more of the operations of the method500are performed by one or more components of the validation system200ofFIG.2.

At operation510, the validation system200may configure a corresponding set of requests320for each programming stack client310in a plurality of programming stack clients310based on usage metrics of a cloud service260. In some example embodiments, the cloud service260may comprise a DBaaS. However, other types of cloud services260are also within the scope of the present disclosure. Each programming stack client310may be configured to be used to build a software application for the cloud service260. The usage metrics may indicate historical requests executed on the cloud service260via the plurality of programming stack clients310. The usage metrics may be restricted to only those usage metrics that correspond to historical requests executed within a most recent period of time, such as restricting the usage metrics to those usage metrics that correspond to historical requests that were executed within the last 30 days.

The corresponding set of requests320for each programming stack client310may comprise a set of queries. Each query may comprise a request for data results from one or more databases hosted by the cloud service260(e.g., a data retrieval query) or for an action to be performed on data stored by the cloud service260(e.g., an action query) or for both. However, other types of requests320are also within the scope of the present disclosure.

In some example embodiments, the configuring of the corresponding set of requests320for each programming stack client310may comprise specifying test data on which to execute a set of queries based on the usage metrics. For example, in a scenario where the usage metrics indicate that the historical requests executed on the cloud service260via a particular programming stack client310comprised queries that were performed on a particular portion of a database (e.g., specific tables, rows, columns, partitions, etc.), the validation system200may configure the set of requests320for that particular programming stack client310to comprise queries to be performed on the same particular portion of the database.

Additionally or alternatively, the configuring of the corresponding set of requests320for each programming stack client310in the plurality of programming stack clients310may comprise configuring a corresponding volume of the set of requests320for each programming stack client310based on a corresponding volume of the historical requests for the programming stack client310indicated by the usage metrics. For example, in a scenario where the usage metrics indicate that the total number of times that a particular type of query was executed on the cloud service260via a particular programming stack client310within the last 30 days is 1,328, the validation system200may configure the set of requests320for that particular programming stack client310to comprise 1,328 queries of that particular type of query.

The validation system200may then, for each programming stack client310in the plurality of programming stack clients310, trigger execution of the corresponding set of requests320for the programming stack client310on the cloud service260via the programming stack client310, at operation520. For example, the validation system200may send one or more instructions to each programming stack client310to execute the corresponding set of requests32for the programming stack client310on the cloud service260. The one or more instructions may include the set of requests320.

Next, the validation system200may, at operation530, obtain results of the execution of the sets of requests320for the plurality of programming stack clients310on the cloud service260. For example, the sets of requests320may be executed on the cloud service260, and the results of the execution of the sets of requests320may be stored in the database250. The validation system200may obtain the stored results from the database250, such as for use in further validation operations.

At operation540, the validation system200may then cause an indication of the results to be displayed on a computing device. For example, the validation system200may perform one or more validation operations to compute a summary of the results for the different requests320for each programming stack client310, such as corresponding indications of whether each request320passed or failed a validation standard (e.g., one or more criteria). As previously discussed,FIG.4illustrates an example of results of execution of sets of requests320being displayed within the graphical user interface400. Other ways of causing the indication of the results to be displayed are also within the scope of the present disclosure.

FIG.6is a flowchart illustrating an example method of configuring a corresponding set of requests for each programing stack client in a plurality of programming stack clients. The method600can be performed by processing logic that can comprise hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions run on a processing device), or a combination thereof. In one example embodiment, one or more of the operations of the method600are performed by one or more components of the validation system200ofFIG.2.

At operation610, the validation system200may map one or more request types to each programming stack client310in the plurality of programming stack clients310based on corresponding request types of the historical requests executed on the cloud service260via the programming stack client310. The one or more request types may comprise one or more types of queries. For example, the one or more request types may comprise a combination of one or more of collection SQL queries, graph SQL queries, transactional SQL queries, or analytical SQL queries. However, other types of request types and other types of queries are also within the scope of the present disclosure.

Next, the validation system200may, at operation620, configure the corresponding set of requests320for the programming stack client310to be of the one or more request types mapped to the programming stack client310. For example, the validation system200may configure the corresponding set of requests320for a first programming stack client310to include collection SQL queries and transactional SQL queries based on collection SQL queries and transaction SQL queries being mapped to the first programming stack client310, while the validation system200may configure the corresponding set of requests320for a second programming stack client310to include collection SQL queries and analytical SQL queries based on collection SQL queries and analytical SQL queries being mapped to the second programming stack client310.

Example 1 includes a computer-implemented method performed by a computer system having a memory and at least one hardware processor, the computer-implemented method comprising: configuring a corresponding set of requests for each programming stack client in a plurality of programming stack clients based on usage metrics of a cloud service, each programming stack client in the plurality of programming stack clients being configured to be used to build a software application for the cloud service, the usage metrics indicating historical requests executed on the cloud service via the plurality of programming stack clients; and, for each programming stack client in the plurality of programming stack clients, triggering execution of the corresponding set of requests for the programming stack client on the cloud service via the programming stack client.

Example 2 includes the computer-implemented method of example 1, wherein the cloud service comprises a database-as-a-service (DBaaS).

Example 3 includes the computer-implemented method of example 1 or example 2, wherein the corresponding set of requests for each programming stack client in the plurality of programming stack clients comprises a set of queries.

Example 4 includes the computer-implemented method of any one of examples 1 to 3, wherein the configuring the corresponding set of requests for each programming stack client in the plurality of programming stack clients comprises: specifying test data on which to execute the set of queries based on the usage metrics.

Example 5 includes the computer-implemented method of any one of examples 1 to 4, wherein the configuring the corresponding set of requests for each programming stack client in the plurality of programming stack clients comprises: configuring a corresponding volume of the set of requests for each programming stack client in the plurality of programming stack clients based on a corresponding volume of the historical requests for the programming stack client indicated by the usage metrics.

Example 6 includes the computer-implemented method of any one of examples 1 to 5, wherein the configuring the corresponding set of requests for each programming stack client in the plurality of programming stack clients comprises: mapping one or more request types to each programming stack client in the plurality of programming stack clients based on corresponding request types of the historical requests executed on the cloud service via the programming stack client; and configuring the corresponding set of requests for the programming stack client to be of the one or more request types mapped to the programming stack client.

Example 7 includes the computer-implemented method of any one of examples 1 to 6, wherein the one or more request types comprise one or more types of queries.

Example 8 includes the computer-implemented method of any one of examples 1 to 7, further comprising: obtaining results of the execution of the sets of requests for the plurality of programming stack clients on the cloud service; and causing an indication of the results to be displayed on a computing device.

Example 9 includes a system comprising: at least one processor; and a non-transitory computer-readable medium storing executable instructions that, when executed, cause the at least one processor to perform the method of any one of examples 1 to 8.

Example 10 includes a non-transitory machine-readable storage medium, tangibly embodying a set of instructions that, when executed by at least one processor, causes the at least one processor to perform the method of any one of examples 1 to 8.

Example 11 includes a machine-readable medium carrying a set of instructions that, when executed by at least one processor, causes the at least one processor to carry out the method of any one of examples 1 to 8.

The example computer system700includes a processor702(e.g., a central processing unit (CPU), a graphics processing unit (GPU) or both), a main memory704, and a static memory706, which communicate with each other via a bus708. The computer system700may further include a graphics or video display unit710(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system700also includes an alphanumeric input device712(e.g., a keyboard), a user interface (UI) navigation (or cursor control) device714(e.g., a mouse), a storage unit (e.g., a disk drive unit)716, an audio or signal generation device718(e.g., a speaker), and a network interface device720.

The storage unit716includes a machine-readable medium722on which is stored one or more sets of data structures and instructions724(e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. The instructions724may also reside, completely or at least partially, within the main memory704and/or within the processor702during execution thereof by the computer system700, the main memory704and the processor702also constituting machine-readable media. The instructions724may also reside, completely or at least partially, within the static memory706.

This detailed description is merely intended to teach a person of skill in the art further details for practicing certain aspects of the present teachings and is not intended to limit the scope of the claims. Therefore, combinations of features disclosed above in the detailed description may not be necessary to practice the teachings in the broadest sense, and are instead taught merely to describe particularly representative examples of the present teachings.