Patent ID: 12198050

DETAILED DESCRIPTION

The illustrative embodiments recognize and take into account that, in frequent software development cycles, automated testing of application programming interfaces is desirable, but often inefficient. Currently, test coverage focuses on development and code.

Illustrative embodiments use the power of artificial intelligence to transform traditional approaches to software and application development and testing to accelerate the pace of production. Illustrative embodiments provide a system and method for using an artificial intelligence algorithm and model to identify the application programming interface relationships. Illustrative embodiments provide an appropriate scope or the minimal set of application programming interfaces for testing.

Illustrative embodiments utilize a graph attention network to derive an application programming interface relationship network that further expands test coverage and provides the minimal set of application programming interfaces for testing, making development and testing more efficient. Illustrative embodiments extend the coverage of application programming interface relationships to including characteristics of a software product to be tested, such as application programming interfaces, software functionality, software users, business requirements of the software, etc.

Illustrative embodiments establish an application programming interface relationship network by a graph attention network learner and builder. From the application programming interface relationship network, a minimal set of application programming interfaces for testing may be determined, based on various kinds of scenarios. Illustrative embodiments provide for analyzing the application programming interface relationship network and exploiting the business insights.

Illustrative embodiment thus may improve application programming interface automated testing in frequent development cycles by use of a graph attention network learner and builder. Illustrative embodiments may help customers to reduce the business risk in production of software applications, including internet of things application products.

With reference now to the figures and, in particular, with reference toFIG.1, a pictorial representation of a network of data processing systems is depicted in which illustrative embodiments may be implemented. Network data processing system100is a network of computers in which the illustrative embodiments may be implemented. Network data processing system100contains network102, which is the medium used to provide communications links between various devices and computers connected together within network data processing system100. Network102may include connections, such as wire, wireless communication links, or fiber optic cables.

In the depicted example, server computer104and server computer106connect to network102along with storage unit108. In addition, client computer110, client computer112, and client computer114connect to network102. Client computers110,112, and114can be, for example, computers, workstations, or network computers. In the depicted example, server computer104provides information, such as boot files, operating system images, and applications to client computers110,112, and114. In this illustrative example, server computer104, server computer106, storage unit108, and client computers110,112, and114are network devices that connect to network102in which network102is the communications media for these network devices.

Client computers110,112, and114are clients to server computer104in this example. Network data processing system100may include additional server computers, client computers, and other devices not shown. Client computers110,112, and114connect to network102utilizing at least one of wired, optical fiber, or wireless connections.

Program code located in network data processing system100can be stored on a computer-recordable storage medium and downloaded to a data processing system or other device for use. For example, program code can be stored on a computer-recordable storage medium on server computer104and downloaded to client computers110,112, or114over network102for use on client devices110,112, or114.

In the depicted example, network data processing system100is the Internet with network102representing a worldwide collection of networks and gateways that use the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers consisting of thousands of commercial, governmental, educational, and other computer systems that route data and messages. Of course, network data processing system100also may be implemented using a number of different types of networks. For example, network102can be comprised of at least one of the Internet, an intranet, a local area network (LAN), a metropolitan area network (MAN), or a wide area network (WAN).FIG.1is intended as an example, and not as an architectural limitation for the different illustrative embodiments.

As used herein, “a number of” when used with reference to items, means one or more items. For example, “a number of different types of networks” is one or more different types of networks.

The phrase “at least one of,” when used with a list of items, means different combinations of one or more of the listed items can be used, and only one of each item in the list may be needed. In other words, “at least one of” means any combination of items and number of items may be used from the list, but not all of the items in the list are required. The item can be a particular object, a thing, or a category.

For example, without limitation, “at least one of item A, item B, or item C” may include item A, item A and item B, or item B. This example also may include item A, item B, and item C or item B and item C. Of course, any combinations of these items can be present. In some illustrative examples, “at least one of” can be, for example, without limitation, two of item A; one of item B; and ten of item C; four of item B and seven of item C; or other suitable combinations.

Turning toFIG.2, a block diagram of a continuous integration continuous deployment environment is depicted in accordance with an illustrative embodiment. Continuous integration continuous deployment environment200also may be referred to as a continuous integration continuous delivery environment. In this illustrative example, continuous integration continuous deployment environment200includes components that may be implemented in hardware, such as the hardware shown in network data processing system100inFIG.1.

Continuous integration continuous deployment environment200features the use of CI/CD pipeline202for the development, deployment, and operation of product204. Product204is a software product or includes software206. For example, without limitation, product204may be a software application.

CI/CD pipeline202includes various processes for the development, deployment, and operation of product204. Processes included in CI/CD pipeline202include, without limitation, development208, software configuration management210, building212, packaging214, and deployment216. Deployment216of product204may be to various different environments. For example, without limitation, deployment216may be to development environment218, integration environment220, staging environment222, or production environment224.

Product204may be tested at various points in CI/CD pipeline202. For example, without limitation, product204may be tested before building212and packaging214of product204and before deployment216of product204to one of development environment218, integration environment220, staging environment222, or production environment224.

Testing of product204may be performed by an appropriate testing system226. Testing of product204may include, for example, without limitation, testing application programming interfaces228of product204.

In accordance with an illustrative embodiment, testing system226may include relationship identifier230. Relationship identifier230is configured to use artificial intelligence to determine application programming interfaces relationships232of application programming interfaces228in product204. In accordance with an illustrative embodiment, application programming interfaces relationships232includes relationships between application programming interfaces228of product204and other appropriate characteristics234of product204. For example, without limitation, application programming interfaces relationships232may include relationships between application programming interfaces228of product204and one or more of functions236performed by product204, users238of product204, business requirements240of product,204or other characteristics242of product204.

Application programming interfaces relationships232may be used by test generator244to generate appropriate test parameters246for test248to be conducted on product204. For example, without limitation, test parameters246may include threshold250level of relationships for application programming interfaces228to be tested. Test parameters246may specify focus252of application programming interfaces228related to particular characteristics234to be tested. Testing system226may be configured to conduct test248of product204using test parameters246to generate test results254.

Testing system226may include any appropriate user interface256to provide for interaction of operator258with testing system226. Operator258may be any appropriate human operator of testing system226. Operator258may interact with user interface256via any appropriate user interface devices260.

The illustration of continuous integration continuous deployment environment200inFIG.2is not meant to imply physical or architectural limitations to the manner in which an illustrative embodiment can be implemented. Other components in addition to or in place of the ones illustrated may be used. Some components may be unnecessary. Also, the blocks are presented to illustrate some functional components. One or more of these blocks may be combined, divided, or combined and divided into different blocks when implemented in an illustrative embodiment.

With reference toFIG.3, a block diagram of a relationship identifier is depicted in accordance with an illustrative embodiment. Relationship identifier300is an example of one implementation of relationship identifier230in continuous integration continuous deployment environment200inFIG.2. Relationship identifier300includes entity builder302, entity builder tools304, relationship learner306, relationship extractor308, trainer310, relationship explorer312, and relationship reporter314.

Relationship identifier300may be implemented in software, hardware, firmware or a combination thereof. When software is used, the operations performed by relationship identifier300may be implemented in program code configured to run on hardware, such as a processor unit. When firmware is used, the operations performed by relationship identifier300may be implemented in program code and data and stored in persistent memory to run on a processor unit. When hardware is employed, the hardware may include circuits that operate to perform the operations in relationship identifier300.

In the illustrative examples, the hardware may take a form selected from at least one of a circuit system, an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device, or some other suitable type of hardware configured to perform a number of operations. With a programmable logic device, the device can be configured to perform the number of operations. The device can be reconfigured at a later time or can be permanently configured to perform the number of operations. Programmable logic devices include, for example, a programmable logic array, a programmable array logic, a field programmable logic array, a field programmable gate array, and other suitable hardware devices. Additionally, the processes can be implemented in organic components integrated with inorganic components and can be comprised entirely of organic components excluding a human being. For example, the processes can be implemented as circuits in organic semiconductors.

Entity builder302is configured to allow operator316to generate entity information318for a software product, such as product204in continuous integration continuous deployment environment200inFIG.2. Entity information318defines various characteristics of the software product. For example, without limitation, entity information318may include information defining entities320, attributes of entities322, and relationships between entities324of the product.

Entities320may include application programming interfaces326and one or more of functions328, users330, business requirements332, or other entities334. Relationships between entities324may be defined in relationships dictionary336.

Entity builder tools304may include various tools to help operator316generate entity information318by automating the process of generating entity information318, in whole or in part. For example, without limitation, entity builder tools304may include one or more of code scanner338, natural language processing tool340, definition parser342, and other appropriate tools344.

Learner306is configured to process entity information318by graph attention network346to generate application programming interface relationship network348. Application programming interface relationship network348indicates degrees of relationships between application programming interfaces326and entities320of the product defined by entity information318.

Processing entity information318by graph attention network346includes defining the classes of the entities in entity information318as:
T={t1,t2,t3, . . . }

The attributes of the entities are:

A=[[attr⁢for⁢entity⁢1],[attr⁢for⁢entity⁢2],[attr⁢for⁢entity⁢3],…].

V={V1, V2, V3, . . . }∈RFiis the attributes vectors for each of the entities.

Using the features conversion matrix:

Wti∈RF′×Fi, results in the converted entities:
X=W·V

The relationships defined in entity information318are:
R={r1,r2,r3}

The attention parameter is defined as:

αij=exp⁡(LeakyReLU⁡(Warm)T[Xi⁢Xj]))∑S∈Nirmexp⁡(LeakyReLU⁡(Warm)T[Xi⁢Xs])).

The embedded form is calculated as:

zirm=σ⁡(∑j∈Nirmαijrm·Xj).

With the same steps, the final embedded form is calculated as:
zirm=σ(Σz∈rmβrm·Zj)

Relationship extractor308is configured to determine correlation values352for the relationships between application programming interfaces326and entities320from application programming interface relationship network348.

For example, relationship extractor308may define each of the embedded form:
z′i=sigmoid(Wc·zi)=[f1,f2, . . . fc]

With c=2 (related or not), determine correlation values352as the confidence:

Pifx=exp⁡(fx)∑c=1Cexp⁡(fc).

Trainer310is configured to provide semi-supervised training354of graph attention network346using known relationships356. Known relationships356may be any knowledge of relationships between application programming interfaces326and entities320from any appropriate source. For example, without limitation, known relationships356may be from operator316, based on customer support cases, or from any other appropriate source.

Trainer310may define a loss function, such as cross-entropy function:
L=−Σl∈YLΣx=1Cylfxln(Plfx)

The loss function is used in back-propagation to update the W parameters used in graph attention network346.

Relationship explorer312is configured to identify undiscovered relationships358in application programming interface relationship network348. Undiscovered relationships358are relationships between application programming interfaces326and entities320that are not initially defined in entity information318. Relationship explorer312uses an algorithm to identify such new relationships.

Relationship reporter314generates relationship report360. Relationship report360may report degrees of relationships350between application programming interfaces326and entities320in any appropriate format. For example, without limitation, relationship report360may be tabular362report of correlation values352that indicates degrees of relationships350between application programming interfaces326and entities320. Undiscovered relationships358identified by relationship explorer312may be included in relationship report360.

The illustration of relationship identifier300inFIG.3is not meant to imply physical or architectural limitations to the manner in which an illustrative embodiment can be implemented. Other components in addition to or in place of the ones illustrated may be used. Some components may be unnecessary. Also, the blocks are presented to illustrate some functional components. One or more of these blocks may be combined, divided, or combined and divided into different blocks when implemented in an illustrative embodiment.

Turning toFIG.4, an illustration of an example of graph attention network processing is depicted in accordance with an illustrative embodiment.FIG.4illustrates an example of the conversion of entity X1400to entity z1402by a graph attention network, such as graph attention network346in relationship identifier300inFIG.3, such that the attributes of entities X2-X12404that are related to entity X1400, and the relationship between entity X1400and entities X2-X12404are used in determining the attributes for entity z1402.

Turning toFIG.5, an illustration of an example of identifying an undiscovered relationship by a relationship explorer is depicted in accordance with an illustrative embodiment. Process500inFIG.5may be performed by relationship explorer312in relationship identifier300inFIG.3.

Turning toFIG.6, an illustration of a flowchart of process600for identifying application programming interface relationships is depicted in accordance with an illustrative embodiment. For example, without limitation, process600may be implemented in relationship identifier300inFIG.3.

Process600may begin with generating entity information for a product (operation602). The entity information is processed by a graph attention network to generate an application programming interface network (operation604). Correlation values for the application programming interfaces of the product are generated from the application programming interface network (operation606).

It may be determined whether training of the graph attention network is needed or desirable (operation608). In response to a determination that training of the graph attention network is needed or desirable, the graph attention network may be trained using known relationships (operation610). Process600then may continue with processing the entity information by the trained graph attention network at operation604.

In response to a determination at operation608that training of the graph attention network is not needed or desirable, undiscovered relationships may be identified (operation612). A relationship report then may be generated (operation614). The relationship report may be used to select test parameters for testing the product (operation616), with the process terminating thereafter.

Turning toFIG.7, an illustration of a flowchart of process700for identifying undiscovered relationships is depicted in accordance with an illustrative embodiment. For example, without limitation, process700may be implemented in relationship explorer312in relationship identifier300inFIG.3. Process700is an example of one implementation of operation612in process600inFIG.6.

Process700may begin with identifying the shortest path between an application programming interface entity and another entity with an intervening entity (operation702). The minimum correlation value between adjacent entities along the shortest path is then identified (operation704). It is then determined whether the identified minimum correlation value is greater than or equal to a threshold (operation706). In response to a determination that the minimum correlation value is greater than or equal to the threshold, the relationship between the application programming interface and the other entity is added to the relationship report with the correlation value of the relationship being the minimum correlation value identified in operation704(operation708), with the process terminating thereafter.

The flowcharts and block diagrams in the different depicted embodiments illustrate the architecture, functionality, and operation of some possible implementations of apparatuses and methods in an illustrative embodiment. In this regard, each block in the flowcharts or block diagrams may represent at least one of a module, a segment, a function, or a portion of an operation or step. For example, one or more of the blocks can be implemented as program code, hardware, or a combination of the program code and hardware. When implemented in hardware, the hardware may, for example, take the form of integrated circuits that are manufactured or configured to perform one or more operations in the flowcharts or block diagrams. When implemented as a combination of program code and hardware, the implementation may take the form of firmware. Each block in the flowcharts or the block diagrams can be implemented using special purpose hardware systems that perform the different operations or combinations of special purpose hardware and program code run by the special purpose hardware.

In some alternative implementations of an illustrative embodiment, the function or functions noted in the blocks may occur out of the order noted in the figures. For example, in some cases, two blocks shown in succession can be performed substantially concurrently, or the blocks may sometimes be performed in the reverse order, depending upon the functionality involved. Also, other blocks can be added in addition to the illustrated blocks in a flowchart or block diagram.

Turning toFIG.8, a block diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system800can be used to implement server computer104, server computer106, client computer110, client computer112, and client computer114inFIG.1. Data processing system800can also be used to implement testing system226inFIG.2or relationship identifier300inFIG.3. In this illustrative example, data processing system800includes communications framework802, which provides communications between processor unit804, memory806, persistent storage808, communications unit810, input/output (I/O) unit812, and display814. In this example, communications framework802takes the form of a bus system.

Processor unit804serves to execute instructions for software that can be loaded into memory806. Processor unit804includes one or more processors. For example, processor unit804can be selected from at least one of a multicore processor, a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a network processor, or some other suitable type of processor. For example, further, processor unit804can may be implemented using one or more heterogeneous processor systems in which a main processor is present with secondary processors on a single chip. As another illustrative example, processor unit804can be a symmetric multi-processor system containing multiple processors of the same type on a single chip.

Memory806and persistent storage808are examples of storage devices816. A storage device is any piece of hardware that is capable of storing information, such as, for example, without limitation, at least one of data, program code in functional form, or other suitable information either on a temporary basis, a permanent basis, or both on a temporary basis and a permanent basis. Storage devices816may also be referred to as computer-readable storage devices in these illustrative examples. Memory806, in these examples, can be, for example, a random-access memory or any other suitable volatile or non-volatile storage device. Persistent storage808may take various forms, depending on the particular implementation.

For example, persistent storage808may contain one or more components or devices. For example, persistent storage808can be a hard drive, a solid-state drive (SSD), a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage808also can be removable. For example, a removable hard drive can be used for persistent storage808.

Communications unit810, in these illustrative examples, provides for communications with other data processing systems or devices. In these illustrative examples, communications unit810is a network interface card.

Input/output unit812allows for input and output of data with other devices that can be connected to data processing system800. For example, input/output unit812may provide a connection for user input through at least one of a keyboard, a mouse, or some other suitable input device. Further, input/output unit812may send output to a printer. Display814provides a mechanism to display information to a user.

Instructions for at least one of the operating system, applications, or programs can be located in storage devices816, which are in communication with processor unit804through communications framework802. The processes of the different embodiments can be performed by processor unit804using computer-implemented instructions, which may be located in a memory, such as memory806.

These instructions are referred to as program code, computer usable program code, or computer-readable program code that can be read and executed by a processor in processor unit804. The program code in the different embodiments can be embodied on different physical or computer-readable storage media, such as memory806or persistent storage808.

Program code818is located in a functional form on computer-readable media820that is selectively removable and can be loaded onto or transferred to data processing system800for execution by processor unit804. Program code818and computer-readable media820form computer program product822in these illustrative examples. In the illustrative example, computer-readable media820is computer-readable storage media824.

In these illustrative examples, computer-readable storage media824is a physical or tangible storage device used to store program code818rather than a medium that propagates or transmits program code818.

Alternatively, program code818can be transferred to data processing system800using a computer-readable signal media. The computer-readable signal media can be, for example, a propagated data signal containing program code818. For example, the computer-readable signal media can be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals can be transmitted over connections, such as wireless connections, optical fiber cable, coaxial cable, a wire, or any other suitable type of connection.

The different components illustrated for data processing system800are not meant to provide architectural limitations to the manner in which different embodiments can be implemented. In some illustrative examples, one or more of the components may be incorporated in or otherwise form a portion of, another component. For example, memory806, or portions thereof, may be incorporated in processor unit804in some illustrative examples. The different illustrative embodiments can be implemented in a data processing system including components in addition to or in place of those illustrated for data processing system800. Other components shown inFIG.8can be varied from the illustrative examples shown. The different embodiments can be implemented using any hardware device or system capable of running program code818.

The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.

Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be accomplished as one step, executed concurrently, substantially concurrently, in a partially or wholly temporally overlapping manner, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiment. The terminology used herein was chosen to best explain the principles of the embodiment, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed here.