Method and system for enabling self-maintainable test automation

The present disclosure relates to a method and a system for enabling self-maintainable test automation. In one embodiment, the system creates a test automation suite using historical test scenarios and automatically updates the test automation suite based on functional changes in one or more related applications. The system determines one or more reusable automation units and one or more test data units that are affected by the functional changes identified in test scenarios received as input and accordingly updates the one or more reusable automation units and one or more test data units. Thus, the system enables self-maintainable test automation, thereby eliminating the effort and expertise required to create automation test suite, build automation scripts, and modify automation scripts for future enhancements.

FIELD OF THE DISCLOSURE

The present subject matter is related, in general to automated testing of computer applications, and more particularly, but not exclusively to a method and a system for enabling self-maintainable test automation.

BACKGROUND

Traditional testing of a computer program can include the external monitoring of integrity of the program and performance of the program, either subjectively based upon end user impression, or objectively based upon independently acquired metrics. Often, to test an application, testing personnel must establish and configure a testing environment. Within the testing environment, a test protocol can be defined for exercising a computing application. The individual steps and portions of the testing protocol, in turn, can be automated through operating system scripts, batch files and the like. In this regard, for a test protocol, a typical test cycle can include multiple phases that test a system from various angles, such as functional, performance, and system test phases.

Test automation brings down Test cycles resulting in faster and frequent release of software to market. However, current Test Automation frameworks are highly technical and require automation experts to deliver Automated Test Suites. Once an automated Test Suite is created, modifying it to incorporate future changes in application is very tedious, effort intensive and requires involvement of both Automation and Domain expertise. Therefore, there is a need for a method and a system that enables self-maintainable test automation eliminating the effort and expertise required to create, modify and maintain Test Automation Suites.

SUMMARY

One or more shortcomings of the prior art are overcome and additional advantages are provided through the present disclosure. Additional features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the disclosure are described in detail herein and are considered a part of the claimed disclosure.

Accordingly, the present disclosure relates to a method of enabling self-maintainable test automation. The method comprising the step of receiving one or more input test scenarios associated with one or more applications executing in a real time environment. The method further comprising comparing the one or more input test scenarios with one or more corresponding historical test scenarios to identify one or more functional changes associated with the one or more input test scenarios. Based on the comparison, one or more reusable automation units of the one or more input test scenarios are determined. Further, one or more test data units corresponding to the one or more reusable automation units are determined. The one or more reusable automation units and the one or more test data units thus determined above are updated based on the one or more functional changes thus identified.

Further, the present disclosure relates to a system for enabling self-maintainable test automation. The system comprises at least a processor and a test automation repository coupled with the processor and configured to store a test automation suite and one or more historical test scenarios. The system further comprises a memory communicatively coupled with the processor, wherein the memory stores processor-executable instructions, which, on execution, cause the processor to receive one or more input test scenarios associated with one or more applications executing in a real time environment. The processor is configured to compare the one or more input test scenarios with the one or more corresponding historical test scenarios to identify one or more functional changes associated with the one or more input test scenarios. Based on the comparison, the processor is configured to determine one or more reusable automation units of the one or more input test scenarios and one or more test data units corresponding to the one or more reusable automation units. The processor further updates the one or more reusable automation units and the one or more test data units based on the one or more functional changes thus identified.

Furthermore, the present disclosure relates to a non-transitory computer readable medium including instructions stored thereon that when processed by at least one processor cause a system to perform the act of one or more input test scenarios associated with one or more applications executing in a real time environment. Further, the instructions cause the processor to compare the one or more input test scenarios with one or more corresponding historical test scenarios to identify one or more functional changes associated with the one or more input test scenarios. Based on the comparison, the processor determines one or more reusable automation units of the one or more input test scenarios. The processor is further configured to determine one or more test data units corresponding to the one or more reusable automation units and update the one or more reusable automation units and the one or more test data units based on the one or more functional changes thus identified.

DETAILED DESCRIPTION

The present disclosure relates to a method and a system for enabling self-maintainable test automation. In one embodiment, the system creates a test automation suite using historical test scenarios and updates the test automation suite automatically based on functional changes in one or more related applications. The system determines one or more reusable automation units and one or more test data units that are affected by the functional changes identified in test scenarios received as input and accordingly updates the one or more reusable automation units and one or more test data units with the functional changes. Thus, the system enables self-maintainable test automation, thereby eliminating the effort and expertise required to create automation test suite, build automation scripts, and modify automation scripts for future enhancements.

FIG. 1illustrates an architecture diagram of an exemplary system for enabling self-maintainable test automation in accordance with some embodiments of the present disclosure.

As shown inFIG. 1, the exemplary system100comprises one or more components configured for enabling self-maintainable test automation. In one embodiment, the exemplary system100comprises a test automation system (hereinafter referred to as TAS)102, and a test automation repository104connected via a communication network106. The test automation repository104is configured to store one or more historical test scenarios108and one or more test automation suite110.

The one or more test automation suite110is a reusable test automation framework comprising one or more reusable automation units112and one or more test data units114associated with the one or more reusable automation units112. The one or more reusable automation units (RAUs)112may be for example, a collection or logical grouping of one or more steps that can be performed frequently. The one or more reusable test data units (TDUs)114may be for example, a collection or logical group of one or more test data that can be used by the one or more RAUs112. The one or more RAUs112and the one or more TDUs114are arranged in one or more orders to generate one or more test automation suite110and stored in the test automation repository104. In one embodiment, the test automation repository104may be integrated within the TAS102. In another embodiment, the test automation repository104may be configured independent without being integrated within the TAS102.

The TAS102comprises at least a processor116and a memory118coupled with the processor116. The TAS102further comprises a test automation suite creation module (hereinafter referred to as “TAS creation module”)120and a test automation suite updating module (hereinafter referred to as “TAS updation module”)122coupled with the processor116. The TAS creation module120is configured to create one or more test automation suite110and the TAS updation module122is configured to update the one or more test automation suite110based on one or more functional changes associated with execution of one or more applications in a real time environment.

In one embodiment, the TAS102may be a typical TAS as illustrated inFIG. 2a. The TAS102comprises the processor116, the memory118, and an I/O interface202. The1/O interface202is coupled with the processor116and an I/O device. The I/O device is configured to receive inputs via the I/O interface202and transmit outputs for displaying in the I/O device via the I/O interface202. The TAS102further comprises data204and modules206. In one implementation, the data204and the modules206may be stored within the memory118. In one example, the data204may include one or more input test scenarios208, the one or more functional changes210, one or more input user actions212, one or more object information214and other data216. In one embodiment, the data204may be stored in the memory118in the form of various data structures. Additionally, the aforementioned data can be organized using data models, such as relational or hierarchical data models. The other data216may be also referred to as reference repository for storing recommended implementation approaches as reference data. The other data216may also store data, including temporary data and temporary files, generated by the modules206for performing the various functions of the TAS102.

The modules206may include, for example, the TAS creation module120, the TAS updating module122, and a test automation graphical user interface module (TAGUI)218. The modules206may also comprise other modules218to perform various miscellaneous functionalities of the TAS102. As illustrated inFIG. 2b, the TAS creation module120includes sub-modules such as an application monitoring agent220, an action segregator agent222, a data profiler agent224and an expert agent226. Further, as illustrated inFIG. 2c, the TAS updating module122includes sub-modules such as an application change monitoring agent228and a test automation suite (TAS) managing agent230. It will be appreciated that such aforementioned modules may be represented as a single module or a combination of different modules. The modules206may be implemented in the form of software, hardware and/or firmware.

In operation, the TAS102creates a self-maintainable test automation suite110for automating test scenarios. In one embodiment, the TAS creation module120creates the one or more test automation suites110based on one or more historical test scenarios108. The one or more historical test scenarios108can include a collection of test scenarios, each test scenario may be arranged from an ordered selection of steps and supplemental test data based on one or more test user actions. For example, a test scenario for an e-commerce application to register a user can include the ordered combination of application specific user login function, an application specific user registration function and an application specific verification function. In addition, the test scenario may also include specific user credentials and user details for use when a user is registered through the user registration scenario. In one implementation, the one or more historical test scenarios108may be created by a conventional test scenario generator known in the art.

In another example, a test scenario for an e-commerce application to place order for a product can include the ordered combination of application specific user login function, an application specific search function, add to bag function, checkout function, shipping and billing function, payment function, place order function and verification function. In addition, the test scenario may also include specific user credentials, specific product details, specific user details like address, payment details and so on.

The application monitoring agent220is configured to track one or more applications that are loaded into a test environment comprising the TAS102for testing purposes. The test environment enables loading and execution of the one or more applications for testing. One or more applications may include one or more versions or releases of test applications that may be executed for testing purposes. The application monitoring agent220monitors the one or more applications, for example an e-commerce application that are executed by the TAS102. In one embodiment, the application monitoring agent220records one or more training test scenarios of the one or more applications such as user registration, place order by search and credit card and place order by browse and gift card etc. and clusters the one or more training test scenarios based on logical similarity among the one or more training test scenarios. In one implementation, the application monitoring agent220derives the one or more user actions corresponding to the one or more training test scenarios received during the execution of the one or more applications. For example, the application monitoring agent220derives the one or more user actions such as login, registration, browse, search, add to bag, checkout, payment, shipping, billing, place order and verify and so on. The action segregator agent222processes and groups the one or more user actions into one or more automated system actions based on logical similarity among the one or more user actions thus derived. For example, the action segregator agent222creates the one or more automated system actions based on grouping of the one or more similar user actions like for example, browse and search grouped together, shipping and billing grouped together etc. Based on the one or more automated system actions thus created, the expert agent226determines the one or more RAUs112and the one or more TDUs114.

In one embodiment, the expert agent226compares the one or more training test scenarios with historical test scenarios108recorded in the past and determines one or more similar automated system actions based on comparison. The expert agent226further determines the one or more RAUs112comprising the one or more similar automated systems, for example, login, browse, and payment and so on. The data profiler agent224determines the one or more test data associated with the one or more similar automated system actions for example, user credentials, product details and credit card or gift card details etc., and creates the one or more TDUs114based on the one or more test data thus determined. Upon creating the one or more RAUs112and the one or more TDUs114, the expert agent226creates the one or more test automation suite110comprising the one or more RAUs112and the one or more TDUs114and stores the one or more test automation suite110in the test automation repository104. The one or more test automation suite110thus created shall be used to enable self-maintainable test automation in real time environment.

In addition, the expert agent226is also configured to store information associated with one or more objects or controls associated with the one or more applications thus executed. In one embodiment, the expert agent226receives and records key strokes associated with the one or more user actions associated with the one or more training test scenarios and associates the recorded key strokes with one or more objects. For each object, the expert agent226determines the one or more object information comprising at least a location or a unique resource identifier and one or more properties associated with each object and store the one or more object information in the test automation repository104.

In real time environment, the TAS updating module112is configured to automatically update the test automation suite110at any time when there is one or more functional changes identified in one or more applications. In one embodiment, the application change monitoring agent (ACMA)228receives one or more input test scenarios208associated with one or more application executing in the real time environment. In one example, the one or more input test scenarios includes one or more input user actions212associated with the one or more input test scenarios208. Upon receiving, the ACMA228determines the one or more RAUs corresponding to the one or more input test scenarios208. In one implementation, the ACMA228compares the one or more input test scenarios208with one or more corresponding historical test scenarios108stored in the test automation repository104and identifies one or more functional changes210associated with the one or more input test scenarios208based on the comparison. Upon the identification, the ACMA228determines the one or more RAUs affected by the one or more functional changes. In one implementation, the ACMA228determines the one or more RAUs comprising one or more user actions similar to the one or more input user actions212using cognitive and machine learning methods. Based on the determination of the one or more RAUs, the ACMA228further determines the one or more TDUs corresponding to the one or more RAUs thus determined. The TAS managing agent230updates the one or more RAUs and the one or more TDUs with the one or more identified functional changes210and accordingly updates the one or more test automation suite110with the updated RAUs and TDUs.

Further, the ACMA228determines one or more objects associated with the one or more input user actions212and updates the one or more object information related with the one or more determined objects. For example, the ACMA228receives and records key strokes associated with the one or more input user actions212and identifies one or more associated objects or controls. The ACMA228determines the one or more object information corresponding to the one or more associated objects or controls and updates the object information with the one or more functional changes210.

In one embodiment, the ACMA228updates the one or more object information such as location and/or properties associated with the one or more objects and stores the updated object information in the test automation repository104. In one embodiment, the one or more object information may be stored along with the one or more RAUs112and the one or more TDUs114of the one or more test automation suite110in the test automation repository104. In another embodiment, the one or more object information may be stored independent of the one or more test automation suite110along with rules defining the mapping of the one or more object information with the one or more test automation suite110.

Thus, the above disclosed system enables self-maintainable test automation, thereby eliminating the effort and expertise required to create automation test suite, build automation scripts, and modify automation scripts for future enhancements.

FIG. 3aillustrates a flowchart of a method of creation of test automation suite in accordance with some embodiments of the present disclosure.

As illustrated inFIG. 3a, the method300comprises one or more blocks implemented by the processor116for creation of test automation suite. The method300may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform particular functions or implement particular abstract data types.

At block302, receive training test scenario. In one embodiment, the application monitoring agent220is configured to track one or more applications that are loaded into a test environment comprising the TAS102for testing purposes. The test environment enables loading and execution of the one or more applications for testing. One or more applications may include one or more versions or releases of test applications that may be executed for testing purposes. The application monitoring agent220monitors the one or more applications, for example an e-commerce application that are executed by the TAS102.

At block304, determine automated system actions. In one embodiment, the application monitoring agent220records one or more training test scenarios of the one or more applications such as user registration, place order by search and credit card and place order by browse and gift card etc. and clusters the one or more training test scenarios based on logical similarity among the one or more training test scenarios. In one implementation, the application monitoring agent220derives the one or more user actions corresponding to the one or more training test scenarios received during the execution of the one or more applications. For example, the application monitoring agent220derives the one or more user actions such as login, registration, browse, search, add to bag, checkout, payment, shipping, billing, place order and verify and so on. The action segregator agent222processes and groups the one or more user actions into one or more automated system actions based on logical similarity among the one or more user actions thus derived. For example, the action segregator agent222creates the one or more automated system actions based on grouping of the one or more similar user actions like for example, browse and search grouped together, shipping and billing grouped together etc. Based on the one or more automated system actions thus created, the expert agent226determines the one or more RAUs112and the one or more TDUs114.

At block306, determine reusable automation units (RAU) and test data units (TDU). In one embodiment, the expert agent226compares the one or more training test scenarios with historical test scenarios108recorded in the past and determines one or more similar automated system actions based on comparison. The expert agent226further determines the one or more RAUs112comprising the one or more similar automated systems, for example, login, browse, and payment and so on. The data profiler agent224determines the one or more test data associated with the one or more similar automated system actions for example, user credentials, product details and credit card or gift card details etc., and creates the one or more TDUs114based on the one or more test data thus determined. Upon creating the one or more RAUs112and the one or more TDUs114, the expert agent226creates the one or more test automation suite110.

At block308, create test automation suite. In one embodiment, the expert agent226creates the one or more test automation suite110comprising the one or more RAUs112and the one or more TDUs114and stores the one or more test automation suite110in the test automation repository104. The one or more test automation suite110thus created shall be used to enable self-maintainable test automation in real time environment.

In addition, the expert agent226is also configured to store information associated with one or more objects or controls associated with the one or more applications thus executed. In one embodiment, the expert agent226receives and records key strokes associated with the one or more user actions associated with the one or more training test scenarios and associates the recorded key strokes with one or more objects. For each object, the expert agent226determines the one or more object information comprising at least a location or a unique resource identifier and one or more properties associated with each object and store the one or more object information in the test automation repository104.

Thus, the above disclosed method300enables creation of self-maintainable test automation, thereby eliminating the effort and expertise required to create automation test suite, build automation scripts, and modify automation scripts for future enhancements.

FIG. 3billustrates a flowchart of a method of enabling self-maintainable test automation in accordance with some embodiments of the present disclosure.

As illustrated inFIG. 3b, the method310comprises one or more blocks implemented by the processor116for enabling self-maintainable test automation. The method310may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform particular functions or implement particular abstract data types.

The order in which the method310is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method310. Additionally, individual blocks may be deleted from the method310without departing from the spirit and scope of the subject matter described herein. Furthermore, the method310can be implemented in any suitable hardware, software, firmware, or combination thereof.

At block312, receive input test scenarios. In real time environment, the TAS updating module112is configured to automatically update the test automation suite110at any time when there is one or more functional changes identified in one or more applications. In one embodiment, the application change monitoring agent (ACMA)228receives one or more input test scenarios208associated with one or more application executing in the real time environment. In one example, the one or more input test scenarios includes one or more input user actions212associated with the one or more input test scenarios208. Upon receiving, the ACMA228determines the one or more RAUs corresponding to the one or more input test scenarios208.

At block314, identify RAU and TDU based on input test scenarios. In one embodiment, the ACMA228determines the one or more RAUs corresponding to the one or more input test scenarios208. In one implementation, the ACMA228compares the one or more input test scenarios208with one or more corresponding historical test scenarios108stored in the test automation repository104and identifies one or more functional changes210associated with the one or more input test scenarios208based on the comparison. Upon the identification, the ACMA228determines the one or more RAUs affected by the one or more functional changes. In one implementation, the ACMA228determines the one or more RAUs comprising one or more user actions similar to the one or more input user actions212using cognitive and machine learning methods. Based on the determination of the one or more RAUs, the ACMA228further determines the one or more TDUs corresponding to the one or more RAUs thus determined.

At block316, update test automation suite. In one embodiment, the TAS managing agent230updates the one or more RAUs and the one or more TDUs with the one or more identified functional changes210and accordingly updates the one or more test automation suite110with the updated RAUs and TDUs.

Further, the ACMA228determines one or more objects associated with the one or more input user actions212and updates the one or more object information related with the one or more determined objects. For example, the ACMA228receives and records key strokes associated with the one or more input user actions212and identifies one or more associated objects or controls. The ACMA228determines the one or more object information corresponding to the one or more associated objects or controls and updates the object information with the one or more functional changes210.

In one embodiment, the ACMA228updates the one or more object information such as location and/or properties associated with the one or more objects and stores the updated object information in the test automation repository104. In one embodiment, the one or more object information may be stored along with the one or more RAUs112and the one or more TDUs114of the one or more test automation suite110in the test automation repository104. In another embodiment, the one or more object information may be stored independent of the one or more test automation suite110along with rules defining the mapping of the one or more object information with the one or more test automation suite110.

Thus, the above disclosed method310enables self-maintainable test automation, thereby eliminating the effort and expertise required to create automation test suite, build automation scripts, and modify automation scripts for future enhancements.

FIG. 4is a block diagram of an exemplary computer system for implementing embodiments consistent with the present disclosure.

Variations of computer system401may be used for implementing all the computing systems that may be utilized to implement the features of the present disclosure. Computer system401may comprise a central processing unit (“CPU” or “processor”)402. Processor402may comprise at least one data processor for executing program components for executing user- or system-generated requests. The processor may include specialized processing units such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. The processor402may include a microprocessor, such as AMD Athlon, Duron or Opteron, ARM's application, embedded or secure processors, IBM PowerPC, Intel's Core, Itanium, Xeon, Celeron or other line of processors, etc. The processor402may be implemented using mainframe, distributed processor, multi-core, parallel, grid, or other architectures. Some embodiments may utilize embedded technologies like application-specific integrated circuits (ASICs), digital signal processors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.

Using the I/O interface403, the computer system401may communicate with one or more I/O devices. For example, the input device404may be an antenna, keyboard, mouse, joystick, (infrared) remote control, camera, card reader, fax machine, dongle, biometric reader, microphone, touch screen, touchpad, trackball, sensor (e.g., accelerometer, light sensor, GPS, gyroscope, proximity sensor, or the like), stylus, scanner, storage device, transceiver, video device/source, visors, etc. Output device405may be a printer, fax machine, video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, or the like), audio speaker, etc. In some embodiments, a transceiver406may be disposed in connection with the processor402. The transceiver may facilitate various types of wireless transmission or reception. For example, the transceiver may include an antenna operatively connected to a transceiver chip (e.g., Texas Instruments WiLink WL1283, Broadcom BCM47501UB8, Infineon Technologies X-Gold 618-PMB9800, or the like), providing IEEE 802.11a/b/g/n, Bluetooth, FM, global positioning system (GPS), 2G/3G HSDPA/HSUPA communications, etc.

In some embodiments, the processor402may be disposed in communication with a communication network408via a network interface407. The network interface407may communicate with the communication network408. The network interface407may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10/40/400 Base T), transmission control protocol/internet protocol (TCP/IP), token ring, IEEE 802.11a/b/g/n/x, etc. The communication network408may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface407and the communication network408, the computer system401may communicate with devices409,410, and411. These devices may include, without limitation, personal computer(s), server(s), fax machines, printers, scanners, various mobile devices such as cellular telephones, smartphones (e.g., Apple iPhone, Blackberry, Android-based phones, etc.), tablet computers, eBook readers (Amazon Kindle, Nook, etc.), laptop computers, notebooks, gaming consoles (Microsoft Xbox, Nintendo DS, Sony PlayStation, etc.), or the like. In some embodiments, the computer system401may itself embody one or more of these devices.

As described above, the modules208, amongst other things, include routines, programs, objects, components, and data structures, which perform particular tasks or implement particular abstract data types. The modules208may also be implemented as, signal processor(s), state machine(s), logic circuitries, and/or any other device or component that manipulate signals based on operational instructions. Further, the modules208can be implemented by one or more hardware components, by computer-readable instructions executed by a processing unit, or by a combination thereof.