Abstract:
The present invention provides a method and a system for a universal software quality assurance automation framework. The three reusable components of this framework are composed of a test resource comprising of a test module-entity driver-entity communication. The test module provides an opportunity to create several case scenarios and test logics. The entity driver enables the test environment software entities to be accessible to the test module, without prior knowledge of where those entities are located. The entity communication enables the drivers to communicate with various entities inside the test environment. The combination of the three reusable components enable the framework to be product agnostic. Multiple tests may be performed in parallel. Test cases are presented in the integrated graphical user interface as a hierarchical managing structure. The framework is collaborative and multiple users may use it simultaneously.

Description:
FIELD OF TECHNOLOGY 
       [0001]    This disclosure relates generally to software testing and more particularly to provide a universal framework for software quality assurance testing automation. 
       BACKGROUND 
       [0002]    Software quality assurance (QA) testing is performed to evaluate an attribute or capability of a program or system and determining that it meets its required results. Software quality assurance testing has lot of challenges both in manual as well as in automation. Although crucial to software quality and widely deployed by quality assurance engineers, software quality assurance testing still remains an art, due to limited availability of single piece of framework that would be product and operating system agnostic. Software testing may be a trade-off between budget, time and quality. 
         [0003]    Software quality assurance testing for different products and platforms is done using disparate tools and custom built modules. This creates an undue burden on the quality assurance engineer and is difficult for the quality assurance engineer to go back and evaluate the testing code. Each quality assurance engineer has their own method of coding test scripts which creates a deficit of uniform test code monitoring system. 
         [0004]    There is currently no defined process to start software quality assurance test automation. Quality assurance engineers tend to mix the test case logic and the product functionality together, create a monolithic piece of application, which makes readability, understanding and reuse of existing code very difficult. 
       SUMMARY 
       [0005]    Embodiments of the present invention address deficiencies of the art in respect to quality assurance testing framework for software. An embodiment may provide a novel and non-obvious method and system for reusable software for a universal quality assurance automation framework. 
         [0006]    In one embodiment, the test module provides an opportunity for the quality assurance engineer to implement test logic for various test cases. The code generated in another embodiment is transparently communicated to the entities in the test environment. 
         [0007]    In one embodiment, the entity driver exposes the functionality of the product under test, and other entities in the test environment, to the test module. In another embodiment, the connection between the entity driver and underlying entity is facilitated by the entity communication module. 
         [0008]    In one embodiment, the quality assurance automation framework provides integrated graphical user interface to define the test resource and generate template source code. 
         [0009]    In one embodiment, the quality assurance automation framework provides a hierarchical authoring tool. In another embodiment, the quality assurance framework provides a management interface, comprehensive test report and activity logging system. 
         [0010]    In one embodiment, the quality assurance automation framework is supported by a database to store, retrieve and present the entire framework activities. In another embodiment, the test infrastructure is extensible and easy to maintain. 
         [0011]    In one embodiment, the quality assurance automation framework comprises of sequential and parallel test case execution engines. In another embodiment, the present quality assurance framework client side does not require any end-user configuration to start with. The ease of authoring and execution enable the quality assurance engineer to focus on product specific functionalities and test logic. 
         [0012]    In one embodiment the quality assurance framework may be product and operating system agnostic. In another embodiment, the test resource code is accessible as a shared library and may be reused by quality assurance engineers. 
         [0013]    The methods and systems disclosed herein may be implemented in any means for achieving various aspects, and may be executed in a form of a machine-readable medium embodying a set of instructions. Other features will be apparent from the accompanying drawings and from the detailed description that follows. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    Example embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which: 
           [0015]      FIG. 1  is a diagrammatic illustration of the universal quality assurance automation framework  100 . 
           [0016]      FIG. 1A  is an expanded view of test module  104 . 
           [0017]      FIG. 1B  is another view of automation framework  300 . 
           [0018]      FIG. 2  is a network view of the software QA automation  200  in a network environment for performing the test. 
           [0019]      FIG. 3  is a process flow view of the method for quality assurance test (QAT)  300 . 
           [0020]      FIG. 4  is an exploded view of the computer device  600  and its various components being used for quality assurance testing. 
           [0021]      FIG. 5  is a view of the integrated graphical user interface (IGUI)  102 A. 
           [0022]      FIG. 6  is another view of the IGUI  102 A showing hierarchical view of test case management. 
           [0023]      FIG. 7  is another illustration of the IGUI  102 A showing various test resource parameters. 
           [0024]      FIG. 8  is a view of IGUI  102 A showing one test module of test logic for various test scenarios. 
           [0025]      FIG. 9  is a view of IGUI  102 A showing execution results for a sequence of test case scenarios. 
           [0026]      FIG. 10  is a view of IGUI  102 A showing detailed reports for a sequence of test case scenarios. 
       
    
    
       [0027]    Other features of the present embodiments will be apparent from the accompanying drawings and from the detailed description that follows. 
       DETAILED DESCRIPTION 
       [0028]    Several methods and systems for quality assurance testing of the software using test modules, entity driver and entity communication as test resources are disclosed. Although the present embodiments have been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the various embodiments. 
         [0029]      FIG. 1  is a diagrammatic illustration of the universal quality assurance automation framework  100 . The universal quality assurance automation framework  100  comprises of a test resource  102 . The test resource  102  comprises of three parts, test module  104 , entity driver  106  and entity communication  108 . The test module  104  comprises of various test scenarios for individual test cases. The test module  104  enables the user (e.g., quality assurance engineer, manager, enterprise data management team personal etc.) to create a test case and save the test case in the universal quality assurance automation framework  100 . The entity driver  106  enables a test module to access functionality of the quality assurance software and any other software that being tested with, and seamlessly executes the task. The entity communication  108  is the enabling link between the entity driver  106  and the quality assurance software being tested and the entity software used for testing. The entity software could be any established software such as Turbo Tax®, any web application such as Google® Gmail application, etc. 
         [0030]    The three components of the testing software, test module  104 , entity driver  106 , and entity communication  108 , together with the management system, are executed using a hardware having a processor  110 , a memory  114  connected to a database  112 . The universal quality assurance automation framework  100  may be employed for quality assurance testing software tools, which execute tests without manual intervention. This may also be applied in functional, system, performance, API, etc. testing. The universal quality assurance automation framework  100  may be invoked directly through user interaction, or accessed through the use of computer programs and/or scripts. Universal quality assurance automation framework  100  is suitable for use both by developers for unit tests and quality assurance for functional, system, and other testing of any software. 
         [0031]    The test module  104  in  FIG. 1A  comprises of an integrated graphical user interface (GUI)  102 A that allows the user to input test data  106 A, test execution logic, create and view test page. The GUI also provides a transparent connection to the entity driver  108 A. The results page A may be viewed by the user after running a software test using the universal quality assurance automation framework  100 . The script page  122 A enables the user to input test execution logic for a series of given test scenarios being tested for quality assurance testing. These script pages  122 A can be reused, edited and quality assurance testing for the software can be done. Various users such as a quality assurance engineer might have a different view such as an engineer view  124 A that may be different from that of a manager view  126 A. The engineer view may allow the quality assurance engineer to enter test execution logic and new test cases, whereas the manager view  126 A may be viewed by the manager and may not have write functions. The manager may for example have results view function and write comments for the quality engineer to review. 
         [0032]    The universal quality assurance automation framework  100  may be used by a single user or multiple users in a collaborative view  128 A. Multiple users may be able to use the same universal quality assurance automation framework  100  throughout the company and may be collaboratively use scripts for reuse, modify existing scripts for improving or changing the test execution logic. 
         [0033]      FIG. 1B  shows a process flow for the universal quality assurance automation framework  100  (framework  100 B). Multiple entities are test subjects and are located outside the framework  100 B and are connected to the test resource  102  via the framework  100 B. The framework  100  may have an environment management  102 B component that comprises of an entity communication  108 . Several entity communications may be present as shown in  FIG. 1B  as  108 B to  108 B-N. The entity communication  108  may communicate with entity drivers  106  on one end and communicate and open up the functionality of several entities  110 B to  110 B-N on one end to facilitate the quality assurance testing. Several entity drivers are shown as  106 B to  106 B-N. The test modules listed as  104 B-A to  104 B-N may reference to single entity driver  106  or multiple drivers at the same time during quality assurance testing. The test case management  104 B may be enabled by integrated graphical user interface  102 A (IGUI). The entity driver  106 B-A to  106 B-N, collectively test case management  104 B and environment management  102 B may also communicate with the database  112  to access information and parameters for the quality assurance test. The test case management may be managed by IGUI comprises of several test modules ( 104 ) in one embodiment. These test modules  104  are managed with entity driver  106  and entity communication  108  for testing using outside resources such as entity  110 . The data produced by the quality assurance test, used by the test resources such as environment management  102 B and test case management  104 B, is stored into and retrieved from the database  112 . 
         [0034]    In another embodiment, the entity driver&#39;s function may be translated by the entity communication  108  to drive to the entity ( 110 B,  104 B-C,  110 B-N). The entity  110 B-A-N may be classified in two categories such as entity-under-test and/or entity-for-test. They may be treated functionally separate and may comprise of but not limited to a website (UI or API), a web service provider, a GUI software application, a firmware image, a piece of test equipment, data generator and a power switch. 
         [0035]      FIG. 2  is a network view of the software quality assurance automation  200  for performing the test. The test scenario  210 , test logic  212  and test automation  214  as a part of display on the GUI as a test module may reside on a server  216 , and execute on client computers  202  (A, B, N . . . ) using an internet  220 . A database  112  may also be connected to the computers and/or the server using the internet  220 . The connection may be WAN and/or LAN. 
         [0036]    In one embodiment,  FIG. 3  is a view of the method for quality assurance test (QAT)  300 . The quality assurance engineer (QAE)  330  uses the collaborative test automation GUI or individual GUI and uses the test module  104  to create a quality assurance software test scenario. The user QAE  330  creates a test scenario based on the need for the type of quality assurance test that needs to be performed. In  FIG. 3 , the use of software to control the execution of tests using the run QAT  312 , the comparison of actual outcomes to predicted outcomes using data from database  112 , the setting up of test preconditions in the test module  104 , and other test control and test reporting functions in the report generation  314  is done. Once the quality assurance test is performed, after observing the report generation  314  step the user can change parameters  316  if the results are not optimal such as fail or if pass  318  then the test concludes at step  320  and the user can start the new test all over again. The execution of test module  104 , together with entity driver  106 , is facilitated by entity communication  108  to expose the entity functionality  310  to be used for test logic and for testing. The test results may be stored in the database  112  for future retrievals and audit. 
         [0037]    In one embodiment, a quality assurance engineer may create a quality assurance test case  308  for checking the functionality of a mail service, for example Gmail®, Yahoo Mail® or Hotmail® using the universal quality assurance automation framework  100 . The quality assurance engineer may create a set of test scenario  210 , such as invoking the correct browser, checking for logging in function, entering the user name and password, sending text and/or email, receiving the text and/or email, opening the email, opening the attachment, deleting the email, storing the email, composing an email, and logging out of account. Using the GUI  302 , the QAE  330  would write the test logic  212  against entity driver  106 . Once the whole steps of process are saved, the automation framework runs the quality assurance test (QAT) and stores the information in the database. The entity communication  108  further facilitates the entity driver  106  to get access to entity functionality  310 , such as an URL access in this example and all the fields required by the mail server to be filed in before all the functions of that particular software may be tested for quality assurance. The run QAT  312  in the invention is novel because test module  104  runs against the entity driver  106  without the need to know where the entity  310  is actually located. The implementation of entity driver  106  and entity communication  108  may be independent to test module  104 . 
         [0038]    The universal quality assurance automation framework  100  is product agnostic and the is capable of accessing any product from the test station computer using protocols such as http, soap, telnet, ssh, sql, snmp, serial, gpib etc. It is not limited to the stated examples. The universal quality assurance automation framework  100  is also platform agnostic and may be used using any test station operating system and/or test development programming language. 
         [0039]    Report data is generated throughout the run of QAT  312  function, and stored in the database  112 . The test may have met the criteria set by the quality assurance engineer and it may state as pass in the report. If the criteria is not met or some other parameter may need to be added for testing then the change of parameter at step  316  may be performed and pass/fail  318  results may be obtained. The test may be finished at step  320  or another new test may be started at  320 . 
         [0040]      FIG. 4  is an exploded view of the computer device  400  and its various components being used for quality assurance testing.  FIG. 4  is a diagrammatic representation of a data processing system of the universal quality assurance automation framework  100  of  FIG. 1  capable of processing a set of instructions to perform one or more of methodologies described herein, according to one embodiment. In various embodiments, the data processing system operates as a standalone device and for may be connected (e.g., networked through the network  220 ) to other machines. In a network deployment, the data processing system may operate as a workstation (e.g., to provide a graphic user interface) which connects to the universal quality assurance automation framework  100  of  FIG. 1  directly and/or via the network  220 . The data processing system may be any one or any collection of a personal computer (PC), a tablet PC, a set-top box (STB), a web appliance, a network router, a switch and/or a bridge, an embedded system, and/or any machine capable of executing a set of instructions (sequential and/or otherwise) that specify actions to be taken by the machine. 
         [0041]    One example of the data processing system may include a processor  112  (e.g., a central processing unit (CPU) or the CPU and a graphics processing unit (GPU)), a main memory  404 , and a static memory  406 , which communicate to each other via a bus  434 . The data processing system may further include a video display unit  420  (e.g., a liquid crystal display (LCD) and/or a cathode ray tube (CRT)), an alpha-numeric input device  422  (e.g., a keyboard), a cursor control device  424  (e.g., a mouse), a drive unit  426 , a signal generation device  428  (e.g., a speaker), and a network interface device  408 . 
         [0042]    The drive unit  426  may include a machine-readable medium  430  on which is stored one or more sets of instructions (e.g., instruction  432 ) embodying any one or more of the methodologies and/or functions described herein. The instruction  432  may also reside, completely and/or at least partially, within the main memory  404  and/or within the processor  402  during the execution thereof by the data processing system, wherein the main memory  404  and the processor  402  may also constitute machine-readable media. 
         [0043]    The instruction  432  may further be transmitted and/or received over the network  110  via the network interface device  408 . While the machine-readable medium  430  is shown in an example embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium and/or multiple media (e.g., a centralized and/or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” should also be taken to include any medium that is capable of storing, encoding, and/or carrying a set of instructions for execution by the machine and that causes the machine to perform any one or more of the methodologies of the various embodiments. The “machine-readable medium” shall accordingly be taken to include, but not limited to, solid-state memories, optical and magnetic media, and carrier wave signals. 
         [0044]      FIG. 5  is a view of the integrated graphical user interface (IGUI)  102 A. IGUI  102 A is the tool for quality assurance engineer to view and have a collaborative environment to create and execute the test cases for test module  104 . In  FIG. 5 , an example of the IGUI is shown as a split panel. The left side panel shows a hierarchical tree  512  for various test cases organized in test project and test script, such as  502  and  508  in one view. The collaborative view enables the quality assurance engineer to reuse the existing test resource and modify the test execution logic  212  for further use. The right panel is the authoring interface for various levels of details, and execution interface a test script. The detailed time stamp, for example, is one of the properties that may be stored in the database  112  for future reference for the manager view  126 A, collaborative view  128 A or engineer view  124 A. A new project can be added to the hierarchical tree by clicking on Add project  510  button. 
         [0045]      FIG. 6  is another view of the IGUI  102 A showing further details of the hierarchical view of a test case scenario. A test script such as  602  shows the various scripts created and stored for testing in a hierarchical tree format on the left panel. Correspondingly, on the right panel in the IGUI  102 A the sequence of the test case  602  are displayed. Test Id  606  may show machine generated unique Id for a particular test case. Run  608  column shows the status of the test case such as run or not run as a check box. In another embodiment a separate column test case name shows for example shows “Check all links on home page”  604  for the test case scenario. The module type  610  is displayed for the Test Id  606  and test case name  604  as object  612 . Test Automated  614  shows the implementation status of the test script, such as automated 100%. The percentage of automation is the statistics of current quality assurance progress. 
         [0046]      FIG. 7  is another illustration of the IGUI  102 A showing various entity driver parameters. Parameter editor  702  in  FIG. 3  allows the user to change parameter  316  step is shown in more detail for specific entity driver in  FIG. 7 . Parameter name  704 , parameter value  706  and type  708  are shown as one embodiment. The quality assurance engineer may use this information for evaluating the test result and may decide to change in one embodiment, in one another embodiment the user may decide to observe the results and reuse the test parameter for some other test scenario. The manager view may enable the supervisor of the quality assurance manager, for example review the parameters and decide appropriateness of the test parameters and approve or disapprove the test case. 
         [0047]      FIG. 8  is a view of IGUI  102 A showing various steps for test execution logic of a list of test scenarios. This script execution interface has additional columns to control the execution logic. The Iterations column is to show how many iterations the test case runs to pass and/or fail the quality assurance software test. This may be useful to monitor the failures and success of the particular test logic for a test scenario. The script execution IGUI page  802  enables the user to see the sequence of test cases being executed. For a particular script, the sequence of test cases can be listed and seen in column  804 , and number of iterations it may run in column  804 . Column  806  allows a quality engineer to run just checked test cases. Column  808  shows the test module that actually implements the test logic for the corresponding test scenario. 
         [0048]      FIG. 9  is a view of IGUI  102 A showing execution results for the test script. In one embodiment, the script result Id  910  for script  912  shows a failed result in column execution results  901  or pass  902  status for a particular script. This IGUI shows the history of the test script result that is displayed and also stored for future quality assurance tasks, collaborative view and manager view. The start time  907 , the date stamp  908  and end time  903  are shown for each test script execution result such as  906  and  904 . It enables the user to determine an effective time test took  906  in the time used column  905  to run and the details of the results may enable the user to determine the effectiveness of the test logic and the test script used by the quality assurance engineer. 
         [0049]      FIG. 10  is a view of IGUI  102 A showing detailed reports for the test script. In another embodiment, a much more detailed view of  FIG. 9  is shown as another IGUI in  FIG. 10 . A particular script may be expanded as  1010 . Report details  1002  is shown for that particular script result. The expanded result IGUI shows which specific test case shows a pass  1008  result and which one shows a fail  1006  result in the execution result  1004  column. Ease of readability and use enables the user and upper management to provide a better quality modular quality assurance test using this universal quality assurance automation framework  100 . 
         [0050]    In addition, it will be appreciated that the various systems and methods disclosed herein may be embodied in a machine-readable medium and/or a machine accessible medium compatible with a data processing system (e.g., a computer system), and may be performed in any order (e.g., including using means for achieving the various operations). Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.