System, method, apparatus and computer program for automatic evaluation of user interfaces in software programs

A method includes inputting an application program to be tested to a data processing system; linking the application program to a software library; performing, in cooperation with the software library, a static analysis of a user interface of the application program, without executing the application program, to generate a set of static analysis results; performing, in cooperation with the software library, a dynamic analysis of the user interface of the application program while executing the application program to generate a set of dynamic analysis results and, based on the set of static analysis results and the set of dynamic analysis results, a step of determining if the user interface of the application program violates one or more user interface policy rules. Also disclosed is a computer program product that implements the method and a system configured to execute the computer program product in accordance with the method.

TECHNICAL FIELD

The various examples of the embodiments of this invention relate generally to data processing systems and computer software and, more specifically, relate to user interface (UI) design, evaluation and analysis for an application program such as one intended to be executed on a mobile platform.

BACKGROUND

Mobile application testing typically requires manual interaction by an individual with the application program (application or simply app) being tested/evaluated. As the tester performs a test case on a mobile application, the interaction and the application's response can be captured and stored for offline analysis. However this approach is limited since human error can contaminate the test results such as by not adequately testing all possible program flow paths. Furthermore this approach is inefficient since each individual test case must be manually performed.

SUMMARY

In accordance with one aspect thereof the embodiments of this invention provide a method that includes a step of inputting an application program to be tested to a data processing system; a step of linking the application program to a software library; a step of performing, in cooperation with the software library, a static analysis of a user interface of the application program, without executing the application program, to generate a set of static analysis results; a step of performing, in cooperation with the software library, a dynamic analysis of the user interface of the application program while executing the application program to generate a set of dynamic analysis results; and based on the set of static analysis results and the set of dynamic analysis results, a step of determining if the user interface of the application program violates one or more user interface policy rules.

In accordance with another aspect thereof the embodiments of this invention provide a computer program product comprised of software instructions on a computer-readable medium. Execution of the software instructions in a data processing system results in performing operations that comprise inputting an application program to be tested to the data processing system; linking the application program to a software library; performing, in cooperation with the software library, a static analysis of a user interface of the application program without executing the application program to generate a set of static analysis results; performing, in cooperation with the software library, a dynamic analysis of the user interface of the application program while executing the application program to generate a set of dynamic analysis results and, based on the set of static analysis results and the set of dynamic analysis results, determining if the user interface of the application program violates one or more user interface policy rules.

In accordance with yet another aspect thereof the embodiments of this invention provide a system comprised of at least one data processor connected with at least one memory that stores software instructions. In the system the execution of the software instructions by the at least one data processor causes the system to input an application program to be tested; link the application program to a software library; perform, in cooperation with the software library, a static analysis of a user interface of the application program without executing the application program to generate a set of static analysis results; perform, in cooperation with the software library, a dynamic analysis of the user interface of the application program while executing the application program to generate a set of dynamic analysis results and, based on the set of static analysis results and the set of dynamic analysis results, to determine if the user interface of the application program violates one or more user interface policy rules.

DETAILED DESCRIPTION

The use of embodiments of this invention enables an automatic evaluation and testing of a UI within an application. One significant advantage that is realized by the use of the invention is that no direct user interaction is required to perform the evaluation/testing.

The use of this invention enables the automatic evaluation of a UI implemented within an application and thus can have a direct impact on the application design of a native UI (native to the application) by providing an application developer with feedback in order to determine if any existing UI controls violate any predetermined application parameters.

The use of this invention also enables an injection of analytical code into an existing application to achieve the analysis directly on the native UI widgets/buttons/controls of an application, without requiring the direct involvement of a person which could otherwise potentially introduce incompleteness and errors.

The use of this invention also enables a system allowing an application tester, such as an application developer, to perform at least the following tasks on a mobile application or mobile application prototype:

(A) visually review top-level views of the mobile application as they would be presented on a screen to a user of the application; and

(B) respond to alerts automatically generated for the application tester that indicate a presence of any abnormality in the views or the logical flow between the views that would violate pre-established guidelines/rules.

In operation, and referring toFIG. 1, a mobile application10(a completed application or a prototype application) to be tested is uploaded to a system12via a suitable system interface14. The application10can be one written by an enterprise or organization for internal use, or it can be a third party application written for use by a particular enterprise or application, or it could be a third party application written to be published in an application store to be downloaded for use by any number of potential users. The application10can be one developed to be executed on a mobile platform device, such as a smart phone or a tablet, or on a PC or a workstation type device, or on any combination of types of host devices. While the invention is described below primarily in the context of the testing of mobile application (i.e., applications intended to be run on a mobile device platform) the invention is not restricted for use with mobile applications.

It can be noted that uploading the application10to the system12is but one suitable technique to make the application10available for testing at the system12. In other embodiments, and by example, the system12could take the application10as input from a transmission process that could be considered as a ‘peer’ transmission from an application distribution system (e.g., an enterprise application store). In still other embodiments a memory device, such as a memory stick or a portable disk, that has the application10recorded therein could be connected to the system12and the application read from the memory device to a memory of the system12.

By whatever means the application10is provided to the system12the system12injects a software library16into the uploaded application10and software routines of the library16are linked to the software of the application10. This forms an augmented application referred to below as Application A1. An operation contained within the library16is executed by the system12, resulting in the execution of analysis algorithms18on the application10. The analysis algorithms include at least one or more static analysis algorithms18A and one or more dynamic analysis algorithms18B.

First, a static analysis is performed on the application10, resulting in the collection of a first data set20A that includes statically resolved top-level views, their contained controls, and their associated targeted execution flow.

In operation the static analysis algorithms18extract a view hierarchy of the application10. A partial example of a representative view hierarchy is depicted inFIG. 2. InFIG. 2under UI View is a UI View Controller that in turn includes three components: UI Tab Bar Controller, UI Table View Controller and UI Navigation Controller. Each of these can be considered as a software code unit or module that performs some certain UI function. Under each there will typically be at least one UI Button (three are shown by example under the UI Tab Bar Controller) by which a user of the application is enabled to exert control over the functionality of the respective UI software module.

Next a dynamic analysis is performed on the application10, resulting in the collection of a second data set20B that includes a collection of dynamically resolved top-level views, their contained controls, and their associated target execution paths.

In operation the dynamic analysis algorithms18B capture automated running application screen views or screenshots, and example of which is shown inFIG. 3. InFIG. 3the application10is exercised and it generates resulting screen views, three of which are shown. For example, selecting one of the meetings presented under the screen ‘My Meetings’ generates a second screen ‘Attendees’. Selecting the ‘All Participants’ under ‘Attendees’ produces the third screen where a message can be composed to be sent to all of the participants listed under ‘Attendees’. Representative simulated user data can be generated to populate the various text-containing blocks of the screens.

The dynamic analysis algorithms18B can thus automatically simulate input data to navigate between screens and also exercise the various UI buttons associated with each screen to cause the application10to operate and generate output that is captured by the system12. An analysis reconciliation process is performed to detect false negative results and false positive results.

More specifically, the results from the collected first data20A and second data20B are compared by a comparison function22such that false-positive values can be eliminated from the collected data sets. The remaining results formed from the combination of both the static and the dynamic analyses are extracted, and the results are processed for validation against a pre-established set of UI rules24. Any detected abnormalities are cataloged and output by the system12as a list of detected results26for review by an application tester or by the application developer.

Note that in some embodiments the application tester could be person while in other embodiments the application tester could be embodied as software running on the system12or on another computer system.

Describing now the process and method in accordance with this invention in further detail, the application10is uploaded, either in source form or in binary (compiled) form, to the system12.

Upon receipt of the application10the system12detects a presence of any ‘metadata’ information of the application, including (but not limited to): the Application Program Interface/Software Development Kit (API SDK) version that the application10supports; the date/version information of the application10; the file type/extension and the file size and characteristics. The metadata can be extracted and be stored and is useful when analyzing the application software.

Subsequent to receipt of the application10and the detection of the metadata associated with the application10, referred to now as uploaded Application A, the system12performs a modification to the uploaded Application A to produce the updated and augmented Application A1. The updated and augmented Application A1contains the original application logic and also the injected additional algorithmic logic. The injected additional algorithmic logic can be application-agnostic. The additional algorithmic logic is embodied in the form of the application library16. Algorithms, including the static analysis algorithms18A and the dynamic analysis algorithms18B, are precompiled within the library16. Once the code embodied in the application library16is included as part of a linking stage during the creation/assembly of Application A1the algorithms within the application library16are called and invoked upon the execution of the native code contained in the originally uploaded Application A. This is achieved by virtue of the application library16containing application initialization execution logic code that seamlessly takes precedent over the original (native) initialization execution logic code of Application A1. The application initialization execution logic code in the application library16is detected and invoked by a mobile platform operating system when executing Application A1. The linked software routines embodied in the library16thus take control over the Application A1and characterize and exercise the Application A1.

Once the platform/version is determined the Application A is augmented with the library16and calls to the library16are weaved (injected) into the existing code of the Application A to provide the augmented Application A1. The weaving process can involve adding the library calls in the application code where there were none before. This process can differ as to whether the Application A was originally supplied in source or binary form, and what programming language(s) were used to build the Application A. This process can include known techniques such as byte code injection, application program interface (API) method swizzling, or otherwise modifying the binary and/or source code at one or more key points to alter the original flows of the Application A with the injected calls to the library16. API method swizzling is a known procedure that can be used with Objective-C to exchange the implementation of two methods, i.e., the implementation of one method can be exchanged with an implementation of another method.

For testing purposes an applicable mobile platform operating system can be embodied at the system12and can be executed so as to simulate the software/hardware environment of the particular mobile platform(s) for which the Application A is intended. The modified Application A1, containing the library16, is thus not required to be executed on or by a target mobile platform, and no results are required to be uploaded from any target mobile platform to the system12for analysis.

It is noted that the application initialization execution logic in library16, as part of the execution of Application A1, in no way compromises the original functionality of Application A; rather, the application initialization execution logic in library16replicates the original initialization execution logic code of the Application A, while additionally adding the ability for subsequent invocations/execution paths during the lifecycle of Application A1to be rerouted to the library16for algorithmic processing before returning to the original execution path.

After Application A1is created the system12invokes an algorithmic process on the Application A1, referred to herein as the static analysis algorithm(s)18A. One distinguishing feature of the static analysis algorithm18A is that it functions without actually executing the Application A1. Instead the static analysis algorithm18A systematically inspects all (native) components of the Application A1, including all available invocations/execution flow paths, all visually codified entities/views and their sub-view components, and all resource files. A result of the operation of the static analysis algorithm18A when completed is the generation of an exhaustive list or collection of itemized entities relating to the UI of the Application A. This list, L1, is stored by the system12as the static analysis results20A for later use.

The system12next invokes a second distinctive algorithmic process on Application A1embodied as the dynamic analysis algorithm(s)18B. In contradistinction to the static analysis algorithm18A the dynamic analysis algorithm18B operates by direct execution of the Application A1, which in this case is an automated (simulated) execution of the Application A1. The dynamic analysis algorithm18B systematically executes portions of the Application A1, beginning with the initial launching of the Application A1, and continues with the subsequent execution of any controls and buttons that can be invoked by a user during the execution of the Algorithm A when it is resident on a suitable mobile platform. The buttons and controls can be determined by an analysis of the static analysis results20A. This ability to perform automated execution of the Application A, as modified to become the Application A1, is one of the features of the algorithmic logic provided by application library16.

The dynamic analysis algorithm18B systematically inspects all components of the application, including all available invocations/execution flow paths, visually presented entities/views and their sub-view components, and resources used during the execution of the application. For example, every screen that is invoked is tested by algorithmically activating each button/control that is present in each screen, which can result in invoking another screen containing one or more buttons/controls, etc. The resultant hierarchy of screen images is captured, and the controls used to invoke the screen images are recorded, as part of the dynamic analysis results20B. The hierarchy of captured screens can be subsequently presented to an application tester or the application developer for visual verification of screen correctness and order. Further by example, if a text entry box is invoked by activating a certain button then simulated user input data can be entered into the text entry box and the box then closed using another control to verify that the simulated user input data is sent intact to a designated application buffer or other designated location for the entered alphanumeric data. Further by example, if a particular text entry box has been declared to be a date box having a format mm/dd/yyyy then the dynamic analysis algorithm18B operates to verify that only in-range numbers are accepted for entry into the date box.

The result of the dynamic analysis algorithm18B when complete is a list L2stored as the dynamic analysis results20B. The list L2represents an exhaustive collection of itemized application entities relating to the dynamic operation of the UI of the Application A.

Upon completion of both the static analysis algorithm(s)18A and the dynamic analysis algorithm(s)18B the system12proceeds to a processing stage referred to as analysis reconciliation (compare22inFIG. 1) where a comparison algorithm is executed on the lists L1and L2. Each of the application entities within both lists are compared and categorized. The comparison results in filtering out imprecision in the reports produced by the analysis, for example “false positives”. A false positive in the static analysis occurs when the static analysis reports a result that is infeasible at run time. In this case, for example, the static analysis results20A contain an entity but there is no dynamic analysis result20B to corroborate the entity. Conversely, a particular screen image/view may be invoked and recorded but is not declared in the static analysis results20A. This would be a false negative for the static analysis20B, and it would occur because not all the views can be created without executing the code, particularly those views that are generated based on dynamic input provided by the user at run time. Also by example, the static analysis results20A may indicate that some number n of static image views/components have been declared and it can thus be expected that the dynamic analysis results20B show that all of the n static image views/components were invoked at least once.

The resulting list L3is then compared against the pre-determined list of UI policy violations and rules24pre-established for the application. The policy violation rules24can range from widely accepted industry standard UI guidelines to organization-specific UI rules. The policy violation rules24can be language-specific and/or country-specific. The list of UI policy violation rules24is in a form that is tractable for comparison to L3.

FIG. 4shows an example of a data processing system100that can be used to embody one or more of the components of the system12shown inFIG. 1. For example, the data processing system100includes at least one data processor102connected with at least one computer-readable medium such as a memory104that stores a program106configured to perform the augmentation of the received application10and execute the static analysis algorithms18A, the dynamic analysis algorithms18B, the comparison algorithm22and report the results26as described above. The memory104is also configured to store the static analysis results20A and the dynamic analysis results20B including the hierarchy of screen images that result from the execution of the dynamic analysis algorithms18B. The data processing system100further includes a first interface (I/F)108(shown inFIG. 1as the interface14) that is configured to receive the application10from some application source (e.g., a third party application developer or an internal, enterprise-associated application developer) and a second interface110configured to output results data to a user of the system100via any suitable system user interface that includes a display112. The user can, for example, review the stored hierarchy of screen images that result from the execution of the dynamic analysis algorithms18B. The interfaces108and110can each be directly or indirectly connected to an associated network that can be embodied as a wired network or as a wireless network.

In some embodiments the data processing system100can be physically located and instantiated at an enterprise, while in some other embodiments some or all of the data processing system100can be a virtual system hosted in the cloud either by the enterprise itself or by a third party cloud provider.

FIG. 5is a logic flow diagram that is descriptive of the execution of a method, and a result of execution of computer program code such as program code hosted by the data processing system100, in accordance with the embodiments of this invention.

At Block5A there is a step of inputting an application program to be tested to a data processing system. At Block5B there is a step of linking the application program to a software library. At Block5C there is a step of performing, in cooperation with the software library, a static analysis of a user interface of the application program, without executing the application program, to generate a set of static analysis results. At Block5D there is a step of performing, in cooperation with the software library, a dynamic analysis of the user interface of the application program while executing the application program to generate a set of dynamic analysis results. At Block5E there is a step performed, based on the set of static analysis results and the set of dynamic analysis results, of determining if the user interface of the application program violates one or more user interface policy rules.

As was noted above the embodiments of this invention do not require the instantiation of a process on a mobile device or mobile platform. Instead the system12takes as input the mobile application10, executes analysis processes on the application, processes the differing results achieved through the analysis and validates the results against pre-established metrics.

The embodiments of this invention do not require that the system12intercept and substitute values during the execution of the mobile application10. Instead the system12obtains the first stage of analysis results (the static analysis results20A) without executing the Application A, and can obtain the second stage of analysis results (the dynamic analysis results20B) by recording results during the execution of the augmented Application A1.

The embodiments of this invention do not require or rely on any source/target ‘shims’ to intercommunicate and step through interlaced analysis operations. The system12leverages the dynamic binding of the library16to the existing uploaded application10, which does not modify the UI behavior of the mobile application under evaluation. The system10performs two distinct analytical operations on the mobile application10, i.e., the “static” and “dynamic” analysis process phases, and produces a reconciled evaluation as a result. Furthermore, the results are compared to a predetermined set of established UI parameters, resulting in the creation of a problem-list of issues contained within the UI of the application10.

The embodiments of this invention do not require recording any interactions of a user that would serve to compose a structure of actions. Instead, the system12analyzes the UI of the uploaded application10statically with no actionable input, and then dynamically with the algorithm18B that attempts to iterate through all potential user execution paths within the mobile application10. The results of the analysis are not linked to any action that a user has taken; rather, the reconciled analytical results are compared with a set of pre-established metrics, whereby a presence of any problems within the UI of the mobile application can be determined.

The embodiments of this invention dynamically link the analytical library16to the uploaded mobile application in a non-invasive manner without requiring any changes or alterations to the original, native code of the application10.

The embodiments of this invention, during the dynamic analysis phase, automate the execution of the mobile application10without a user's directed input in an effort in order to obtain all of the potential UI objects that the application10could potentially present to a random user.

As such, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. As but a few examples it should be clear that similar or equivalent hardware embodiments, metadata and/or programming languages may be used by those skilled in the art. However, all such and similar modifications of the teachings of this invention will still fall within the scope of this invention.