Automated screen captures

A computer-implemented method, computer program product, and computing system is provided for automatically creating screen captures of a user interface. In an implementation, a method may include identifying a portion of a user interface display associated with a page object. The method may also include creating a screen capture image including the portion of the user interface display associated with the page object. The method may further include associating the screen capture image with a documentation record for the page object.

TECHNICAL FIELD

The present disclosure generally relates to systems and method for creating screen capture images of user interface features.

BACKGROUND

Software user interfaces, for example, user interfaces of browser-based software products, and the like, may often be tested using various automated testing tools. Such automated software testing tools may test the software product by interacting with the user interface. For example, the software testing tools may enter text into text input fields, click on links and buttons in the user interface, and similarly test various other aspects of the user interface. Such testing may be directed at ensuring that all of the included features of the user interface operate correctly. For example, when a button within a user interface is clicked by the testing tool, the testing tool may then evaluate the result of the interaction with the user interface. That is, did the software product perform the intended action or provided an appropriate result in response to the button being clicked. The automated nature of the testing may allow many and/or all of the features of a user interface to be tested to ensure proper operation, while at the same time reducing the time and manual labor that would be required to manually test and evaluate all of the features of the user interface.

SUMMARY OF DISCLOSURE

According to an implementation, a computer-implemented method may include identifying a portion of a user interface display associated with a page object. The method may also include creating a screen capture image including the portion of the user interface display associated with the page object. The method may also include associating the screen capture image with a documentation record for the page object.

One or more of the following features may be included. The method may include identifying a screen capture annotation associated with the page object. Creating the screen capture image may be in response to identifying the screen capture annotation. Identifying the portion of the user interface display associated with the page object may include identifying an explicitly defined parent element in a document object model associated with the page object. Identifying the portion of the user interface display associated with the page object may include calculating a parent element in a document object model associated with the page object.

The method may also include cropping the screen capture image to isolate the page object. Cropping the screen capture image may include extracting a user interface screen location associated with the page object. Associating the screen capture image with the documentation record for the page object may include replacing a prior screen capture image.

According to another implementation, a computer program product includes a computer readable medium having a plurality of instructions stored on it. When executed by a processor the instructions may cause a processor to perform operations including identifying a portion of a user interface display associated with a page object. Instructions may also be included for creating a screen capture image including the portion of the user interface display associated with the page object. Instructions may also be included for associating the screen capture image with a documentation record for the page object.

One or more of the following features may be included. Instructions may be included for identifying a screen capture annotation associated with the page object. Creating the screen capture image may be in response to identifying the screen capture annotation. The instructions for identifying the portion of the user interface display associated with the page object may include instructions for identifying an explicitly defined parent element in a document object model associated with the page object. The instructions for identifying the portion of the user interface display associated with the page object may include instructions for calculating a parent element in a document object model associated with the page object.

Instructions may also be included for cropping the screen capture image to isolate the page object. The instructions for cropping the screen capture image may include instructions for extracting a user interface screen location associated with the page object. The instructions for associating the screen capture image with the documentation record for the page object may include instructions for replacing a prior screen capture image.

According to another implementation, a computing system may include a processor device configured to identify a portion of a user interface display associated with a page object. The processor device may also be configured to create a screen capture image including the portion of the user interface display associated with the page object. The processor device may further be configured to associate the screen capture image with a documentation record for the page object.

One or more of the following features may be included. The processor device may be further configured to identify a screen capture annotation associated with the page object. Creating the screen capture image may be in response to identifying the screen capture annotation. The processor device configured to identify the portion of the user interface display associated with the page object may be configured to identify an explicitly defined parent element in a document object model associated with the page object. The processor device configured to identify the portion of the user interface display associated with the page object may be configured to calculate a parent element in a document object model associated with the page object.

The processor device may be further configured to crop the screen capture image to isolate the page object. The processor device configured to crop the screen capture image may be further configured to extract a user interface screen location associated with the page object. The processor device configured to associate the screen capture image with the documentation record for the page object may be further configured to replace a prior screen capture image.

DETAILED DESCRIPTION

Referring toFIG. 1, there is shown screen capture process10. For the following discussion, it is intended to be understood that screen capture process10may be implemented in a variety of ways. For example, screen capture process10may be implemented as a server-side process, a client-side process, or a server-side/client-side process.

For example, screen capture process10may be implemented as a purely server-side process via screen capture process10s. Alternatively, screen capture process10may be implemented as a purely client-side process via one or more of client-side application10c1, client-side application10c2, client-side application10c3, and client-side application10c4. Alternatively still, screen capture process10may be implemented as a server-side/client-side process via screen capture process10sin combination with one or more of client-side application10c1, client-side application10c2, client-side application10c3, and client-side application10c4. In such an example, at least a portion of the functionality of screen capture process10may be performed by screen capture process10sand at least a portion of the functionality of screen capture process10may be performed by one or more of client-side application10c1,10c2,10c3, and10c3.

Accordingly, screen capture process10as used in this disclosure may include any combination of screen capture process10s, client-side application10c1, client-side application10c2, client-side application10c3, and client-side application10c4.

Referring also toFIG. 2, and as will be discussed in greater detail below, screen capture process10may generally identify100a portion of a user interface display associated with a page object. Screen capture process10may also create102a screen capture image including the portion of the user interface display associated with the page object. Screen capture process10may further associate104the screen capture image with a documentation record for the page object.

Screen capture process10smay be a server application and may reside on and may be executed by computing device12, which may be connected to network14(e.g., the Internet or a local area network). Examples of computing device12may include, but are not limited to: a personal computer, a server computer, a series of server computers, a mini computer, a mainframe computer, or a dedicated network device.

The instruction sets and subroutines of screen capture process10s, which may be stored on storage device16coupled to computing device12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within computing device12. Examples of storage device16may include but are not limited to: a hard disk drive; a tape drive; an optical drive; a RAID device; an NAS device, a Storage Area Network, a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices.

Examples of client-side applications10c1,10c2,10c3,10c4may include but are not limited to an automated software testing application, a web browser, or a specialized application (e.g., an application running on a mobile platform). The instruction sets and subroutines of client-side application10c1,10c2,10c3,10c4, which may be stored on storage devices20,22,24,26(respectively) coupled to client electronic devices28,30,32,34(respectively), may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into client electronic devices28,30,32,34(respectively). Examples of storage devices20,22,24,26may include but are not limited to: hard disk drives; tape drives; optical drives; RAID devices; random access memories (RAM); read-only memories (ROM), and all forms of flash memory storage devices.

Examples of client electronic devices28,30,32,34may include, but are not limited to, personal computer28, laptop computer30, mobile computing device32, notebook computer34, a netbook computer (not shown), a server computer (not shown), a gaming console (not shown), a data-enabled television console (not shown), and a dedicated network device (not shown). Client electronic devices28,30,32,34may each execute an operating system.

Users36,38,40,42may access screen capture process10directly through network14or through secondary network18. Further, screen capture process10may be accessed through secondary network18via link line44.

The various client electronic devices (e.g., client electronic devices28,30,32,34) may be directly or indirectly coupled to network14(or network18). For example, personal computer28is shown directly coupled to network14. Further, laptop computer30is shown wirelessly coupled to network14via wireless communication channels46established between laptop computer30and wireless access point (WAP)48. Similarly, mobile computing device32is shown wirelessly coupled to network14via wireless communication channel50established between mobile computing device32and cellular network/bridge52, which is shown directly coupled to network14. WAP48may be, for example, an IEEE 802.11a, 802.11b, 802.11g, 802.11n, Wi-Fi, and/or Bluetooth device that is capable of establishing wireless communication channel46between laptop computer30and WAP48. Additionally, personal computer34is shown directly coupled to network18via a hardwired network connection.

In some embodiments, screen capture process10may communicate with, interact with, and/or include a component or module of an automated software testing application (e.g., test framework application54). In an embodiment, test framework application54(for example, Selenium browser automation framework, or other automated software testing application) may facilitate automated software testing, including, for example, testing web browser-based user interface and/or other (e.g., not web browser-based) user interface. For example, test framework application may allow authoring and execution of software tests, e.g., that may be run via test framework application54, a client application (e.g., one or more of client applications10c1-10c4, which may include web browsers and/or other suitable applications). One or more of users36,38,40,42may access test framework application54via one or more respective client electronic devices (e.g., client electronic devices28,30,32,34) to author and/or execute a software test, e.g., of a user interface, etc.

In some embodiments, screen capture process10may generally allow images of select portions of a user interface display to be automatically captured and/or maintained, for example, as by automatically providing an updated screen capture image of the select portions of the user interface display in a situation in which the select portion of the user interface display has changed since the screen capture image was created. In some embodiments, the screen capture image including the select portion of the user interface display may be made available to an individual, e.g., who may use the image in connection with designing and/or authoring a test that may be executed relative to the user interface.

For example, a user interface of a software product may include one, or often many, different elements that may be presented to a user (e.g., in the form of a user interface display that may be presented to the user via an appropriate display device). The user may interact with various elements of the user interface for the purpose of operating the software product and/or utilizing functionality provided by the software product. For example, and referring also toFIG. 3, there is shown example user interface display150that may be generated by a software product. User interface display150may include a plurality of elements that may be graphically displayed to a user (e.g., user36) by the software product in a particular state of the software product. As shown, user interface display150may include various elements that user36may interact with for the purpose of utilizing the software product. The various elements may include, for example, information that may be presented to user36(e.g., text, graphics, etc.), links that may be selected by user36(e.g., for navigating to various locations within the software product and/or external to the software product, for displaying information, etc.) buttons (or the like) that may be selected by user to cause the software product to perform various operations and/or provide various functionality. For example, in the illustrated example, user interface display150may generally include top menu152(e.g., which may include various elements), side bar154(e.g., which may include various elements) and login156(e.g., which may include various elements, such as login dialog158, username field identifier160, username text entry box162, password field identifier164, password text entry box166, and sign in button168). It will be appreciated that various different software product may include user interfaces including a wide variety of elements, combinations of elements, and graphical arrangements. In some embodiments, user interface display150may be a user interface associated with a web browser-based software product. As such, user interface display150may include a web browser display. In other embodiments, the user interface display may be associated with a non-web browser based software product and may, therefore, be a non-web browser based user interface.

When building (e.g., designing and/or authoring) an automated test for a user interface (e.g., which may include user interface display150, for example), a user interface display may be logically considered to be a group of page objects. That is, for example, in such automated testing, a common pattern may be to utilize an object to encapsulate a particular page in a user interface, or a part of a page in a user interface, as a page object. The page object may provide an abstraction of the fields, such as button and text fields, and also the actions that can be performed on the fields, such as clickButton( ) enterText(“myText”).” Therefore, in general, a page object may include an object in an object oriented language (such as Java, or the like) that encapsulates part of the user interface display in a particular state. (Java is a trademark of Oracle Corporation in the United States, other countries, or both.) For example, in user interface display150login156may be a page object that encapsulates login dialog158, username field identifier160, username text entry box162, password field identifier164, password text entry box166, and sign in button168. As will be appreciated, in testing user interface display150, login page object156(e.g., including the various elements thereof) may be used in various test scenarios, for example, as all of the elements generally relate to the same functionality (e.g., the ability to login via the software product).

As described, user interfaces in general, and page object in particular, may be very visual in nature, for example, in as much as users may interact with graphically displayed features of the user interface and page objects to utilize the software product and/or execute functionality provided by, or in connection with, the software product. For example, the page objects associated with the user interface may form part of the application programming interface (API) that may be, at least in part, the basis of the tests performed by the testing framework application. Accordingly, in order to build a test for the software product it may be useful for a tester (e.g., a user building the test) to have a clear understanding of the graphical elements included within the page object so that the tester may write tests using the page object API. According to an embodiment, screen capture process10may automatically capture an image of a page object as it may be displayed in a user interface. As such, when designing a test that implements an API associated with the page object, the tester may be able to visually reference the image of the page object. Further, in some embodiments, screen capture process10may also maintain the image of the page object, e.g., which may include capturing a new image of the page object in the event that the user interface changes.

As generally discussed above, screen capture process10may identify100a portion of a user interface display associated with a page object. Screen capture process10may also include create102a screen capture image including the portion of the user interface display associated with the page object. Screen capture process10may further associate104the screen capture image with a documentation record for the page object.

For example, screen capture process10may identify100a portion of a user interface display associated with a page object. Generally, identifying100a portion of the user interface display associated with the page object may include identifying which portion of the user interface display it is that the page object encapsulates. Referring to the example shown inFIG. 3, the page object may include login page object156included within user interface display150(e.g., which may be a currently displayed user interface state). In such an embodiment, login page object156may generally encapsulate login dialog158, username field identifier160, username text entry box162, password field identifier164, password text entry box166, and sign in button168(as generally designated by the broken line box inFIG. 3).

In an embodiment, screen capture process10may identify the portion of the user interface display associated with the page object in response to identifying106a screen capture annotation associated with the page object. For example, during development of the software product (and/or at sometime thereafter) the developer of the software product (and/or another individual) may add an annotation to the software code associated with the page object that may indicate a request that a screen capture image of the page object (with which the annotation is associated) be taken and/or maintained. For example, for a software product written in Java, an example annotation may include:@AUTOGENERATEJAVADOCSCREENCAPTUREpublic class Login extends PageObject

In one such embodiment, the annotation may be defined as follows:

/*** Mark a class with this annotation to take a screencapture of the page object and insert it* into the javadoc.*/@Retention (RetentionPolicy.RUNTIME )@Target (ElementType.TYPE )public @interface AUTOGENERATEJAVA-DOCSCREENCAPTURE {// no properties}

In an example embodiment, when a test is run that tests user interface display150including login page object156, the test (e.g., which may be carried out via test framework application56) may acquire all page objects included within user interface display150. When test framework application56acquires the page objects included within user interface display150, screen capture process10may examine the various page objects from the presence of the annotation (e.g., the annotation “@AUTOGENERATEJAVADOCSCREENCAPTURE” in the above example). In an embodiment in which the annotation is included associated with a page object (e.g., associated with login page object156in the example embodiment), screen capture process10may identify106the screen capture annotation associated with login page object156. In response, at least in part, to identifying106the screen capture annotation, screen capture process10may identify100the portion of user interface display150associated with login page object156, and may create104a screen capture image including the portion of user interface display150associated with login page object156.

As discussed above, screen capture process10may identify a portion of a user interface display associated with a page object, which may, for example, include identifying which portion of the user interface display it is that the page object encapsulates. In various embodiments, the scope of a page object may be either explicitly defined or implicitly defined. The scope of the page object may define the parent element in a Document Object Model (DOM) structure that defines the outer bounds of the user interface that the page object represents. According to an embodiment, identifying100the portion of the user interface display associated with the page object may include identifying108an explicitly defined parent element in a document object model associated with the page object. For example, in some embodiments the page object (e.g., login page object156) may explicitly define the scope of the page object. For example, the scope may be explicitly defined using a declaration in XPath or CSS, or other technologies that may be utilized for defining a DOM element. For example, the scope may include an HTML <div> element, which may contain all of the HTML markup for the login dialog included in login page object156. In Selenium (as an example of test framework application56), a scope may be defined as a org.openqa.selenium.WebElement object.

In an embodiment in which the scope of the page object is not explicitly defined, identifying100the portion of the user interface display associated with the page object may include calculating110a parent element in a Document Object Model associated with the page object. For example, calculating110the parent element in the DOM associated with the page object may include calculating110a lowest identified parent DOM element of defined elements of the page object. For example, login page object156may include login dialog158, username field identifier160, username text entry box162, password field identifier164, password text entry box166, and sign in button168. Screen capture process10may calculate110the scope of login page object156as the deepest element that is a parent to all of login dialog158, username field identifier160, username text entry box162, password field identifier164, password text entry box166, and sign in button168. Since the DOM may generally include a tree structure, the parent may be uniquely defined, and may be calculated110using the DOM API. For example, screen capture process10may calculate110the parent element using JavaScript and JQuery (e.g., a commonly used JavaScript library). Continuing with the above example of login page object156, example code of calculating110the parent element may include:

Suppose element1 is login dialog158, element2 is username field identifier160, element3 is username text entry box162, element4 is password field identifier164, element5 is password text entry box166, and element6 is sign in button168. Then the common parent element of all of these elements may be calculated by repeatedly using the method findCommonParent( ) for each pair of elements, for example:

In an example in which test framework application56may include Selenium, the calculated parentId may be converted into a Selenium WebElement using the code:
Org.openqa.selenium.WebElement scope=Driver.findElement(By.id(commonParentId))

Screen capture process10may also include create102a screen capture image including the portion of the user interface display associated with the page object. In an example embodiment, screen capture process10may create102the screen capture image (e.g., in an example in which test framework application56includes Selenium) using the following Selenium API:

Utilizing such an example API, screen capture process10may create a screen capture image including the portion of the user interface display associated with the page object.

In some implementations, it may be desirable to isolate only the portion of the user interface display including the page object (e.g., login page object156shown encompassed in broken line inFIG. 3). In such an embodiment, screen shot process10may, for example, crop112the screen capture image to isolate the page object. In an embodiment, cropping112the screen capture image may include extracting114a user interface screen location associated with the page object. In an embodiment, screen capture process10may extract114the user interface screen location associated with the page object prior to creating102the screen capture image. In an example embodiment, extracting114the user interface screen location may include using the DOM API to extract the screen location associated with the page object DOM element (i.e., the scope of the page object). For example, in an embodiment in which test framework application56includes Selenium, extracting114the user interface screen location may be achieved using the following code:
Scope.getLocation( )

In an embodiment, the extracted114user interface screen location associated with the page object may include the x and y coordinates and the width and height of the page object. Screen capture process10may crop112the screen capture image to include the portion of the user interface display identified as the location of the page object. In an embodiment, screen capture process10may crop112the screen capture image using the AWT (i.e., abstract window toolkit) API using the following code:

Wherein x and y may include the extracted114x and y coordinates associated with the page object, and width and height may include the extracted114width and height associated with the page object.

Screen capture process10may further associate104the screen capture image with a documentation record for the page object. For example, screen capture process10may save the cropped screen capture image including the image of the page object to a file system, or other appropriate repository. Further, associating104the screen capture image with a documentation record for the page object may include associating saved screen capture image with a documentation record for the page object. For example, a documentation record may include information (e.g., explanations and/or descriptions regarding portions of the software product) associated with a software product (e.g., which may include comments written in a language such as JavaDoc, and which may be included with the code of the software product). Screen capture process10may, for example, automatically associate104the screen capture image with the documentation record for the page object (e.g., with a portion of the code for the software product for the page object), and/or a user may manually associated the screen capture image with the documentation record.

In an embodiment, screen capture process10may automatically associate104the screen capture image with a documentation record for the page object, via an integrated development environment (IDE) plugin. In an embodiment in which the screen capture image may be stored to file system, an example of associating104the screen capture image with the documentation record may be:

/*** Represents the Login dialog.** Here's what it looks like: <p><img src=login.png”/></p>*/public class Login extends PageObject

In a related manner, for a screen capture image stored in a repository, an example of associating104the screen capture image with the documentation record may include:

/***Represents the Login dialog.*Here's what is looks like: <p><img src=“http://www.mycompany.com/PageObjectScreenCaptures/Login.png”/><p>*/public class Login extends PageObject

Referring also toFIG. 4, when a tester and/or developer accesses the documentation record associated with login page object156, he may see image200of isolated login page object156displayed. As such, when a tester accesses the documentation record associated with login page object156, e.g., to write a test of the API at least partially embodied by login page object156, the tester may be able to utilized image200, to assess visual elements of the API.

As generally described above, screen capture process10may also enable maintaining page object images, e.g., such that images of page objects associated with a documentation record for the page object may remain up to date, for example, in the event of a change to the user interface. In an embodiment, associating104the screen capture image with the documentation record for the page object may include replacing116a prior screen capture image. For example, as described above, screen capture process10may associate the screen capture image with a documentation record associated with the page object such that a developer and/or tester may be able to see image200of the page object. In an embodiment, screen capture process10may evaluate whether the screen capture image is different than a currently stored screen capture image associated within a documentation record for the page object. In a situation in which the created102screen capture image is different from a current image associated within the documentation record (and/or no image is associated with the documentation record for the page object), screen capture process10may replace116a prior screen capture image. Replacing116the prior screen capture image may include, for example, overwriting the prior screen capture image with the newly created102screen capture image, and/or changing a reference (e.g., location identifier) for the image included within the documentation record with a reference corresponding to the newly created screen capture image.

While various example embodiments are described in the context of test framework application56including Selenium, similar implementations utilizing other test framework applications will be appreciated by those having skill in the art. Such additional/alternative implementations are considered to be included in the instant disclosure.

Referring also toFIG. 5, there is shown a diagrammatic view of computing system12. While computing system12is shown in this figure, this is for illustrative purposes only and is not intended to be a limitation of this disclosure, as other configuration are possible. For example, any computing device capable of executing, in whole or in part, screen capture process10may be substituted for computing device12withinFIG. 5, examples of which may include but are not limited to client electronic devices28,30,32,34.

Computing system12may include microprocessor250configured to e.g., process data and execute instructions/code for screen capture process10. Microprocessor250may be coupled to storage device16. As discussed above, examples of storage device16may include but are not limited to: a hard disk drive; a tape drive; an optical drive; a RAID device; an NAS device, a Storage Area Network, a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices. IO controller252may be configured to couple microprocessor250with various devices, such as keyboard254, mouse256, USB ports (not shown), and printer ports (not shown). Display adaptor258may be configured to couple display260(e.g., a CRT or LCD monitor) with microprocessor250, while network adapter262(e.g., an Ethernet adapter) may be configured to couple microprocessor250to network14(e.g., the Internet or a local area network).

As will be appreciated by one skilled in the art, the present disclosure may be embodied as a method (e.g., executing in whole or in part on computing device12), a system (e.g., computing device12), or a computer program product (e.g., encoded within storage device16). Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, the present disclosure may take the form of a computer program product on a computer-usable storage medium (e.g., storage device16) having computer-usable program code embodied in the medium.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor (e.g., processor250) of a general purpose computer/special purpose computer/other programmable data processing apparatus (e.g., computing device12), such that the instructions, which execute via the processor (e.g., processor250) of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory (e.g., storage device16) that may direct a computer (e.g., computing device12) or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.