Patent ID: 12222846

DETAILED DESCRIPTION

The terminology used in this disclosure is to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the present disclosure. Indeed, certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this Detailed Description section. Both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the features, as claimed.

In this disclosure, the term “based on” means “based at least in part on.” The singular forms “a,” “an,” and “the” include plural referents unless the context dictates otherwise. The term “exemplary” is used in the sense of “example” rather than “ideal.” The term “or” is meant to be inclusive and means either, any, several, or all of the listed items. The terms “comprises,” “comprising,” “includes,” “including,” or other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, or product that comprises a list of elements does not necessarily include only those elements, but may include other elements not expressly listed or inherent to such a process, method, article, or apparatus. Relative terms, such as, “substantially” and “generally,” are used to indicate a possible variation of ±10% of a stated or understood value.

In general, the present disclosure provides methods and systems for utilizing a testing extension, such as a browser extension, to record test scenarios on a live website. The testing extension may permit a user to define a test scenario by controlling a web browser application to record actions performed by the user on the live website and record characteristics of elements of the website selected by the user.

FIG.1illustrates an application environment according to the present disclosure. The application environment may include a testing extension110, a communicator115, a host application120, a data storage130, and a testing framework140.

The host application120may be a standalone program in which the testing extension110is run. The host application may be executed by one or more processors of a computer or computer system. The host application120may also be referred to as a “host app,” and the testing extension110may also be referred to as a “remote app.” The testing extension110may be a component that extends, increases, and/or improves the functionalities of the host application120. Additionally, in this disclosure, any component that is referred to by the term “extension” may also be referred to as a plug-in or add-on. Therefore, the testing extension110may also be referred to as a plug-in or add-on of the host application120.

The communicator115may provide an interface by which the testing extension110and the host application120communicate with one another. For example, the testing extension110may send messages to the host application120through the communicator115, in order to request the host application120to perform a specified action.

The testing extension110may be configured to record test scenarios that can be automatically run (e.g., executed) by an automated testing engine (e.g., test automation engine), which may also be referred to as a test runner. In some arrangements, the testing extension110may itself be configured to run the test scenarios, in which case the automated testing engine may be part of and/or included within the testing extension110and may utilize functionalities of the host application120. Alternatively or additionally, a testing framework140that is separate from the testing extension110may be used to run the test scenarios. In other words, the automated testing engine may be provided by either the testing extension110, the testing framework140, or both. Accordingly, as used herein, a “testing extension” is not required to be capable of executing a test scenario unless expressly defined otherwise.

As noted above, the testing framework140may be an automated testing framework that includes an automated testing engine configured to run test scenarios recorded and generated by the testing extension110. The testing framework may include test automation tools such as Selenium, Cucumber, and/or Protractor. According to one possible implementation, the testing framework integrates Cucumber and Selenium. The testing extension110may be configured to generate files specifying a test scenario to be run by the testing framework140.

According to various embodiments of this disclosure, the host application120may be a web browser application, and the testing extension110may be an extension of the web browser application. A web browser application may be a web browser, such as Chrome, Firefox, Safari, Internet Explorer, or Edge, etc. The communicator115may be a JS (JavaScript) communicator115used with the web browser application to establish a connection between the testing extension110and the web browser application of the host application120. For example, the testing extension110may send a message to the web browser application of the host application120through the JS communicator115requesting the web browser application to perform one or more operations such as recording events and DOM (document object model) elements. The web browser application of the host application120may “listen” to such requests from the testing extension110via the JS communicator115. The web browser application of the host application120may return click events and target elements in accordance with the request. The JS communicator115may provide the functionality of overcoming cross-site scripting that may be needed to carry out various processes discussed in this disclosure.

Through the JS communicator115, the testing extension110is able to access various aspects of the web browser application of the host application120. This includes listening to messages between other extensions and websites, in addition to accessing HTML DOM elements of websites loaded in the web browser application of the host application120. As discussed in more detail below, the testing extension110offers a tester the ability to click on an element desired to be tested, and record a validation associated with that element as a part of a test scenario.

The data storage130may be a database or file system for collecting, retrieving, and updating test scenarios that are recorded and generated by the testing extension110. Users may save test scenarios on the data storage130together with tags used by various project management tools. In one possible implementation, the data storage130may be provided locally, on the same device as the host application120. In other possible implementations, the data storage130is a remote storage or a cloud-based storage. A cloud-based storage may be implemented by, for example, a web service such as Amazon S3, Amazon Elastic Compute Cloud (EC2), and/or Amazon Relational Database Service (RDS), etc.

The data storage130, particularly if implemented as a remote or cloud-based storage, may be part of a central repository accessible by multiple users. Each user may have an instance of the testing extension110installed on a local computing device and may have his or her own account for purposes of tracking which user generated a certain test. A user may generate and run his or her own test scenarios, and may also share the test scenarios or results with other users. Centralized storage and management of test scenarios in such a manner may help facilitate collaboration among users.

The elements of the application environment shown inFIG.1may be provided by a single computing device or computer system or by a plurality of computing devices or computer systems. For example, the host application120, the communicator115, the testing extension110, and the testing framework140may be locally installed on the same computer computing device or computer system, and the data storage130may be provided by at least one other computing device or computer system.

FIGS.2A and2Billustrate processes involving the application environment elements illustrated inFIG.1. As shown inFIGS.2A and2B, the testing extension110may generate and display a testing dashboard. The testing dashboard may be a graphical user interface or control panel providing controls to record, save, and run test scenarios (step210).FIG.2Aillustrates processes pertaining to recording a test scenario, whileFIG.2Billustrates processes pertaining to the usage of recorded test scenarios.

Referring to step211ofFIG.2A, a user may input a record command through the testing dashboard generated by the testing extension110, serving as a trigger event for the host application120to start (or resume) recording of a test scenario or data used to create the test scenario. The record command may also be used as a trigger by the testing extension110to establish a connection with the host application120. While recording of the test scenario is activated, the user may also input a stop command to the testing dashboard, serving as a trigger event for the host application120to stop the recording (step212ofFIG.2A).

As indicated in step220, the communicator115may listen for the record or stop command (e.g., events), and relay the record or stop command to the host application120by sending a message to the host application120requesting the activation or termination of a recording process. The host application120then performs the indicated action in response to receiving the message from the communicator115.

In response to receiving a message requesting activation of the recording process, the host application120may collect data describing click events, and DOM elements and their properties (step221ofFIG.2A). For example, the web browser application of host application120may listen to events occurring on any web page loaded in the web browser application, and collect (e.g., record) data of events and information associated with those events.

An event may be any event detectable by a web browser application. Examples of such events include keyboard events (e.g., key presses to type content into a DOM element that is a text input field), focus events (e.g., when a DOM element receives or loses focus), mouse movement events (e.g., when the mouse hovers over a DOM element), mouse click events (e.g., a mouse click on a DOM element), load events (e.g., when a DOM element is loaded), drag and drop events (e.g., a DOM element is selected and dragged), and any other event detectable by the web browser application involving one or more DOM elements. It is noted that the term “DOM elements” may include the root element (e.g., root node) of an HTML document. The types of events that are listened to and recorded may be specified. For example, the web browser application may be specified to listen to only mouse click and/or key press events.

The aforementioned information associated with an event may further include information describing any DOM elements associated with the event, including the xPath and properties of the DOM elements. A DOM element associated with an event may be a DOM element that a user interacted with during the event, such as a DOM element that is interacted with in the manners described in the above list of event types. For example, if the event is a mouse click on a DOM element, the information associated with the event may include the xPath of the DOM element that was clicked on, along with one or more characteristics of the clicked-on DOM element.

A characteristic of a DOM element may be any property of a DOM element. A property may be a combination of the name and value of a property. A property may include or indicate an attribute, which may be an HTML attribute or a CSS (cascading style sheet) attribute defining a visual appearance of the DOM object, such as color or font size. A characteristic of a DOM element may also be the data of an external content referenced by the DOM element, such as the contents of an image. A characteristic a DOM element may also be the mere presence/existence of the DOM object.

The collected data may include one or more lists of events, one or more lists of DOM elements, and one or more lists of DOM elements and their properties or characteristics. The collected data may be organized so as to maintain association between an event and any DOM elements associated with the event so that when the collected data is received by the testing extension110, the testing extension110or the web browser application of host application120is able to determine which particular DOM elements were involved with which particular elements.

The host application120may send the collected data223to the communicator115(step222). This step may be performed by the host application120in response to receiving a message requesting termination of the recording process. Alternatively or additionally, the host application120may send the data223to communicator115in response to other triggers. For example, the host application120may send the data223to the testing extension110, via communicator115, whenever a specified event has occurred, in which case the data may include only data pertaining to that specified event and its associated DOM elements.

In response to receiving the collected data, the communicator115may send a message to the testing extension110so that the testing extension receives the collected data223. The message sent by the communicator115may include the testing data223.

Based on the data223, the testing extension110may generate and record a test scenario. The test scenario may be based on only part of the collected data223. The test scenario may specify one or more testable outcomes, such as the occurrence of a DOM element having a certain characteristic. The test scenario may optionally specify one or more preconditions (e.g., initial states) for execution the test scenario, and/or one or more actions to be taken during execution of the test scenario.

A precondition of the test scenario may be presence of a DOM element having a specified xPath and property. An action of the test scenario may be an action, specified by an event, that is to be performed on a DOM element having a specified xPath. A testable outcome of the test scenario may be an occurrence of a DOM element having a specified xPath and property as a result of the event. As will be discussed in greater detail later in this disclosure, any preconditions, actions, and testable outcomes may be defined by a user through the testing dashboard of the testing extension. Specifically, the recording process performed by the host application120may be used to return the xPath and DOM element properties that are used to specify a precondition, action, or testable outcome of the test scenario.

The format of the data specifying the test scenario may be different from that of the data223collected by the host application120. For example, the data specifying the test scenario may include a hash of part of the data223collected by the host application120.

Referring now toFIG.2B, after the test scenario has been recorded (as a recorded test scenario213), the testing dashboard may enable the user to save the recorded test scenario213in the data storage130. Through the dashboard, the user may also load the recorded test scenario213from the data storage130. For example, if the data storage130is a remote storage (e.g., a cloud storage), the testing extension110may transmit a request to retrieve the test scenario (step214). Once a recorded test scenario213is retrieved, the test scenario213may be executed by the testing extension110(step240) by using an automated testing engine included in, or implemented by, the testing extension110. The results250aof the execution of the recorded test scenario213may be displayed by the testing extension110.

To execute the recorded test scenario213, the testing extension110may use browser extension APIs (application programming interfaces) to control a page and run through steps or events defined by the recorded test scenario213.

Test results250amay also be obtained using the testing framework140. In this case, the testing extension110may convert the recorded test scenario213into a test scenario213aconverted into specified format that usable by the testing framework140. The test scenario213ais then sent to the testing framework140, which, in response to receiving the test file (step231), executes the test scenario213aand returns the test results250b(step232). The test results250bmay be sent back to the testing extension110during executing of the test scenario or afterwards for display by the testing extension110.

It is noted that in the processes shown inFIGS.2A and2B, the recorded test scenario213may be generated and stored as part of a larger element, such as a test comprising multiple recorded test scenarios and other elements.

FIGS.3A and3Billustrate an example of the testing dashboard300generated by the testing extension110. As shown inFIG.3A, the testing dashboard300may include a button310to create a new test, and a button320to display an existing test.

Activating the button310may display a dashboard interface300afor creating a test. Interface300amay include a device link input field311, which is a text input field and/or a selection menu to input or select an identifier of a device to connect to. This identifier may be the identifier (ID) of another extension of the web browser application that is to be tested, and may be specified to enable the testing extension110to listen to messages between the web browser application (of host application120) and the browser extension that is to be tested. The ID may include a URL.

The interface300amay further include a first status indicator312indicating whether the testing extension110is currently in a recording mode, and a second status indicator313indicating whether the web browser application is currently connected to a “device” (such as the browser extension that is to be tested). The interface300amay further include text input fields314,315, and316enabling the user to input a name of the feature, a description of the test, and a name of a scenario, respectively. These fields314,315,316may be used for commenting or project management purposes, and their contents need not play any role in the actual execution of the test scenario. The interface300amay further include user interaction elements302,330, and340enabling the user to add a scenario with the name specified in text input field316, save the current test, and record a test scenario for the test, respectively.

In the example shown inFIG.3B, the test denoted as “test1” may include data of all of the fields shown in the interface300a, as well as data defining any test scenarios of the test. A test scenario may be saved as a component of a test, which may include multiple test scenarios.

FIGS.4A-4Billustrate examples of web pages that may be tested according to the methods and systems of the present disclosure.FIG.4Ais a view of a web browser screen of the web browser application (host application120). InFIG.4A, the web browser screen is displaying, among other things, a first web page410that is loaded in the web browser application (host application120). In this particular example, the web page410may be part of a merchant website operated by a merchant, and may include a form412containing various fields for a user to input credit card information, an address form413, and a button414for submitting the entered information.

As shown, the web browser screen may also display a web page interface450generated by a second browser extension of the web browser application. The web page interface450may also be referred to as a web page, and may be generated by the second browser extension based on data received by the second browser extension from a web resource such as a server. For example, the second browser extension may be a browser extension for using a service that provides a virtual credit card number to be used to purchase an item from the merchant website. In order to provide usage of the service, the second browser extension may communicate with a server of a credit card issuer, such as a bank. One or more screens of the web page interface450may include, or be based on, data (e.g., HTML code) received from the server.

In the example shown inFIG.4A, the web page interface450generated by the second browser extension may include a login screen comprising a logo452, a heading453, an image457, a username input field454, a password input field455, and a sign-in user button458. The screen may additionally contain other elements451and456, as desired or needed. Upon inputting a username and password of a valid user account and then selecting the “sign-in” button458, the second browser extension submits the login information to the server of the credit card issuer. In response to receiving the login information, the server of the credit card issuer may then transmit data to the second browser extension to display the post-sign-in screen shown inFIG.4B. This data may be, for example, an HTML document for the post-sign-in screen or data instructing the second browser extension to display the screen. The post-sign-in screen may include a heading461, text462, a statement463regarding terms and conditions, and an “agree” button464used to proceed with the virtual credit card number service.

In the instant example, the web page interface450generated by the second browser extension may be a floating interface overlaying the web page410on the web browser screen. The web page interface450may automatically pop up and as a result of a user interaction performed on the first web page410. This user interaction may, for example, be a click or a mouse hover performed on the “card number” text input field or some other element or portion of the form312, or the activation of a button or link on the web page410that caused the form312to be displayed in response to the activation. The second browser extension may have a listener to detect this user interaction, and display the log-in screen as a pop-up in response to the user interaction.

The testing dashboard300shown inFIG.3A, along with its interfaces, may be displayed on the browser screen shown inFIG.4A(e.g., as a container overlaying the web page410), or in its own tab or window.

FIG.5is a flowchart for a process for recording a test scenario. The process may be performed by one or more processors of a computer system. The process ofFIG.5may include activating, by the testing extension, a recording mode during which user interaction events detected by the web browser application are recorded (step501).

The activating the recording mode may be performed in response to a record command received from a user. As discussed above, the record command may be received through a testing dashboard interface of the testing extension110. The step of activating the recording mode may comprise sending a record command to the web browser application (host application120), according to steps211and202ofFIG.2A. The web browser application (host application120), in response to receiving the command, initiates a process of detecting events and recording events that are detected, along with associated DOM elements. The types of events that the web browser application is to detect and record may be specified by a configuration setting, and may for example, be limited to user interaction events such as click events and key press events.

The process shown inFIG.5further includes: recording one or more actions performed by a user on a web page, the action resulting in a web page element having a characteristic being displayed in the web browser application (host application120) (step502); recording an identifier of the web page element, in response to detecting a user input indicating selection of the web page element (step503); recording the characteristic of the web page element (step504); and generating, by the testing extension, a test scenario specifying a testable outcome resulting from the one or more actions, the testable outcome being an occurrence of an element having the identifier and the characteristic (step505).

As a first example for illustrating these steps ofFIG.5, the “web page” in step502may be the web page410shown inFIG.4A. The user may perform an action on the web page410triggering the web page interface450to appear on the screen of the web browser application (host application120). The triggering action may, for example, be one of the actions discussed earlier. The “web page element” in step502, which is displayed in response to the user action, may be the web page interface450or any element within web page interface450. For example, the web page element may be the image457and the characteristic of this element may be or represent image data of the image. The web page element may also be the sign-in button458and the characteristic of this element may be a green color of the sign-in button, which may be a characteristic defined by a DOM attribute.

Therefore, in the above example, a process of triggering the web page interface450is recorded. The recorded information may subsequently be used to generate a test scenario designed to test whether the web page interface450is properly triggered and properly displayed as a result of a user action on website410.

As a second example, the “web page” in step502may be the web page interface450generated by the second browser extension. The user may perform a sequence of actions comprising inputting a username into field454, inputting a password into field455, and clicking the sign-in button458. This sequence of actions results in the loading and display of the post-sign-in screen shown inFIG.4B. Accordingly, the “web page element” in step502may be any element of the post-sign in screen ofFIG.4B. For example, the web page element may be a DOM element containing the text “by continuing, you agree to our terms & conditions,” as shown inFIG.4B.

Therefore, in the second example, a process of using the web page interface450is recorded. The recorded information may be subsequently used to generate a test scenario213used to test whether the second browser extension is operating as intended.

In step502, the recording of each action among the one or more actions performed on the web page may include recording an event and information describing the DOM element(s) associated with the event, as discussed above forFIG.2A. An action may include a combination of an event and an identifier that the user element interacted with during the event. For example, in the second example discussed above, the web browser application may record the key presses used to type the username into field454, the key presses used to type the password into field455, and the mouse click on sign-in button458as events. The web browser application may further record the xPath of the field454, the xPath of field455, and the xPath of sign-in button458. The xPath of these elements may be recorded so that it is possible to identify the elements for automated interaction with the elements when the test scenario is run.

In step503, the input indicating selection of the web page element may be performed by a user to identify the web page element so that the web page element can be added to the test scenario as an element to be validated when the test scenario is run. The selection of the web page element may be performed by selecting the web page element on a browser screen, which, accordingly, may be a WYSIWYG (“what you see is what you get”) interface.

The user input in step503may be a mouse click to select the web page element, in which case the mouse click may be recognized and recorded by the web browser application as an event, in the same manner that the one or more actions in step502includes an event. Events corresponding to a user inputs intended to define an action of the test scenario (e.g., an event of the one or more actions in step502) may be distinguished from events corresponding to user inputs intended to define an outcome of the test scenario (e.g., an event of the user input selecting the web page element) by an automatic procedure, a manual procedure, or a combination of automatic and manual procedures.

For example, events that occur when defining an action of the test scenario may be automatically distinguished from events that occur when defining an outcome of the test scenario automatically based on the type of element (e.g., whether the element is a static element rather than an interactive element used to submit data or trigger another event), the type of interaction, and/or some other context. It is also possible for a user to manually select whether an interaction is to be recorded as an input action or a selection of a feature. For example, when a user clicks on an element, which may be an interactive element, the user may be prompted to select whether the click is to be processed as an input action performed on to the web page, or instead as an identification of a feature for validation.

In step503, the identifier of the web page element may be the xPath of the DOM element corresponding to the web page element.

In step504, a characteristic of the web page element is recorded, and, in step505, the recorded characteristic is used to define a testable outcome of the test scenario. It is noted that a web page element (e.g., a DOM element) may have many characteristics, such as color, font-size, container size etc. Therefore, the web browser application (host application120) may record a plurality of characteristics of the DOM element (in performing steps221,222, and230shown inFIG.2A) and transmit the plurality of characteristic to the testing extension110. The testing extension110may then provide a selection interface to the user to select one or a subset among the plurality of characteristics of the DOM element. Alternatively, or additionally, the web browser application (host application120) may prompt the user to select one or a subset of the plurality of characteristic of the DOM element to be recorded as part of step221ofFIG.2A.

FIGS.6A and6Billustrate the testing dashboard during and after recording of the test scenario, respectively. In this example the test scenario is labeled “case1.” As shown inFIG.6A, during the recording of the test scenario, the testing dashboard may display a list of recorded steps in a step list610. The steps may be displayed in the step list610in the order of execution when the test scenario is run. Each step may be defined by a step keyword and a definition of the step (including DOM element properties used to define the step).

The step keyword indicates a step type, such as whether the step is a precondition (initial state) of the test scenario, an action to be performed when the test scenario is run, or a testable outcome. The step keyword may, for example, be a Gherkin keyword such as “given”, “when,” “then,” “and,” or “but.” Steps labeled with the “given” keyword define a precondition of the test scenario. Steps labeled with the “when” keyword define an action to be performed during the test. Steps labeled with the “then” keyword define a testable outcome resulting from the actions to be performed during the test. Steps may also be labeled by “and” and “but,” as used in Gherkin. For example, “and” indicates a conjunction with the previously listed step. The step list610may include interface elements611to adjust the type of step.

Using the “second example” discussed above for testing the interface450of the second browser extension, the test file interface of the testing dashboard may be represented as a scenario having the following steps: a step marked by keyword “when” and indicating an action of inputting a username into a field having an xPath used to identify field454ofFIG.4A; a subsequent step marked by keyword “and” indicating an action of inputting a password into a field having an xPath used to identify field455ofFIG.4A; a subsequent step marked by keyword “and” indicating a click on the sign-in button458, and a subsequent step marked by keyword “then” indicating an element containing the text “by continuing, you agree to our terms & conditions,” as shown inFIG.4B.

The keyword or step type may be automatically assigned by the testing extension and/or manually selected through the testing dashboard. For example, the keyword may initially have a blank or default value, and the user may assign the keyword as the user records the test scenario using a selection list. For example, a step may be added to the list every time an event is detected by the web browser application (in accordance with step221ofFIG.2A), and the user may then specify the step type by selecting one of the keywords from a drop-down menu611.

The precondition of the test scenario may be automatically defined by the testing extension110. For example, the testing extension may automatically record the URL of the web page on which the initial user interaction is performed. Alternatively or additionally, a user may record the precondition in the manner of steps503and504ofFIG.5. That is, the user may select a DOM element on a web page (which may be the entire document), and select the document URL as the characteristic of this DOM element. Upon doing so, a step may appear in step list610, and the user may designate the step as a precondition by selecting the keyword “given.”

Precondition steps may be optional, and may be omitted from the test scenario. For example, the user may navigate to a certain web page and run the test scenario on that website. In such a usage, it is not necessary for preconditions to be defined. Similarly, action steps may be optional. For example, if there are neither precondition steps nor action steps, the execution of the test scenario may simply validate whether a current display of the web browser application (host application120) includes a DOM element having the identifier and characteristic specified in the testable outcome.

The sequence of steps in the step list610may be the sequence as recorded. However, the step list610may include and interface elements611to adjust the order of steps. The step definitions may be automatically generated. However, the user may edit the definition of each step, such as the selection of the DOM element and its DOM element properties to be validated during execution of the test scenario.

As shown inFIG.6A, the user may click a user interaction element620to stop recording of the test.FIG.6Bshows an example of a test file interface testing dashboard after having recorded and generated two test scenarios. As shown, the testing dashboard shows a step list610afor the test scenario “case1” and another step list610bfor the test scenario “case2.”

In the interface shown inFIG.6B, the user may click a button630to return to a another interface, such as the interface shown inFIG.3A. The user may also click on a button650to edit the test, thereby opening the edit test interface shown in FIG.6C. The edit test interface shown inFIG.6Cpermits the user to edit the test, save the test and record additional scenarios for the test. The button340may be used to record additional test scenarios or test scenario steps for the test that is being edited.

The process shown inFIG.5further includes executing the test scenario (step506). This step may include executing a replication of the one or more actions on a web page to be tested, and determining whether the testable outcome resulted from the replication of the one or more actions. The processes of executing the test scenario and evaluating the results may be performed by the testing extension110or by the testing framework140that is external to the testing extension110, as shown inFIG.2B.

For example, a test scenario may have a precondition, an action, and a testable outcome. To validate a precondition of the test scenario, the testing extension110or testing framework140may determine whether a current web page loaded in the web browser application (host application120) includes a DOM element having the xPath and property specified by the precondition. If such a DOM element exist, then the precondition is validated. Next, the testing extension110or testing framework140may perform the action of the test scenario by finding a DOM element having a specified xPath, and performing the event on that DOM element specified by the action. If such a DOM element exists, and the event is performable, then the action is validated. Finally, to test the testable outcome, the testing extension110or testing framework140may determine whether any current web page loaded in the web browser application (at a time subsequent to the action) includes a DOM element having the xPath and property specified by the precondition. If such a DOM element exist, then the testable outcome is validated.

FIGS.6D-6Fillustrate interfaces of the testing dashboard related to execution of a test scenario.FIG.6Dshows a display of the testing dashboard during execution of the test. The testing dashboard may distinguish steps that have been run (e.g., steps622a,622b) and steps that have not been run (e.g., step662c) by displaying the latter in a different visual manner (e.g., in a faded manner). Additionally, among steps that have been run, those that were successfully validated and those that were not successfully validated may be marked in different visual manner (e.g., different colors, such as of green and red, for successful and unsuccessful results, respectfully). Additionally, if there is any step in a test scenario that did not validate successfully, the testing dashboard may display a marker663indicating a failed test.

FIG.6Eshows the testing dashboard upon completion of the test.FIG.6Fdisplays a summary list670of multiple test scenarios of the test, including an indicator of whether the test scenario was successful, was not successful, or has not yet been run, as well as the dates and times of the latest execution of the test scenarios.

The testing extension110may provide the functionality of taking screenshots of failed or successful test scenarios, manually or automatically. Screenshots may be saved on the data storage130.

The interfaces shown inFIGS.6D-6Fmay be generated whether the processes of executing the test scenario and evaluating the results is performed by the testing extension110or by the testing framework140. If the processes are performed by the testing framework, the testing framework may supply results at the end of the execution of the test, and/or during the execution of the test.

Accordingly, various embodiments of this disclosure provide a testing extension permitting users to record, save, and run test scenarios through a graphical user interface, without the need for coding, thereby allowing non-technical personnel to more easily become involved with the testing of websites. As discussed above, components of the test scenario (precondition, action, and testable outcome) may be defined based on selection of web page elements in a web browser. In particular, an action of the test scenario may be recorded on a live website. Additionally, the testing extension may have the ability to connect to other browser extensions, to thereby provide automated testing of browser extensions as well as testing of typical websites.

In general, any process discussed in this disclosure that is understood to be performable by a computer may be performed by one or more processors. Such processes include, but are not limited to: the processes shown inFIGS.2A-2B and5; processes involving operations of the testing extension110, the communicator115, the host application120, and the testing framework140; the generation of a user interface described in this disclosure; and processes involving data processing, storage, or analysis. The one or more processors may be configured to perform such processes by having access to instructions (computer-readable code) that, when executed by the one or more processors, cause the one or more processors to perform the processes. The instructions may also be stored on a non-transitory computer-readable medium.

It should be appreciated that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

Thus, while certain embodiments have been described, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the invention, and it is intended to claim all such changes and modifications as falling within the scope of the invention. For example, functionality may be added or deleted from the block diagrams and operations may be interchanged among functional blocks. Steps may be added or deleted to methods described within the scope of the present invention.

The above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other implementations, which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description. While various implementations of the disclosure have been described, it will be apparent to those of ordinary skill in the art that many more implementations and implementations are possible within the scope of the disclosure. Accordingly, the disclosure is not to be restricted except in light of the attached claims and their equivalents.