Patent Publication Number: US-8977739-B2

Title: Configurable frame work for testing and analysis of client-side web browser page performance

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     The present application claims priority to U.S. provisional patent application No. 61/330,838 filed May 3, 2010, entitled CONFIGURABLE FRAME WORK FOR TESTING AND ANALYSIS OF CLIENT-SIDE WEB BROWSER PAGE PERFORMANCE and identifying James Troup as inventor. This aforementioned patent application is incorporated by reference herein. 
    
    
     COPYRIGHT NOTICE 
     A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. 
     BACKGROUND 
     Web-based applications typically use a web browser to support execution. These web-based applications are required to be tested against their specified functionalities in order to verify that execution will proceed as intended. For large web-based applications that have many lines of code, automated testing is preferable, because automated testing saves substantial labor as compared to manual testing. 
     One example of an automated testing system is available under the trade name Selenium. Selenium is a portable software testing framework for web applications that provides a record/playback tool for authoring tests without learning a test scripting language. Selenium provides a test domain specific language (DSL) to write tests in a number of popular programming languages, including various versions of C, Java, Ruby, Groovy, Python, PHP, and Perl. Test playback is possible in most modern web browsers. Selenium deploys on Windows, Linux, and Macintosh platforms. Selenium is open source software released under the Apache 2.0 license and can be downloaded and used without charge. 
     Another tool used in conjunction with automated testing systems, such as Selenium is known as Jiffy-web, or Jiffy. Jiffy is advertised as a web page instrumentation and measurement suite that was first released in June of 2008. Combined these tools facilitate the testing of computer applications in an efficient manner. However, these tools fail to demonstrate the manner by which to implement the same for testing web-based computer network performance. 
     A need exists, therefore, to provide testing techniques for web-based computer network performance. 
     BRIEF SUMMARY 
     The present invention features methods, computer program products and apparatuses for measuring client-side computer system performance that features identifying one of a plurality of uniform resource locator addresses contained on a server computing system, with each of the uniform resource locator addresses being associated with computing resources. The computing resources associated with any one of the plurality of uniform resource locator addresses are different from the computer resource corresponding to the remaining uniform resource locator addresses. The computing resources are accessible through a web-browser that accesses the same computing resources associated with the one of the plurality of uniform resource locator addresses. Measured are metrics of the interactions between the web-browser and the computing resources associated with one of the plurality of uniform resource locator addresses. It is determined whether the metrics satisfy pre-determined operational requirements. These and other embodiments are discussed more fully below. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a simplified plan view of a computer network in which the current invention is practiced; 
         FIG. 2  is a plan view showing a representative architecture in which a multi-tenant database system, shown in  FIG. 1 , is employed; 
         FIG. 3  is a plan view of computer system employed by a user to communicate with the multi-tenant database shown in  FIG. 2 ; 
         FIG. 4  is a detailed view of a configuration testing framework shown in  FIG. 1 ; and 
         FIG. 5  is a graphical display showing web page load time trends employing the current invention. 
     
    
    
     DETAILED DESCRIPTION 
     Referring to  FIG. 1 , a computer network  10  includes a multi-tenant database architecture  12  in data communication with client side facilities  14  and a configurable test framework (CTF)  16 . Components of computer network  10  may be in data communication over any type of known data communication network  18  or combination of networks of devices that communicate with one another. Data communication network  18  can be any one or any combination of a LAN (local area network), WAN (wide area network), telephone network, wireless network, point-to-point network, star network, token ring network, hub network, or other appropriate configuration. As the most common type of computer network in current use is a TCP/IP (Transfer Control Protocol and Internet Protocol) network, such as the global inter-network of networks often referred to as the “Internet”, it will be used in many of the examples herein. However, it should be understood that the networks that the present invention might use are not so limited, although TCP/IP is a frequently implemented protocol. As a result the components of network  10  may be co-located in a common geographic area and/or building or spread across a diverse area of the globe, e.g., on several different continents. Typically, client side facilities  14  and CTF  16  are in data communication with architecture  12  over the Internet using suitable computer systems. However, in other configurations CTF  16  may be included in architecture  12 . Architecture  12  includes a multi-tenant database system (MTS) in which various elements of hardware and software are shared by one or more multiple users  20 ,  22  and  24  associated with client side facilities  14 . 
     A given application server of MTS may simultaneously process requests for a great number of users, and a given database table may store rows for a potentially much greater number of users. To that end, and as shown in  FIG. 2 , architecture  12  includes a processor sub-system  28 , memory space  30 , in data communication therewith, and network interface resources  32  in data communication with both memory space  30  and processor sub-system  28 . Processor sub-system  28  may be any known processor sub-system in the art, e.g., the CORE DUO® or the CORE 2 DUO® from Intel Corporation of Santa Clara, Calif. Memory space  30  includes drive storage  34 , shown as one or more hard drives  36  and  38 , as well as data and instruction registers, shown as  40 , and volatile and non-volatile memory shown as  42 . 
     Architecture  12  provides access to a database  44  by multiple users  20 ,  22  and  24  of client side facilities  14  over data communication network  18  using standard computer systems (not shown). To that end, network interface resources  32  include a plurality of virtual portals  45 - 47 . Each virtual portal  45 - 47  provides an “instance” of a portal user interface coupled to allow access to database  44 . Typically, tenants obtain rights to store information, referred to as tenant information  48  and  50 , on database  44  and make the same accessible to one or more users  20 ,  22  and  24  to whom the tenant provides authorization. This is typically achieved by rental agreements between the tenant and an owner/provider of architecture  12 . In this manner, architecture  12  provides an on-demand database service to users  20 ,  22  and  24  that is not necessarily concerned with building and/or maintaining the database system; rather, these functions are addressed between the tenant and the owner/provider. 
     With architecture  12 , multiple users  20 ,  22  and  24  may access database  44  through a common network address, in this example a universal resource locator (URL). In response, web-pages and other content may be provided to users  20 ,  22  and  24  over data communication network  18 . The resources of database  44  that users  20 ,  22  and  24  may access can be different, depending on user&#39;s  20 ,  22  and  24  security or permission level and/or tenant association. As a result, data structures included in tenant information  48  and  50  are managed so as to be allocated at the tenant level, while other data structures might be managed at the user level. Because architecture  12  supports multiple tenants including possible competitors, security protocols  52  and other system software  54 , stored for example on hard drive  38 , maintain applications and applications&#39; use to only those users  20 ,  22  and  24  with proper access rights. Also, because many tenants may desire access to architecture  12  rather than maintain their own system, redundancy, up-time, and backup are additional functions that may be implemented in architecture  12 . 
     Referring to both  FIGS. 2 and 3 , to facilitate web-based CRM, a user system  55  employed by one of users  20 ,  22  and  24  typically communicates with architecture  12  using TCP/IP and, at a higher network level, other common Internet protocols to communicate, such as HTTP, FTP, AFS, WAP, etc. To that end, user system  55  may be any computing device capable of interfacing directly or indirectly to the Internet or other network connection, such as desktop personal computer, workstation, laptop, PDA, cell phone, or any wireless access protocol (WAP) enabled device and the like running an HTTP client. An example of a user system  55  includes a processor system  56 , a memory system  57 , an input system  58 , and output system  59 . Processor system  56  may be any combination of one or more processors. Memory system  57  may be any combination of one or more memory devices, volatile, and/or non-volatile memory. A portion of memory system  57  is used to run operating system  60  in which an HTTP client  61  executes. Input system  58  may be any combination of input devices, such as one or more keyboards, mice, trackballs, scanners, cameras, and/or interfaces to networks. Output system  59  may be any combination of output devices, such as one or more displays  63 , printers, and/or interfaces to networks. HTTP client  61  allows users  20 ,  22  and  24  of users systems  55  to access, process and view information, pages and applications available to it from server system architecture  12  over network  18 . Examples of HTTP client  61  include various browsing applications, such as Microsoft&#39;s Internet Explorer browser, Netscape&#39;s Navigator browser, Opera&#39;s browser, or a WAP-enabled browser in the case of a cell phone, PDA or other wireless device, or the like. Access is gained to requisite tenant information  48  and  50  by entering the URL (not shown) into the URL box  62  of HTTP client  61 . The URL directs users  20 ,  22  and  24  to the appropriate virtual portal for to determine authorization and permission level to access the requisite tenant information  48  and  50 . In one embodiment, users  20 ,  22  and  24  gain access to web pages stored on database  44 . The web pages are rendered in HTTP client  61 . 
     Referring to both  FIGS. 2 and 3 , an important aspect with respect to providing desired user experience is to avoid the functional latency of the interaction of users  20 ,  22  and  24  with respect to database  44 . One manner in which functional latency may detract from user experience is by HTTP client  61  requiring too great of time to parse and render a web pages. This may occur in response to changes implemented to architecture  12  and/or user system  55 . One advantage for a tenant utilizing architecture  12  is that functional and/or computational improvements to the same and/or user system  55  may be provided with minimal and/or no deleterious effects to a user&#39;s experience of database  44 . 
     Referring to  FIGS. 2 ,  3  and  4 , to minimize deleterious effects on a user&#39;s experience when implementing changes to either architecture  12  and/or user system  55 , testing of the proposed changes is undertaken to determine whether the use experience is degraded as a result of the changes. The tests may be implemented on CTF  16 . CTF  16  may include hundreds of computer systems (not shown), colloquially referred to as a server farm, upon which an emulation  72  of network  10  is implemented. Emulation  72  mimics the operational interaction and operational characteristics of client side facilities  14  and architecture  12 . It is desired to minimize the number of man-hours required to perform the testing. Given the complexity of architecture  12  and the number of users  20 ,  22  and  24  that may access the same at any given time automated testing is employed. To that end, an open source browser testing framework (BTF)  74  is employed in CTF  16 . One example of BTF  74  is sold under the trade name Selenium RC that is available from http://seleniumhq.org/. An instance  76  of BTF  74  is run in a browser  78  of emulation  72 . Emulation  72  is established to mimic operations of user system  55  of one of users  20 ,  22  and  24 . As a result, browser  78  is an emulation of HTTP client  61 . Browser  78  executes a sequence of test methods  79 ,  80  and  81 , referred to as a test group  82 , on a web-based application under test, and referred to as a test configuration file  84 , which is run on emulation  72 . Specifically, BTF  74  opens web browser  78  and provides methods  79 - 81  that interact with browser  78  programmatically, i.e., without the need for human intervention once BTF  74  has launched. BTF  74  navigates to a specific URL address and queries the content of a web page under test  88  (WUT). 
     Test configuration file  84  includes test configuration data  85  and environmental setup mechanism  86 . Test configuration data  85  configures individual tests and includes metadata to facilitate execution of the test. Examples of the metadata that may be included in test configuration data  85  includes a universal resource locator (URL) address of WUT  88 . This may be an explicit/static URL address or it may include information that a web application (not shown) may employ to derive the URL address at which the WUT  88  is located. Additionally, metadata may include environmental configuration information so that emulation  72  accurately mimics the interactions of architecture  12  with client side facilities  14 . Metadata is also included identifying metrics that must be satisfied by the test, e.g., a failure threshold for events that occur on client-side facilities  14 . One example of a failure threshold would be web-page loading time, i.e., whether WUT  88  is rendered upon browser  78  in a predetermined amount of time. The time is typically measured in milliseconds. Were the interactions of client side facilities  14  and architecture  12  to occur outside of the allotted time then the test would be reported as a failure. Other metadata that may be included in configurable test data  85  identifies the owner of the test, an e-mail or other electronic address to which a communication is transmitted indicating the results of the test, e.g., whether a failure had occurred; test case tags; logical grouping of test cases or any other associated metadata specific to a given test. Finally, the metadata in test configuration data  85  is provided to communicate to environmental setup mechanism  86 . An example test configuration data  85  is as follows: 
                                &lt;TestCases xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”       xsi:noNamespaceSchemaLocation=“TestCases.xsd”&gt;                         &lt;TestCase name=“testCase1” owner=“jtroup” urlType=“staticUrl”                 url=“/home.jsp” setupSteps=“enableHomePage”&gt;                         &lt;MeasuredEvent name=“pageLoadTime”           failureThreshold=“200”/&gt;                         &lt;/TestCase&gt;           &lt;TestCase name=“testCase2” owner=“jtroup” urlType=“derived”                 DAO=“CustomerObject” view=“edit”       setupSteps=“enableCustomerObjects”&gt;                         &lt;MeasuredEvent name=“pageLoadTime”           failureThreshold=“200”/&gt;                         &lt;/TestCase&gt;                 &lt;/TestCases&gt;                    
The test configuration data  85  recited above includes two test cases: testCase1 and testCase2. Typically, test configuration data  85  is stored on CTF  16  in extended markup language (XML) The first test case, testCase1, is directed to a static URL address for BTF  64 . The second test case, testCase2 provides metadata that will be used by environment setup mechanism  86  to create a new CustomerObject and construct the URL address for that object&#39;s “edit” view. Both testCase1 and testCase2 pass named steps to environmental setup mechanism  86 , “enableHomePage” and “enableCustomerObjects”, respectively. The metrics established by test configuration data  65  that WUT  88  must satisfy is that WUT  88  must be rendered by browser  78  in no greater than 200 ms.
 
     Environmental setup mechanism  86  performs any desired web application pre-test setup of emulation  72 . This may include configuration to accurately emulate a desired web server configuration; database population; and web application metadata configuration that may affect the generated content of the WUT  88 . Environmental setup mechanism  86  essentially allocates the computer resources of CTF  16  that are accessible by browser  78  by virtue of it establishing emulation  72 . As a result, the environmental configuration metadata included in test configuration data  85  determines which steps are performed by environmental setup mechanism  86 . An example of environmental setup mechanism  86  is as follows: 
                                &lt;SetupSteps xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance”       xsi:noNamespaceSchemaLocation=“SetupSteps.xsd”&gt;                         &lt;SetupStep name=“enableHomePage”&gt;                         &lt;JavaCall method=“PageUtilities.enableHomePage”/&gt;                         &lt;/SetupStep&gt;           &lt;SetupStep name=“enableCustomerObjects”&gt;                         &lt;JavaCall           method=“ObjectPermissions.enableCustomerObjects”/&gt;                         &lt;/SetupStep&gt;                 &lt;/SetupSteps&gt;                    
As with the test configuration data  85 , the environmental setup mechanism  66  is written in XML. This environmental setup mechanism  66  calls architecture  12  logic in Java. The Java code will perform to enable the specified features.
 
     BTF  74  is also written in Java code to communicate with Selenium to opening the WUT in browser  78  determine the time to render WUT. An example of BTF Java code is as follows: 
                                            public int getPageLoadTime( ) {                         String url = getUrlToTest( );           open(url);           return getLoadTimeFromClient( );                         }                        
In short, the test case configuration data  85  provides a data access object (DAO) name and the name of a view for that object, e.g., “detail view”, “edit view”, etc. to environmental setup mechanism  86 . Environmental setup mechanism  86  operates on the DAO to mimic, or create a mock instance, of the specified DAO and constructs a URL address to the DAO at the specified view. BTF  74  navigates to the constructed URL address gathers performance metrics on the generated page that represents a view of the DAO.
 
     Included in WUT  88  is a client-side data provider  90  to measure the performance metrics. Client-side data provider  90  analyzes the actions of web browser  78  and generates reports concerning the same. One metric of interest is load time of WUT  88 , representing the time taken for browser  78  to parse and render an HTML, used to define WUT  88 , into a usable web page by browser  78 . One manner in which to gather performance timings would involve comparing JavaScript timestamps at various points within client-side code. To that end, client side data provider  90  uses JavaScript to gather page load time and stores it in a table of measurement that may be retrieved by BTF  74 . An example of client side data provider is Jiffy-web, or Jiffy. Jiffy is advertised as a web page instrumentation and measurement suite that was first released in June of 2008. Support for Jiffy is available from http://code.google.com/p/jiffy-web/wiki/Jiffy_js. 
     Also included in CTF  16  is a data storage and retrieval mechanism (DSRM)  92 . DSRM  92  provides a persistent storage place for test case measurements and metadata, e.g., the time a particular test was performed, the version of browser  78 , system hardware on which the test was run and the like. DSRM  70  may be stored as a plain text file on CTF  16  or architecture  12 , or as any number of persistent databases like MySQL. DSRM  92  may also provide trending data in virtually any format, such as graph  72 , shown in  FIG. 5 . As shown, graph  94  includes an x-axis representing the times individual test have been performed, and a y-axis representing event duration. The slope of line  96  illustrates an increase in page load time occurring over time. Typically the DSRM  92  is implemented in Javascript, as example of which used to generate graph  94  is as follows: 
     
       
         
           
               
             
               
                   
               
             
            
               
                 &lt;TestCaseResults 
               
               
                 xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” 
               
               
                 xsi:noNamespaceSchemaLocation=“TestCaseResults.xsd”&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;TestCaseResult testCaseName=“testCase1” dateRun=“03/20/2010” 
               
            
           
           
               
            
               
                 failed=“false”&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;MeasuredEvent name=“pageLoadTime” value=“60” 
               
            
           
           
               
            
               
                 failureThreshold=“200”/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/TestCaseResult&gt; 
               
               
                   
                 &lt;TestCaseResult testCaseName=“testCase1” dateRun=“03/21/2010” 
               
            
           
           
               
            
               
                 failed=“false”&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;MeasuredEvent name=“pageLoadTime” value=“75” 
               
            
           
           
               
            
               
                 failureThreshold=“200”/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/TestCaseResult&gt; 
               
               
                   
                 &lt;TestCaseResult testCaseName=“testCase1” dateRun=“03/22/2010” 
               
            
           
           
               
            
               
                 failed=“false”&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;MeasuredEvent name=“pageLoadTime” value=“184” 
               
            
           
           
               
            
               
                 failureThreshold=“200”/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/TestCaseResult&gt; 
               
               
                   
                 &lt;TestCaseResult testCaseName=“testCase1” dateRun=“03/23/2010” 
               
            
           
           
               
            
               
                 failed=“true”&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;MeasuredEvent name=“pageLoadTime” value=“250” 
               
            
           
           
               
            
               
                 failureThreshold=“200”/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/TestCaseResult&gt; 
               
            
           
           
               
            
               
                 &lt;/TestCaseResults&gt; 
               
               
                   
               
            
           
         
       
     
     Referring again to  FIG. 4  although the CTF  16  has been described with respect to a single data configuration file  84 , in practice a plurality of data configuration files would be included, three of which are shown as  84 ,  98  and  99 . Each of the data configuration files  84 ,  98  and  99  would being associated with a uniform resource locator address that is different from the uniform resource locator addresses associated with the remaining data configuration files  84 ,  98  and  99 . It should be understood that each data configuration files  84 ,  98  and  99  includes corresponding test configuration data,  85 ,  100  and  101 , respectively and environmental setup mechanism  86 ,  102  and  103 , respectively. These will allocate different resources of CTF  16  to be accessible by a browser  78 , because each generates a different emulation  72 . Thus, computing resources of CTF  16  allocated to browser  78  is dependent upon the uniform resource locator address to which browser  78  is directed so that different emulations  72  may occur. Thus, the computing resources associated with any one of the plurality of uniform resource locator addresses is different from the computer resource corresponding to the remaining uniform resource locator addresses of the plurality of uniform resource locators defined by data configuration files  84 ,  98  and  99 . 
     The Computer code for operating and configuring network  10  to intercommunicate and to process web pages, applications and other data and media content as described herein is preferably downloaded and stored on a hard disk, but the entire program code, or portions thereof, may also be stored in any other volatile or non-volatile memory medium or device as is well known, such as a ROM or RAM, or provided on any media capable of storing program code, such as any type of rotating media including floppy disks, optical discs, digital versatile disk (DVD), compact disk (CD), microdrive, and magneto-optical disks, and magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data. Additionally, the entire program code, or portions thereof, may be transmitted and downloaded from a software source over a transmission medium, e.g., over the Internet, or from another server, as is well known, or transmitted over any other conventional network connection as is well known (e.g., extranet, VPN, LAN, etc.) using any communication medium and protocols (e.g., TCP/IP, HTTP, HTTPS, Ethernet, etc.) as are well known. It will also be appreciated that computer code for implementing embodiments of the present invention can be implemented in any programming language that can be executed on a client system and/or server or server system such as, for example, C, C++, HTML, any other markup language, Java™, JavaScript, ActiveX, any other scripting language, such as VBScript, and many other programming languages as are well known may be used. (Java™ is a trademark of Sun Microsystems, Inc.). Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.