Abstract:
A system and method for replaying a user&#39;s interaction with a webpage by requesting playback of user interaction data from a replay server, receiving a media asset from the replay server to allow playback the requested user interaction data, the media asset being pre-fetched for use by a proxy application, periodically requesting and receiving the user interaction data from the replay server, recreating an interaction visualization from the user interaction data, and playing back said interaction visualization based on data received from the replay server.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application is a continuation of U.S. patent application Ser. No. 13/075,636, filed on Mar. 30, 2011, which claims priority to U.S. Provisional Application Ser. No. 61/319,907, filed Apr. 1, 2010, the disclosures of each being incorporated herein by reference in their entirety. 
         [0002]    This application hereby incorporates by reference in its entirety U.S. patent application Ser. No. 12/421,695 filed Apr. 10, 2009. 
     
    
     BACKGROUND 
       [0003]    It is desirable for companies conducting business online, such as by operating business related webpages over the Internet, to be able to observe and/or collect and analyze the interaction of certain users with their webpages. Commercial software for collecting and analyzing aspects of user interaction with webpages is known in the art. However, such known tools typically do not permit the collection or analysis of details of individual user interactions with a webpage, such as how an individual user completes a form on a webpage, or which parts of a webpage a user interacts with the most easily, or how a user navigates from one element of a webpage to another, for example. 
         [0004]    Other software for remotely collecting and analyzing certain other details of individual user interactions with a webpage is known in the art. However, such known software typically requires that the host or service provider collect user interaction details and also provide the webpage to the user&#39;s computer. Such configurations increase the processing overhead on the webpage server and limits the ability of a third party service provider, such as a user interaction analysis service provider, from being able to collect user interaction data from the host server. Moreover, other approaches may provide a premade video rendering of such user interactions, which are typically large in size. 
         [0005]    As a result of such limitations, it would be desirable to provide systems and methods that provide remote capture and replay of user interaction data that allows for the capture and serving of replay information by a service provider, and the replay of user interaction data with the webpage by a second remote computer, which is separate from the service hosting and serving the webpage to the user. 
       SUMMARY 
       [0006]    It is one object of the present invention to provide systems and methods for remote tracking of user interaction with a webpage that addresses some of the limitations of the prior art. 
         [0007]    According to an embodiment of the invention, a computer readable medium is provided, comprising executable instructions to replay a user&#39;s interaction with a webpage, comprising: requesting playback of user interaction data from a replay server; receiving a media asset from the replay server to allow playback the requested user interaction data; the media asset being pre-fetched for use by a proxy application; periodically requesting and receiving the user interaction data from the replay server; recreating an interaction visualization from the user interaction data; and playing back said interaction visualization based on data received from the replay server. 
         [0008]    Further advantages of the invention will become apparent when considering the drawings in conjunction with the detailed description. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]    The system and method of the present invention will now be described with reference to the accompanying drawing figures, in which: 
           [0010]      FIG. 1  illustrates an exemplary networked operating environment for implementing an embodiment of the present invention. 
           [0011]      FIG. 2  illustrates an exemplary user computer architecture configured according to an embodiment of the invention. 
           [0012]      FIG. 3  illustrates an exemplary web server computer architecture configured according to an embodiment of the invention. 
           [0013]      FIG. 4  illustrates an exemplary tracking server computer architecture configured according to an embodiment of the invention. 
           [0014]      FIG. 5  illustrates a series of processing operations associated with an embodiment of the invention. 
           [0015]      FIG. 6  illustrates a series of processing operations implemented by a user computer associated with an embodiment of the invention. 
       
    
    
       [0016]    Like reference numerals refer to corresponding parts throughout the several views of the drawings. 
       DETAILED DESCRIPTION 
       [0017]      FIG. 1  illustrates an exemplary networked operating environment in which embodiments of the present invention may be implemented. The networked environment includes a user computer  10  connected to a communication network  50 , which may include, for example, one or more of: a local area network (LAN), wide area network (WAN), world wide web (WWW), the Internet, such that user computer  10  may communicate with other computers similarly connected to network  50 . Other computers connected to network  50  may include a web server  20 , tracking server  30 , and an analysis computer  40 , which may each communicate with any other computer connected to network  50 . User computer  10  includes standard computing components known in the art for transmitting and receiving data to and from other computers connected to the user computer  10  through network  50 . 
         [0018]    Similarly, web server  20  includes standard computing components known in the art for serving web pages to other computers over the network  50 . In particular, web server  20  is connected to a webpage storage repository  22  which stores one or more webpage resources  24 . Webpage storage repository  22  may include a conventional data storage device such as a hard disk or solid-state memory located with and connected directly to web server  20 , or may include a remote data storage facility connected to web server  20 . Webpage may include a single document, such as a hypertext document commonly written in a markup language such as HTML or XML, or may include multiple documents such as a multi-page website, which may be accessed by a user computer  10  connected to the web server  20  through network  50 . Webpage may also include one or more scripts or commands for executing operations expressed in one or more known scripting languages such as JavaScript for example. Such scripts may be included in the webpage  24  when it is served to other computers by web server  20 . 
         [0019]    Tracking server  30  includes standard computing components known in the art for sending and receiving data to and from other computers connected to network  50 . In particular, tracking server  30  is connected to a tracking storage repository  32 . Tracking storage repository  32  may comprise a conventional data storage device such as a hard disk or solid-state memory located with and connected directly to tracking server  30 , or may comprise a remote data storage facility connected to tracking server  30 , such as a database. 
         [0020]    Tracking storage repository  32  stores a tracking script  34  which may be expressed in a known scripting or command language such as JavaScript for example, and may include computer executable instructions to record and transmit interaction data related to the interaction of a user with a webpage, as described in greater detail below in reference to  FIGS. 2-6 . In some embodiments, the script may include a function call to one or more applications that may be used to communicate interaction data across network  50 . 
         [0021]    Tracking storage repository  32  may also store one or more tracking records  36  including such interaction data, which may be received from or transmitted to one or more computers connected to tracking server  30  through network  50 , such as a user computer  10  or an analysis computer  40 , for example. 
         [0022]    It will be appreciated from the above that tracking server  30  and tracking storage  32  may be constructed in many forms (not shown). For example, these resources may be constructed from multiple, physically remote, stand alone platforms, with functions that may be distributed across such resources or may be integrated into one or more “single platforms” configured to perform the various different required functions as further described herein (e.g., through a partition or logical assignment). 
         [0023]    Moreover, it will be further understood that server  30  and storage  32  may perform a variety of roles. For example, server  30  and storage  32  may receive tracking information regarding a user&#39;s interaction with a webpage as further described herein. However, server  30  and storage  32  may also perform certain other functions such as the replay of interaction data stored in storage  30  in response to a request for playback. 
         [0024]    For example, a system analyst may desire to view user interactions at analysis computer  40  based on tracking data stored in storage  32 . In this case, tracking server  30  may act as a “replay server” and fetch the tracking records from storage  32  and replay them for viewing on analysis computer  40 . Thus, tracking server  30  and tracking storage  32  may be sometimes by referred to herein as “replay server”  30  or “replay storage”  32  depending on the role they perform. 
         [0025]    Furthermore, it will be understood that server  30  and storage  32  may play dual roles, performing both replay and capture functions as needed. However, others may operate as substantially dedicated servers performing either replay or storage functions (not shown). 
         [0026]    Similar to user computer  10 , analysis computer  40  may includes standard computing components for transmitting and receiving data to and from other computers connected to the analysis computer  40  through network  50 . 
         [0027]      FIG. 2  illustrates an exemplary computer architecture for a user computer  10  configured in accordance with an embodiment of the invention. According to an embodiment of the invention, analysis computer  40  as described above may also be configured similarly to the exemplary architecture of user computer  10  as illustrated in  FIG. 2 . 
         [0028]    Computer  10  may include standard components, including a central processing unit  102  and input/output devices  104 , which are linked by a bus  108 . Input/output devices  104  may comprise a keyboard, mouse, touch screen, monitor, printer, and the like, for example. A network interface  106  is also connected to the bus  108 . Network interface  106  provides connectivity to a network  50 , such as the exemplary network  50  described above, thereby allowing the computer  100  to operate in a networked environment. Also connected to bus  108  is a computer-readable memory  110 . Memory  110  stores executable instructions to implement some or all of the functions of the invention. Computer-readable memory  110  may comprise any available computer-readable media or device that can be accessed by computer  100 . 
         [0029]    In an embodiment of the invention, one or more of the following program modules and data files may be stored in the memory  110  of the computer  10 : an operating system module  112 , a Web browser module  114  and a tracking script module  116 . 
         [0030]    Operating system module  112  may be suitable for controlling the operation of a networked user or analysis computer, and in particular may include instructions for handling various system services, such as file services or for performing hardware dependant tasks. Operating system module  112  may also comprise instructions for standard computer operation, including receiving input from input devices such as a keyboard or mouse, and for displaying output in a graphical format on a monitor, for example. 
         [0031]    Web browser module  114  may include instructions for browsing webpages provided by a web server or other source, such as instructions for requesting and receiving a webpage from a web server and displaying the webpage on a display device such as a monitor. Web browser module  114  also include instructions for receiving input from a user&#39;s interaction with a webpage such as from input devices like a keyboard and mouse for example, and for transmitting such user input to a web server. Web browser module  114  may also comprise instructions for executing processing commands comprised in webpages or other files such as scripts. 
         [0032]    Tracking script module  116  may include instructions for recording interaction data input by a user in the process of interacting with a webpage, such as mouse movements, scrolling, clicks and keyboard entries, for example. In an embodiment of the invention, tracking script module  116  may also include instructions for processing such interaction data, and for transmitting processed interaction data to a remote tracking server  30 , such as over network  50 . 
         [0033]    In some embodiments, tracking script module  116  may include instructions for receiving interaction data from a remote replay server  30 , and for recreating and displaying an interaction visualization from the received interaction data (e.g., using certain known multimedia applications such as Windows Media Player or through the use of a client player application described further herein). 
         [0034]    The above described program modules incorporate instructions to implement processing operations associated with the invention. Various embodiments of the processing operations of the above-described program modules are described below. The modules stored in memory  110  are exemplary, and additional modules can be included. It should be appreciated that the functions of the presented modules may be combined. In addition, a function of a module need not be performed on a single machine, instead, the function may be distributed across a network to one or more other computers if desired. For example, a playback application may be stored on a client computer, with interaction data provided to that computer for playback. This may be accomplished by the creation of a video or graphic rendering of user interaction at the client side based on interaction data provided. 
         [0035]      FIG. 3  illustrates an exemplary computer architecture for a web server computer  20 , such as illustrated in the computer system of  FIG. 1 , configured in accordance with an embodiment of the invention. Computer  20  may include standard components known in the art, including a central processing unit  202  and input/output devices  204 , which are linked by a bus  208 . Input/output devices  204  may comprise a keyboard, mouse, touch screen, monitor, printer, and the like, for example. A network interface  206  is also connected to the bus  208 . Network interface  206  provides connectivity to a network  50 , such as the exemplary network  50  described above, thereby allowing computer  20  to operate in a networked environment. 
         [0036]    Web server  20  also comprises a webpage storage repository  22  capable of storing one or more webpage resources. Webpage storage repository  22  may comprise a conventional data storage device such as a hard disk or solid-state memory located with and connected to web server  20  such as by bus  208  as shown in  FIG. 3 , or may alternately comprise a remote data storage facility accessibly connected to web server  20 . Also connected to bus  208  is a computer-readable memory  210 . Memory  210  stores executable instructions to implement functions of the invention. Computer-readable memory  210  may comprise any available computer-readable media or device that can be accessed by the computer  20 . 
         [0037]    In an embodiment of the invention, one or more of the following program modules and data files may be stored in memory  210  of the web server computer  20 : an operating system module  212 , a web server module  214 , a tracking tag code module  216  and a webpage storage module  218 . 
         [0038]    Similar to module  112  described above, operating system module  212  may include instructions for handling various system services for web server computer  20 , such as file services or for performing hardware dependant tasks. Operating system module  212  may also include instructions for standard computer operation, including receiving input from input devices such as a keyboard or mouse, and for displaying output in a graphical format on a monitor, for example. 
         [0039]    Web server module  214  includes instructions for serving webpages requested by another computer, such as instructions for receiving a request for a webpage from a user computer, accessing the webpage, and serving the webpage to the requesting user computer, for example. Web server module  214  may also include instructions for receiving input from a user&#39;s interaction with a webpage, processing such input, and serving additional webpage content to a user computer in response. 
         [0040]    In one embodiment, tracking tag code module  216  may be stored as a sub-module of web server module  214 , and includes instructions for requesting a tracking script from a tracking server  30 . The tracking tag code module may also include further instructions to add or attach instructions for requesting a tracking script from tracking server  30  to one or more webpages before they are served to user computers. 
         [0041]    Webpage storage module  218  includes instructions for accessing webpage storage repository  22 , such as for accessing and retrieving stored webpages and/or other stored resources, in response to requests for such webpages and/or resources from user computers, for example. 
         [0042]      FIG. 4  illustrates an exemplary computer architecture for a tracking server computer  30 , such as illustrated in the computer system of  FIG. 1 , configured in accordance with an embodiment of the invention. Computer  30  may include standard components known in the art, including a central processing unit  302  and input/output devices  304 , which are linked by a bus  308 . The input/output devices  304  may include a keyboard, mouse, touch screen, monitor, printer, and the like, for example A network interface  306  is also connected to the bus  308 . The network interface  306  provides connectivity to a network  50 , such as the exemplary network  50  described above, thereby allowing the computer  30  to operate in a networked environment. Tracking server  30  also comprises a tracking storage repository  32  capable of storing one or more tracking scripts and/or tracking record resources, for example. 
         [0043]    Tracking storage repository  32  may include one or more conventional data storage devices such as a hard disk, solid-state or optical memory located with and connected directly to web server such as by bus  308  as shown in  FIG. 4 , or may alternately include a remote data storage facility accessibly connected to web server  30 . Also connected to bus  308  is a computer-readable memory  310 . Memory  310  stores executable instructions to implement functions of the invention. Computer-readable memory  310  may include any available computer-readable media or device that can be accessed by computer  30 . 
         [0044]    In an embodiment of the invention, one or more of the following program modules and data files may be stored in memory  310  of the tracking server computer  30 : an operating system module  312 , a tracking server module  314 , a tracking recording module  316 , and a tracking replay module  318 . 
         [0045]    Similar to module  112  described above, operating system module  312  may include instructions for handling various system services for tracking server  30 , such as file services or for performing certain hardware or application dependant tasks. Operating system module  312  may also include instructions for standard computer operation, including receiving input from input devices such as a keyboard or mouse, and for displaying output in a graphical format on a monitor, for example. 
         [0046]    Tracking server module  314  includes instructions for receiving a request for a tracking script from a user computer, accessing the tracking script, and transmitting a tracking script to the user computer. In one embodiment, the tracking script or other suitable tracking code may be stored in tracking storage repository  32 . Furthermore, tracking server module  314  includes instructions for accessing tracking storage repository  32  to retrieve the tracking script or other suitable tracking code. 
         [0047]    As shown in  FIG. 4 , tracking server module  314  may further include tracking server module  316  as a sub-module or embedded application. Tracking recording module  316  includes instructions to receive user interaction data transmitted by a user computer, and to record such data as one or more tracking records or data structures. In one embodiment, such tracking records may be stored in tracking storage repository  32 , for example. In other embodiments, tracking recording module  316  may also include instructions for analyzing user interaction data received from a user computer. 
         [0048]    Tracking server module  314  may also further include a tracking replay module  318  as a sub-module or embedded application. Tracking replay module  318  includes instructions for receiving a request to provide one or more tracking records from an analysis computer, accessing the one or more tracking records, such as from tracking storage repository  32 , and for transmitting recorded user interaction data to analysis computer  40 . In one embodiment, tracking replay module  318  may further include instructions to process and/or transform user interaction data retrieved from one or more tracking records, prior to transmitting the data to the an analysis computer. For example, in some embodiments, tracking replay module  318  may include code or routines that facilitate the playback of captured interaction data. 
         [0049]    Tracking replay module  318  may further include code or routines that improve the quality or comprehensiveness of the replay experience. For example, data captured by recording module  316  may be in varying formats and/or may be captured with limited metadata or background information. This may limit the quality and/or ability to consistently or accurately replay user session data. 
         [0050]    Accordingly, the system may capture certain information relating to the details regarding the browsing session to be recreated. Thus, when it is desired to playback the interaction data at a playback computer, this information may be provided initially to recreate the playback environment at the playback web browser. This information may include, but is not limited to: The web browser was used to record the session; when the session was recorded; How the long the browsing session lasted; The initial screen dimensions; How much mouse movement there was; Whether or not there was keyboard involvement in the session; What pages were visited and times Spent on each page; The amount of activity on a certain time-basis with which to generate an activity histogram of the session; The account ID of the customer to whom the session belongs, etc. Any other similar information may also be provided, if desired. 
         [0051]    Such information may be provided to the playback computer prior to the replay of user interaction data as a browser initialization file. In some embodiments, this may be called an .FSR file and may include metadata captured from the source webpage. When the client player loads such an initialization file, it deserializer this information and makes a request on replay module  318  to deliver the actual web page content and animation data. A media player on the playback computer may then use a standard web browser or embedded web browser to replay the session and display it to a user. 
         [0052]    Tracking replay module  318  may further include routines that normalize the different data formats received by module  316  to permit accurate or consistent replay and may further include certain “proxy” routines to collect information to further facilitate and improve the playback experience. 
         [0053]    Such normalization functions may include, but are not limited to: taking screenshots of the page, rewriting image URL&#39;s to point to the correct location of an image; document write calls are removed or masked; AJAX requests are rewritten to use custom AJAX class requests rather than native browser HTTP requests; JavaScript errors known to cause errors are rewritten using heuristics; known third party components such as JQuery are supplanted with customized versions that function better in the replay environment, etc. 
         [0054]    In some embodiments, proxy routines may retrieve certain additional information such as environment, content, background, driver or other related webpage information such as a document object model “DOM” (the “additional features” information) not related to user interaction. In some embodiments, this information id captured at substantially the same time as the user interaction data, and may be stored as part of the initialization file described above. 
         [0055]    In other embodiments, this information may not have been captured along with the user interaction due to the size of the information, based on the relatively static nature of the information, due the information already existing in tracking storage  32 , due to the specific coding of the tracking script, or based on other factors. 
         [0056]    In this case, such proxy routines may seek this additional features information from various sources such as the Internet or other network locations to facilitate replay of user interaction data. This may be referred to as a “remote proxy.” In some embodiments, such additional features data may be automatically obtained at some point after associated user interaction data is transmitted to tracking storage  32 . In other embodiments, such additional features data may be obtained when playback of that user interaction data is requested (or somewhat before, based on expected replay to prevent excessive system latency). 
         [0057]    Moreover, the client playback computer may perform certain scaling functions in order to ensure the recreated webpage is properly rendered in view of the, size resolution, and aspect ratio of the client computer. Because the source webpage may differ in these respects from that found in the playback computer, the conversion is desirable. One way this may be accomplished is to position a browser window at the playback computer off-screen and then copy bitmaps from the browser to the player windows at a high framerate, scaling them in the process. Another way this may be accomplished is to use the screenshots of the page taken during the normalization process and rescale them. In some embodiments, known interpolation techniques may be also be used, if desired. 
         [0058]    In some embodiments, as described above, the additional features data may have been captured at the time the user interaction data was captured. In this case, this information may be retrieved from tracking storage  32  when replay is requested. This may be referred to as a “local proxy.” In other embodiments, some additional features information may be captured with user interaction data, while others are not. In this case, proxy routines in tracking replay module  318  may first poll storage  32  for the additional features data. If it is not present in storage  32 , the information may be automatically sought as described above (i.e., start with a local proxy and expand to remote proxy, if necessary (or desired)). 
         [0059]      FIG. 5  illustrates a series of processing operations that may be implemented by the system illustrated in  FIG. 1 , and the exemplary computers illustrated in  FIGS. 2-4 , according to an embodiment of the invention. In the first processing operation  510  of  FIG. 5 , a user requests to playback file from the replay module  318 , using, for example, a local web browsing application. Next, at step  512 , module  318  provides an initialization file described above to the requesting computer. At step  514 , the initialization file is loaded, launches a playback application and the playback computer. 
         [0060]    In some embodiments, this may involve the use of a specialized browser or player designed to accept and process the initialization file for creating an appropriate playback rendering construct for interaction data. 
         [0061]    At step  516 , the playback application loads the data and/or metadata from the initialization file, which at least partially recreates the native browsing environment. Next, at step  518 , webpage and animation information is downloaded from replay module  318 . Afterwards, playback begins at step  520 . 
         [0062]      FIG. 6  illustrates a series of processing operations that may be implemented by web browser module  114  of user computer  10  illustrated in  FIG. 2 , in accordance with an embodiment of the invention. More particularly, once interaction data is captured as described above at step  612 , it may be preprocessed at step  614 , such that the playback data, and any associated metadata, may provided to the playback computer as further described herein. 
         [0063]    However, to ensure that sufficient data has been captured to allow playback, aspects of the present invention may further employ certain asset capture steps to acquire certain necessary information. These steps are preferably be performed substantially at the time of data capture to improve the likelihood the acquired information is useful for rendering interaction data. Nevertheless, any missing information may be acquired afterwards if necessary. For example, at step  616 , the system of the present invention may attempt to pre-play stored interaction data to determine whether sufficient information has, in fact, been captured. 
         [0064]    At step  618 , a high speed playback of a given interaction record may be attempted, which HTML code representing the webpage delivered to the playback browser (step  620 ). Next, at step  622 , required image and other media assets are downloaded through a proxy server. At step  624 , If the media assets are present in module  318 , it is provided to the playback computer at step  630  and playback continues (step  632 ). If the media asset is not present, the proxy application retrieves it from the web (step  626 ) through a custom proxy server in module  318  and stores is for future use in module  318  (step  628 ). 
         [0065]    Once this process has been completed for a given playback record, the asset capture process is complete (step  634 ). At this point, there is a high degree of confidence that the user will have the required media assets to playback the playback record. 
         [0066]    More specifically, at this point, every part of the replay including any images, javascript files, CSS documents, Flash movies, Silverlight movies or Screenshots should be stored in a replay database for subsequent use. When replays are requested by the client player application, these versions of the assets are delivered instead of contemporaneously seeking the originals from the web. In the case of what are called “relative requests” or requests that do not specify a fully qualified URL (eg: “../img.gif” instead of “http://mysite.com/img.gif”) a process called a “404 Handler” may be employed which catches these requests in a proxy server and relays them to the browser. In the process they are stored in the database for next time. 
         [0067]    An embodiment of the present invention relates to a computer storage product with a computer-readable medium having computer code thereon for performing various computer-implemented operations. The computer-readable media and computer code may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well known and available to those having skill in the computer software arts. Examples of computer-readable media include, but are not limited to: magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and holographic devices; magneto-optical media such as optical disks; and hardware devices that are specially configured to store and execute program code, such as application-specific integrated circuits (“ASICs”), programmable logic devices (“PLDs”) and ROM and RAM devices. Examples of computer code include machine code, such as produced by a compiler, and files containing higher-level code that are executed by a computer using an interpreter. For example, an embodiment of the invention may be implemented using XML, JavaScript, C, C++, or other scripting, markup and/or programming languages and development tools. Another embodiment of the invention may be implemented in hardwired circuitry in place of, or in combination with, machine-executable software instructions. 
         [0068]    The exemplary embodiments herein described are not intended to be exhaustive or to limit the scope of the invention to the precise forms disclosed. They are chosen and described to explain the principles of the invention and its application and practical use to allow others skilled in the art to comprehend its teachings. 
         [0069]    As will be apparent to those skilled in the art in light of the foregoing disclosure, many alterations and modifications are possible in the practice of this invention without departing from the spirit or scope thereof. Accordingly, the scope of the invention is to be construed in accordance with the substance defined by the following claims.