Diagnostic system and method

A computer-implemented method, computer program product and computing system for recording video information on the computing device during a monitored event. Execution information is recorded on the computing device during the monitored event. The video information and the execution information are temporally synchronized to form temporally-synchronized diagnostic content.

TECHNICAL FIELD

This disclosure relates to diagnostic systems and, more particularly, to diagnostic systems for use with computing systems.

BACKGROUND

During the normal operation of computing devices, problems may be encountered. For example, web applications may not operate properly, resulting in websites not loading properly. Unfortunately, diagnosing and reporting these issues is often highly frustrating, as the user who experienced the issue would need to describe the issue that they experienced to a remote technician so that the technician may attempt to diagnose the problem remotely. Alternatively and in order to avoid such a situation, the user may have a technician make a “house call” to locally resolve the problem, but that typically incurs considerable expense.

SUMMARY OF DISCLOSURE

In one implementation, a computer-implemented method is executed on a computing device and includes recording video information on the computing device during a monitored event. Execution information is recorded on the computing device during the monitored event. The video information and the execution information are temporally synchronized to form temporally-synchronized diagnostic content.

One or more of the following features may be included. The temporally-synchronized diagnostic content may be provided to a third party for analysis. Recording video information on the computing device during a monitored event may include recording a timestamped video portion. The execution information may include network traffic information. The execution information may include diagnostic information. The video information may include web browser content rendered within a web browser by the computing device. The monitored event may be a computer malfunction event.

In another implementation, a computer program product resides on a computer readable medium and has a plurality of instructions stored on it. When executed by a processor, the instructions cause the processor to perform operations including recording video information on the computing device during a monitored event. Execution information is recorded on the computing device during the monitored event. The video information and the execution information are temporally synchronized to form temporally-synchronized diagnostic content.

One or more of the following features may be included. The temporally-synchronized diagnostic content may be provided to a third party for analysis. Recording video information on the computing device during a monitored event may include recording a timestamped video portion. The execution information may include network traffic information. The execution information may include diagnostic information. The video information may include web browser content rendered within a web browser by the computing device. The monitored event may be a computer malfunction event.

In another implementation, a computing system includes a processor and a memory system configured to perform operations including recording video information on the computing device during a monitored event. Execution information is recorded on the computing device during the monitored event. The video information and the execution information are temporally synchronized to form temporally-synchronized diagnostic content.

One or more of the following features may be included. The temporally-synchronized diagnostic content may be provided to a third party for analysis. Recording video information on the computing device during a monitored event may include recording a timestamped video portion. The execution information may include network traffic information. The execution information may include diagnostic information. The video information may include web browser content rendered within a web browser by the computing device. The monitored event may be a computer malfunction event.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

System Overview

Referring toFIG.1, there is shown diagnostic process10. Diagnostic process10may be implemented as a server-side process, a client-side process, or a hybrid server-side/client-side process. For example, diagnostic process10may be implemented as a purely client-side process via reward process10c. Alternatively, diagnostic process10may be implemented as a hybrid server-side/client-side process via reward process10cin combination with diagnostic process10s. Accordingly, diagnostic process10as used in this disclosure may include any combination of diagnostic process10sand diagnostic10c.

Diagnostic process10cmay be a client application and may reside on and may be executed by computing device12, which may be coupled to network14(e.g., the Internet or a local area network). Examples of computing device12may include, but are not limited to: a smartphone, a tablet computer, a notebook computer, a laptop computer, a personal computer, and a server computer. Computing device12may execute an operating system, examples of which may include but are not limited to Microsoft Windows™, Mac OS™, Android™, iOS™, Linux™, or a custom operating system.

The instruction sets and subroutines of diagnostic process10c, which may be stored on storage device16coupled to computing device12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within computing device12. Examples of storage device16may include but are not limited to: a hard disk drive; a RAID device; a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices.

Diagnostic process10smay be a server application and may reside on and may be executed by computing device18, which may also be coupled to network14(e.g., the Internet or a local area network), thus allowing diagnostic process10cand diagnostic process10sto communicate. Examples of computing device18may include, but are not limited to: a personal computer, a server computer, a series of server computers, a mini computer, a mainframe computer, or a cloud-based computing platform/service/network. Computing device18may execute an operating system, examples of which may include but are not limited to Microsoft Windows™, Mac OS™, Android™, iOS™, Linux™, or a custom operating system.

The instruction sets and subroutines of diagnostic process10s, which may be stored on storage device20coupled to computing device18, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within computing device18. Examples of storage device20may include but are not limited to: a hard disk drive; a RAID device; a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices.

Diagnostic Process

Assume for illustrative purposes that computing device12is a personal computer that is used by user22to execute various computer applications including, but not limited to, web browser24. For this example, user22may be any type of computer user. Accordingly, user22may be an ordinary computer user who uses computing device12for their personal enjoyment or for work-related tasks. Alternatively, user22may be an IT professional who uses computing device12to test various programs and/or computing platforms.

Assume, for this example, that while user22is using web browser24, a computer malfunction event occurs. Examples of such a computer malfunction event may include but are not limited to web browser24freezing, web browser24failing to load a webpage, or web browser24simply operating in an improper fashion. Accordingly, user22may utilize diagnostic process10to analyze this computer malfunction event.

Therefore, upon realizing that a computer malfunction event has occurred, user22may initiate diagnostic process10and repeat the procedures that led up to the computer malfunction event so that it could be analyzed by diagnostic process10. For example, web browser24(or an application program interface of web browser24) may be utilized to allow user22to conveniently initiate diagnostic process10. For example, a default toolbar (not shown) incorporated into web browser24may be modified to include a “start” button (not shown) that allows user22to initiate diagnostic process10. Web browser24may include any web browser including mobile web browsers.

Referring also toFIG.2, diagnostic process10may record100video information (e.g. video information26) on computing device12during a monitored event (e.g. the above-described computer malfunction event). While the following discussion concerns the event being monitored by diagnostic process10as a computer malfunction event, this is for illustrative purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible. For example, user22may utilize diagnostic process10to simply monitor the operation of web browser24, regardless of whether web browser24is operating properly or is operating improperly.

Video information26may include web browser content that is rendered within web browser26by computing device12. For example, video information26may be the content that was rendered within web browser26during the recording100process described above. Therefore, video information26may include static images (e.g. words, pictures, tables, images) or moving images (e.g. videos, animations).

When diagnostic process10effectuates the recording100of video information26rendered (in this example) within web browser24, this may be accomplished using one or more functionalities included within (or incorporated into) web browser24. For example, Google Chrome™ and Microsoft Edge™ may include native functionality that may allow for the recording of content rendered within these web browsers. Accordingly, diagnostic process10may be configured to utilize such native functionalities within these web browsers to effectuate the recording100of video information26.

While the following discussion concerns video information26being content that is rendered by a specific application (e.g. web browser26), this is for illustrative purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible. For example, the recording100of video information26may be broader in scope and may include the recording100of all content shown on display28of computing device12.

When recording100video information26on computing device12during a monitored event (e.g. the above-described computer malfunction event), diagnostic process10may record102a timestamped video portion. For example, diagnostic process10may record the content of web browser24one portion (or chunk) at a time. The granularity of these portions may be adjusted based upon system needs, capabilities, or the desired level of granularity. For example, assume that video information26is recorded100in one second portions by diagnostic process10. When recording these one second portions of video, diagnostic process10may apply a time stamp to each of these one second video portions so that the time at which a specific video portion was recorded is known. The time included in these timestamps may be obtained from e.g. web browser26and/or a system clock (not shown) included within computing device12. For example, diagnostic process10may utilize a MediaRecorder API (e.g., of web browser24), to save the exact timestamp and file size of a particular video portion.

While an example of a user initiating diagnostic process10in response to computer malfunction event and repeating the procedures that led up to the computer malfunction event has been described above, it will be appreciated that diagnostic process10may continuously record100video information26as a user interacts with web browser content rendered within a web browser and may temporally synchronize video information and execution information after the computer malfunction event.

Referring also toFIG.3A, diagnostic process10may record100a continuous video (e.g., video recording300) of user interactions with the web browser content rendered within the web browser24. Continuous recording may generally include passively recording all user interactions with a web browser.

Diagnostic process10may include a custom embedded video which may append video segments of the continuous video recording to a buffer. For example, diagnostic process10may parse video recording300into a plurality of video segments (e.g., video segments302,304). In some embodiments, diagnostic process10may parse video recording300into a plurality of video segments based upon, at least in part, one or more keyframes. As is known in the art, a keyframe may generally include a point in time where changes to one or more parameters of video content begin or end. In the example ofFIG.3A, video recording300is shown as a video content beginning at time “t0” through time “t2”. Accordingly, diagnostic process10may parse video recording300into video segments302,304based upon, at least in part, keyframes at time “t1” and “t2”.

Suppose that at time “t1” (e.g., thirty seconds from time “t0”), a keyframe is detected by diagnostic process10. In this example, diagnostic process10may parse a first video segment (e.g., video segment302) between time “t0” and time “t1” from video recording300based upon, at least in part, the keyframe at time “t1”. Continuing with this example, suppose diagnostic process10detects another keyframe at time “t2” (e.g., two minutes from time “t0”). In this example, diagnostic process10may parse a second video segment (e.g., video segment304) between time “t1” and time “t2” from video recording300based upon, at least in part, the keyframes at time “t1” and time “t2”. In this example, diagnostic process10may record video information26including video segments302and304of continuous video recording300.

As discussed above, diagnostic process10may maintain a running buffer of the continuous video recording to enable a user to record video information of their interactions with a web browser. In this manner, diagnostic process10may trim at least one video segment from the plurality of video segments of the video recording based upon, at least in part, a predefined minimum video recording length, thus defining the video information. Diagnostic process10may identify locations in the continuous video recording where it is safe to trim the continuous video recording. In some embodiments, diagnostic process10may trim the continuous video recording based upon, at least in part, the one or more keyframes. For example, when a new keyframe is detected, diagnostic process10may determine the length (e.g., in seconds) of a first keyframe to keyframe video segment in the running buffer to determine if the first video segment may be trimmed. As will be discussed in greater detail below, because the trimming of video segments from the continuous video recording may be based upon, at least in part, one or more keyframes, the buffered video may not be kept to a consistent length. Accordingly, a predefined minimum video recording length may determine whether diagnostic process10trims at least one video segment from the continuous video recording. In this manner, diagnostic process10may provide video information including a video recording of at least the predefined minimum video recording length.

Referring also toFIG.3B, suppose that diagnostic process10detects another keyframe at time “t3” (e.g., eleven minutes from time “t0”). In this example, diagnostic process10may record100additional video content and parse a third video segment (e.g., video segment306) between time “t2” and time “t3” from video recording300based upon, at least in part, the keyframes at time “t2” and time “t3”. Diagnostic process10may determine the length (e.g., in seconds) of a first keyframe to keyframe video segment (e.g., first video segment302) in the running buffer to determine if the first video segment may be trimmed. In this example and as discussed above, suppose first video segment302is thirty seconds long (e.g., thirty seconds between the keyframes at time “t1” and time “t0”). Further suppose that the predefined minimum video recording length is e.g., ten minutes. While an example of ten minutes has been provided, it will be appreciated that the predefined minimum video recording length may be any length within the scope of the present disclosure. For example, the predefined minimum video recording length may be user-defined (e.g., via a user interface) and/or automatically defined (e.g., a default predefined minimum video recording length).

Returning to the above example, diagnostic process10may determine that because trimming first video segment302would result in a total video recording length of e.g., ten minutes and thirty seconds, process10may trim first video segment302from the running buffer. Accordingly, diagnostic process10may record video information26′ including video segments304and306.

Referring also toFIG.3C, suppose that diagnostic process10detects another keyframe at time “t4” (e.g., twenty minutes from time “t0”). In this example, diagnostic process10may parse a fourth video segment (e.g., video segment308) between time “t3” and time “t4” from video recording300based upon, at least in part, the keyframes at time “t3” and time “t4”. Diagnostic process10may determine the length (e.g., in seconds) of a first keyframe to keyframe video segment (e.g., second video segment304) in the running buffer to determine if the first video segment may be trimmed. In this example and as discussed above, suppose second video segment302is one minute and thirty seconds long (e.g., one minute and thirty seconds between the keyframes at time “t2” and time “t1”). Further suppose that the predefined minimum video recording length is e.g., ten minutes. In this example, diagnostic process10may determine that because trimming second video segment304would result in a total video recording length of e.g., eighteen minutes (e.g., video segment306has a length of nine minutes and video segment308has a length of nine minutes), process10may trim second video segment304from the running buffer. Accordingly, diagnostic process10may record video information26″ including video segments306and308.

While several examples have been provided for trimming video segments based upon, at least in part, the one or more keyframes, it will be appreciated that diagnostic process10may trim at least one video segment from the plurality of video segments of the video recording without relying on the detection of keyframes and/or by forcing keyframes at desirable trimming locations within the video segment. For example, suppose diagnostic process10is configured to maintain a running video buffer with a predefined minimum video recording length. In this example, diagnostic process10may force a keyframe at the predefine minimum video length to define a running video buffer with at least the predefined minimum video recording length.

Diagnostic process10may also be configured to store each of the one or more video segments of the continuous video recording. For example, suppose diagnostic process10is executed on a computing device with large amounts of storage space for storing video segments of the continuous video recording. In this example, diagnostic process10may store each video segment as discussed above. In some embodiments, diagnostic process10may delete the oldest video segments after a predefined amount of time (e.g., a predefined number of minutes, hours, days, months, etc.). In this manner, diagnostic process10may store one or more video segments for a user to access without having to repeat the procedures that caused a monitored event (e.g., a computer malfunction event).

In addition to recording100video information26, diagnostic process10may record104execution information30on computing device12during the monitored event (e.g. the above-described computer malfunction event). For example, diagnostic process10may record104various pieces of execution information30that are available from computing device12and/or web browser24so that execution information30may be used (in conjunction with video information26) to aid in the understanding/diagnosis/repair of the above-described computer malfunction event. Examples of execution information30may include, but are not limited to, network traffic information and diagnostic information.

As discussed above, web browser24may be configured to provide one or more native functionalities. For example, Google Chrome™ and Microsoft Edge™ may include native functionalities that gather execution information (e.g. execution information30). Additionally/alternatively, computing device12(or the operating system/applications executed on computing device12) may be configured to gather execution information30. Accordingly, diagnostic process10may be configured to utilize such native functionalities within e.g. web browser24, computing device12, the operating system of computing device12, or one or more applications executed on computing device12to obtain execution information30.

For example and when recording104execution information30, diagnostic process10may enable the console domain (e.g., of web browser24) to collect console messages that were already in the buffer, as well as collect new console message as they arrive. Further, diagnostic process10may enable the debugger domain (e.g., of web browser24) to collect scriptParsed and scriptFailedToParse events, wherein these scriptParsed events may allow diagnostic process10to determine whether scripts have associated source maps and (if so) store these associated source maps for stack traces. Further, diagnostic process10may enable the network domain (e.g., of web browser24) to collect all network events as they occur, which may be logged. When the loadingFinished event is received, if the request type is ‘XHR’ (which typically indicates a dynamic resource being loaded), diagnostic process10may retrieve the response body from web browser24(which may also be logged).

In a fashion similar to that of video information26, execution information30may be timestamped (in the event that execution information30does not include native time indicators).

Diagnostic process10may temporally synchronize106(using the above-described timestamps) video information26and execution information30to form temporally-synchronized diagnostic content32. The process of temporal synchronizing106video information26and execution information30to form temporally-synchronized diagnostic content32may be done locally (e.g. on computing device12) or remotely (e.g. on computing device18). As discussed above, examples of computing device18may include, but are not limited to: a personal computer, a server computer, a series of server computers, a mini computer, a mainframe computer, or a cloud-based computing platform/network. Accordingly, diagnostic process10may upload video information26and execution information30to a cloud-based platform (e.g. the Google cloud) where video information26and execution information30may be temporally synchronized106to form temporally-synchronized diagnostic content32. In the event that video information26and/or execution information30is provided to a third party (e.g. the Google cloud), one or more of video information26and/or execution information30may be encrypted.

As discussed above, diagnostic process10may allow a user to detect a monitored event (e.g., a computer malfunction event) and temporally synchronize a running buffer of recorded video information and execution information to form the temporally-synchronized diagnostic content. For example, diagnostic process10may receive a capture command in response to a monitored event. For example, a default toolbar (not shown) incorporated into web browser24may be modified to include a “capture” button (not shown) to generate a capture command for diagnostic process10. In response to receiving the capture command, diagnostic process10may temporally synchronize the video information and the execution information to form temporally-synchronized diagnostic content32. In this manner, diagnostic process10may keep a running buffer of past activity rather than requiring a user to actively recreate an issue or monitored event the user came across.

Temporally-synchronized diagnostic content32may be played (e.g., in a web browser or media player) by e.g. user22, which may allow user22to ascertain the cause of the above-described computer malfunction. For example, temporally-synchronized diagnostic content32(when played) may allow user22to monitor the status of the above-described execution information30while watching video information26. Accordingly, when the above-described computer malfunction occurs (as evidenced by video information26), user22may be able to determine what was going on within web browser24via execution information30. Temporally-synchronized diagnostic content32may be configured to allow user22to pause the play back of temporally-synchronized diagnostic content32so that certain areas of temporally-synchronized diagnostic content32may be scrutinized (e.g. the area in which the above-described computer malfunction event begins to occur).

While temporarily-synchronized diagnostic content32is described above as being reviewed locally (e.g. on computing device12by user22), other configurations are possible are considered to be within the scope of this disclosure. For example, diagnostic process10may be configured to provide108temporally-synchronized diagnostic content32to a third party for analysis. Examples of such a third party may include but are not limited to user34. For example, user34may be a remote diagnostic technician that is either works with user22or works for a company that produced e.g. computing device12or by browser24. Accordingly, in the event of the above-described computer malfunction, user22may utilize diagnostic process10to capture video information26and execution information30, which may be temporally synchronized106to form temporally-synchronized diagnostic content32that may be provided to users34(who works for a technical support line for e.g. web browser24). Therefore and when diagnosing the above-described computer malfunction event, user34may simply review temporally-synchronized diagnostic content32determine a cause of the above-described computer malfunction event (as opposed to requiring that user22describe or replicate the above-described computer malfunction event for user34).

General