Generating profiling data

Systems and methods for generating profiling data include monitoring the execution of a task on a thread of a program. Profiling data may be generated that includes metrics regarding the execution of the task. A profiling data aggregation service may aggregate generated profiling data from a plurality of devices that execute the program.

BACKGROUND

The present disclosure relates generally to generating profiling data. The present disclosure more specifically relates to generating profiling data regarding the execution of tasks within a multi-threaded executable program.

Various levels of abstraction often separate source code written by a programmer and the actual instructions executed by a computing device. For example, a programmer may write source code for a computer program in a high-level programming language, such as C++ or C. The source code may then be transformed by a compiler into assembly code. Likewise, an assembler may then convert the assembly code into an object file having assembly code with offsets. A linker may then merge one or more object files or libraries into an executable file. During linking, references to external symbols may be resolved and finalized addresses to functions and variables may be generated.

SUMMARY

Implementations of the systems and methods for generating profiling data are described herein. One implementation is a computerized method for profiling a program. The method includes receiving, at a processing circuit, a request to execute a task on a thread of the program, the program being a commercially-released version of a software product. The method also includes monitoring, by the processing circuit, the execution of the task and generating profiling data indicative of one or more metrics regarding the execution of the task. The method further includes providing, over a network, the profiling data to a profiling data aggregation service.

Another implementation is a system for profiling a program. The system includes a processing circuit operable to receive a request to execute a task on a thread of a program, the program being a commercially-released version of a software product. The processing circuit is also operable to monitor the execution of the task and to generate profiling data indicative of one or more metrics regarding the execution of the task. The processing circuit is further operable to provide, over a network, the profiling data to a profiling data aggregation service.

A further implementation is a computer-readable storage medium having instructions therein, the instructions being executable by a processor to cause the processor to perform operations. The operations include receiving a request to execute a task on a thread of a program, the program being a commercially-released version of a software product. The operations also includes monitoring the execution of the task and generating profiling data indicative of one or more metrics regarding the execution of the task. The operations further include providing, over a network, the profiling data to a profiling data aggregation service.

Another implementation is a method for aggregating profiling data. The method includes providing a program to a plurality of electronic devices, the program being a commercially-released version of a software product for use by consumers. The program is also a multi-threaded application configured to generate profiling data indicative of one or more metrics regarding the execution of a task on a program thread. The method also includes receiving, at a profiling data aggregation server, profiling data from the plurality of electronic devices regarding the execution of the task by the plurality of electronic devices. The method further includes aggregating, by the profiling data aggregation server, the profiling data from the plurality of electronic devices regarding the execution of the task. The method yet further includes generating, by the profiling data aggregation server, a report comprising the aggregated profiling data and providing the report to an electronic display.

A further implementation is a system for aggregating profiling data. The system includes a processing circuit operable to provide a program to a plurality of electronic devices, the program being a commercially-released version of a software product for use by consumers. The program is also a multi-threaded application configured to generate profiling data indicative of one or more metrics regarding the execution of a task on a program thread. The processing circuit is also operable to receive profiling data from the plurality of electronic devices regarding the execution of the task by the plurality of electronic devices. The processing circuit is further operable to aggregate the profiling data from the plurality of electronic devices regarding the execution of the task and to generate a report comprising the aggregated profiling data. The processing circuit is yet further operable to provide the report to an electronic display.

These implementations are mentioned not to limit or define the scope of this disclosure, but to provide examples of implementations to aid in understanding thereof.

DETAILED DESCRIPTION

Various utilities may be used during the conversion of source code into an executable program, to identify bugs and other conditions that may impact the performance of the program. For example, a compiler may be configured to utilize different compiling modes that balance compilation time with optimizations that improve the performance of the compiled code. However, certain conditions that affect the performance of an executable program may remain undetectable until after the program is linked and run. In one example, assume that a program attempts to read a configuration file when the program is launched. Also assume that a user has placed the configuration file on a slow network drive, thereby increasing the amount of time needed for the program to launch. Since the program is functioning as designed, the unintentional delay in loading may not be detected until the program is fully linked and executed under those conditions. Moreover, the programmatic cause of the delay may not be immediately discernable to a user, only that the program is loading slowly.

In some implementations, profiling data may be generated regarding an executable program. As used herein, profiling data refers to data regarding the execution of the various portions of a program (e.g., processes, tasks, functions, code blocks, instructions, etc.). In some cases, profiling data may include statistics regarding the execution of the various portions of the program. For example, profiling data for a particular task may include statistics regarding the lifecycle of the task or function (e.g., execution time, creation thread, execution thread, queuing delay, resource utilization such as memory allocation, etc.). In some implementations, samples may be preserved of some profiling data for runs of substantially similar tasks (e.g., recording one or more samples of the statistics noted for a task). In some implementations, profiling data may be aggregated for substantially similar tasks or functions (e.g., count of executions, total execution time, minimum execution time, maximum execution time, maximum queuing delay, total resource utilization, etc.). In some implementations, substantial similarity of tasks may include agreement on one or more of the identifying statistics, such as the name or code location of a top-level function, or the thread on which the task was created, or the thread on which the task was executed. Profiling data differs from crash data in that crash data is generated in a reactionary manner, i.e., crash data may only be generated after a program crashes or hangs for a certain amount of time. As opposed to profiling data, crash data may be limited to information surrounding the crash and not the execution of the program during normal operation. For example, crash data does not include data regarding the successful completion of a task.

Profiling data for a program may be generated in a number of different ways. In some implementations, an executable file may be instrumented by inserting hooks into the code that record when different portions of the code are executed. However, generating profiling data is subject to the observer effect (i.e., the act of profiling a program may affect how the program would normally operate). For example, the greater the number of instrumentation instructions inserted into the program, the greater the detrimental impact on the performance of the program. Thus, a heavily instrumented program may be run in a test environment before the program is released for use by actual users, but may prove unacceptable for an actual user due to instrumentation induced performance reduction. However, how an instrumented program is used by a tester in a controlled environment may differ from how an actual user in the field may operate the program.

In various implementations, profiling data may be collected from devices operated by actual users in the field, if the users have opted in to allowing such data to be provided. The program may be configured, for example, to generate profiling data at a user's device and transmit the data to a profiling data aggregation service (e.g., a service operated by the entity that wrote the program or another similar entity). In some implementations, the degree of profiling may be such that the performance of the program may be minimally impacted, making the profiling process transparent to a user. Data collected by the service may then be used by developers of the program to diagnose errors, identify code portions at which performance may be improved, and gain a greater understanding of how the program operates in the field.

Referring toFIG. 1, a block diagram of a computer system100in accordance with a described implementation is shown. System100may include one or more devices104-108(e.g., a first device, a second device, etc., through an nth device) which communicate with other computing devices via a network110. In various implementations, devices104-108may execute copies of a program, such as a web browser, video game, word processor, spreadsheet application, communication application, or similar. The program may be configured to generate profiling data during normal functioning of the program. For example, a word processor may be configured to generate profiling data when the program is started, a new document is created, the document is saved, and the program is closed. The program may also be configured to transmit generated profiling data to a profiling data aggregation service102via network110, if a user of the device has opted in to providing such data. Data aggregation service102may aggregate profiling data generated by devices104-108, to store metrics on the execution of the program.

Network110may be any form of computer network that relays information between devices104-108, network110, and profiling data aggregation service102. For example, network110may include the Internet and/or other types of data networks, such as a local area network (LAN), a wide area network (WAN), a cellular network, satellite network, or other types of data networks. Network110may also include any number of computing devices (e.g., computer, servers, routers, network switches, etc.) that are configured to receive and/or transmit data within network110. Network110may further include any number of hardwired and/or wireless connections. For example, device104may communicate wirelessly (e.g., via WiFi, cellular, radio, etc.) with a transceiver that is hardwired (e.g., via a fiber optic cable, a CATS cable, etc.) to other computing devices in network110.

Devices104-108may be of any number of different types of user electronic devices configured to communicate via network110(e.g., laptop computers, desktop computers, tablet computers, smart phones, digital video recorders, set-top boxes for televisions, video game consoles, combinations thereof, etc.). Each of devices104-108may include a processing circuit configured to perform the operations described with regard to devices104-108. In general, a processing circuit includes one or more processors and one or more memory devices. For example, devices104-108are shown to include processors114-118and memories122-126, respectively. Memories122-126may store machine instructions that, when executed by processors114-118, cause processors114-118to perform operations. Processors114-118may include microprocessors, ASICs, FPGAs, other forms of processing devices, or combinations thereof. Memories122-126may include, but is not limited to, electronic, optical, magnetic, or any other storage or transmission devices capable of providing processors114-118with program instructions. Memories122-126may include a floppy disk, CD-ROM, DVD, magnetic disk, memory chip, ROM, RAM, EEPROM, EPROM, flash memory, optical media, or any other suitable memory from which processor114can read instructions. The instructions may include code from any suitable computer programming language such as, but not limited to, C, C++, C#, Java, JavaScript, Perl, HTML, XML, Python and Visual Basic.

Devices104-108may include one or more user interface devices. A user interface device may be any electronic device that conveys data to a user by generating sensory information (e.g., a visualization on a display, one or more sounds, etc.) and/or converts received sensory information from a user into electronic signals (e.g., a keyboard, a mouse, a pointing device, a touch screen display, a microphone, etc.). The one or more user interface devices may be internal to the housings of devices104-108(e.g., a built-in display, microphone, etc.) or external to the housings of devices104-108(e.g., a monitor connected to device104, a speaker connected to device106, etc.), according to various implementations. For example, devices104-108may include displays128-132.

Profiling data aggregation service102may be one or more electronic devices connected to network110configured to receive profiling data from devices104-108. For example, profiling data aggregation service102may be a computer server (e.g., an FTP server, file sharing server, web server, etc.) or a combination of servers (e.g., data centers, cloud computing platforms, etc.). Similar to devices104-108, profiling data aggregation service102include a processing circuit. For example, profiling data aggregation service102may include processor112and memory120that stores program instructions executable by processor112. Where profiling data aggregation service102is a combination of devices, processor112may represent the collective processors of the devices and memory120may represent the collective memories of the devices.

According to various implementations, copies of a particular program on devices104-108may be configured to generate profiling data during execution. The profiling data may be generated continuously during execution, at certain periods of time during execution (e.g., during startup, during shutdown, etc.), or based on one or more configuration settings (e.g., based on a configuration parameter that turns a profiling mode on and off). The generated profiling data may be sent to profiling data aggregation service102using either a push or pull methodology. For example, device104may transmit profiling data to profiling data aggregation service102in response to receiving a request for profiling data from profiling data aggregation service102. In other examples, device104may transmit profiling data to profiling data aggregation service102without first receiving a request for the data (e.g., periodically, when the program is launched, when the program is closed, as a background process at night, etc.).

The program executed by devices104-108may use a multi-process architecture. In a multi-process architecture, a program may be divided into a set of processes that act as independent programs and work in cooperation to implement the full functionality of the program. Different processes may be allocated to different memory spaces (e.g., address spaces), when the program is executed. Processes may communicate between one another, but require the use of an inter-process communication (IPC) channel that passes inter-process messages.

The program executed by devices104-108may also use a multi-threaded design, in some implementations. In general, a thread differs from a process in that multiple threads may share the same memory space. Multiple threads may belong to the same process, i.e., the memory space allocated to a certain process may be used by different threads. Threads may be executed at different times, allowing a more efficient use of hardware resources. For example, a task may be executed on thread B while a task on thread A is waiting for a reply from a peripheral device. Tasks may be posted to different threads for the same process. In some implementations, an inter-process task may be spawned on a thread of a first process for execution on a different thread of another process.

In one example, a text editor may run as a single process that includes multiple threads. One thread may execute a “File Open” task to open an existing text document from a file system. Another thread may execute a “File Save” task to save changes to a text document. Both threads exist within the same process, allowing the memory space allocated to the process to be shared between the threads.

According to various implementations, the program may be profiled at the task and/or function level. Tasks that are posted to a thread within the same process and/or IPC tasks may be tracked to generate profiling data regarding the execution of the tasks. For example, data regarding how long the task took to execute, how long the task waited in a scheduler queue to execute, the source and/or destination threads for the task (e.g., on which thread the task was spawned, on which thread the task was executed, etc.), the process that executed the task, and other such data may be gathered by the program as profiling data. Statistics may also be generated regarding the collected data, such as the average amount of time taken by the task to execute, the average amount of time the task waited in the scheduler queue, etc. Profiling at the task level allows for profiling data to be gathered without significantly affecting the performance of the program. Thus, profiling data may be generated and collected when the program is run by actual users in the field (i.e., the program is run on devices104-108). The profiling data may then be collected by profiling data aggregation service102, if the users of devices104-108have opted in to sharing such data.

In some implementations, profiling data aggregation service102may be configured to generate one or more reports using the aggregated profiling data. For example, profiling data aggregation service102may receive profiling data from devices104-108regarding their respective executions of the same task of the program. In such a case, profiling data aggregations service102may aggregate the profiling data from devices104-108and provide aggregated metrics for the task as part of a generated report. For example, such a report may reflect the average execution time of the task by any number of users' devices in the field.

Referring now toFIG. 2, an illustration of an example multi-process architecture200for a web browser is shown. The web browser may be divided into a number of processes, allowing for the execution of the program to be compartmentalized. Doing so may protect the entire browser from crashing, if a particular process crashes. For example, browser tabs (e.g., different visual areas of the program in which to view different webpages and other forms of content) may be implemented to use different processes. Thus, a bug in one webpage or plug-in that causes a particular tab to crash would not cause the other open tabs in the browser to crash. While architecture200is shown with regard to web browser implementations, the described methodologies may be applied to any multi-threaded application (e.g., a video game, a text editor, a messaging application, etc.).

As shown, the web browser program may be subdivided into a browser process202and one or more renderer processes206-208. Browser process202may oversee the primary user interface functions of the browser, as well as coordinating inter-process communications with renderer processes206-208. Renderer processes206-208may render content within different tabs of the web browser. For example, renderer processes206-208may use layout engines250-252, which are configured to interpret a webpage markup language, such as the hypertext markup language (HTML), and to determine the visual layout of the webpage. Layout engines250-252may be any type of layout engine, such as the open-source engines WebKit and Gecko or the proprietary engines Trident and Presto.

Any number of threads may be allocated to browser process202and renderer processes206-208. For example, browser process202may include a main thread212on which tasks associated with the main browser window are executed. Browser process202may also include an I/O thread210that handles communications between browser process202and other processes, such as renderer processes206-208. Similarly, renderer processes206-208may include renderer threads216,220, on which layout engines250,252execute, respectively, to interpret and render webpage layout data. Renderer processes206-208may also include threads214,218, on which IPC communications are handled for their respective renderer processes.

A resource dispatcher host240may execute on I/O thread210that coordinates the passing of messages between renderer processes206-208and main thread212of browser process202. IPC pipes246-248may be assigned to renderer processes206-208, respectively, over which IPC communications between renderer processes206-208and I/O thread210of browser process202may be passed. IPC pipes246-248may pass communications within channels242-244in I/O thread210, which proxy messages through proxy channels to main thread212. Resource dispatcher host240may control channels242-244, to manage the passing of messages between renderer processes206-208and main thread212of browser process202. Render process objects222-224executing on threads214,218, respectively, may handle requests between render threads216,220and corresponding render process hosts226-228executing on main thread212of browser process202. Render process hosts226-228may pass browser state information and other data to their corresponding render process objects222-224.

On each of render threads216,220, resource dispatchers254-256may coordinate resource requests with resource dispatcher host240via render process objects222-224. Render view objects230,232,236may represent the contents of a webpage within a particular tab or browsing area of the program, as laid out by layout engines250-252. Render view hosts234,235,236may execute on main thread212of process202and use data from render view objects230,232,236to display webpages within different tabs in the web browser window.

According to various implementations, profiling data may be generated regarding tasks posted to threads210-220and/or inter-process tasks posted via IPC pipes246-248. For example, a task executed on render thread216of renderer process206may be programmatically monitored to determine the source thread from which the task was initiated, how long the task took to execute, and similar data. In another example, render process host226executing on main thread212may dispatch a “message send” task for execution on I/O thread210to send IPC data to renderer process206. Profiling the web browser during the dispatch of tasks to its various threads allows the profiling data to be generated in a minimally-intrusive manner, so as not to significantly affect the performance of the web browser.

Profiling data may be generated at all times or may be generated only at certain times. In some implementations, the dispatch of tasks within a program may be randomly sampled. For example, n-number of samples may be recorded for some or all tasks executed within a thread and/or process. The samples may include data regarding the resources used by an executing task. For example, samples regarding the wall-clock and/or CPU time used to execute the task, the amount of memory used, the disk writes made, I/O operations performed during execution of the task, combinations thereof, or the like. In some implementations, a particular task or a part of a task (e.g., a function executed as part of the task) may be sampled. For example, only certain task or functions may be profiled.

Referring now toFIG. 3, a flow diagram of a process300for generating profiling data is shown, according to various implementations. Process300may be implemented by a computing device executing a program configured to generate profiling data. For example, process300may be implemented by devices104-108shown inFIG. 1, or by any other form of device that includes a processor that executes instructions stored within a memory. The instructions may include a program configured to generate profiling data during execution of the program. Since the program is profiled at the task level, the generation of profiling data may have minimal impact on the performance. Thus, the program may be a commercial version of the program distributed for use by consumers in the field, in some cases. In some implementations, profiling data generated by process300may be provided to a device that stores and aggregates profiling data from multiple devices, such as profiling data aggregation service102. Therefore, profiling data from the devices of multiple consumer devices may be collected and aggregated to generate metrics on the performance of the program on actual consumers' devices.

Process300includes receiving a request to execute a task on a thread (block302). In a multi-threaded program, a task may be dispatched for execution on a particular thread. In general, a task refers to a set of one or more functions (e.g., portions of executable code) that may be executed on a particular thread. For example, a thread may execute a function that corresponds to sending a request for a webpage to a URL address. A task may be dispatched to the same thread, a different thread within the same process, or even to a thread of another process. For example, a task may be dispatched from a thread in a first process to a thread in another process via an IPC channel.

Process300includes monitoring the execution of the task on the thread (block304). Various metrics may be collected regarding the execution of the task. Monitoring the task may include recording when the task was dispatched for execution on the thread, when the task was executed, when the task completed execution, and the resources used by the task while executing. In some implementations, programmatic hooks may be inserted into the program to monitor the execution of a task. For example, profiling hooks may be inserted into a module that handles IPC messages, to determine when an IPC task is dispatched. In another example, a program counter of a thread may be monitored to determine how tasks are executed on the thread. In general, monitoring the execution of the program at the task level allows for the generation of profiling data without significantly affecting the performance of the program. In some implementations, the program may be configured to always monitor the execution of a task and to generate profiling data. In other implementations, the program may be configured to only profile the execution of a task in response to receiving a request (e.g., based on a user-set parameter, based on a request from a profiling data aggregation service, etc.).

Process300includes generating profiling data regarding the execution of the task (block306). For example, profiling data may include data indicative of the number of times the task was executed, the wall-clock and/or CPU time taken to execute the task, the amount of time the task waited in a scheduling queue before executing, the thread that dispatched the task, the thread on which the task executes, the process that executes the task, the name of a function executed as part of the task, memory used by the task, disk writes by the task, I/O operations performed by the task, and similar metrics. In some implementations, a task may be monitored randomly and/or only at certain times when the task is executed. Profiling data may also include statistics based on recorded metrics. For example, data regarding average metrics, maximum and/or minimum metrics (e.g., the maximum time to execute, etc.), and similar statistics may be calculated and included as part of the profiling data. Thus, the profiling data may be updated each time the task is executed.

According to various implementations, blocks302-306may be repeated any number of times within process300. For example, a particular task may be dispatched to a thread and executed a multitude of times. Each time the task is dispatched and executed, its execution may be monitored and the resulting profiling data aggregated with other profiling data regarding the task. In non-limiting examples, such aggregated profiling data may include the average run time for the task, the maximum run time for the task, the average queue time for the task, or any other metric regarding the execution of the task.

Process300includes providing the generated profiling data (block308). In some implementations, the generated profiling data may be provided to a user interface device, such as a display. For example, the profiled program may be configured to display one or more screens, allowing a user of the program to review the profiling data. In another example, a separate application may receive the profiling data as input and display the profiling data on a display. In some implementations, the generated profiling data may be provided to a profiling data aggregation service, which receives and stores profiling data from a plurality of devices that run copies for the profiled program. The profiling data may be provided in response to receiving a request from the aggregation service, at a certain point in the execution of the program (e.g., during startup, when the program is closed, etc.), as a background process, in response to receiving a command from a user interface device (e.g., from a keyboard, a mouse, touch screen display, etc.), or at any other time. In other implementations, the profiling data first may be aggregated locally (e.g., by repeating blocks302-306a number of times and locally aggregating the profiling data for the task) and then sent to the profiling aggregation service.

Referring now toFIG. 4, an illustration of an example GUI showing profiling data on a display device400is depicted. The GUI may be configured to display various metrics and statistics from the profiling data. In some implementations, the displayed profiling data may be derived from the profiled program running on a single device. For example, the GUI may be part of the program on a user's device that executes the program. In further implementations, the GUI may display aggregated profiling data from different instances of the program running on a plurality of devices. For example, a profiling data aggregation service may generate the GUI depicted inFIG. 4using profiling data aggregated from any number of users' devices in the field.

In some implementations, the GUI may include columns402-424that include information about the execution of a particular task. For example, the GUI may include column402that includes an execution count for a task (e.g., the number of times the task was executed since starting the profiling). Column404may display the total run time for the task (e.g., a wall-clock count, a CPU count, etc.). Column406may display the average run time for the task (e.g., the quotient of the value in column404divided by the value in column402. Column408may display the maximum time a single run of the task took to execute. Column410may display the average time the task took to queue before execution (e.g., measured from when the task was constructed to when the task was run, measured from when a delayed task was asked to run to when the task was actually run, measured as the inter-process latency for an IPC task, etc.). Column412may display the maximum queue time over all runs of the task. Column414may display the thread on which the task was constructed. Column416may display the thread on which the task was executed. Column418may display the type of process associated with the thread on which the task was executed. Column420may display the process ID of the process that executed the task. Column422may display a function executed as part of the task. Finally, column424may display the name and/or a link to the source code location of the function indicated by column422. For example, a programmer may click on a link in column424to navigate to the portion of the source code that corresponds to the task.

Any number of tasks may be represented in the GUI as row entries, in the example shown. For example, the GUI may include rows426-432to represent different tasks executed by the program. As shown, row430may indicate via columns414,416,422that a task that includes the function, “ContentBrowserClient::AllowGetCookie,” may be spawned from the thread, “IOThread,” and is executed on the thread, “BrowserMain.” Similarly, column424may indicate that the function is defined within the source file, “content browser client.cc.” Thus, a programmer or other interested party that wishes to make changes to the function or see how the function is defined may click on the link in column424. For example, assume that the average run time for the task is higher than expected. In such a case, a programmer may navigate to the corresponding location in the source code to diagnose the error.

The features disclosed herein may be implemented on a smart television module (or connected television module, hybrid television module, etc.), which may include a processing circuit configured to integrate internet connectivity with more traditional television programming sources (e.g., received via cable, satellite, over-the-air, or other signals). The smart television module may be physically incorporated into a television set or may include a separate device such as a set-top box, Blu-ray or other digital media player, game console, hotel television system, and other companion device. A smart television module may be configured to allow viewers to search and find videos, movies, photos and other content on the web, on a local cable TV channel, on a satellite TV channel, or stored on a local hard drive. A set-top box (STB) or set-top unit (STU) may include an information appliance device that may contain a tuner and connect to a television set and an external source of signal, turning the signal into content which is then displayed on the television screen or other display device. A smart television module may be configured to provide a home screen or top level screen including icons for a plurality of different applications, such as a web browser and a plurality of streaming media services, a connected cable or satellite media source, other web “channels”, etc. The smart television module may further be configured to provide an electronic programming guide to the user. A companion application to the smart television module may be operable on a mobile computing device to provide additional information about available programs to a user, to allow the user to control the smart television module, etc. In alternate embodiments, the features may be implemented on a laptop computer or other personal computer, a smartphone, other mobile phone, handheld computer, a tablet PC, or other computing device.