Mechanism for maintaining detailed trace information relevant to the current operation being processed

A system, method, computer program product, and program storage device for storing trace information of a program is disclosed. Upon entering or calling a subroutine, a memory buffer is created. Whenever a nested subroutine is called inside the subroutine, a subordinate memory buffer is created. Upon completion of a subroutine execution, a corresponding memory buffer is deleted. When encountering an event (e.g., an error, a defect, a failure, a warning) during execution, all data in currently existing memory buffers are transferred to a secondary memory storage device (e.g., a disk).

BACKGROUND OF THE INVENTION

1. Fields of the Invention

The present invention generally relates to debugging an error in a program. More particularly, the present invention relates to generating and storing trace information (e.g., parameters passed to a called subroutine, return values of a subroutine, trace data) of a program to debug an error in the program.

2. Description of the Prior Art

Application tracing is important to provide trace information when a program has failed. For example, the application tracing has following requirements;trace as much usefull detail as possible;do not store too much information;trace as little information as possible that is unrelated to an event (e.g., a failure, a defect, an error, a warning) to diagnose;

Currently, a lot of large applications use proprietary tracing software, but the tracing software is often based on common systems. For example, a programming language Java comes with a built-in tracing application (e.g., java.util.logging (also known as JSR 47)).

When the application tracing detects an event (e.g., an error, a failure, or a problem), traditional solutions turn up a tracing level (e.g., a tracing level OFF indicates gathering no information to trace out; a tracing level LOW indicates gathering a very high level view information; a tracing level HIGH indicates gathering large amounts of debug information, but may affect a system performance due to large amounts of I/O

operation and consume lots of storage space) for the detected event to gather information necessary to recreate the event.These traditional solutions have two disadvantages:The traditional solutions require the event (e.g., an error, a failure, or a problem) to be recreated, so that a sufficient amount of data is collected. However, recreating the event is not always possible in a customer environment.The traditional solutions gather all information traced from where an event occurred.However, the gathered information may be a substantial amount, causing a difficulty in analyzing the gathered information by a user (e.g., a product developer, an application developer). Furthermore, gathering information impacts a system performance by increasing resource usages (e.g., a memory usages increase, a CPU usage increase, a hard disk usage increase).

In traditional solutions, circular buffers are often used to store a limited amount of detailed trace data (e.g., brief summary of recently executed operation). When an event occurs, the detailed trace data in the circular buffer is transferred to a secondary storage device (e.g., a disk) for a later analysis. However, using the circular buffer has limitations:The circular buffer will often wrap around, loosing important information (e.g., initial configuration and setup information)The circular buffer may be full of lots of detailed data. However, most of the data in the circular buffer may be unrelated information to an event (e.g., an error, a defect, a failure).

Alexander, III et al (U.S. Pat. No. 6,604,210 B1) discloses a method and system for detecting and recovering from errors in trace data. The trace data records selected events for executing routines and the routines corresponding to the events are represented as one or more nodes in a tree structure. The events may be entries and exits to executing methods.

A non-patent literature entitled “Trace Cache Sampling Filter”, Michael Behar et al., Proceedings of the 14thInternational Conference on Parallel Architectures and Compilation Techniques (PACT'05), 2005 IEEE, IEEE Computer Society, discloses a technique for efficient usage of small trace caches. A trace cache can significantly increase the performance of wide out-of-order processors, but to be effective, the size of the trace cache should be large.

It would be desirable to provide a system and method for maintaining detailed trace information relevant to a current operation being processed in a program.

SUMMARY OF THE INVENTION

The present invention is a system and method for storing trace information relevant to an event encountered and removes trace information that is not necessary to analyzing the event.

For one aspect, a memory buffer is created when a subroutine in an executing program is called. Trace data generated during executing the subroutine is stored in the created memory buffer. If the subroutine calls a nested subroutine, a subordinate memory buffer is created. Trace data generated during executing the nested subroutine is stored in the subordinate memory buffer. When a subroutine completes its execution (e.g. returns a value at the end of execution), a corresponding memory buffer (i.e. the memory buffer that is created when the subroutine is called) is deleted. When an event occurs during an execution, all contents in currently existing memory buffers are transferred to a secondary storage device (e.g. a disk).

Thus, there is provided a system for storing trace information of a program to debug an error in the program comprising:

a PC (Program Counter) register for traversing program codes in the program in an executable order;

a main memory buffer for storing one or more of: trace data of the program, an entry data of a subroutine, and an exit data of the subroutine;

a first-level memory buffer, being created when the subroutine is called for execution, for storing trace data related to the subroutine execution;

means for linking the main memory buffer and the first-level memory buffer, the first-level memory buffer being a subordinate of the main memory buffer; and

means for deleting the first-level memory buffer and any stored trace data when the subroutine completes execution.

Thus, there is provided a method for storing trace information of a program to debug an error in the program comprising:

traversing program codes in the program in an executable order;

creating a main memory buffer for storing one or more of: trace data of the program, an entry data of a subroutine, and an exit data of the subroutine;

upon calling the subroutine for execution, creating a first-level memory buffer for storing trace data related to the executing subroutine;

linking the main memory buffer and the first-level memory buffer, the first-level memory buffer being a subordinate of the main memory buffer; and

deleting the first-level memory buffer and any stored trace data when the subroutine completes execution.

In one embodiment, the present invention removes unnecessary trace information in non-linear format by storing trace information in a tree of memory buffers (e.g., in a cache memory or a main memory). A memory buffer is discarded, when the trace information, which the memory buffer stores, becomes sufficiently unimportant (e.g., if a subroutine completes its execution without an event, a corresponding memory buffer, (i.e., a memory buffer which is created when the subroutine is called), is discarded).

The present invention has advantages over traditional solutions:Data in existing memory buffers are always relevant to a current operation being executed. When an event occurs and transferring data to a secondary storage device (e.g., a disk) is requested, the data transferred to the secondary storage device will be specific to the event.Only a relatively small amount of trace information (e.g., trace data, a subroutine entry data, a subroutine exit data) is stored in a main memory device. Therefore, transferring trace information to the secondary storage device can be quick. Less main memory spaces are required to store trace information.The trace information stored in memory buffers is closely related to a current stack trace.The data transferred to the secondary storage device is pre-pruned (e.g., when a subroutine completes its execution, a corresponding memory buffer is discarded).A high level of tracing is maintained (e.g., by recording a subroutine entry data and exit data in a superior memory buffer).At the time an error occurs, trace information in currently existing memory buffers provide enough information to debug an error or a failure without trying to recreate the error or the failure and without gathering trace information at a higher tracing level (e.g., tracing level HIGH).

DETAILED DESCRIPTION

For purpose of description, and in a non-limiting way, a program, which comprises subroutines, as referenced to herein includes and exhibits at least the following characteristics:Record an entry data (e.g., When a first subroutine is called by a second subroutine, the second subroutine lists or records parameters passed to the first subroutine);Perform some operations (e.g., mathematical operations);Record trace data about what it is doing (e.g., brief summary of recently executed operation);Call other subroutines;Record an exit data (e.g., When a subroutine returns a value, the subroutine lists or records the return value);Return some information to the subroutine that called it (e.g., a first subroutine, which is called by a second subroutine, returns a value to the second subroutine at the end of execution).

FIGS. 1-7depicts an exemplary embodiment of the present invention. Lines100-165in theFIGS. 1-7illustrates exemplary program codes in a program called Sub A (i.e., Subroutine A). AtFIG. 1, when the Sub A is called or initiated by executing line100, a memory buffer A (200) is created. In one embodiment, a memory buffer (e.g., a memory buffer A (200)) is directly created by a tracing application (e.g., java.util.logging (also known as JSR 47)). An executing program informs the tracing application which code in the program is executed (e.g., Sub A is called or Sub is executed). Then, the tracing application creates a memory buffer for the called or executed subroutine. A size of a memory buffer (e.g., a memory buffer A (200)) is an implementation choice. In one embodiment, a size of a memory buffer can be flexible or adjustable to grow, as more space is needed. In one embodiment, the Sub A is a main function, and the memory buffer A is a main memory buffer. In one embodiment, a program counter (PC) register is utilized to point to a currently executing code (e.g., a starting address of Sub A).

FIG. 2illustrates the program execution from line105to line115. When line105is executed, a trace data0001(205) of the Sub A is recorded to the memory buffer A (200). At line110, when Sub A1(i.e., a subroutine A1) is called, Sub A1's entry data (210) (e.g., parameters passed to Sub A1) is recorded to the memory buffer A (200) and a memory buffer A1(300) (e.g., a first-level memory buffer) is created. In one embodiment, parameters passed to a called subroutine (e.g., Sub A1) are dependent on programming languages. However, generally, almost anything that is touched by a currently executing program can be passed to a subroutine as a parameter. The memory buffer A1(300) is linked to the memory buffer A (200) as a subordinate memory buffer. When line115is executed, a trace data0002(305) of the Sub A1is recorded to the memory buffer A1(300).

FIG. 3illustrates the program execution at line120. When line120is executed, Sub A1completes its execution by returning a value (e.g., 1). After line120is executed, the return value (215) of Sub A1is recorded to the memory buffer A (300). If Sub A1is completed without an event (e.g., an error, a failure, a warning) by executing lines110-120successfully, the memory buffer A1(300) is discarded after line120is executed. If an event occurs during the program execution, all contents of currently existing memory buffers are transferred to a secondary storage device (e.g., a disk). For example, if an error occurs at line115, the trace data of Sub A (205), Sub A1entry data (210), and the trace data of Sub A1(305) (i.e., as shown inFIG. 2) are transferred to a secondary storage device.

FIG. 4illustrates the program execution from line125to line140. When line125is executed, a trace data0003(220) of Sub A is recorded to the memory buffer A (200). When Sub A2(i.e., a Subroutine A2) is called at line130, a Sub A2entry data (225) is recorded to the memory buffer A (200) and a memory buffer A2(400) is created. The memory buffer A2(400) is linked to the memory buffer A (200) as a subordinate memory buffer. When Sub A2i(i.e., a Subroutine A2i) is called at line135, a Sub A2ientry data (405) is recorded to the memory buffer A2(400) and a memory buffer A2i(500) is created. The memory buffer A2i(500) is linked to the memory buffer A2(400) as a subordinate memory buffer. When line140is executed, a trace data0004(505) of Sub A2iis recorded in the memory buffer A2i(500). If an error occurs during an execution, all contents of currently existing memory buffers are transferred to a secondary storage device. For example, when an error occurs at line140, trace data of Sub A (205,220), Sub A1entry data (210), Sub A1exit data (215), Sub A2entry data (225), Sub A2ientry data (405), trace data of Sub A2i(505) (i.e., as shown inFIG. 4) are transferred to a secondary storage device.

FIG. 5illustrates the program execution from line145to line150. After line145is executed, the memory buffer A2i(500) is discarded and a Sub A2iexit data (410) is written to the memory buffer A2(400). When line150is executed, a trace data0005(415) of Sub A2is recorded to the memory buffer A2(400).

FIG. 6illustrates the program execution from line155to line160. After line155is executed, the memory buffer A2(400) is discarded and a Sub A2exit data (230) is recorded to the memory buffer A (200). When line160is executed, a trace data0006(235) of Sub A is recorded to the memory buffer A (200).

FIG. 7illustrates the program execution at line165. After line165is executed, the memory buffer A (200) is discarded. If Sub A was called by a superior subroutine, Sub A exit data may be recorded to a superior memory buffer. Otherwise, the program finishes execution.

FIG. 8is a flow chart depicting a methodology according to one embodiment of the present invention. At step10, when a subroutine B entry event (i.e., Subroutine B is called by a program to be executed) occurs, a memory buffer B is created and starts to store trace data of the subroutine B. At step12, it is checked whether a nested subroutine is called in the subroutine B. If a nested subroutine B′ is called, at step14, a memory buffer B′ is created. At step16, the memory buffer B′ is linked to the memory buffer B as a child memory buffer (i.e., a subordinate memory buffer). How to link memory buffers is an implementation choice. In exemplary embodiment, memory buffers are linked via a linked list and a data structure having references or pointers. Therefore, there is a memory pointer at the end of a memory buffer. The memory pointer stores an address of another memory buffer. A memory pointer can be obtained by a memory allocation request to an operating system. Returning to theFIG. 8, trace data of the nested subroutine B′ is stored in the memory buffer B′ at step18. At step20, when the nested subroutine B′ completes its execution (e.g., returns a value), an exit data of the nested subroutine B′ is recorded to the memory buffer B. At step22, the memory buffer B′ is discarded. At step24, more trace data of the subroutine B is recorded to the memory buffer B. If a nested subroutine is not called at step12, it is checked whether the subroutine B completes execution (e.g., executes a return command) at step26. If the subroutine B continues its execution, at step28, trace data of the subroutine B is recorded to the memory buffer B and more trace data (T) of the subroutine B is recorded to the memory buffer B at step24. If the subroutine B completes its execution (e.g., executes a return command), at step30, an exit data of the subroutine B is recorded to a superior memory buffer (if the subroutine B is called by a superior subroutine). The memory buffer B is discarded.

In one embodiment, trace information (e.g., trace data, subroutine exit data, subroutine exit data) is removed from memory buffer(s) in a non-linear format (e.g., pruning out trace information in a memory buffer when the trace information becomes unnecessary (e.g., when a subroutine completes its execution)). In this embodiment, the trace information is stored in a tree of memory buffers:Each node in the tree is a memory buffer. (Adding and deleting a memory buffer is exactly like adding and deleting a node in a tree.)A node can have only one child node at a certain moment.Adding and deleting a node (i.e., adding and deleting a memory buffer) occur only at the deepest level of the tree (e.g., a leaf node).Regular trace information (e.g., trace data) is stored on a current node (i.e., a newly created memory buffer).A subroutine call makes a child node under the current node. Then, the child node is set to the current node. Trace data of the subroutine is stored in the current node.When a subroutine completes its execution (e.g., return a value), the current node traverse to its parent node and set the parent node to the current node. The child node of the current node is deleted. An exit data of the completed subroutine is stored in the current node.In another embodiment, especially executing a parallel application written by a parallel programming language (e.g., F#, parallel C++, Ocamlp31, occam, Charm++, Unified Parallel C), a node (i.e., a memory buffer) in the tree can have more than one child node (i.e., more than one subordinate memory buffers) at a certain time. In addition, if a program is executed an out-of-order (i.e., not sequentially), adding and deleting a node can occur at any level of the tree (e.g., a superior memory buffer can be deleted before its subordinate memory buffer is discarded). In an alternative embodiment, memory buffers are connected each other in the form of a linked list or a stack.

In one embodiment, a Program Counter (PC) register is implemented to traverse program codes in a program in an executable order (e.g., sequentially or concurrently). In this embodiment, the Program Counter is always in a current node (i.e., a newly created memory buffer) and is in the deepest level node (i.e., the most subordinate memory buffer) of the tree. In a multi-threaded environment, each thread has its own Program Counter. In another embodiment, especially executing a parallel application, a plurality of Program Counters points to concurrently executing program codes and exists in corresponding memory buffers. For example, atFIG. 4, if line125and line140are concurrently executed, a Program Counter is in the memory buffer A (200) and another Program Counter is in the memory buffer A2i(500).

In one embodiment, trace information (e.g., trace data, subroutine exit data, subroutine exit data) is closely related to how a stack trace may look like at the moment of execution. For example, atFIG. 4, when lines100-140are executed, memory buffers (i.e., a memory buffer A, a memory buffer A2, a memory buffer A2i) stores trace data of Sub A (205,220), Sub A1entry data (210), Sub A1exit data (215), Sub A2entry data (225), Sub A2ientry data (405), and trace data of Sub A2i(505) as shown atFIG. 4. A stack trace at the point of executing line140may look like:data of Sub A2ientry data of Sub A2ientry data of Sub A2data of Sub Adata of Sub A
Therefore, when an error occurs, memory buffers stores direct nested trace information (e.g., in memory buffers) for a subroutine where the error occurs. The direct nested trace information looks like a stack trace as shown above.

In one embodiment, trace information (e.g., trace data, a subroutine entry data, a subroutine exit data) is stored in a main memory device (e.g., DRAM, SRAM, Flash Memory) and generates a file only on an event (e.g., occurred in a subroutine). The file is transferred to a secondary storage device for a future analysis. This embodiment is called “first failure data capture”.

Although the preferred embodiments of the present invention have been described in detail, it should be understood that various changes and substitutions can be made therein without departing from spirit and scope of the inventions as defined by the appended claims. Variations described for the present invention can be realized in any combination desirable for each particular application. Thus particular limitations, and/or embodiment enhancements described herein, which may have particular advantages to a particular application need not be used for all applications. Also, not all limitations need be implemented in methods, systems and/or apparatus including one or more concepts of the present invention.

The present invention can be realized in hardware, software, or a combination of hardware and software. A typical combination of hardware and software could be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein. The present invention can also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which—when loaded in a computer system—is able to carry out these methods.