Abstract:
A tracing facility which is useful in a multiprocessing environment on a single processor is disclosed. Tracing statements are encoded within each process, each statement having arguments to indicate its location in the code and a message which is logged when the sybsystem and function corresponding to that tracing statement are selected for debugging. Each process includes a trace library that is called by each trace statement, and the trace library includes a local sync word and a local trace table having a list of the subsystems and functions currently selected for tracing. Each process has access to a global sync word and a global trace table having a list of the processes, subsystems and functions selected for tracing. A change in the trace operation is made by changing a parameter in the global trace table and changing the value of the global sync word to indicate that a change has been made in the global table. Each call to the trace library causes a comparison to be made between the local sync word and the global sync word to determine whether the local trace table should be updated from parameters in the global trace table. Each trace statement also has an argument that indicates its level of importance in the debugging operation, and both tables are caused to have a parameter which indicates the level at which trace statements are currently required to be logged.

Description:
BACKGROUND OF THE INVENTION 
     The development of software in computer based systems has become very time-consuming and, therefore, an expensive part of the initial cost of these systems. One difficulty is that many of the transactions between various pieces of software are not fully tested at the time the software is written. Accordingly, running the software causes logic errors to occur that are unexpected and these errors frequently produce undesirable results. These unexpected logic errors and undesirable results are frequently said to be due to bugs in the software. 
     One technique for locating the bugs in software is to intersperse coded statements within the software that provide the developer with an indication as to what values key parameters have during the execution of the software at the various points in the software at which the coded statements are inserted. The parameter values provided in this way will frequently provide the developer with an indication as to what is wrong with the present state of the software. These debugging statements frequently referred to as tracing statements take time to execute, and if a large number of them are interspersed within the code, the execution time for a particular piece of software can be increased significantly. It is therefore desirable to have only those tracing statements which are essential to a particular debugging operation to be executed. Inasmuch as there is usually no foresight as to which part of the software code or which function within the code will experience difficulties in execution, the tracing statements are usually interspersed at all points in a process that will be expected to give useful output parameters. This is found to be far more efficient than subsequently inserting statements only in the portions of code that are presently experiencing difficulties. The latter procedure would require constant rewriting of the source code and recompilation of that code every time a new bug is uncovered. 
     With trace statements interspersed throughout the code, it is desirable to have some way of deactivating the trace statements that are not presently required. One technique of achieving this mode of operation is to cause each tracing statement to contain arguments which indicate the subsystem and function with which it is associated in the code and to have a table loaded at the beginning of the process execution which specifies the subsystems and functions within the code presently requiring tracing. The inclusion of a subsystem or function within the table will cause the trace statements associated with those subsystems and functions to produce an output of the parameters specified in the trace statement. One difficulty with this approach is that the table must be changed in order to change the subsystem or function under investigation and the process must then be rerun from its starting point. This approach is especially difficult in cases where the processes execute over long periods of time. For example, in a communication system where there are many interrelated processes that are executed and continue to live while the system is active, a change in the table would require that the entire system be deactivated while a change be made in the table of each process. The system would then have to be reinitialized by bringing up all the processes and thereby consuming a large amount of time solely for the purpose of debugging a new subsystem or function. 
     Still another approach to the selective activation of trace statements in a multiprocessing environment is to provide a global table rather than a local table within each process. This global table can be associated with an operating system which is present during the execution of every process in the system. One feature that may be used to construct a table in connection with the VAX11/780 computer from Digital Equipment Corporation is the &#34;logical name table&#34; associated with its VMS operating system. Each process during its execution may then check the subsystem name and function name within the logical name table in order to determine whether a particular trace statement should be executed. The difficulty with this approach is that accessing the logical name table in the operating system is far more time consuming than accessing a table within the process itself. In addition, the logical name table may frequently have a much larger number of entries than a local table within the process and for this reason requires even more time than access to a local table. 
     SUMMARY OF THE INVENTION 
     A highly efficient trace facility is provided in accordance with the present invention wherein a process in a multiprocessing environment has a trace library within which a local trace table provides a list of the subsystems and functions whose trace statements are to be activated during any execution of the process. A global trace table is also provided containing a list of the processes, subsystems and functions whose trace statements are to produce results during execution. The trace library also has a local sync word which can be compared to a global sync word in order to determine whether any changes had been made in the global table that have not yet been updated within the local table. Each call to the trace library of the process causes such a comparison to be made between the two sync words. If no difference exists the local table is used to determine whether that particular call to the trace library should result in the output of the parameters associated with that call. The parameters in the local table are therefore replaced by their values in the global table only when a difference is detected between the local sync word and the global sync word. As a result, the much longer period of time required for reading the global trace table is avoided except when changes are made in the global table. 
     It is a feature of the present invention that the global trace table and the local table in the trace library also include a level value. This level value permits different degrees of investigation into the operation of each process during development of the code. Each trace statement is assigned a value to indicate the degree of importance of that trace statement in the debugging of that process. Level 1 statements, for example, could be reserved for the trace statements that are essential in determining an overview of the operation of that process (for example, inputs to and outputs from the process). At the other extreme, a higher number level, level four for example, can be assigned to those trace statements which provide detailed information about the operation of the subroutines within the process. As a result, the trace facility can be caused to permeate the execution of the processes to varying degrees depending on the level assigned in the global trace table. This listing of proceses, subsystems, functions and levels within a global trace table provides an extremely flexible debugging facility that can dynamically interact with the processes being run to a degree which varies with time depending on the level of investigation that is necessary in order to uncover the bug that is encountered. 
    
    
     BRIEF DESCRIPTION OF THE DRAWING 
     The invention will be more readily understood when the following detailed description is read in conjunction with the drawings wherein: 
     FIG. 1 is a block diagram which illustrates the relationship between a process and the global sync word and trace table on a processor which utilizes the present invention; 
     FIG. 2 is a block diagram which illustrates how two processes on a single processor utilize the global sync word and trace table during their execution on a single processor which utilizes the present invention; and 
     FIGS. 3 through 6 provide a flowchart for a process which practices the present invention. 
    
    
     DETAILED DESCRIPTION 
     The present invention was implemented on a VAX11/780 computer runner processes that are useful in a communications environment. Many of the terms used in this specification are terms that are defined in the VAX hardware and software handbooks for the VAX11/750 and VAX11/780. See the VAX Hardware Handbook by Digital Equipment Corporation, copyright 1980 and the VAX Software Handbook by Digital Equipment Corporation, copyright 1981, both of which are incorporated by reference for the purpose of providing support for the many terms used in this specification. 
     As pointed out in the handbooks, a process is the basic entity scheduled by system software that provides the context which an image executes. A process consists of an address space and both hardware and software contexts. As pointed out hereinabove, this invention is most useful in an environment where several processes are executing on a single central processing unit and these processes are long lived. Process 101 in FIG. 1 is representative of each of the processes operating in the multiprocessing environment. As also shown in FIG. 1, process 101 will frequently have several subsystems shown illustratively as subsystem A and subsystem B in FIG. 1. The subsystems of process 101 will have access to a trace library 102 which is the software that provides the heart of the present invention. This access is gained to the trace library by inserting statements in the software of the subsystems that call the trace library for execution of a debugging display. One format that a typical statement might take when written in the C language as defined in the text &#34;The C Programming Language&#34; by B. W. Kernighan and D. M. Ritchie, Prentice-Hall, Inc., 1978, is as follows: 
     trace(subsys --  name, function --  name, level, format --  string, arg1, arg2, . . . ). 
     In this statement, the portion within the parenthesis represents arguments in a call to the trace routine. The subsys --  name argument represents the name of a subsystem within the process. This subsystem name used as the first argument in a trace call in the embodiment constructed is simply a two-character prefix that is assigned to the subsystem and is unique for each identical subsystem in the many processes run in the multiprocessing environment. The next argument function --  name is supplied by the caller and will permit the caller to uniquely identify some segment of his subsystem. There should, of course, be some consistent strategy used in designating this argument in order to avoid confusion either within the process or from process to process in the total environment. The third argument is a trace level which, in the embodiment constructed, was given any one of four levels, TRLEV --  ONE THROUGH TRLEV --  FOUR. The lowest trace level was used to identify entries to or from major routines of the subsystem whereas the higher level trace calls were used for information that was embedded in lower level routines. As will be apparent hereinafter, when a particular trace level is turned ON in the sense that it will produce a trace information output, all levels at or below the specified level are also caused to produce information output. That is, of course, providing the particular process, subsystem and function specified in the trace call are also turned on in the sense that an information output should be provided. 
     A typical use of a trace statement is illustrated in the following C language statement: 
     if (dept --  no&lt;0||dept --  no&gt;9999){trace (&#34;mm&#34;, &#34;label&#34;, TRLEV --  TWO, &#34;Bad department number=%d&#34;, dept --  no)} 
     In this particular statement, a check is made to determine whether the specified department number indicated by dept --  no is less than 0 or greater than 9999. If this condition is satisfied, a call is made to the trace library and this call specifies to the library that the call is being made from a function entitled &#34;label&#34; in a subsystem entitled &#34;mm&#34; with a trace level of 2. If this subsystem and function are ones that are presently specified as being on in the sense that they will produce an information output and further if the trace level is presently set to a level of 1 or 2, a department number of 10,000, for example, would produce an output either in a file or display with the information &#34;Bad department number=10,000&#34;. 
     As indicated in FIG. 1, the multiprocessing environment also includes a trace administration process 105 whose sole function is to update a global sync word 106 in that it increments its value and to modify and/or display a global trace table 107. Both the global sync word 106 and the global trace table 107 can be created during an initialization process by using facilities available in the VMS operating system of the VAX11/780 which facilities are clearly specified in the above-identified VAX hardware and software handbooks. In the embodiment constructed the global trace table 107 was created during an initialization through use of the &#34;logical name table&#34; provided by VMS. As shown in FIG. 1, this global trace table 107 is caused to include a process list, subsystem list, function list and level. A process, subsystem or function is included in the list only if the specified function, subsystem and process are intended to produce information outputs during a trace call. Hence only the items included in the list are considered to be ON. The level is, of course, any one of the levels specified hereinabove which will cause statements at various levels within the software to produce information outputs. 
     In the embodiment which was constructed each of the lists was permitted to include a single name &#34;all&#34; to indicate that all of the items encountered should produce information output statements. In addition, the specification of &#34;all&#34; was permitted to be followed by a list of items, each one of which is prefixed by an &#34;!&#34; to indicate items that should be excluded from the all inclusive &#34;all&#34; statement. Each of the lists were limited in the VAX11 780 to a total of 64 characters which is a VMS limit for the logical name table. 
     As indicated in FIG. 1, the parameters from the global trace table 107 are copied into a local trace table 108 within the trace library 102 on the first trace call execution. This local trace table 108 is accessed during all subsequent trace calls. If the trace parameters are modified in the global trace table 107 by the trace administration process 105, this process 105 also increments the global sync word 106. As will be apparent hereinafter in connection with the discussion of the flowchart shown in FIGS. 3-5, each call to the trace library causes a comparison to be made of the global sync word 106 to a local sync word 109. If the global sync word and local sync word differ, then the trace library is caused to get a new copy of the parameters in the global trace table 107. 
     If a call to the trace library 102 finds that all of its arguments are specified within the local trace table 108 either specifically or by inclusion by way of an &#34;all&#34; statement, and the trace level present within the table 108 is equal to or greater than the trace level specified in the call, the statement specified in the format string of the trace call will be coupled as an input to a trace data file 110 in FIG. 1. There is, of course, a trace data file for each of the processes executing on the single processor. By reading the trace data file associated with a given process it can be determined whether the expected parameters are being produced by the code of the process. As is well known to those skilled in the computing science art, this type of information is extremely useful in determining the location of bugs within the software. 
     The nature of the software used to provide the trace library function is illustrated by the flowchart shown in FIGS. 3-7. As indicated in step 301 of FIG. 3, entry to the trace library immediately causes a comparison to be made between the local sync word 109 and the global sync word 106. If these words are equal in value, it is known that the local trace table has parameters that correspond exactly to the parameters contained in the global trace table, and therefore the software may immediately jump to a point designated as A in FIG. 5 at which point the determination as to whether or not that particular trace call should produce an information output is begun. For the moment assume that these two sync words are not equal in value. This causes the software to first set them equal in value as indicated by step 302 in FIG. 3 and then the local trace table receives a copy of the global trace table as indicated by the step 303 in FIG. 3. A determination is then made as to whether the process name is in the process list as indicated by the decision step 304 in FIG. 3. If the process name is not in the process list of the table, the process flag is set to its false state, and a determination is then made as to whether the file-open flag is set as indicated by decision step 306 in FIG. 3. If the file-open flag 114 is not set thereby indicating that the trace data file 110 is not open, there is an immediate return from the trace library routine. If on th other hand it is set, the trace data file is closed as indicated by step 307 in FIG. 3, and the flag is set to its false condition before returning from the trace library. 
     If, on the other hand, the process name is in the process list of the local trace table, the process flag 111 is set to its true condition as indicated by step 308 in FIG. 3, and a determination is then made by step 401 in FIG. 4 as to whether the subsystem limit begins with an &#34;all&#34; statement. If the subsystem list does not begin with the statement &#34;all&#34;, the software immediately proceeds to step 403 in FIG. 4 to determine whether the function list begins with the statement &#34;all&#34;. If the subsystem list does begin with the statement &#34;all&#34;, the allsubsys flag 112 is first set to its true condition before a test is made on the function list. Similarly, if the function list begins with the statement &#34;all&#34;, the allfunc flag 113 is set to its true condition in step 404. The trace data file name is then created in step 405 with a name that uniquely identifies the process that will be providing information to the data file. The trace file itself is not created until the first trace record is generated. This prevents the proliferation of &#34;empty&#34; files when tracing is not enabled. 
     After the construction of the data file name in step 405 or if the local sync word equals the global sync word in step 301, the trace library software then provides a decision in step 501 of FIG. 5 to determine whether the process flag is set to its true state. If it is not, this indicates that the process is not currently undergoing debugging and therefore an immediate return should be made from the trace library. If on the other hand the process flag is set to its true condition as determined by decision step 501 in FIG. 5, a decision is then made in step 502 as to whether the trace level of the calling code is less than or equal to the trace level present in the local trace table 108. If the trace level of the calling code is greater than the trace level specified in the table, a return is executed from the trace library. This could correspond to a condition where only major parts of the code are undergoing debugging and the deeply embedded parts of the code are permitted to execute without producing output information in order to reduce overhead on the system. 
     If the trace level of the calling code is less than or equal to the trace level present in the trace table a decision is then made as to whether the allsubsys flag is true, as indicated by decision step 503 in FIG. 5. If it is set to its true condition, a determination is then made in step 504 as to whether the subsystem specified in the calling code is in the excluded list that follows an &#34;all&#34; statement. If it is so excluded, an immediate return is made from the trace library. If it is not, the code proceeds to decision statement 506 in FIG. 5. If the allsubsys flag is not set to it true state, a determination is made as to whether the subsystem is specifically included in the subsystem list of the local trace table in decision step 505. If it is not, an immediate return is made from the trace library. If it is so included, software proceeds to decision step 506. This step determines whether the allfunc flag is set to its true condition. If it is, a determination is made as indicated by decision step 508 as to whether the function is specifically excluded. If it is excluded, there is an immediate return. If it is not, the code proceeds to point C at which point it is ready to provide information output relating to this trace call. If the allfunc flag is not set to its true condition, a determination is first made as indicated by decision step 507 as to whether or not the function is specifically included in the function list. If it is not, there is an immediate return from the trace library. If it is specifically included, the code also proceeds to point C at which point information output is to be generated. 
     At point C in the software, the code of the trace library causes a string to be formatted that includes the time on the system, the subsystem and function that are presently causing an output to be executed to the trace data file, and the trace level that corresponds to the calling trace code as indicated by step 601 in FIG. 6. The message text present in the calling trace statement is then formatted as indicated by step 602 in FIG. 6. This formatting of the message text usually includes the replacement of token words that are present in the calling code with definitions that correspond to the particular situation encountered by this code. For example, in the above illustrated trace statement relating to a particular department number, the specific department number that was encountered by the code is included within the message text. 
     The code of the trace library then makes a determination as indicated by decision step 603 in FIG. 6 as to whether or not a file is open to receive the trace data. If it is, the trace message is immediately coupled to that file by way of a code that is represented by step 606. If the file is not open, however, a file is first opened as indicated by step 604 and the file-open flag 114 is set to its true condition as indicated by step 605 prior to outputting a trace message. After the message is delivered to the data file there is a return from the trace library as indicated in FIG. 6. 
     At this point the properties of the trace library should be evident and the dynamics of its use in a multiprocessing environment can be illustrated by the diagram presented in FIG. 2. As indicated in FIG. 2, several processes in a multiprocessing environment, only two of which are shown as process 210 and 220, can be simultaneously providing outputs from their trace library to trace data files that are uniquely associated with each of the processes. The overall action of the debugging process for the individual processor, however, is controlled by the global trace table which is present in the processor at a point in its memory that is available to all operating processes. As indicated hereinabove, the present embodiment includes a global trace table within the part of the memory that is allocated to the VMS operating system. The global sync word is also present in a portion of memory that is available to each of the processes. All of the processes in this multiprocessing environment can continue to execute without re-initialization even though they are taken in and out of the debugging process. The parameters of the global trace table are simply changed in order to exclude or include specific processes or to include or exclude subsystems and functions within those processes. Finally, the intensity of the debugging and therefore the time consumed on the processor by the debugging trace facility can be changed by modifying the trace level within the global trace table. The trace facility can be completely deactivated by excluding all active processes from the process list. In summary, the present invention provides a trace facility that can dynamically interact with continuously executing processes in a multiprocessing environment by permeating the code of these processes to varying degrees depending on the parameters specified in the global trace table. 
     It should be readily apparent to those skilled in the art that numerous changes can be made in the present embodiment without departing from the spirit and scope of the present invention. For example, the information provided as an output from the trace library may also be delivered to a terminal in addition to, or in place of, the trace data file. The global and local trace tables may also include other lists that permit further specification of the kind of code that is to cause an output from the trace facility. Obviously, additional levels may also be included in order to change the degrees to which the tracing facility is permitted to permeate the processes operating in the multiprocessing environment.