Examining Execution History Using the History Tool

The history tool can be used with any program. It produces a modified version of your original program which can record a history of the source lines it executes. When you subsequently run the modified program under debugger control, you can examine the execution history of the program at any point during its execution, even if the program has put itself into a state in which it is executing random memory and has lost access to the real stack. This means you can find out what the program was doing just before it got lost.

Because the recording is done by your program as it runs, and done without any I/O, the time overhead to do the recording is far less than it would be if the recording were done by a debugger or by using printf statements.

Because the records are put into a ring buffer, the memory overhead does not grow with time for single-threaded applications. For multi-threaded applications, the tool uses one ring buffer per thread; the memory overhead will depend on the number of threads.

The tool is currently a unsupported prototype and may change. It is available in source form; the buffer size and other factors can be changed by editing the source.

This note discusses the following topics:

• Obtaining the history tool prototype
• Using the history tool
• History tool commands
• History tool options
• Restrictions
• Example

Obtaining the History Tool Prototype

1. Retrieve the prototype via anonymous FTP at:

   	system:     ftp.compaq.com
           directory:  /pub/products/software/ntenterprise
           file:       ldb-ring.tar
   

   Or via a web browser at the URL:

   ftp://ftp.compaq.com/pub/products/software/ntenterprise/ldb-ring.tar

2. Copy the tar file into a temporary directory. For example:

   % mkdir ~/ring
   % mv ldb-ring.tar ~/ring
   % cd ~/ring
   

3. Extract the prototype files:

   % tar -xvf ldb-ring.tar
   

4. View the enclosed README and ring.faq files for the most-recent information on the prototype.

How to Use the History Tool

1. Obtain and set up the history tool prototype.
2. From the directory in which the prototype files reside, run the ringify-app script on your application (see the Options section). The command will look something like the following:

      % ringify-app -t -o executable-file.ring executable-file
   

3. Load the application into the debugger using the command output by the ringify-app script. This command represents the simplest method for invoking the debugger. It may be necessary for you to use a modified version of this command for your particular application. The command will look something like the following:

      % env _RLD_LIST=libringstubs.so.executable-file.atom:DEFAULT \
             ladebug -c ring.alias.cmd executable-file.ring
   

4. Debug your application as normal.

5. At the point where you wish to examine the execution history for your application, issue one or more of the prototype commands.

History Tool Commands

Command	Description
hhcur	Displays the current execution history record.
hhdump hhdumpn count	Displays the last count entries in the execution history for the current thread, most-recent first. If no argument is specified, the number of entries defaults to 12.
hhfile	Displays the name of the source file containing the source line of the current execution history record.
hhfirst	Sets the current record to be the oldest execution history record for the current thread.
hhlast	Sets the current record to be the newest execution history record for the current thread.
hhlist	Displays the source line of the current execution history record.
hhnext	Advances the current record to the next execution history record forward in time at the same or higher call depth.
hhprev	Retreats the current record to the previous execution history record backward in time at the same or higher call depth.
hhproc	Displays the name of the procedure containing the source line of the current execution history record.
hhrecent hhrecentn count	Displays a window of count execution history entries, centered around the current execution history record. If no argument is specified, the window size defaults to 12.
hhshowthread hhshowthreadn thread_identifier	Displays the list of threads for which at least one execution history record exists. Specifiying the thread_identifier argument to hhshowthreadn results in only that thread being displayed.
hhstep	Advances the current record to the next execution history record forward in time.
hhstepprev hhprevstep	Retreats the current record to the previous execution history record backward in time.
hhupnext	Advances the current record to the next execution history record forward in time for the caller of the current procedure.
hhupprev	Retreats the current record to the previous execution history record backward in time for the caller of the current procedure.
hhw	Displays a window of 10 source lines centered around the source line of the current execution history record.
hhW HHW	Displays a window of 20 source lines centered around the source line of the current execution history record.
hhwhere	Displays the historical procedure call sequence for the current thread.

Options

Option	Description
-a	Instruments all statically loaded shared libraries in the shared executable. By default, no shared libraries are instrumented except the "stub" function shared library produced by this script and, in the case of threaded applications, libpthread.so. The instrumented shared libraries are written to the same directory as the instrumented user executable.
-d	Produces debugging output for the script. This option allows the end user to view the compile and atom commands issued by the script.
-f flag	Specifies additional flags to be passed to the atom tool during instrumentation. See man atom for a list of the valid options for the atom tool.
-e shared-library	Excludes the named shared library from instrumentation. This option can be used more than once to specify several shared libraries. In the case of threaded applications, this option has no effect on libpthread.so.
-h	Outputs the usage list for this script. As an alternative, you may also use ringify-app help.
-i shared-library	Includes the named shared library for instrumentation. This option can be used more than once to specify several shared libraries. The instrumented shared libraries are written to the same directory as the instrumented user executable.
-o output-file	Names the instrumented executable file output-file. The default name of the output file is generated by appending .ring to the name of the user's executable file and is placed in the same directory.
-t	Specifies that thread-safe support is required. This option should be used when instrumenting threaded applications.
-x	Excludes use of the regular expression lists in the ring.inst.patterns.h file. Empty lists, as defined in the ring.inst.patterns.template.h file, are used instead.

NOTE: When using options that produce instrumented versions of shared libraries (-a, -i ,-t), you may need to modify your LD_LIBRARY_PATH setting before debugging.

History Tool Example

   Welcome to the Ladebug Debugger Version 4.0-61 (built May  2 2000)
        ------------------
        object file name: c_ringProtoTest1.ring
        Reading symbolic information ...done
        (ladebug) run
        user: [Joe User]   pswd: [jellybeans]   date: [05-01-2000 at 11:15:22]
        Thread received signal ILL
        stopped at [ 0x120030004]
        (ladebug) where
        >0  0x120030004 in c_ringProtoTest1.ring
        #1  0x12002fffc in c_ringProtoTest1.ring
        (ladebug) hhdump
        Last 12 lines executed were (most recent first):

                        <return>
                        main: 107
                            <return>
                            outputLoginData: 99
                            outputLoginData: 98
                        <call of outputLoginData>
                        main: 106
                            <return>
                            captureLoginEntry: 94
                            captureLoginEntry: 93
                                <return>
                                terminateAndFillBuffer: 69
                                terminateAndFillBuffer: 68
                                terminateAndFillBuffer: 66
                                terminateAndFillBuffer: 65
                                terminateAndFillBuffer: 60
                                terminateAndFillBuffer: 59
        (ladebug) hhstepprev
          main: 107
            107 }
        (ladebug) hhstepprev
          <return>
             99 }
        (ladebug) hhstepprev
          outputLoginData: 99
             99 }
        (ladebug) hhstepprev
          outputLoginData: 98
             98     printf ("%s\n", outBuffer);
        (ladebug) hhstepprev
          <call of outputLoginData>
            106     outputLoginData   (mainBuffer);
        (ladebug) hhstepprev
          main: 106
            106     outputLoginData   (mainBuffer);
        (ladebug) hhstepprev
          <return>
             94 }
        (ladebug) hhstepprev
          captureLoginEntry: 94
             94 }
        (ladebug) hhstepprev
          captureLoginEntry: 93
             93     strcpy (outBuffer, workBuffer);
        (ladebug) hhw
             88                          "05-01-2000 at 11:15:22",
             89                           nextIdxToWrite);
             90
             91     terminateAndFillBuffer(workBuffer, nextIdxToWrite, 138);
             92
             93     strcpy (outBuffer, workBuffer);
             94 }
             95
             96 void outputLoginData (char * outBuffer)
             97 {
        (ladebug) stop at 93
        [#1: stop at "c_ringProtoTest1.c":93 ]
        (ladebug) hhstepprev
          <return>
             69 }
        (ladebug) hhstepprev
          terminateAndFillBuffer: 69
             69 }
        (ladebug) hhstepprev
          terminateAndFillBuffer: 68
             68     memset(&buffer[bufIdx], 0, bufLen - bufIdx);
        (ladebug) hhw
             63     }
             64
             65     buffer[bufIdx] = '\0';
             66     bufIdx = bufIdx + 1;
             67
             68     memset(&buffer[bufIdx], 0, bufLen - bufIdx);
             69 }
             70
             71 void captureLoginEntry (char * outBuffer)
             72 {
        (ladebug) stop at 68
        [#2: stop at "c_ringProtoTest1.c":68 ]
         [#2: stop at "c_ringProtoTest1.c":68 ]
        (ladebug) rerun
        Process has exited
        [2] stopped at [void terminateAndFillBuffer(char*, int, int):68 0x120035680]
             68     memset(&buffer[bufIdx], 0, bufLen - bufIdx);

Restrictions

• This prototype requires Atom version 2.59 or higher (atom).
• This prototype requires the C compiler (cc).
• This prototype requires the Ladebug debugger Version 4.0-54 or higher (ladebug).
• The script ringify-app must be run from its home directory.