From: marc@mercutio.ultra.com (Marc Kwiatkowski {Host Software-AIX})
Subject: Re: Anyone debugged AIX before?
Organization: Ultra Network Technologies
Date: Fri, 21 Aug 92 18:20:33 GMT

In article <1992Aug21.144448.64350@ans.net> sangster@news.ans.net (Paul Sangster) writes:


>   Greetings,
>
>   I am trying to do alittle kernel hacking on the TCP/IP
>   component of AIX.  My question is of someone who had done
>   this ... how did you getting debugging info printed out
>   on the console?  I see lots of printfs floating around in
>   the code, but they don't seem to do anything when compiled
>   into the netinet.  Is their a special kernel library
>   with its own printf (or does it just use libc)?
>
>   Please direct follow ups via e-mail.
>
>   Thanx,
>

Below is a copy of my cheat sheet for kernel debugging.  Everything
you need to know is in InfoExplorer databases, but it's hard to find
it.  If someone wants to stick this into an FAQ, feel free.

================================================================================

Installing the AIXv3.2 kernel debugger.

To configure your AIXv3.2 kernel for debugging, run the following
command.

 $ bosboot -a -D -d <boot-device>

Where boot device is the hard-disk you boot off of, typically, hdisk0.

Configure tty0 for 9600 baud, 8 bit, no-parity, 1 stop-bit.  Connect serial port
S1 (ie tty0)  from your machine to a dumb terminal capable of supporting 
the above line mode.  If you have another Unix workstation you can connect to 
its serial port with tip or cu.

Reboot your machine.  To enter the debugger, strike the following
keys on the console keyboard simultaneously.

  Cntl-Alt-<Numeric-key-pad 4>
 
That is, the number 4 on the numeric keypad as opposed to the number
4 above the E and R key.  You should see a register dump displayed
on your dumb terminal.  Entering "go" or "quit" will allow the kernel
to resume execution.  See the InfoExplorer on-line documentation for
more on the kernel debugger commands.

Kernel Printfs

Kernel printfs are only displayed to the debug terminal.  If you do
not have the kernel debugger configured or you don't have your terminal
hooked up you will not see kernel printfs, period.

Whe your kernel is configured with debugging on, all kernel printfs
will go to your kernel-debug dumb terminal.  Note that there is
no way to get around the 9600 baud rate, so too many printfs and
printfs in time-critical areas will significantly alter the behaviour
of your kernel.

Panics

When your kernel is configured with debugging on, any panic will trap
into the debugger.  When this happens look to see if any error messages 
have been displayed to the dumb terminal.

Stack traces.

When your machine has panicked, you may want to trace the thread of
exuction by examining the stack.  Because the AIXv3 kernel debugger
doesn't maintain a full-blown symbol table, traces are a little painful
but can be done.  The current stack pointer is always stored in register
1, and so one can display the current stack by entering the following
in the debugger dumb terminal.

> d r1
001F1CCC   001F1D1C 00000010 013DB7C0 20202072   |.........=..   r|
           ^^^^^^^^          ^^^^^^^^
           Previous          Calling
           Stack             Function
           Pointer           Return
                             Address

This shows the first 16 bytes of the stack.  The first four bytes
are the address of the previous stack-pointer, and the third
four bytes are the return address of the calling function.  Thus
by first displaying r1, and then dumping the first four bytes of
each successive output one can get a quick complete dump.

For example:
> d r1
001F1CCC   001F1D1C 00000010 013DB7C0 20202072   |.........=..   r|
> d r1
001F1CCC   001F1D1C 00000010 013DB7C0 20202072   |.........=..   r|
> d 001F1D1C
001F1D1C   001F1D8C 01484238 013B1ECC 009AB300   |.....HB8.;......|
> d 001F1D8C
001F1D8C   001F1DCC 0000608C 013B172C 70000000   |......`..;.,p...|
> d 01F1DCC
001F1DCC   001F1EAC 007FFFFF 013B1240 00000002   |.........;.@....|
> d 01F1EAC
001F1EAC   001F1F2C 24000024 0001C21C 001F2F64   |...,$..$....../d|
> d 1F1F2C
001F1F2C   00000000 48002084 00004498 00000000   |....H. ...D.....|

Converting addresses to symbol names

The AIXv3 kernel debugger provides minimal symbol table support.
The map command can be used to locate the address of global
symbols (ie those that are exported by their module) and nothing
else.  

To figure out what each function is, a quick and dirty trick is
to dump the return address in the stack or the current IAR (pc).  
The name of each function is encoded
as an ASCII string at the end of each function, so if you dump
enough you will eventually see it.  Note:  this doesn't always
work.  Low-level and/or hardware panics may leave current registers
in a meaningless state, thus attempts to dump stacks and return 
addresses are useless.


> d 013DB7C0 200
.
.
.
.
013DB890   81810098 30210090 7D8803A6 BB61FFEC   |....0!..}....a..|
013DB8A0   4E800020 00000000 00002041 80050300   |N.. ...... A....|
013DB8B0   00000000 00000218 000D656E 745F786D   |..........ent_xm|
013DB8C0   69745F64 6F6E6500 7C0802A6 BF81FFF0   |it_done.|.......|
.
.
. 

Note the ent_xmt_done in the ASCII output column.  That is the name
of the last calling function.
================================================================================
-- 
Marc P. Kwiatkowski                  /|/|            Ultra Network Technologies
internet: marc@ultra.com     \______/ | | W.A.S.T.E. 101 Daggett Drive
uucp: ...!ames!ultra!marc    / (__) \ | |            San Jose, CA 95134 USA
voice: 408 922 0100 x249             \|\|