# Presentation: 827815

## Tool on IBM SP

- Majdi Baddourah
- May 23, 2002
- NERSC

## Objective

- ZEROFAULT (Memory Tool)
- HPMCOUNT (Performance)
- HPMVIZ (Performance)
- VAMPIR (Performance)
- TOTALVIEW (Debugging)

## ZEROFAULT

- Works only on a single processor.
- Detects any memory problem in a program
- Memory leak
- Attempts to free unallocated memory
- Write/Read beyond array size.
- Works with Fortran and C/C++ code.

## How to Use ZEROFAULT

- module load ZeroFault
- Compile your code with –g option
- Xlf –g main.f –o a.out
- Run the code with the zf option:
- zf –u ./a.out
- The run will produce an X window

## ZEROFAULT Window

## Memory Leaks

## ZEROFAULT Other Commands

- zf –d none –u ./a.out
- This will produce a.out.1.zfo
- To use X-window to see the output file use.
- zf_ui a.out.1.zfo
- To generate output in a file
- Zf_prt node.1.zfo

## HPMCOUNT

- Easy to use
- Works with any parallel or serial job
- Little overhead
- Produce performance results:
- Flop Rate
- TLB misses
- MIPS

## How to Use HPMCOUNT

- Compile code normal
- At run time:
- poe hpmcount ./a.out –nodes 2 –procs 32
- This will produce output to the screen
- Each processor will produce an output

## HPMCOUT Output

-   PM_TLB_MISS (TLB misses)                   	:         4598441
-   PM_ST_CMPL (Stores completed)              	:        74253113
-   PM_LD_CMPL (Loads completed)               	:       193501740
-   PM_FPU0_CMPL (FPU 0 instructions)          	:        18000019
-   PM_FPU1_CMPL (FPU 1 instructions)          	:               0
-   PM_EXEC_FMA (FMAs executed)                	:               0
-   Utilization rate                           		:          86.691 %
-   Avg number of loads per TLB miss           	:          42.080
-   Load and store operations                  		:         267.755 M
-   Instructions per load/store                		:           2.487
-   MIPS                                       		:         370.220
-   Instructions per cycle                     		:           1.139
-   HW Float points instructions per Cycle     	:           0.031
-   Floating point instructions + FMAs         	:          18.000 M
-   Float point instructions + FMA rate        	:          10.006 Mflip/s
-   FMA percentage                             		:           0.000 %
-   Computation intensity                      		:           0.067

## HPMCOUNT Output

-   PM_TLB_MISS (TLB misses)                   	:          377882
-   PM_ST_CMPL (Stores completed)              	:        25994728
-   PM_LD_CMPL (Loads completed)               	:      1710293853
-   PM_FPU0_CMPL (FPU 0 instructions)          	:      1695086721
-   PM_FPU1_CMPL (FPU 1 instructions)          	:      1682219124
-   PM_EXEC_FMA (FMAs executed)                	:      3375005980
-   Utilization rate                           		:          96.110 %
-   Avg number of loads per TLB miss           	:        4526.000
-   Load and store operations                  		:        1736.289 M
-   Instructions per load/store                		:           3.083
-   MIPS                                       		:         955.341
-   Instructions per cycle                     		:           2.650
-   HW Float points instructions per Cycle     	:           1.672
-   Floating point instructions + FMAs         	:        6752.312 M
-   Float point instructions + FMA rate        	:        1204.991 Mflip/s
-   FMA percentage                             		:          99.966 %
-   Computation intensity                      		:           3.889

## HPMLIB

- Add fortran/C calls within your code
- Monitor performance on segment of the code
- Easy to use

## Example

-          implicit real*8(a-h,o-z)
-          parameter (n=1500)
- #include "f_hpm.h"
-          dimension a(n,n),b(n,n),c(n,n)
-          call f_hpminit( taskID, "my program" )
-          call f_hpmstart( 1, "init   "  )
-          do i = 1, n
-               do j = 1, n
-                  a(i,j) = rand()
-                  b(i,j) = rand()
-               enddo
-          enddo
-          call f_hpmstop( 1  )
-          call f_hpmstart( 2, "dgemm  "  )
-          call dgemm('N', 'N', n, n, n, 1.0,A,n,B,n,0.0, C, n)
-          call f_hpmstop ( 2  )
-          call f_hpmterminate( taskID )
-          end

## How to Compile and Run

- module load hpmtoolkit/2.3
- xlf main.F -qsuffix=cpp=F –lessl $HPMTOOLKIT
- ./a.out
- This will produce 2 files:
-  *my program*.viz
-  perfhpm*.*
- Pefhpm*.* is a text file of the results
- *myprogram*.viz is binary file which can be used with hpmviz

## HPMVIZ Window

## HPMVIZ Window

- Right Click on dgemm

## Section Information

## VAMPIR

- Used to look at communication and Computation times
- Harder to understand

## How to Compile and Run

- module load vampir
- xlf90 –g main.f $VTLIB
- Run the code  poe ./a.out –nodes 2 –procs 32
- The execution will generate a file *.bpv (a.out.bpv)
- To run vampir: vampir a.out.bpv

## Vampir Window

## Global Displays

## Processor Information

## Global Activity

- Left mouse click

## Activity for Processor 0

- right mouse click

## Another Example

## VAMPIR Configuration File

- Vampir will create a configuration file under:
- $HOME/.VAMPIR_defaults
- Called VAMPIR2.cnf

## TOTALVIEW

- Debugging tool
- Supported by ETNUS
- Works well most of the time
- Line mode and x-windows mode
- Used to debug FORTRAN and C

## Usage

- module load totalview
- mpxlf –g main.f
- totalview poe –a ./a.out –nodes 2 –procs 32

## TOTALVIEW Window

## POE Window

## Source Code Window

## Set Breakpoint

- right mouse click

## Breakpoint options

## Evaluation Function

## Variable Value

## Looking at the vector x

## Looking at a Slice of an Array

## Attaching a Running Job

- Run Totalview using this command
- totalview &
- poe ./a.out –nodes 2 –procs 32
- Click on the poe job

## Attaching a Job

## Running the Job

- Click Go

## Detaching a Job

- Click detach

## TOTALVIEW X-Resources

- totalview*searchPath:/usr/common/homes/m/majdi/NETCDF4/src/libsrc, \
- /usr/common/homes/m/majdi/NETCDF4/src/fortran, \
- /usr/common/homes/m/majdi/NETCDF4/src/f90, \
- /usr/common/homes/m/majdi/NETCDF4/src, \
- /scratch/scratchdirs/majdi, /scratch/scratchdirs/majdi/NimRot