N-WAYS GPU BOOTCAMPNSIGHT SYSTEM PROFILINGNSIGHT SYSTEM
What to expect?
•Basic introduction to Nsight family of tools•Using Nsight Systems

This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)2

DEVELOPMENT CYCLEAnalyze§ Analyze your code to determine
most likely places needing
parallelization or optimization.
§ Parallelize your code by startingwith the most time consuming partsand check for correctness.
§ Optimize your code to improveobserved speed-up fromparallelization.

Optimize

ParallelizeThis material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

3

PROFILING SEQUENTIAL CODE
Profile Your CodeObtain detailed information about how
the code ran.
This can include information such as:§ Total runtime§ Runtime of individual routines§ Hardware counters

Hotspot Analysis
Total Runtime: 55.87 seconds

func2
26.07sfunc1 29.78s

Identify the portions of code that took
the longest to run. We want to focus on
these “hotspots” when parallelizing.
This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

4

NVIDIA NSIGHT FAMILYThis material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

5

Nsight Product Family
Workflow

Nsight Systems - Analyze

application algorithm systemwide

Nsight Compute -Debug/optimize CUDA kernel

Nsight Graphics -Debug/optimize graphics
workloads

This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

6

Thread/core
migration

Processes
andthreads

Thread state

CUDA andOpenGL API trace
cuDNN andcuBLAS trace
Kernel and memorytransfer activities

Multi-GPU
This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

7

GPU IDLE AND LOW UTILIZATION LEVEL OF DETAILThis material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

12

CUDA MEMORY TRANSFER COLOR PALLETTESHOW DIRECTION AND PAGEABLE MEMORY HAZARDSThis material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

14

PROFILING WITH NSIGHTSYSTEM
AND NVTXThis material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)16

PROFILING SEQUENTIAL CODE
Using Command Line Interface (CLI)

NVIDIA Nsight Systems CLI provides§ Simple interface to collect data
§ Can be copied to any system and analysed later§ Profiles both serial and parallel code
§ For more info enter nsys --help on the terminal
To profile a serial application with NVIDIA Nsight Systems, we use NVIDIA Tools Extension(NVTX) API functions in addition to collecting backtraces while sampling.

This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

17

PROFILING SEQUENTIAL CODENVIDIA Tools Extension API (NVTX) library

What is it?
§ A C-based Application Programming Interface (API) for annotating events§ Can be easily integrated to the application§ Can be used with NVIDIA Nsight Systems
Why?
§ Allows manual instrumentation of the application§ Allows additional information for profiling (e.g: tracing of CPU events and time ranges)
How?
§ Import the header only C library nvToolsExt.h§ Wrap the code region or a specific function with nvtxRangePush() andnvtxRangPop()This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

18

#include <string.h>#include <stdio.h>#include <stdlib.h>#include <omp.h>#include "laplace2d.h"#include <nvtx3/nvToolsExt.h>
int main(int argc, char** argv){
const int n = 4096;
const int m = 4096;
const int iter_max = 1000;-t

Selects the APIs to be traced (nvtx in this example)--status

if true, generates summary of statistics after the collection-b

Selects the backtrace method to use while sampling. The option dwarf
uses DWARF's CFI (Call Frame Information).

--force-overwrite

if true, overwrites the existing results-osets the output (qdrep) filename

const double tol = 1.0e-6;
double error = 1.0;
double *restrict A= (double*)malloc(sizeof(double)*n*m);
double *restrict Anew = (double*)malloc(sizeof(double)*n*m);
nvtxRangePushA("init");initialize(A, Anew, m, n);nvtxRangePop();printf("Jacobi relaxation Calculation: %d x %d mesh\n", n, m);double st = omp_get_wtime();int iter = 0;
nvtxRangePushA("while");while ( error > tol && iter < iter_max ){nvtxRangePushA("calc");
error = calcNext(A, Anew, m, n);nvtxRangePop();nvtxRangePushA("swap");swap(A, Anew, m, n);nvtxRangePop();if(iter % 100 == 0) printf("%5d, %0.6f\n", iter, error);

NVTX range
statistics

iter++;}
nvtxRangePop();
double runtime = omp_get_wtime() - st;printf(" total: %f s\n", runtime);deallocate(A, Anew);return 0;}

jacobi.c(starting and ending of ranges arehighlighted with the same color)“calc” region (calcNext function) takes 26.6%“swap” region (swap function) takes 23.4% of
total execution time
Open laplace-seq.qdrep withNsight System GUI to view thetimeline

This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

19

USER ANNOTATIONS APIS FOR CPU & GPU
NVTX, OPENGL, VULKAN, AND DIRECT3D PERFORMANCE MARKERSEXAMPLE: VISUAL MOLECULAR DYNAMICS (VMD) ALGORITHMS VISUALIZED WITH NVTX ON CPU
This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

20

REFERENCES

https://docs.nvidia.com/nsight-systems
https://developer.nvidia.com/hpc-sdk

This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)25

THANK YOU