Package: nvtx-plugins
NVTX Plugins for Deep LearningNVTX Plugins allows users to add their own NVIDIA Tools Extension (NVTX)
events and time ranges to a TensorFlow graph. Applications which integrate NVTX can use NVIDIA
Nsight Systems and Nsight Compute to capture and visualize
these events and time ranges.The NVTX ranges are added by wrapping regions of the computation graph with
nvtx start and end operations.NVTX Plugins also provides Keras callbacks and session hooks.More about:NVTX:https://docs.nvidia.com/cuda/profiler-users-guide/index.html#nvtxNSight systems:https://developer.nvidia.com/nsight-systemsTable of ContentsInstalling or building NVTX PluginsQuick start guideDocumentationAcknowledgmentsDisclaimerContributionsCopyright and LicenseInstalling or building NVTX PluginsPrerequisitesLinuxPython 3.4+NVIDIA GPU + CUDA toolkit 10.0 or newerTensorFlow 1.13 or newerInstalling NVTX-PluginsThe package can be installed from PyPI:# Stable releasepipinstallnvtx-plugins# Pre-release (may present bugs)pipinstallnvtx-plugins--preThe package is also available for download on github:https://github.com/NVIDIA/nvtx-plugins/releasespipinstallnvtx-plugins*.tar.gzInstalling from sourceYou can build and install the package from source:pythonsetup.pysdistpipinstalldist/nvtx-plugins*.tar.gzFor development objectives, you can install the package directly from source with:pythonsetup.pyinstallWe recommend building the package inside NVIDIA’s NGC TensorFlow container:https://ngc.nvidia.com/catalog/containers/nvidia:tensorflowFor more information about how to get started with NGC containers, see the
following sections from the NVIDIA GPU Cloud Documentation and the Deep
Learning DGX Documentation:Getting Started Using NVIDIA GPU
Cloud,Accessing And Pulling From The NGC container registryandRunning TensorFlow.Building the documentationThe documentation is built by running:cddocspipinstall-rrequirements.txtmakehtmlThe documentation files will be generated indocs/build/htmlBuilding the documentation does not require NVTX Plugins to be installed.
Nonetheless, due to an issue in Sphinxonly Python 3.7 is supportedto build the documentation.Quick start guideAdding markers to the graphMarkers are added by wrapping parts of the computation graph with start and end
operations. The operations are identity ops (passing the input to the output
without modification) but they have a side effect of generating nvtx markers.importnvtx.plugins.tfasnvtx_tfx,nvtx_context=nvtx_tf.ops.start(x,message='Dense 1-3',domain_name='Forward',grad_domain_name='Gradient')x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_1')x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_2')x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_3')x=nvtx_tf.ops.end(x,nvtx_context)x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_4')For convenience, the package also provides a function dectorator:@nvtx_tf.ops.trace(message='Dense Block',domain_name='Forward',grad_domain_name='Gradient')defdense_block(x):x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_1')x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_2')x=tf.layers.dense(x,1000,activation=tf.nn.relu,name='dense_3')returnxMore detailed examples can be found inexamples/, also, check theDocumentationfor more information about other workflows including
session hooks, Keras layers and callbacks.Visualizing the rangesNVTX requires a logger to register the generated events and ranges, we will use
NVIDIA Nsight Systems to capture these event.Run your code withnsys(pre-installed in NVIDIA’s NGC TensorFlow container) to
generate aqdrepfile:nsysprofile-d60\-wtrue\--sample=cpu\-t'nvtx,cuda'\-o./generated_timeline\python./network.pyThe generated qdrep can be viewed using Nsight Systems.Nsight Systems andnsyscan also be downloaded and from theNVIDIA’’s developer website.More details about nsys and Nsight Systems can be foundhere.DocumentationMore details about NVTX Plugins can be found on here:https://nvtx-plugins.readthedocs.io/en/latest/AcknowledgmentsThe project structure is heavily influenced by the TensorFlow custom-op example:https://github.com/tensorflow/custom-opDisclaimerThe project is in beta stage, breaking changes are to be expected in the future.ContributionsContributions to NVTX Plugins are more than welcome. To contribute code,
please submit apull requestagainst the master branch from a local fork.We appreciate feedback, questions or bug reports. If you need help
with the code, create aGitHub issue.
Please follow the process outlined in the Stack Overflowhttps://stackoverflow.com/help/mcvedocument. Ensure that the
posted examples are:minimal: Use as little code as possible that still produces the same problem.complete: Provide all parts needed to reproduce the problem.
Check if you can strip external dependency and still show the problem.
The less time we spend on reproducing the problems, the more time we
can dedicate to the fixes.verifiable: Test the code you are about to provide, to make sure
that it reproduces the problem. Remove all other problems that are not
related to your request.Copyright and LicenseThis project is released under the Apache License, Version 2.0