Package: deepview-predict
DeepView.PredictA Runtime-Based Computational Performance Predictor for Deep Neural Network TrainingInstallationBuilding from sourceUsage exampleDevelopment Environment SetupRelease processRelease historyLicenseResearch paperContributingDeepView.Predict is a tool that predicts a deep neural network's training iteration execution time on a given GPU. It currently supports PyTorch. To learn more about how DeepView.Predict works, please see ourresearch paper.InstallationTo run DeepView.Predict, you need:Python 3.6+Pytorch 1.1.0+A system equiped with an Nvidia GPU with properly configured CUDACurrently, we have predictors for the following Nvidia GPUs:GPUGenerationMemoryMem. TypeSMsP4000Pascal8 GBGDDR514P100Pascal16 GBHBM256V100Volta16 GBHBM280RTX 2070Turing8 GBGDDR636RTX 2080TiTuring11 GBGDDR668T4Turing16 GBGDDR640RTX 3090Ampere24 GBGDDR6X82A100Ampere40 GBHBM2108A40Ampere48 GBGDDR684RTX A4000Ampere16 GBGDDR648RTX 4000Turing8 GBGDDR636Building locallyInstalling from pipInstall via pip with the following commandpipinstalldeepview-predictInstalling from sourceInstall CUPTICUPTI is a profiling interface required by DeepView.Predict. Select your version of CUDAhereand follow the instructions to add NVIDIA's repository. Then, install CUPTI with:sudoapt-getinstallcuda-cupti-xx-xwherexx-xrepresents the version of CUDA you have installed.Alternatively, if you do not have root access on your machine, you can usecondato install CUPTI. Select your version of CUDAhereand follow the instructions. For example if you have CUDA 11.6.0, you can install CUPTI with:condainstall-c"nvidia/label/cuda-11.6.0"cuda-cuptiAfter installing CUPTI, add$CONDA_HOME/extras/CUPTI/lib64/toLD_LIBRARY_PATHto ensure the library is linked.Install CMake 3.17+.Note that CMake 3.24.0 and 3.24.1 has a bug that breaks DeepView.Predict as it is not able to find the CUPTI directory and you should not use those versionshttps://gitlab.kitware.com/cmake/cmake/-/merge_requests/7608/diffsRun the following commands to download and install a precompiled version of CMake 3.24.2wgethttps://github.com/Kitware/CMake/releases/download/v3.24.2/cmake-3.24.2-linux-x86_64.sh
chmod+xcmake-3.24.2-linux-x86_64.sh
mkdir/opt/cmake
shcmake-3.24.2-linux-x86_64.sh--prefix=/opt/cmake--skip-license
ln-s/opt/cmake/bin/cmake/usr/local/bin/cmakeYou can verify the version of CMake you installed with the following commandcmake--versionInstallGit Large File StorageClone the DeepView.Predict packagegitclonehttps://github.com/CentML/DeepView.PredictcdDeepView.PredictGet the pre-trained models used by DeepView.Predictgitsubmoduleinit&&gitsubmoduleupdate
gitlfspullFinally build DeepView.Predict with the following command./analyzer/install-dev.shBuilding with DockerDeepView.Predict has been tested to work on the latest version ofNVIDIA NGC PyTorch containers.To build DeepView.Predict with Docker, first run the NGC container wheredockerrun--gpusall-it--rmnvcr.io/nvidia/pytorch:XX.XX-py3Inside the container, clone the repository then build and install DeepView.Predict Python package:gitclone--recursivehttps://github.com/CentML/DeepView.Predict
./habitat/analyzer/install-dev.shNote:DeepView.Predict needs access to your GPU's performance counters, which requires special permissions if you are running with a recent driver (418.43 or later). If you encounter aCUPTI_ERROR_INSUFFICIENT_PRIVILEGESerror when running DeepView.Predict, please follow the instructionshereand inissue #5.Usage exampleYou can verify your DeepView.Predict installation by running the simple usage example:# example.pyimporthabitatimporttorchimporttorchvision.modelsasmodels# Define model and sample inputsmodel=models.resnet50().cuda()image=torch.rand(8,3,224,224).cuda()# Measure a single inferencetracker=habitat.OperationTracker(device=habitat.Device.RTX2080Ti)withtracker.track():out=model(image)trace=tracker.get_tracked_trace()print("Run time on source:",trace.run_time_ms)# Perform prediction to a single target devicepred=trace.to_device(habitat.Device.V100)print("Predicted time on V100:",pred.run_time_ms)python3example.pySeeexperiments/run_experiment.pyfor other examples of DeepView.Predict usage.Release HistorySeeReleasesLicenseThe code in this repository is licensed under the Apache 2.0 license (seeLICENSEandNOTICE), with the exception of the files mentioned below.This software contains source code provided by NVIDIA Corporation. These files
are:The code undercpp/external/cupti_profilerhost_util/(CUPTI sample code)cpp/src/cuda/cuda_occupancy.hThe code mentioned above is licensed under theNVIDIA Software Development
Kit End User License Agreement.We include the implementations of several deep neural networks underexperiments/for our evaluation. These implementations are copyrighted by
their original authors and carry their original licenses. Please see the
correspondingREADMEfiles and license files inside the subdirectories for
more information.Research PaperDeepView.Predict began as a research project in theEcoSystem Groupat theUniversity of Toronto. The accompanying research paper appeared in the proceedings ofUSENIX
ATC'21. If you are
interested, you can read a preprint of the paperhere.If you use DeepView.Predict in your research, please consider citing our paper:@inproceedings{habitat-yu21,author={Yu, Geoffrey X. and Gao, Yubo and Golikov, Pavel and Pekhimenko,Gennady},title={{Habitat: A Runtime-Based Computational Performance Predictor forDeep Neural Network Training}},booktitle={{Proceedings of the 2021 USENIX Annual Technical Conference(USENIX ATC'21)}},year={2021},}ContributingCheck outCONTRIBUTING.mdfor more information on how to help with Habitat.