Package: pynvml
Python bindings to the NVIDIA Management LibraryProvides a Python interface to GPU management and monitoring functions.This is a wrapper around the NVML library.
For information about the NVML library, see the NVML developer pagehttp://developer.nvidia.com/nvidia-management-library-nvmlAs of version 11.0.0, the NVML-wrappers used in pynvml are identical
to those published throughnvidia-ml-py.Note that this file can be run with 'python -m doctest -v README.txt'
although the results are system dependentRequiresPython 3, or an earlier version with the ctypes module.Installationpip install .UsageYou can use the lower level nvml bindings>>>frompynvmlimport*>>>nvmlInit()>>>print("Driver Version:",nvmlSystemGetDriverVersion())DriverVersion:410.00>>>deviceCount=nvmlDeviceGetCount()>>>foriinrange(deviceCount):...handle=nvmlDeviceGetHandleByIndex(i)...print("Device",i,":",nvmlDeviceGetName(handle))...Device0:TeslaV100>>>nvmlShutdown()Or the higher level nvidia_smi APIfrompynvml.smiimportnvidia_sminvsmi=nvidia_smi.getInstance()nvsmi.DeviceQuery('memory.free, memory.total')frompynvml.smiimportnvidia_sminvsmi=nvidia_smi.getInstance()print(nvsmi.DeviceQuery('--help-query-gpu'),end='\n')FunctionsPython methods wrap NVML functions, implemented in a C shared library.
Each function's use is the same with the following exceptions:Instead of returning error codes, failing error codes are raised as
Python exceptions.>>>try:...nvmlDeviceGetCount()...exceptNVMLErroraserror:...print(error)...UninitializedC function output parameters are returned from the corresponding
Python function left to right.nvmlReturn_tnvmlDeviceGetEccMode(nvmlDevice_tdevice,nvmlEnableState_t*current,nvmlEnableState_t*pending);>>>nvmlInit()>>>handle=nvmlDeviceGetHandleByIndex(0)>>>(current,pending)=nvmlDeviceGetEccMode(handle)C structs are converted into Python classes.nvmlReturn_tDECLDIRnvmlDeviceGetMemoryInfo(nvmlDevice_tdevice,nvmlMemory_t*memory);typedefstructnvmlMemory_st{unsignedlonglongtotal;unsignedlonglongfree;unsignedlonglongused;}nvmlMemory_t;>>>info=nvmlDeviceGetMemoryInfo(handle)>>>print"Total memory:",info.totalTotalmemory:5636292608>>>print"Free memory:",info.freeFreememory:5578420224>>>print"Used memory:",info.usedUsedmemory:57872384Python handles string buffer creation.nvmlReturn_tnvmlSystemGetDriverVersion(char*version,unsignedintlength);>>>version=nvmlSystemGetDriverVersion();>>>nvmlShutdown()For usage information see the NVML documentation.VariablesAll meaningful NVML constants and enums are exposed in Python.The NVML_VALUE_NOT_AVAILABLE constant is not used.  Instead None is mapped to the field.NVML PermissionsMany of thepynvmlwrappers assume that the underlying NVIDIA Management Library (NVML) API can be used without admin/root privileges.  However, it is certainly possible for the system permissions to prevent pynvml from querying GPU performance counters. For example:$ nvidia-smi nvlink -g 0
GPU 0: Tesla V100-SXM2-32GB (UUID: GPU-96ab329d-7a1f-73a8-a9b7-18b4b2855f92)
NVML: Unable to get the NvLink link utilization counter control for link 0: Insufficient PermissionsA simple way to check the permissions status is to look forRmProfilingAdminOnlyin the driverparamsfile (Note thatRmProfilingAdminOnly == 1means that admin/sudo access is required):$ cat /proc/driver/nvidia/params | grep RmProfilingAdminOnly
RmProfilingAdminOnly: 1For more information on setting/unsetting the relevant admin privileges, seethese noteson resolvingERR_NVGPUCTRPERMerrors.Release NotesVersion 2.285.0Added new functions for NVML 2.285.  See NVML documentation for more information.Ported to support Python 3.0 and Python 2.0 syntax.Added nvidia_smi.py tool as a sample app.Version 3.295.0Added new functions for NVML 3.295.  See NVML documentation for more information.Updated nvidia_smi.py toolIncludes additional error handlingVersion 4.304.0Added new functions for NVML 4.304.  See NVML documentation for more information.Updated nvidia_smi.py toolVersion 4.304.3Fixing nvmlUnitGetDeviceCount bugVersion 5.319.0Added new functions for NVML 5.319.  See NVML documentation for more information.Version 6.340.0Added new functions for NVML 6.340.  See NVML documentation for more information.Version 7.346.0Added new functions for NVML 7.346.  See NVML documentation for more information.Version 7.352.0Added new functions for NVML 7.352.  See NVML documentation for more information.Version 8.0.0Refactor code to a nvidia_smi singleton classAdded DeviceQuery that returns a dictionary of (name, value).Added filter parameters on DeviceQuery to match query api in nvidia-smiAdded filter parameters on XmlDeviceQuery to match query api in nvidia-smiAdded integer enumeration for filter strings to reduce overhead for performance monitoring.Added loop(filter) method with async and callback supportVersion 8.0.1Restructuring directories into two packages (pynvml and nvidia_smi)Adding initial tests for both packagesSome name-convention cleanup in pynvmlVersion 8.0.2Added NVLink function wrappers for pynvml moduleVersion 8.0.3Added versioneerFixed nvmlDeviceGetNvLinkUtilizationCounter bugVersion 8.0.4Added nvmlDeviceGetTotalEnergyConsumptionAdded notes about NVML permissionsFixed version-check testingVersion 11.0.0Updated nvml.py to CUDA 11Updated smi.py DeviceQuery to R460Aligned nvml.py with latest nvidia-ml-py deploymentVersion 11.4.0Updated nvml.py to CUDA 11.4Updated smi.py NVML_BRAND_NAMESAligned nvml.py with latest nvidia-ml-py deployment (11.495.46)Version 11.4.1Fix comma bugs in nvml.pyVersion 11.5.0Updated nvml.py to support CUDA 11.5 and CUDA 12Aligned with latest nvidia-ml-py deployment (11.525.84)