Package: igpu
iGPUTheigpuis a pythonic cross-platform module for getting the GPU info and status from NVIDA GPU boards using thpynvml(a python wrapper around the NVML library).Quick StartInstall theigpumodule.pipinstalligpuAnd, enjoy it!importigpugpu_count=igpu.count_devices()gpu=igpu.get_device(0)print(f'This host has{gpu_count}devices.')print(f'The first gpu is a{gpu.name}with{gpu.memory.total:.0f}{gpu.memory.unit}.')Thishosthas4devices.
ThefirstgpuisaGeForceGTX1080Tiwith11178.50MiBTable of ContentsRequirementsInstallationExamplesUsage DocumentationAvailable DevicesVisible DevicesGPUInfo Class DescriptionLicenseRequirementsSupports Python 3.4 or higher. Requires least one NVIDIA GPU with latest NVIDIA driver installed.Python standard libraries:os (The Python Standard Library)textwraper (The Python Standard Library)datetime (The Python Standard Library)Third-party libraries:pynvml (Python bindings to the NVIDIA Management Library)psutil (Python process and system utilities)InstallationCurrently it is only supported Python 3.4 or higher. It can be installed through pip:pip install igpuTest the installation:Start a python console by typingpythonin the terminalIn the newly opened python console, type:importigpuprint(igpu.nvidia_driver_version())Your output should look something like following, depending on your nvidia driver version.(430, 34)ExamplesThe following examples illustrates the ease of use of this module. For a complete reference, see theUsage Documentationsection.List the utilization stats of all devicesThe user can access the utilization stats attributes in the following manner:importigpuforgpu_infoinigpu.devices():print(f'GPU board{gpu_info.index}:{gpu_info.name}')print(f'Utilization: GPU{gpu_info.utilization.gpu}% | Memory{gpu_info.utilization.memory}%')print(f'Temperature:{gpu_info.utilization.temperature}C (Fan -{gpu_info.utilization.fan}%)')print()GPUboard0:GeForceGTX1080Ti
Utilization:GPU0%|Memory0%
Temperature:30C(Fan-23%)GPUboard1:GeForceGTX1080Ti
Utilization:GPU65%|Memory58%
Temperature:62C(Fan-36%)GPUboard2:GeForceGTX1080Ti
Utilization:GPU38%|Memory33%
Temperature:63C(Fan-37%)GPUboard3:GeForceGTX1080Ti
Utilization:GPU0%|Memory0%
Temperature:28C(Fan-23%)List the memory and power summary of a specific deviceEach attributes group (memory,utilization,pci,clocks,power,processes) can be summarized simply by calling the string converstion. In the next example, we will print the memory and power summary of the second GPU board:importigpugpu_info=igpu.get_device(1)print(gpu_info.memory)print(gpu_info.power)GPUMEMORY:Total:11178.50MiB(100.00%)Used:6525.00MiB(58.37%)Free:4653.50MiB(41.63%)POWERINFO:Management:SupportedDraw:55.751Limit:250.0MinLimit:125.0MaxLimit:300.0List the processes and the memory usage of a specific deviceIn this example, we will work with thememoryandprocessesattributes of the first GPU board:importigpugpu_info=igpu.get_device(0)print(f'The device has{gpu_info.memory.used}allocated out of a total of{gpu_info.memory.total}')print(f'The{len(gpu_info.processes)}processe(s) running on device are:')forprocingpu_info.processes:print(f'{proc.pid}: created at{proc.create_time}by{proc.user}and using{proc.gpu_memory}')Thedevicehas10799.0allocatedoutofatotalof11178.5
The2processe(s)runningondeviceare:4276:createdat2020-04-1703:41:27byantonioandusing42845764:createdat2020-04-1617:57:12byacnazarejrandusing6515Usage DocumentationTheigpumodule is very versatile and can be used in a different of ways, given below.To include iGPU in your Python code, all you have to do is included it on your script:importigpuOnce included all functions are available. The main functions along with a short description of inputs, outputs, and functionality can be found in the next sections.Available DevicesTheigpucan manipulate information and stats for all available devices. Availability is determined based on the GPU boards installed on the host.igpu.count_devices()Returns the number of available GPU devices installed on the host.>>>igpu.count_devices()4igpu.devices_index()Returns an index list, containing the device index for each available GPU.>>>igpu.devices_index()[0,1,2,3]igpu.get_device(device_index)Given adevice_index, returns aGpuInfoobject containing the device properties and stats. If a nonexistentdevice_indexis provided, an error is thrown.All properties and methods ofGpuInfoclass are described inGPUInfo Class Descriptionsection.>>>gpu_info=igpu.get_device(2)>>>print(gpu_info.index,gpu_info.name)2,GeForceGTX1080Tiigpu.devices()Returns aGpuInfolist containing all available devices.All properties and methods ofGpuInfoclass are described inGPUInfo Class Descriptionsection.>>>forgpu_infoinigpu.devices():...print(gpu_info.index,gpu_info.name)0,GeForceGTX1080Ti1,GeForceGTX1080Ti2,GeForceGTX1080Ti3,GeForceGTX1080TiVisible DevicesBy manually setting the environment variableCUDA_VISIBLE_DEVICES(or automatically, by other software, like SLURM cluster manager), the user can mask which GPUs should be visible to different Deep Learning frameworks (e.g. TensorFlow, PyTorch, Caffee, etc). Seethis stackoverflow questionfor more info.Theigpumodule can deal with this, and consider only the visible devices, by the following methods. This functionality is handy for those who work on environments where the visible devices are defined externally. For example,when the SLURM setstheCUDA_VISIBLE_DEVICESvariable in a job.For the next examples, consider that theCUDA_VISIBLE_DEVICESvariable was set as below:exportCUDA_VISIBLE_DEVICES=1,3igpu.count_visible_devices()Returns the number of visible GPU devices, defined by theCUDA_VISIBLE_DEVICESenvironmnt variable.>>>igpu.count_visible_devices()2igpu.visible_devices_index()Returns an index list, containing the device index for each visible GPU defined by theCUDA_VISIBLE_DEVICESenvironmnt variable.>>>igpu.visible_devices_index()[1,3]igpu.visible_devices()Returns aGpuInfolist containing all visible devices defined by theCUDA_VISIBLE_DEVICESenvironmnt variable.All properties and methods ofGpuInfoclass are described inGPUInfo Class Descriptionsection.>>>forgpu_infoinigpu.visible_devices():...print(gpu_info.index,gpu_info.name)1,GeForceGTX1080Ti3,GeForceGTX1080TiGPUInfo Class DescriptionTheGPUInfois a helper class that handles the attributes of each GPU. The user can access all properties and stats accordingly with the GPU attributes categories. Each category has another helper subclass described below. Also, the class, and consequently, its subclasses, has an implicit conversion to a pretty string.For the all next examples, consider that the a instace of GPUInfo was created as follow:gpu_info=igpu.get_device(0)Also, the class has a method (GPUInfo.update()) to update the device information that can be used as follow:gpu_info.update()TheGPUInfohas the general GPU attributes:Attributesindex(int) - The index of the GPU device.name(str) - The official product name of the GPU.serial(str) - The GPU board serial number. This number matches the serial number physically printed on each board. It is a globally unique immutable alphanumeric value.uuid(str) - The GPU board uuid. This value is the globally unique immutable alphanumeric identifier of the GPU. It does not correspond to any physical label on the board.bios(str) - The BIOS version of the GPU board.Usage>>>gpu_info.index0>>>gpu_info.name'GeForce GTX 1080 Ti'>>>gpu_info.serial'N/A'>>>gpu_info.uuid'GPU-d9bf777c-e49e-e609-f873-d05b3208970b'>>>gpu_info.bios'86.02.39.00.01'String Conversion>>>print(gpu_info)INDEX:0BOARDNAME:'GeForce GTX 1080 Ti'SERIAL:'N/A'UUID:'GPU-115dafb8-26db-32ba-d729-f21b29fc001f'BIOSVERSION:'86.02.39.00.01'GPUMEMORY:Total:11178.50MiB(100.00%)Used:10799.00MiB(96.61%)Free:379.50MiB(3.39%)GPUUTILIZATION:Performance:P2Temperature:'61C'Graphics:[|||||||||||]44.00%Memory:[||||||||]35.00%Fan:[|||||||||]36.00%PCIINFO:Bus:'88'BusID:'0000:88:00.0'Device:'00'DeviceID:'1B0610DE'Sub-SystemID:'1210196E'Generation:'3 (Max: 3)'LinkWidth:'16x (Max: 16x)'GPUCLOCK:Graphics:1911.0MHz(Max:1911.0)SM:1911.0MHz(Max:1911.0)Memory:5005.0MHz(Max:5505.0)POWERINFO:Management:'Supported'Draw:122.23WLimit:250.0WMinLimit:125.0WMaxLimit:300.0WPROCESSESPID|NAME|USER|PARENT|CREATIONTIME|GPUMEM4276|python|antonio|45436|'2020-04-17 03:41:27'|42845764|python|acnazarejr|5759|'2020-04-16 17:57:12'|6515Memory Attributes (GPUMemoryInfo)Helper class that handles the on-board memory attributes. Reported total memory is affected by the ECC state. If ECC is enabled, the total available memory is decreased by several percent, due to the requisite parity bits. The driver may also reserve a small amount of memory for internal use, even without active work on the GPU. These attributes are available for all products.Attributestotal(float) - The total installed GPU memory.used(float) - The total memory allocated by active contexts.free(float) - The total free memory.unit(str) - The memory unit of measurement.Usage>>>gpu_info.memory.total11178.5>>>gpu_info.memory.used10799.0>>>gpu_info.memory.free379.5>>>gpu_info.memory.unit'MiB'String Conversion>>>print(gpu_info.memory)GPUMEMORY:Total:11178.50MiB(100.00%)Used:10799.00MiB(96.61%)Free:379.50MiB(3.39%)Utilization Stats (GPUUtilizationInfo)Helper class that handles the utilization stats of each GPU. Utilization rates report how busy each GPU is over time, and can be used to determine how much an application is using the GPUs in the system.Attributesgpu(float) - The percent of the time over the past sample period during which one or more kernels were executing on the GPU. The sample period may be between 1 second and 1/6 second, depending on the product.memory(float) - The percent of the time over the past sample period during which global (device) memory was being read or written. The sample period may be between 1 second and 1/6 second, depending on the product.fan(float) - For a healthy fan, the percent of fan's speed.temperature(float) - The core GPU temperature. For all discrete and S-class products.performance(str) - The current performance state for the GPU. States range fromP0(maximum performance) toP12(minimum performance).Usage>>>gpu_info.utilization.gpu65>>>gpu_info.utilization.memory59>>>gpu_info.utilization.fan36>>>gpu_info.utilization.temperature61>>>gpu_info.utilization.performance'P2'String Conversion>>>print(gpu_info.utilization)GPUUTILIZATION:Performance:'P2'Temperature:'61C'Graphics:[||||||||||||||||]65.00%Memory:[||||||||||||||]59.00%Fan:[|||||||||]36.00%PCI Attributes (GPUPCIInfo)Helper class that handles the pci attributes of each GPU.The class handles the basic PCI info for the device. Some of this information may change whenever cards are added/removed/moved in a system. It also provides the PCIe link generation and bus width. These attributes are available for all products.Attributesbus(str) -The PCI bus number, in hex.bus_id(str) -The PCI bus id as "domain:bus:device.function", in hex.device(str) -The PCI device number, in hex.device_id(str) -The PCI vendor device id, in hex.sub_system_id(str) -The PCI Sub System id, in hex.current_link_gen(str) -The current link generation. These may be reduced when the GPU is not in use.max_link_gen(str) -The maximum link generation possible with this GPU and system configuration. For example, if the GPU supports a higher PCIe generation than the system supports then this reports the system PCIe generation.current_link_width(str) -The current link width. These may be reduced when the GPU is not in use.max_link_width(str) -The maximum link width possible with this GPU and system configuration. For example, if the GPU supports a higher link width than the system supports then this reports the system PCIe link with.Usage>>>gpu_info.pci.bus'88'>>>gpu_info.pci.bus_id'0000:88:00.0'>>>gpu_info.pci.device'00'>>>gpu_info.pci.device_id'1B0610DE'>>>gpu_info.pci.sub_system_id'1210196E'>>>gpu_info.pci.current_link_gen'3'>>>gpu_info.pci.max_link_gen'3'>>>gpu_info.pci.current_link_width'16x'>>>gpu_info.pci.max_link_width'16x'String Conversion>>>print(gpu_info.pci)PCIINFO:Bus:'88'BusID:'0000:88:00.0'Device:'00'DeviceID:'1B0610DE'Sub-SystemID:'1210196E'Generation:'3'(Max:'3')LinkWidth:'16x'(Max:'16x')Clock Attributes (GPUClockInfo)Helper class that handles the clocks attributes of each GPU.The current frequency at which parts of the GPU are running. All readings are in MHz.Attributesgraphics(float) - The current frequency of graphics (shader) clock.sm(float) - The current frequency of SM (Streaming Multiprocessor) clock.memory(float) - The current frequency of memory clock.max_graphics(float) - The maximum frequency of graphics (shader) clock.max_sm(float) - The maximum frequency of SM (Streaming Multiprocessor) clock.max_memory(float) - The maximum frequency of memory clock.unit(str) - The clock unit of measurement.Usage>>>gpu_info.clocks.graphics1885>>>gpu_info.clocks.sm1885>>>gpu_info.clocks.memory5005>>>gpu_info.clocks.max_graphics1911>>>gpu_info.clocks.max_sm1911>>>gpu_info.clocks.max_memory5505>>>gpu_info.clocks.unit'MHz'String Conversion>>>print(gpu_info.clocks)GPUCLOCK:Graphics:1911.0MHz(Max:1911.0)SM:1911.0MHz(Max:1911.0)Memory:5005.0MHz(Max:5505.0)Power Attributes (GPUPowerInfo)Helper class that handles the power attributes of each GPU.Power readings help to shed light on the current power usage of the GPU, and the factors that affect that usage. When power management is enabled, the GPU limits power draw under load to fit within a defined power envelope by manipulating the current performance state. See below for limits of availability.Attributesmanagement(str) - A flag that indicates whether power management is enabled. Either "Supported" or "N/A". Requires Inforom PWR object version 3.0 or higher or Kepler device.draw(int) - The last measured power draw for the entire board, in watts. Only available if power management is supported. This reading is accurate to within +/- 5 watts. Requires Inforom PWR object version 3.0 or higher or Kepler device.limit(int) - The software power limit, in watts. Set by software such as nvidia-smi. Only available if power management is supported. Requires Inforom PWR object version 3.0 or higher or Kepler device.min_limit(int) - The minimum value in watts that power limit can be set to. Only on supported devices from Kepler family.max_limit(int) - The maximum value in watts that power limit can be set to. Only on supported devices from Kepler family."""unit(str) - The clock unit of measurement.Usage>>>gpu_info.power.management'Supported'>>>gpu_info.power.draw93.067>>>gpu_info.power.limit250.0>>>gpu_info.power.min_limit125.0>>>gpu_info.power.max_limit300.0>>>gpu_info.power.unit'W'String Conversion>>>print(gpu_info.power)POWERINFO:Management:SupportedDraw:93.067WLimit:250.0WMinLimit:125.0WMaxLimit:300.0WProcesseses Attributes(GPUProcessesInfoandGPUProcessInfo)While each process which has compute context on the GPU are handle by theGPUProcessInfo, theGPUProcessesInfogroups all processes. In other words,GPUProcessesInfolists all processes andGPUProcessInfohas the attributes of each process.To access the process list, the user need to use theprocessesproperty ofGPUInfo. This property also has a implicit string conversion:Processes List String Conversion>>>print(gpu_info.processes)PROCESSESPID|NAME|USER|PARENT|CREATIONTIME|GPUMEM4276|python|antonio|45436|'2020-04-17 03:41:27'|42845764|python|acnazarejr|5759|'2020-04-16 17:57:12'|6515Once theprocessesproperty is a list ofGPUProcessesInfo, the user can access each process individually, using their attributes.Attributespid(int) - The process PID.name(str) - The process name.user(str) - The name of the user that owns the process.parent_pid(int) - The process parent PID.parent_name(str) - The process parent name.create_time(str) - The process creation time as a floating point number expressed in seconds since the epoch, in UTC.gpu_memory(int) - The amount of GPU memory allocated by the process.UsageFor example, for the first process in the list.>>>gpu.processes[0].pid5764>>>gpu.processes[0].name'python'>>>gpu.processes[0].user'acnazarejr'>>>gpu.processes[0].parent_pid5759>>>gpu.processes[0].parent_name'slurmstepd'>>>gpu.processes[0].create_time'2020-04-16 17:57:12'>>>gpu.processes[0].gpu_memory10789String Conversion>>>print(gpu_info.processes[0])5764|python|acnazarejr|5759|'2020-04-16 17:57:12'|6515LicenseSeeLICENSE.