Repository: DataDog/integrations-extras

Issue:
[NVML] Nvml integration call the deprecated API (nvmlDeviceGetComputeRunningProcesses_v2) in laetest pynvml dependency.
The latest NVML integration (v.1.0.8) encounters API call issues related to API calls that are deprecated in the latest version of Pynvml.
NVML integration is written based on pynvml 11.4.1, but the pip3 install command example in the installation documentation will always install the latest version of the pynvml package.
Nvidia updated pynvml to 11.5.0 and changed the version of the API calls with v3, and the v2 API calls are no longer possible with this library.
We can simply call the version-compatible function for compatibility in other pynvml version.

- https://github.com/DataDog/integrations-extras/blob/817420c250b360c45ee05e5b342a5b766905fc91/nvml/datadog_checks/nvml/nvml.py#L175

Pynvml has functions, `nvmlDeviceGetComputeRunningProcesses` and `nvmlDeviceGetComputeRunningProcesses_vX`. 
They made the version compatibility method call `nvmlDeviceGetComputeRunningProcesses` to call the latest version of the API in the pynvml.
**_In fact, Nvidia made the change the version suffix in the function name without updating the logic._**

We can simply fix the issue by calling `nvmlDeviceGetComputeRunningProcesses` method rather than use the api call with specific version for compatibility with all pynvml version.

```
    nvml (1.0.8)
    ------------
      Instance ID: nvml:a6050e4fc38cd06 [ERROR]
      Configuration Source: file:/etc/datadog-agent/conf.d/nvml.d/conf.yaml
      Total Runs: 42
      Metric Samples: Last Run: 14, Total: 588
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 297ms
      Last Execution Date : 2023-10-30 06:57:23 UTC (1698649043000)
      Last Successful Execution Date : Never
      Error: module 'pynvml' has no attribute 'nvmlDeviceGetComputeRunningProcesses_v2'
      Traceback (most recent call last):
        File "/opt/datadog-agent/embedded/lib/python3.9/site-packages/datadog_checks/base/checks/base.py", line 1235, in run
          self.check(instance)
        File "/opt/datadog-agent/embedded/lib/python3.9/site-packages/datadog_checks/nvml/nvml.py", line 103, in check
          self.gather(instance)
        File "/opt/datadog-agent/embedded/lib/python3.9/site-packages/datadog_checks/nvml/nvml.py", line 116, in gather
          self.gather_gpu(handle, tags)
        File "/opt/datadog-agent/embedded/lib/python3.9/site-packages/datadog_checks/nvml/nvml.py", line 175, in gather_gpu
          compute_running_processes = NvmlCheck.N.nvmlDeviceGetComputeRunningProcesses_v2(handle)
      AttributeError: module 'pynvml' has no attribute 'nvmlDeviceGetComputeRunningProcesses_v2'
```


Pynvml has `nvmlDeviceGetComputeRunningProcesses_v2` and `nvmlDeviceGetComputeRunningProcesses` less than v.11.4.1.

Latest Pynvml (v.11.5.0) removed the `nvmlDeviceGetComputeRunningProcesses_v2` call and replace it to `v3`  in `nvml.py` file.

Nvidia always keep the `nvmlDeviceGetComputeRunningProcesses` function for compatibility in every pynvml version.

https://github.com/gpuopenanalytics/pynvml/blob/41e1657948b18008d302f5cb8af06539adc7c792/pynvml/nvml.py#L2187-L2223C1

**pynvml 11.4.1.**
```python
# Added in 2.285
def nvmlDeviceGetComputeRunningProcesses_v2(handle):
    # first call to get the size
    c_count = c_uint(0)
    fn = _nvmlGetFunctionPointer("nvmlDeviceGetComputeRunningProcesses_v2")
    ret = fn(handle, byref(c_count), None)

    if (ret == NVML_SUCCESS):
        # special case, no running processes
        return []
    elif (ret == NVML_ERROR_INSUFFICIENT_SIZE):
        # typical case
        # oversize the array incase more processes are created
        c_count.value = c_count.value * 2 + 5
        proc_array = c_nvmlProcessInfo_t * c_count.value
        c_procs = proc_array()

        # make the call again
        ret = fn(handle, byref(c_count), c_procs)
        _nvmlCheckReturn(ret)

        procs = []
        for i in range(c_count.value):
            # use an alternative struct for this object
            obj = nvmlStructToFriendlyObject(c_procs[i])
            if (obj.usedGpuMemory == NVML_VALUE_NOT_AVAILABLE_ulonglong.value):
                # special case for WDDM on Windows, see comment above
                obj.usedGpuMemory = None
            procs.append(obj)

        return procs
    else:
        # error case
        raise NVMLError(ret)

def nvmlDeviceGetComputeRunningProcesses(handle):
    return nvmlDeviceGetComputeRunningProcesses_v2(handle);
```

**pynvml v.1.15.0** 
https://github.com/gpuopenanalytics/pynvml/blob/43a7803c42358a87e765bb66373f3ae536d589a3/pynvml/nvml.py#L2573-L2608

```python
# Added in 2.285
def nvmlDeviceGetComputeRunningProcesses_v3(handle):
    # first call to get the size
    c_count = c_uint(0)
    fn = _nvmlGetFunctionPointer("nvmlDeviceGetComputeRunningProcesses_v3")
    ret = fn(handle, byref(c_count), None)

    if (ret == NVML_SUCCESS):
        # special case, no running processes
        return []
    elif (ret == NVML_ERROR_INSUFFICIENT_SIZE):
        # typical case
        # oversize the array incase more processes are created
        c_count.value = c_count.value * 2 + 5
        proc_array = c_nvmlProcessInfo_t * c_count.value
        c_procs = proc_array()

        # make the call again
        ret = fn(handle, byref(c_count), c_procs)
        _nvmlCheckReturn(ret)

        procs = []
        for i in range(c_count.value):
            # use an alternative struct for this object
            obj = nvmlStructToFriendlyObject(c_procs[i])
            if (obj.usedGpuMemory == NVML_VALUE_NOT_AVAILABLE_ulonglong.value):
                # special case for WDDM on Windows, see comment above
                obj.usedGpuMemory = None
            procs.append(obj)

        return procs
    else:
        # error case
        raise NVMLError(ret)

def nvmlDeviceGetComputeRunningProcesses(handle):
    return nvmlDeviceGetComputeRunningProcesses_v3(handle);
```



Discussion:

Resolving patch:
diff --git a/nvml/datadog_checks/nvml/__about__.py b/nvml/datadog_checks/nvml/__about__.py
--- a/nvml/datadog_checks/nvml/__about__.py
+++ b/nvml/datadog_checks/nvml/__about__.py
@@ -1,4 +1,4 @@
 # (C) Datadog, Inc. 2020-present
 # All rights reserved
 # Licensed under a 3-clause BSD style license (see LICENSE)
-__version__ = '1.0.8'
+__version__ = '1.0.9'
diff --git a/nvml/datadog_checks/nvml/nvml.py b/nvml/datadog_checks/nvml/nvml.py
--- a/nvml/datadog_checks/nvml/nvml.py
+++ b/nvml/datadog_checks/nvml/nvml.py
@@ -172,7 +172,7 @@ def gather_gpu(self, handle, tags):
             self.gauge('fan_speed', fan_speed, tags=tags)
 
         with NvmlCall("compute_running_processes", self.log):
-            compute_running_processes = NvmlCheck.N.nvmlDeviceGetComputeRunningProcesses_v2(handle)
+            compute_running_processes = NvmlCheck.N.nvmlDeviceGetComputeRunningProcesses(handle)
             for compute_running_process in compute_running_processes:
                 self.gauge(
                     'compute_running_process',


Test patch:
diff --git a/nvml/tests/Dockerfile b/nvml/tests/Dockerfile
--- a/nvml/tests/Dockerfile
+++ b/nvml/tests/Dockerfile
@@ -1,13 +1,13 @@
 # As the NVML integration is a community integration, it is not packaged with the Agent image by default. A custom image with the packaged integration should therefore be used in containerized environments.
 # Setup taken from https://docs.datadoghq.com/agent/guide/use-community-integrations/?tab=docker
 
-# Installs the NVML integration version 1.0.8.
+# Installs the NVML integration version 1.0.9.
 # See the changelog for the other versions that are available (https://github.com/DataDog/integrations-extras/blob/master/nvml/CHANGELOG.md)
 FROM gcr.io/datadoghq/agent:latest
-RUN agent integration install -t -r datadog-nvml==1.0.8
+RUN agent integration install -t -r datadog-nvml==1.0.9
 
 # Use versioned link to the requirements to install the PIP libraries used by the check
-RUN curl https://raw.githubusercontent.com/DataDog/integrations-extras/nvml-1.0.8/nvml/requirements.in > /tmp/requirements.in && /opt/datadog-agent/embedded/bin/pip3 install -r /tmp/requirements.in
+RUN curl https://raw.githubusercontent.com/DataDog/integrations-extras/nvml-1.0.9/nvml/requirements.in > /tmp/requirements.in && /opt/datadog-agent/embedded/bin/pip3 install -r /tmp/requirements.in
 
 # Why do you need these variables: See https://github.com/NVIDIA/nvidia-docker/wiki/Usage
 ENV NVIDIA_VISIBLE_DEVICES all
diff --git a/nvml/tests/test_nvml.py b/nvml/tests/test_nvml.py
--- a/nvml/tests/test_nvml.py
+++ b/nvml/tests/test_nvml.py
@@ -76,7 +76,7 @@ def nvmlDeviceGetFanSpeed(h):
         return 14
 
     @staticmethod
-    def nvmlDeviceGetComputeRunningProcesses_v2(h):
+    def nvmlDeviceGetComputeRunningProcesses(h):
         Mock = namedtuple('Mock', ['pid', 'usedGpuMemory'])
         return [Mock(pid=1, usedGpuMemory=11)]