Repository: numba/numba

Issue:
CUDA: remove context query in launch configuration
There's a call made to query the device context every time a kernel is configured for launch as `config.CUDA_LOW_OCCUPANCY_WARNINGS` is on by default.
https://github.com/numba/numba/blob/fa3630edab447f38f81c97cbd943809ebe7a9840/numba/cuda/dispatcher.py#L453-L462

from OOB discussion with @gmarkall, this task is to explore removing this as it's a debug/help message that impacts the performance of the most common use case. Replacement would be to instead just produce the message for suspiciously low grid sizes (based on some arbitrary value) without the context query. Another alternative is to have a `config.CUDA_PRODUCTION` flag to disable the check (and other future similar ones) in production.


Discussion:

Resolving patch:
diff --git a/numba/cuda/dispatcher.py b/numba/cuda/dispatcher.py
--- a/numba/cuda/dispatcher.py
+++ b/numba/cuda/dispatcher.py
@@ -459,14 +459,19 @@ def __init__(self, dispatcher, griddim, blockdim, stream, sharedmem):
         self.sharedmem = sharedmem
 
         if config.CUDA_LOW_OCCUPANCY_WARNINGS:
-            ctx = get_context()
-            smcount = ctx.device.MULTIPROCESSOR_COUNT
+            # Warn when the grid has fewer than 128 blocks. This number is
+            # chosen somewhat heuristically - ideally the minimum is 2 times
+            # the number of SMs, but the number of SMs varies between devices -
+            # some very small GPUs might only have 4 SMs, but an H100-SXM5 has
+            # 132. In general kernels should be launched with large grids
+            # (hundreds or thousands of blocks), so warning when fewer than 128
+            # blocks are used will likely catch most beginner errors, where the
+            # grid tends to be very small (single-digit or low tens of blocks).
+            min_grid_size = 128
             grid_size = griddim[0] * griddim[1] * griddim[2]
-            if grid_size < 2 * smcount:
-                msg = ("Grid size ({grid}) < 2 * SM count ({sm}) "
-                       "will likely result in GPU under utilization due "
-                       "to low occupancy.")
-                msg = msg.format(grid=grid_size, sm=2 * smcount)
+            if grid_size < min_grid_size:
+                msg = (f"Grid size {grid_size} will likely result in GPU"
+                       "under-utilization due to low occupancy.")
                 warn(NumbaPerformanceWarning(msg))
 
     def __call__(self, *args):


Test patch:
diff --git a/numba/cuda/tests/cudapy/test_gufunc.py b/numba/cuda/tests/cudapy/test_gufunc.py
--- a/numba/cuda/tests/cudapy/test_gufunc.py
+++ b/numba/cuda/tests/cudapy/test_gufunc.py
@@ -206,8 +206,7 @@ def numba_dist_cuda(a, b, dist):
                 numba_dist_cuda(a, b, dist)
                 self.assertEqual(w[0].category, NumbaPerformanceWarning)
                 self.assertIn('Grid size', str(w[0].message))
-                self.assertIn('2 * SM count',
-                              str(w[0].message))
+                self.assertIn('low occupancy', str(w[0].message))
 
     def test_efficient_launch_configuration(self):
         @guvectorize(['void(float32[:], float32[:], float32[:])'],
diff --git a/numba/cuda/tests/cudapy/test_warning.py b/numba/cuda/tests/cudapy/test_warning.py
--- a/numba/cuda/tests/cudapy/test_warning.py
+++ b/numba/cuda/tests/cudapy/test_warning.py
@@ -19,7 +19,7 @@ def kernel():
 
         self.assertEqual(w[0].category, NumbaPerformanceWarning)
         self.assertIn('Grid size', str(w[0].message))
-        self.assertIn('2 * SM count', str(w[0].message))
+        self.assertIn('low occupancy', str(w[0].message))
 
     def test_efficient_launch_configuration(self):
         @cuda.jit