Repository: numba/numba

Issue:
CUDA JIT errors when multiple signatures provided
With Numba's CPU JIT, one is able to supply multiple signatures (e.g. single and double precision floating values) for a function. However when trying the same with Numba's CUDA JIT, this results in an error. Limiting the CUDA JIT to a single signature fixes the issue. Code snippet demonstrating this below.

<details>
<summary>Example:</summary>

```python
In [1]: from numba import jit

In [2]: from numba.cuda import jit as cuda_jit

In [3]: @jit(["f4(f4, f4)", "f8(f8, f8)"])
   ...: def f(a, b):
   ...:     return a + b
   ...: 
   ...: 

In [4]: @cuda_jit(["f4(f4, f4)", "f8(f8, f8)"])
   ...: def g(a, b):
   ...:     return a + b
   ...: 
   ...: 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-5a86fdfdf6d9> in <module>()
----> 1 @cuda_jit(["f4(f4, f4)", "f8(f8, f8)"])
      2 def g(a, b):
      3     return a + b

~/.conda/envs/numba/lib/python3.6/site-packages/numba/cuda/compiler.py in __call__(self, *args)
    700         Specialize and invoke this kernel with *args*.
    701         '''
--> 702         kernel = self.specialize(*args)
    703         cfg = kernel[self.griddim, self.blockdim, self.stream, self.sharedmem]
    704         cfg(*args)

~/.conda/envs/numba/lib/python3.6/site-packages/numba/cuda/compiler.py in specialize(self, *args)
    710         '''
    711         argtypes = tuple(
--> 712             [self.typingctx.resolve_argument_type(a) for a in args])
    713         kernel = self.compile(argtypes)
    714         return kernel

~/.conda/envs/numba/lib/python3.6/site-packages/numba/cuda/compiler.py in <listcomp>(.0)
    710         '''
    711         argtypes = tuple(
--> 712             [self.typingctx.resolve_argument_type(a) for a in args])
    713         kernel = self.compile(argtypes)
    714         return kernel

~/.conda/envs/numba/lib/python3.6/site-packages/numba/typing/context.py in resolve_argument_type(self, val)
    288         ValueError is raised for unsupported types.
    289         """
--> 290         return typeof(val, Purpose.argument)
    291 
    292     def resolve_value_type(self, val):

~/.conda/envs/numba/lib/python3.6/site-packages/numba/typing/typeof.py in typeof(val, purpose)
     31         msg = errors.termcolor.errmsg(
     32             "cannot determine Numba type of %r") % (type(val),)
---> 33         raise ValueError(msg)
     34     return ty
     35 

ValueError: cannot determine Numba type of <class 'function'>
```

</details>


Discussion:
Thanks for the report. It seems that this is the case, lists of signatures are not handled, we should either raise with a more specific error message or just fix the problem!

It's also worth noting from your example that `@cuda.jit` functions are GPU kernel functions and should be `void` [by definition](http://numba.pydata.org/numba-doc/dev/cuda/kernels.html#kernel-declaration).

Also, might be worth looking at `@vectorize` and `@guvectorize`, both of which support the kwarg `target='cuda'` (and multiple signatures). Here's a quick example of both ways:

```python
from numba.cuda import jit as cuda_jit
from numba import cuda, vectorize
import numpy as np

@cuda_jit("void(f8[:], f8)")
def g(a, b):
    tx = cuda.threadIdx.x
    ty = cuda.blockIdx.x
    bw = cuda.blockDim.x
    pos = tx + ty * bw
    if pos < a.size:
        a[pos] += b

@vectorize(['f4(f4, f4)', 'f8(f8, f8)'], target='cuda')
def vec_g(a, b):
    return a + b

x = np.arange(64.)
y = 17

g[1, 64](x, y)
print(x)

x = np.arange(64.)
print(vec_g(x, y))
```
It appears that there may be cases when kernels with signatures specified don't catch type errors as they should do, e.g.: https://github.com/numba/numba/pull/5388#issuecomment-603226418 - when looking into adding support for multiple signatures, type checking should also be checked / considered.
It would be **really** cool if this behavior of `cuda.jit` was (explicitly) documented ;) I have just burned a couple of hours on this, thinking that I made an utterly stupid mistake, until I ended up digging into the sources and issues ... In the meantime, some more useful exceptions could also help. When passing a list of strings, `cuda.jit` currently complains: `[...] is not a callable object` ... which kind of makes sense but is inconsistent with `numba.jit` and does not tell the actual story: Only one signature allowed.

Just out of interest: What are the odds (or technical feasibility) for `cuda.jit` to support multiple signatures just like `numba.jit`?
> Just out of interest: What are the odds (or technical feasibility) for `cuda.jit` to support multiple signatures just like `numba.jit`?

Hopefully much better than they were when this issue was first reported, now that the CUDA Dispatcher uses a lot more of the infrastructure that the CPU target does for dispatch. I'll look into this and see what the gaps are.

Resolving patch:
diff --git a/numba/cuda/decorators.py b/numba/cuda/decorators.py
--- a/numba/cuda/decorators.py
+++ b/numba/cuda/decorators.py
@@ -14,21 +14,22 @@
 def jit(func_or_sig=None, device=False, inline=False, link=[], debug=None,
         opt=True, cache=False, **kws):
     """
-    JIT compile a python function conforming to the CUDA Python specification.
-    If a signature is supplied, then a function is returned that takes a
-    function to compile.
+    JIT compile a Python function for CUDA GPUs.
 
-    :param func_or_sig: A function to JIT compile, or a signature of a function
-       to compile. If a function is supplied, then a
-       :class:`numba.cuda.compiler.AutoJitCUDAKernel` is returned. If a
-       signature is supplied, then a function is returned. The returned
-       function accepts another function, which it will compile and then return
-       a :class:`numba.cuda.compiler.AutoJitCUDAKernel`.
+    :param func_or_sig: A function to JIT compile, or *signatures* of a
+       function to compile. If a function is supplied, then a
+       :class:`Dispatcher <numba.cuda.dispatcher.CUDADispatcher>` is returned.
+       Otherwise, ``func_or_sig`` may be a signature or a list of signatures,
+       and a function is returned. The returned function accepts another
+       function, which it will compile and then return a :class:`Dispatcher
+       <numba.cuda.dispatcher.CUDADispatcher>`. See :ref:`jit-decorator` for
+       more information about passing signatures.
 
        .. note:: A kernel cannot have any return value.
     :param device: Indicates whether this is a device function.
     :type device: bool
-    :param link: A list of files containing PTX source to link with the function
+    :param link: A list of files containing PTX or CUDA C/C++ source to link
+       with the function
     :type link: list
     :param debug: If True, check for exceptions thrown when executing the
        kernel. Since this degrades performance, this should only be used for
@@ -83,16 +84,20 @@ def jit(func_or_sig=None, device=False, inline=False, link=[], debug=None,
         raise ValueError("link keyword invalid for device function")
 
     if sigutils.is_signature(func_or_sig):
+        signatures = [func_or_sig]
+        specialized = True
+    elif isinstance(func_or_sig, list):
+        signatures = func_or_sig
+        specialized = False
+    else:
+        signatures = None
+
+    if signatures is not None:
         if config.ENABLE_CUDASIM:
             def jitwrapper(func):
                 return FakeCUDAKernel(func, device=device, fastmath=fastmath)
             return jitwrapper
 
-        argtypes, restype = sigutils.normalize_signature(func_or_sig)
-
-        if restype and not device and restype != types.void:
-            raise TypeError("CUDA kernel must have void return type.")
-
         def _jit(func):
             targetoptions = kws.copy()
             targetoptions['debug'] = debug
@@ -107,14 +112,20 @@ def _jit(func):
             if cache:
                 disp.enable_caching()
 
-            if device:
-                from numba.core import typeinfer
-                with typeinfer.register_dispatcher(disp):
-                    disp.compile_device(argtypes)
-            else:
-                disp.compile(argtypes)
+            for sig in signatures:
+                argtypes, restype = sigutils.normalize_signature(sig)
+
+                if restype and not device and restype != types.void:
+                    raise TypeError("CUDA kernel must have void return type.")
+
+                if device:
+                    from numba.core import typeinfer
+                    with typeinfer.register_dispatcher(disp):
+                        disp.compile_device(argtypes)
+                else:
+                    disp.compile(argtypes)
 
-            disp._specialized = True
+            disp._specialized = specialized
             disp.disable_compile()
 
             return disp
diff --git a/numba/cuda/simulator/api.py b/numba/cuda/simulator/api.py
--- a/numba/cuda/simulator/api.py
+++ b/numba/cuda/simulator/api.py
@@ -7,7 +7,7 @@
 from contextlib import contextmanager
 from .cudadrv.devices import require_context, reset, gpus  # noqa: F401
 from .kernel import FakeCUDAKernel
-from numba.core.typing import Signature
+from numba.core.sigutils import is_signature
 from warnings import warn
 from ..args import In, Out, InOut  # noqa: F401
 
@@ -85,9 +85,11 @@ def jit(func_or_sig=None, device=False, debug=False, argtypes=None,
 
     if link is not None:
         raise NotImplementedError('Cannot link PTX in the simulator')
+
     # Check for first argument specifying types - in that case the
     # decorator is not being passed a function
-    if func_or_sig is None or isinstance(func_or_sig, (str, tuple, Signature)):
+    if (func_or_sig is None or is_signature(func_or_sig)
+            or isinstance(func_or_sig, list)):
         def jitwrapper(fn):
             return FakeCUDAKernel(fn,
                                   device=device,
diff --git a/numba/cuda/simulator/kernel.py b/numba/cuda/simulator/kernel.py
--- a/numba/cuda/simulator/kernel.py
+++ b/numba/cuda/simulator/kernel.py
@@ -1,5 +1,5 @@
 from contextlib import contextmanager
-from functools import reduce
+import functools
 import sys
 import threading
 
@@ -25,7 +25,7 @@ def _push_kernel_context(mod):
     Push the current kernel context.
     """
     global _kernel_context
-    assert _kernel_context is None, "conrrent simulated kernel not supported"
+    assert _kernel_context is None, "concurrent simulated kernel not supported"
     _kernel_context = mod
     try:
         yield
@@ -74,6 +74,7 @@ def __init__(self, fn, device, fastmath=False, extensions=[], debug=False):
         self.block_dim = None
         self.stream = 0
         self.dynshared_size = 0
+        functools.update_wrapper(self, fn)
 
     def __call__(self, *args):
         if self._device:
@@ -93,7 +94,7 @@ def __call__(self, *args):
 
             def fake_arg(arg):
                 # map the arguments using any extension you've registered
-                _, arg = reduce(
+                _, arg = functools.reduce(
                     lambda ty_val, extension: extension.prepare_args(
                         *ty_val,
                         stream=0,


Test patch:
diff --git a/numba/cuda/tests/cudapy/test_dispatcher.py b/numba/cuda/tests/cudapy/test_dispatcher.py
--- a/numba/cuda/tests/cudapy/test_dispatcher.py
+++ b/numba/cuda/tests/cudapy/test_dispatcher.py
@@ -1,7 +1,8 @@
 import numpy as np
 import threading
 
-from numba import boolean, cuda, float32, float64, int32, int64, void
+from numba import boolean, config, cuda, float32, float64, int32, int64, void
+from numba.core.errors import TypingError
 from numba.cuda.testing import skip_on_cudasim, unittest, CUDATestCase
 import math
 
@@ -14,8 +15,8 @@ def add_kernel(r, x, y):
     r[0] = x + y
 
 
-@skip_on_cudasim('Dispatcher objects not used in the simulator')
-class TestDispatcher(CUDATestCase):
+@skip_on_cudasim('Specialization not implemented in the simulator')
+class TestDispatcherSpecialization(CUDATestCase):
     def _test_no_double_specialize(self, dispatcher, ty):
 
         with self.assertRaises(RuntimeError) as e:
@@ -105,7 +106,9 @@ def f(x, y):
         self.assertEqual(len(f.specializations), 2)
         self.assertIs(f_f32c_f32c, f_f32c_f32c_2)
 
-    # The following tests are based on those in numba.tests.test_dispatcher
+
+class TestDispatcher(CUDATestCase):
+    """Most tests based on those in numba.tests.test_dispatcher."""
 
     def test_coerce_input_types(self):
         # Do not allow unsafe conversions if we can still compile other
@@ -135,6 +138,7 @@ def test_coerce_input_types(self):
         c_add[1, 1](r, 123, 456)
         self.assertPreciseEqual(r[0], add(123, 456))
 
+    @skip_on_cudasim('Simulator ignores signature')
     @unittest.expectedFailure
     def test_coerce_input_types_unsafe(self):
         # Implicit (unsafe) conversion of float to int, originally from
@@ -151,6 +155,7 @@ def test_coerce_input_types_unsafe(self):
         c_add[1, 1](r, 12.3, 45.6)
         self.assertPreciseEqual(r[0], add(12, 45))
 
+    @skip_on_cudasim('Simulator ignores signature')
     def test_coerce_input_types_unsafe_complex(self):
         # Implicit conversion of complex to int disallowed
         c_add = cuda.jit('(i4[::1], i4, i4)')(add_kernel)
@@ -159,6 +164,7 @@ def test_coerce_input_types_unsafe_complex(self):
         with self.assertRaises(TypeError):
             c_add[1, 1](r, 12.3, 45.6j)
 
+    @skip_on_cudasim('Simulator does not track overloads')
     def test_ambiguous_new_version(self):
         """Test compiling new version in an ambiguous case
         """
@@ -187,6 +193,7 @@ def test_ambiguous_new_version(self):
         self.assertEqual(len(c_add.overloads), 4, "didn't compile a new "
                                                   "version")
 
+    @skip_on_cudasim("Simulator doesn't support concurrent kernels")
     def test_lock(self):
         """
         Test that (lazy) compiling from several threads at once doesn't
@@ -213,6 +220,238 @@ def wrapper():
             t.join()
         self.assertFalse(errors)
 
+    def _test_explicit_signatures(self, sigs):
+        f = cuda.jit(sigs)(add_kernel)
+
+        # Exact signature matches
+        r = np.zeros(1, dtype=np.int64)
+        f[1, 1](r, 1, 2)
+        self.assertPreciseEqual(r[0], 3)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, 1.5, 2.5)
+        self.assertPreciseEqual(r[0], 4.0)
+
+        if config.ENABLE_CUDASIM:
+            # Pass - we can't check for no conversion on the simulator.
+            return
+
+        # No conversion
+        with self.assertRaises(TypeError) as cm:
+            r = np.zeros(1, dtype=np.complex128)
+            f[1, 1](r, 1j, 1j)
+        self.assertIn("No matching definition", str(cm.exception))
+        self.assertEqual(len(f.overloads), 2, f.overloads)
+
+    def test_explicit_signatures_strings(self):
+        # Check with a list of strings for signatures
+        sigs = ["(int64[::1], int64, int64)",
+                "(float64[::1], float64, float64)"]
+        self._test_explicit_signatures(sigs)
+
+    def test_explicit_signatures_tuples(self):
+        # Check with a list of tuples of argument types for signatures
+        sigs = [(int64[::1], int64, int64), (float64[::1], float64, float64)]
+        self._test_explicit_signatures(sigs)
+
+    def test_explicit_signatures_signatures(self):
+        # Check with a list of Signature objects for signatures
+        sigs = [void(int64[::1], int64, int64),
+                void(float64[::1], float64, float64)]
+        self._test_explicit_signatures(sigs)
+
+    def test_explicit_signatures_mixed(self):
+        # Check when we mix types of signature objects in a list of signatures
+
+        # Tuple and string
+        sigs = [(int64[::1], int64, int64),
+                "(float64[::1], float64, float64)"]
+        self._test_explicit_signatures(sigs)
+
+        # Tuple and Signature object
+        sigs = [(int64[::1], int64, int64),
+                void(float64[::1], float64, float64)]
+        self._test_explicit_signatures(sigs)
+
+        # Signature object and string
+        sigs = [void(int64[::1], int64, int64),
+                "(float64[::1], float64, float64)"]
+        self._test_explicit_signatures(sigs)
+
+    def test_explicit_signatures_same_type_class(self):
+        # A more interesting one...
+        # (Note that the type of r is deliberately float64 in both cases so
+        # that dispatch is differentiated on the types of x and y only, to
+        # closely preserve the intent of the original test from
+        # numba.tests.test_dispatcher)
+        sigs = ["(float64[::1], float32, float32)",
+                "(float64[::1], float64, float64)"]
+        f = cuda.jit(sigs)(add_kernel)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, np.float32(1), np.float32(2**-25))
+        self.assertPreciseEqual(r[0], 1.0)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, 1, 2**-25)
+        self.assertPreciseEqual(r[0], 1.0000000298023224)
+
+    @skip_on_cudasim('No overload resolution in the simulator')
+    def test_explicit_signatures_ambiguous_resolution(self):
+        # Fail to resolve ambiguity between the two best overloads
+        # (Also deliberate float64[::1] for the first argument in all cases)
+        f = cuda.jit(["(float64[::1], float32, float64)",
+                      "(float64[::1], float64, float32)",
+                      "(float64[::1], int64, int64)"])(add_kernel)
+        with self.assertRaises(TypeError) as cm:
+            r = np.zeros(1, dtype=np.float64)
+            f[1, 1](r, 1.0, 2.0)
+
+        # The two best matches are output in the error message, as well
+        # as the actual argument types.
+        self.assertRegexpMatches(
+            str(cm.exception),
+            r"Ambiguous overloading for <function add_kernel [^>]*> "
+            r"\(array\(float64, 1d, C\), float64, float64\):\n"
+            r"\(array\(float64, 1d, C\), float32, float64\) -> none\n"
+            r"\(array\(float64, 1d, C\), float64, float32\) -> none"
+        )
+        # The integer signature is not part of the best matches
+        self.assertNotIn("int64", str(cm.exception))
+
+    @skip_on_cudasim('Simulator does not use _prepare_args')
+    @unittest.expectedFailure
+    def test_explicit_signatures_unsafe(self):
+        # These tests are from test_explicit_signatures, but have to be xfail
+        # at present because _prepare_args in the CUDA target cannot handle
+        # unsafe conversions of arguments.
+        f = cuda.jit("(int64[::1], int64, int64)")(add_kernel)
+        r = np.zeros(1, dtype=np.int64)
+
+        # Approximate match (unsafe conversion)
+        f[1, 1](r, 1.5, 2.5)
+        self.assertPreciseEqual(r[0], 3)
+        self.assertEqual(len(f.overloads), 1, f.overloads)
+
+        sigs = ["(int64[::1], int64, int64)",
+                "(float64[::1], float64, float64)"]
+        f = cuda.jit(sigs)(add_kernel)
+        r = np.zeros(1, dtype=np.float64)
+        # Approximate match (int32 -> float64 is a safe conversion)
+        f[1, 1](r, np.int32(1), 2.5)
+        self.assertPreciseEqual(r[0], 3.5)
+
+    def add_device_usecase(self, sigs):
+        # Generate a kernel that calls the add device function compiled with a
+        # given set of signatures
+        add_device = cuda.jit(sigs, device=True)(add)
+
+        @cuda.jit
+        def f(r, x, y):
+            r[0] = add_device(x, y)
+
+        return f
+
+    def test_explicit_signatures_device(self):
+        # Tests similar to test_explicit_signatures, but on a device function
+        # instead of a kernel
+        sigs = ["(int64, int64)", "(float64, float64)"]
+        f = self.add_device_usecase(sigs)
+
+        # Exact signature matches
+        r = np.zeros(1, dtype=np.int64)
+        f[1, 1](r, 1, 2)
+        self.assertPreciseEqual(r[0], 3)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, 1.5, 2.5)
+        self.assertPreciseEqual(r[0], 4.0)
+
+        if config.ENABLE_CUDASIM:
+            # Pass - we can't check for no conversion on the simulator.
+            return
+
+        # No conversion
+        with self.assertRaises(TypingError) as cm:
+            r = np.zeros(1, dtype=np.complex128)
+            f[1, 1](r, 1j, 1j)
+
+        msg = str(cm.exception)
+        self.assertIn("Invalid use of type", msg)
+        self.assertIn("with parameters (complex128, complex128)", msg)
+        self.assertEqual(len(f.overloads), 2, f.overloads)
+
+    def test_explicit_signatures_device_same_type_class(self):
+        # A more interesting one...
+        # (Note that the type of r is deliberately float64 in both cases so
+        # that dispatch is differentiated on the types of x and y only, to
+        # closely preserve the intent of the original test from
+        # numba.tests.test_dispatcher)
+        sigs = ["(float32, float32)", "(float64, float64)"]
+        f = self.add_device_usecase(sigs)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, np.float32(1), np.float32(2**-25))
+        self.assertPreciseEqual(r[0], 1.0)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, 1, 2**-25)
+        self.assertPreciseEqual(r[0], 1.0000000298023224)
+
+    def test_explicit_signatures_device_ambiguous(self):
+        # Ambiguity between the two best overloads resolves. This is somewhat
+        # surprising given that ambiguity is not permitted for dispatching
+        # overloads when launching a kernel, but seems to be the general
+        # behaviour of Numba (See Issue #8307:
+        # https://github.com/numba/numba/issues/8307).
+        sigs = ["(float32, float64)", "(float64, float32)", "(int64, int64)"]
+        f = self.add_device_usecase(sigs)
+
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, 1.5, 2.5)
+        self.assertPreciseEqual(r[0], 4.0)
+
+    @skip_on_cudasim('CUDA Simulator does not force casting')
+    def test_explicit_signatures_device_unsafe(self):
+        # These tests are from test_explicit_signatures. The device function
+        # variant of these tests can succeed on CUDA because the compilation
+        # can handle unsafe casting (c.f. test_explicit_signatures_unsafe which
+        # has to xfail due to _prepare_args not supporting unsafe casting).
+        sigs = ["(int64, int64)"]
+        f = self.add_device_usecase(sigs)
+
+        # Approximate match (unsafe conversion)
+        r = np.zeros(1, dtype=np.int64)
+        f[1, 1](r, 1.5, 2.5)
+        self.assertPreciseEqual(r[0], 3)
+        self.assertEqual(len(f.overloads), 1, f.overloads)
+
+        sigs = ["(int64, int64)", "(float64, float64)"]
+        f = self.add_device_usecase(sigs)
+
+        # Approximate match (int32 -> float64 is a safe conversion)
+        r = np.zeros(1, dtype=np.float64)
+        f[1, 1](r, np.int32(1), 2.5)
+        self.assertPreciseEqual(r[0], 3.5)
+
+    def test_dispatcher_docstring(self):
+        # Ensure that CUDA-jitting a function preserves its docstring. See
+        # Issue #5902: https://github.com/numba/numba/issues/5902
+
+        @cuda.jit
+        def add_kernel(a, b):
+            """Add two integers, kernel version"""
+
+        @cuda.jit(device=True)
+        def add_device(a, b):
+            """Add two integers, device version"""
+
+        self.assertEqual("Add two integers, kernel version", add_kernel.__doc__)
+        self.assertEqual("Add two integers, device version", add_device.__doc__)
+
+
+@skip_on_cudasim("CUDA simulator doesn't implement kernel properties")
+class TestDispatcherKernelProperties(CUDATestCase):
     def test_get_regs_per_thread_unspecialized(self):
         # A kernel where the register usage per thread is likely to differ
         # between different specializations
@@ -434,21 +673,6 @@ def simple_lmem(ary):
         self.assertIsInstance(local_mem_per_thread, int)
         self.assertGreaterEqual(local_mem_per_thread, N * 4)
 
-    def test_dispatcher_docstring(self):
-        # Ensure that CUDA-jitting a function preserves its docstring. See
-        # Issue #5902: https://github.com/numba/numba/issues/5902
-
-        @cuda.jit
-        def add_kernel(a, b):
-            """Add two integers, kernel version"""
-
-        @cuda.jit(device=True)
-        def add_device(a, b):
-            """Add two integers, device version"""
-
-        self.assertEqual("Add two integers, kernel version", add_kernel.__doc__)
-        self.assertEqual("Add two integers, device version", add_device.__doc__)
-
 
 if __name__ == '__main__':
     unittest.main()