Repository: numba/numba

Issue:
Can't access Typed Dict with (int32, int32) keys in the python interpreter
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

For more general "how do I do X?" type questions, please speak to us in real
time on https://gitter.im/numba/numba or post to the Numba mailing list
https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

<!--

Please include details of the bug here, including, if applicable, what you
expected to happen!

-->

<details>
<summary> output of numba -s </summary>

```bash
$ numba -s
System info:
--------------------------------------------------------------------------------
__Time Stamp__
2019-07-11 09:37:23.968590

__Hardware Information__
Machine                                       : x86_64
CPU Name                                      : skylake
Number of accessible CPU cores                : 8
Listed accessible CPUs cores                  : 0-7
CFS restrictions                              : None
CPU Features                                  : 
64bit adx aes avx avx2 bmi bmi2 clflushopt cmov cx16 f16c fma fsgsbase invpcid
lzcnt mmx movbe pclmul popcnt prfchw rdrnd rdseed rtm sahf sgx sse sse2 sse3
sse4.1 sse4.2 ssse3 xsave xsavec xsaveopt xsaves

__OS Information__
Platform                                      : Linux-4.18.0-25-generic-x86_64-with-Ubuntu-18.04-bionic
Release                                       : 4.18.0-25-generic
System Name                                   : Linux
Version                                       : #26~18.04.1-Ubuntu SMP Thu Jun 27 07:28:31 UTC 2019
OS specific info                              : Ubuntu18.04bionic
glibc info                                    : glibc 2.15

__Python Information__
Python Compiler                               : GCC 5.4.0 20160609
Python Implementation                         : CPython
Python Version                                : 2.7.16
Python Locale                                 : en_GB UTF-8

__LLVM information__
LLVM version                                  : 8.0.0

__CUDA Information__
Error: CUDA device intialisation problem. Error at driver init: 
[100] Call to cuInit results in CUDA_ERROR_NO_DEVICE:
Error class: <class 'numba.cuda.cudadrv.error.CudaSupportError'>

__ROC Information__
ROC available                                 : False
Error initialising ROC due to                 : No ROC toolchains found.
No HSA Agents found, encountered exception when searching:
Error at driver init: 
NUMBA_HSA_DRIVER /opt/rocm/lib/libhsa-runtime64.so is not a valid file path.  Note it must be a filepath of the .so/.dll/.dylib or the driver:

__SVML Information__
SVML state, config.USING_SVML                 : False
SVML library found and loaded                 : False
llvmlite using SVML patched LLVM              : True
SVML operational                              : False

__Threading Layer Information__
TBB Threading layer available                 : True
OpenMP Threading layer available              : False
+--> Disabled due to                          : Unknown import problem.
Workqueue Threading layer available           : True

__Numba Environment Variable Information__
None set.

__Conda Information__
Conda not present/not working.
Error was [Errno 2] No such file or directory

--------------------------------------------------------------------------------
If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.
=============================================================
```

</details>

I created a jitted numba function that created a typed dict keyed on an (int32, int32) tuple. numba successfully creates and returns this dict, but there seems to a problem when accessing the dict within the python interpreter. However, a (int64, int64) tuple works successfully


```python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

from pprint import pprint

import numba
import numpy as np


@numba.njit(nogil=True, cache=True)
def fn(ant1, ant2, data):
    data_avg = {}

    for r in range(ant1.shape[0]):
        a1 = ant1[r]
        a2 = ant2[r]

        key = (a1, a2)

        if key in data_avg:
            avg = data_avg[key]
            avg += data[r]
        else:
            avg = data.dtype.type(data[r])
            data_avg[key] = avg

    return data_avg


# 64 bit indices in ant1 + ant2, this is fine
ant1, ant2 = np.triu_indices(7, 1)
data = np.random.random(ant1.shape)
res = fn(ant1, ant2, data)

for k, v in res.items():
    print(k, v)

# 32 bit indices in ant1 + ant2, this fails!
ant1, ant2 = (a.astype(np.int32) for a in np.triu_indices(7, 1))
data = np.random.random(ant1.shape)
res = fn(ant1, ant2, data)

for k, v in res.items():
    print(k, v)
```

It produces the following sort of traceback:

```traceback
$ python test_numba_int64.py 
(0, 1) 0.53562473169
(0, 2) 0.323425964623
(0, 3) 0.602906503761
(0, 4) 0.0706116917503
(0, 5) 0.698970661917
(0, 6) 0.598562358806
(1, 2) 0.488224070915
(1, 3) 0.641514625944
(1, 4) 0.356949910527
(1, 5) 0.68163141825
(1, 6) 0.702238307709
(2, 3) 0.524306389349
(2, 4) 0.259019818589
(2, 5) 0.454954563449
(2, 6) 0.887542275029
(3, 4) 0.252880237227
(3, 5) 0.43829949257
(3, 6) 0.985091314436
(4, 5) 0.433945981699
(4, 6) 0.185182054301
(5, 6) 0.25504427879
Traceback (most recent call last):
  File "test_numba_int64.py", line 42, in <module>
    for k, v in res.items():
  File "/home/sperkins/venv/xms-plots/lib/python2.7/_abcoll.py", line 414, in items
    return [(key, self[key]) for key in self]
  File "/home/sperkins/venv/xms-plots/local/lib/python2.7/site-packages/numba/typed/typeddict.py", line 137, in __getitem__
    return _getitem(self, key)
  File "/home/sperkins/venv/xms-plots/local/lib/python2.7/site-packages/numba/dispatcher.py", line 351, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/sperkins/venv/xms-plots/local/lib/python2.7/site-packages/numba/dispatcher.py", line 318, in error_rewrite
    reraise(type(e), e, None)
  File "<string>", line 2, in reraise
numba.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<intrinsic _cast>) with argument(s) of type(s): (tuple(int64 x 2), typeref[tuple(int32 x 2)])
 * parameterized
In definition 0:
    TypingError: cannot safely cast tuple(int64 x 2) to tuple(int32 x 2)
In definition 1:
    TypingError: cannot safely cast tuple(int64 x 2) to tuple(int32 x 2)
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: resolving callee type: Function(<intrinsic _cast>)
[2] During: typing of call at /home/sperkins/venv/xms-plots/local/lib/python2.7/site-packages/numba/dictobject.py (873)


File "../../../venv/xms-plots/local/lib/python2.7/site-packages/numba/dictobject.py", line 873:
    def impl(d, key):
        castedkey = _cast(key, keyty)
        ^

[1] During: typing of intrinsic-call at /home/sperkins/venv/xms-plots/local/lib/python2.7/site-packages/numba/typed/typeddict.py (35)

File "../../../venv/xms-plots/local/lib/python2.7/site-packages/numba/typed/typeddict.py", line 35:
def _getitem(d, key):
    return d[key]
    ^

This is not usually a problem with Numba itself but instead often caused by
the use of unsupported features or an issue in resolving types.

To see Python/NumPy features supported by the latest release of Numba visit:
http://numba.pydata.org/numba-doc/dev/reference/pysupported.html
and
http://numba.pydata.org/numba-doc/dev/reference/numpysupported.html

For more information about typing errors and how to debug them visit:
http://numba.pydata.org/numba-doc/latest/user/troubleshoot.html#my-code-doesn-t-compile

If you think your code should work with Numba, please report the error message
and traceback, along with a minimal reproducer at:
https://github.com/numba/numba/issues/new

```


Discussion:
Thanks for the report, I can reproduce. A minimal reproducer is:
```python
from numba import types, typed
import numpy as np

res = typed.Dict.empty(types.UniTuple(types.int32, 2), types.float64)
res[(np.int32(1), np.int32(1))] = 1.10

# problem is here...
print(res.keys())
```

Resolving patch:
diff --git a/numba/errors.py b/numba/errors.py
--- a/numba/errors.py
+++ b/numba/errors.py
@@ -65,6 +65,12 @@ class NumbaParallelSafetyWarning(NumbaWarning):
     might not have parallel semantics.
     """
 
+
+class NumbaTypeSafetyWarning(NumbaWarning):
+    """
+    Warning category for unsafe casting operations.
+    """
+
 # These are needed in the color formatting of errors setup
 
 
diff --git a/numba/typedobjectutils.py b/numba/typedobjectutils.py
--- a/numba/typedobjectutils.py
+++ b/numba/typedobjectutils.py
@@ -1,6 +1,7 @@
 """ Common compiler level utilities for typed dict and list. """
 
 import operator
+import warnings
 
 from llvmlite import ir
 from llvmlite.llvmpy.core import Builder
@@ -11,7 +12,7 @@
 from numba.targets.registry import cpu_target
 from numba.typeconv import Conversion
 from numba.extending import intrinsic
-from numba.errors import TypingError
+from numba.errors import TypingError, NumbaTypeSafetyWarning
 
 
 def _as_bytes(builder, ptr):
@@ -39,18 +40,38 @@ def _sentry_safe_cast(fromty, toty):
     """Check and raise TypingError if *fromty* cannot be safely cast to *toty*
     """
     tyctxt = cpu_target.typing_context
+    fromty, toty = map(types.unliteral, (fromty, toty))
     by = tyctxt.can_convert(fromty, toty)
+
+    def warn():
+        m = 'unsafe cast from {} to {}. Precision may be lost.'
+        warnings.warn(m.format(fromty, toty),
+                      category=NumbaTypeSafetyWarning)
+
+    isint = lambda x: isinstance(x, types.Integer)
+    isflt = lambda x: isinstance(x, types.Float)
+    iscmplx = lambda x: isinstance(x, types.Complex)
+    # Only check against numeric types.
     if by is None or by > Conversion.safe:
-        if isinstance(fromty, types.Integer) and isinstance(toty, types.Integer):
+        if isint(fromty) and isint(toty):
             # Accept if both types are ints
-            return
-        if isinstance(fromty, types.Integer) and isinstance(toty, types.Float):
+            warn()
+        elif isint(fromty) and isflt(toty):
             # Accept if ints to floats
-            return
-        if isinstance(fromty, types.Float) and isinstance(toty, types.Float):
+            warn()
+        elif isflt(fromty) and isflt(toty):
             # Accept if floats to floats
-            return
-        raise TypingError('cannot safely cast {} to {}'.format(fromty, toty))
+            warn()
+        elif iscmplx(fromty) and iscmplx(toty):
+            # Accept if complex to complex
+            warn()
+        elif not isinstance(toty, types.Number):
+            # Non-numbers
+            warn()
+        else:
+            # Make it a hard error for numeric type that changes domain.
+            m = 'cannot safely cast {} to {}. Please cast explicitly.'
+            raise TypingError(m.format(fromty, toty))
 
 
 def _sentry_safe_cast_default(default, valty):


Test patch:
diff --git a/numba/tests/support.py b/numba/tests/support.py
--- a/numba/tests/support.py
+++ b/numba/tests/support.py
@@ -18,6 +18,7 @@
 import io
 import ctypes
 import multiprocessing as mp
+from contextlib import contextmanager
 
 import numpy as np
 
@@ -26,6 +27,7 @@
 from numba.targets import cpu
 import numba.unittest_support as unittest
 from numba.runtime import rtsys
+from numba.six import PY2
 
 
 enable_pyobj_flags = Flags()
@@ -423,6 +425,14 @@ def run_nullary_func(self, pyfunc, flags):
         self.assertPreciseEqual(got, expected)
         return got, expected
 
+    if PY2:
+        @contextmanager
+        def subTest(self, *args, **kwargs):
+            """A stub TestCase.subTest backport.
+            This implementation is a no-op.
+            """
+            yield
+
 
 class SerialMixin(object):
     """Mixin to mark test for serial execution.
diff --git a/numba/tests/test_dictobject.py b/numba/tests/test_dictobject.py
--- a/numba/tests/test_dictobject.py
+++ b/numba/tests/test_dictobject.py
@@ -8,6 +8,7 @@
 from __future__ import print_function, absolute_import, division
 
 import sys
+import warnings
 
 import numpy as np
 
@@ -1336,6 +1337,30 @@ def foo(k, h, v):
             str(raises.exception),
         )
 
+    def test_conflict_key_type_non_number(self):
+        # Allow non-number types to cast unsafely
+        @njit
+        def foo(k1, v1, k2):
+            d = Dict()
+            d[k1] = v1
+            return d, d[k2]
+
+        # k2 will unsafely downcast typeof(k1)
+        k1 = (np.int8(1), np.int8(2))
+        k2 = (np.int32(1), np.int32(2))
+        v1 = np.intp(123)
+
+        with warnings.catch_warnings(record=True) as w:
+            d, dk2 = foo(k1, v1, k2)
+        self.assertEqual(len(w), 1)
+        # Make sure the warning is about unsafe cast
+        self.assertIn('unsafe cast from tuple(int32 x 2) to tuple(int8 x 2)',
+                      str(w[0]))
+
+        keys = list(d.keys())
+        self.assertEqual(keys[0], (1, 2))
+        self.assertEqual(dk2, d[(np.int32(1), np.int32(2))])
+
     def test_ifelse_filled_both_branches(self):
         @njit
         def foo(k, v):
diff --git a/numba/tests/test_listobject.py b/numba/tests/test_listobject.py
--- a/numba/tests/test_listobject.py
+++ b/numba/tests/test_listobject.py
@@ -1451,6 +1451,6 @@ def foo():
         with self.assertRaises(TypingError) as raises:
             foo()
         self.assertIn(
-            'cannot safely cast int32 to unicode_type',
+            'Cannot cast int32 to unicode_type',
             str(raises.exception),
         )
diff --git a/numba/tests/test_typedobjectutils.py b/numba/tests/test_typedobjectutils.py
new file mode 100644
--- /dev/null
+++ b/numba/tests/test_typedobjectutils.py
@@ -0,0 +1,69 @@
+"""
+Unit-tests for `typedobjectutils.py`
+"""
+from __future__ import print_function, absolute_import, division
+
+import warnings
+
+
+from numba import types
+from .support import TestCase
+from numba.typedobjectutils import _sentry_safe_cast
+
+
+class TestTypedObjectUtils(TestCase):
+    def test_sentry_safe_cast_warnings(self):
+        warn_cases = []
+        warn_cases += [
+            # integer cases
+            (types.int32, types.int16),
+            (types.int32, types.uint32),
+            (types.int64, types.uint32),
+            # float cases
+            (types.float64, types.float32),
+            # complex cases
+            (types.complex128, types.complex64),
+            # int to float cases
+            (types.int32, types.float32),
+            (types.int64, types.float32),
+            # tuple-of-ints to tuple-of-floats,
+            (types.Tuple([types.int32]), types.Tuple([types.float32])),
+        ]
+
+        for fromty, toty in warn_cases:
+            with self.subTest(fromty=fromty, toty=toty):
+                with warnings.catch_warnings(record=True) as w:
+                    _sentry_safe_cast(fromty, toty)
+                self.assertEqual(len(w), 1)
+                # Make sure the warning is about unsafe cast
+                self.assertIn(
+                    "unsafe cast from {} to {}".format(fromty, toty),
+                    str(w[0]),
+                )
+
+    def test_sentry_safe_cast_no_warn(self):
+        ok_cases = []
+        ok_cases += [
+            # integer cases
+            (types.int32, types.int64),
+            (types.uint8, types.int32),
+            # float cases
+            (types.float32, types.float64),
+            # complex cases
+            (types.complex64, types.complex128),
+            # int to float cases
+            (types.int32, types.float64),
+            (types.uint8, types.float32),
+            # float to complex cases
+            (types.float32, types.complex128),
+            (types.float64, types.complex128),
+            # tuple-of-ints to tuple-of-ints,
+            (types.Tuple([types.int32]), types.Tuple([types.int64])),
+        ]
+
+        for fromty, toty in ok_cases:
+            with self.subTest(fromty=fromty, toty=toty):
+                with warnings.catch_warnings(record=True) as w:
+                    _sentry_safe_cast(fromty, toty)
+                # Expect no warnings
+                self.assertEqual(len(w), 0)