Repository: numba/numba

Issue:
Caching of functions with keyword arguments
## Reporting a bug

- [X] I have tried using the latest released version of Numba (most recent is
 visible in the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [X] I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

Using keyword arguments somehow results in the cache being recreated for each process.

Using the following script:

```python
from numba import njit
import numpy as np


@njit(cache=True)
def example(x, y, eps=1e-12):
    result = x + y
    result[result < eps] = 0
    return result


if __name__ == '__main__':
    rng = np.random.default_rng(0)
    x = rng.normal(0, 0.1, 50)
    y = rng.normal(0, 0.1, len(x))
    print(example(x, y))
```

Results in this behaviour when called multiple times:

```
❯ NUMBA_DEBUG_CACHE=1 python fitting.py  | grep saved
[cache] index saved to '__pycache__/fitting.example-5.py38.nbi'
[cache] data saved to '__pycache__/fitting.example-5.py38.1.nbc'
❯ NUMBA_DEBUG_CACHE=1 python fitting.py  | grep saved
[cache] index saved to '__pycache__/fitting.example-5.py38.nbi'
[cache] data saved to '__pycache__/fitting.example-5.py38.2.nbc'
❯ NUMBA_DEBUG_CACHE=1 python fitting.py  | grep saved
[cache] index saved to '__pycache__/fitting.example-5.py38.nbi'
[cache] data saved to '__pycache__/fitting.example-5.py38.3.nbc'
```

Observe that the cache is written each time, with an increasing number in the filename. This created problems when running a numba function on a cluster, generating multiple GB of caching data.


Removing the keyword argument solves the issue:

```
from numba import njit
import numpy as np


@njit(cache=True)
def example(x, y):
    eps = 1e-12
    result = x + y
    result[result < eps] = 0
    return result


if __name__ == '__main__':
    rng = np.random.default_rng(0)
    x = rng.normal(0, 0.1, 50)
    y = rng.normal(0, 0.1, len(x))
    print(example(x, y))

```

Observe no "saved" output on second and third run:

```
❯ NUMBA_DEBUG_CACHE=1 python fitting.py  | grep saved
[cache] index saved to '__pycache__/fitting.example-5.py38.nbi'
[cache] data saved to '__pycache__/fitting.example-5.py38.1.nbc'
❯ NUMBA_DEBUG_CACHE=1 python fitting.py  | grep saved
❯ NUMBA_DEBUG_CACHE=1 python fitting.py  | grep saved
```



Discussion:
Actually using the optional argument also solves the caching issue:

```
print(example(x, y, eps=1e-12))
```
After some debugging, it turns out that the `Omitted` type uses the `id` (memory address) as part of it's `key` and  effectively the `id` becomes part of the `hash` -- so that means, that the hash of the argument is different on every instantiation (But only when the keyword argument is omitted, denoted by the `Omitted` type.,

The offending line is:

https://github.com/numba/numba/blob/master/numba/core/types/misc.py#L91

We (the Numba community)  discussed this today at length, and we are not sure yet what to do about it or how to fix this.
> After some debugging, it turns out that the `Omitted` type uses the `id` (memory address) as part of it's `key` and effectively the `id` becomes part of the `hash` -- so that means, that the hash of the argument is different on every instantiation (But only when the keyword argument is omitted, denoted by the `Omitted` type.,
> 
> The offending line is:
> 
> https://github.com/numba/numba/blob/master/numba/core/types/misc.py#L91
> 
> We (the Numba community) discussed this today at length, and we are not sure yet what to do about it or how to fix this.

@esc could you point me to the meeting notes where this issue was discussed or expand on why `id` is necessary? I'd like to work on this issue if possible.
These are the minutes for the meeting, but there are no specific notes on this particular issue: https://github.com/numba/numba/wiki/Minutes_2021_04_27
A comment with some thoughts about the same fix for a related issue: https://github.com/numba/numba/issues/6173#issuecomment-850831761
> A comment with some thoughts about the same fix for a related issue: [#6173 (comment)](https://github.com/numba/numba/issues/6173#issuecomment-850831761)

This comment (and the followup) discuss removing `id(self._value)`, but it seems to me that just `id` should be removed. Is there a good reason we can't just use `self._value`? Rerunning your example:

```python
import numba

@numba.njit
def func(x, y=1.0, z=1):
    return x+x + y + z
print(func(2.2))

@numba.njit
def func2(x, y=1.0, z=1):
    return x+x + y + z
print(func2(2.2))

@numba.njit
def func3(x, y=2.0, z=4):
    return x+x + y + z
print(func3(2.2))

@numba.njit
def func4(x, y=2.0):
    return x+x + y
print(func4(2.2))
```
I stilll get the output:
```python
6.4
6.4
10.4
6.4
```

You cannot guarantee the type is hashable.
```python
import numba

@numba.njit
def func(x, y=[1, 2]):
    return x + y[0]

print(func(2.2))
```
will fail if the key contains the value.
@stuartarchibald Is this really a case for consideration? Mutable default arguments are a "nogo" in python and immutable types generally are hashable.

It seems strange to me to break the normal usecase (immutable, hashable default values) for something that is universally discouraged in the language.

I think it would be a very valid decision for numba to not allow unhashable default parameters.
It might be the only solution for this problem.
> @stuartarchibald Is this really a case for consideration? Mutable default arguments are a "nogo" in python and immutable types generally are hashable.
> 
> It seems strange to me to break the normal usecase (immutable, hashable default values) for something that is universally discouraged in the language.
> 
> I think it would be a very valid decision for numba to not allow unhashable default parameters.
> It might be the only solution for this problem.

A fair point.
> @stuartarchibald Is this really a case for consideration? Mutable default arguments are a "nogo" in python and immutable types generally are hashable.

It was the most trivial thing to demonstrate the issue in `nopython` mode, indeed, don't use mutable default args :-)

> immutable types generally are hashable.

It might be this that needs to be enforced as a requirement to create the necessary guarantees.

I do wonder what would happen in objmode where the hash of the key might involve calling hash on an object which inherits its `__hash__` from `object` itself and so ends up calling `id`. I think this lands us back at the same problem but there's now something that can be done in code controlled by the user to fix it (implement the `__hash__`).
> @stuartarchibald Is this really a case for consideration? Mutable default arguments are a "nogo" in python and immutable types generally are hashable.
> 
> It seems strange to me to break the normal usecase (immutable, hashable default values) for something that is universally discouraged in the language.
> 
> I think it would be a very valid decision for numba to not allow unhashable default parameters.
> It might be the only solution for this problem.

I think we could just provide separate implementations for both situations. Something like this:

```python

class Omitted(Opaque):
    """
    An omitted function argument with a default value.
    """

    def __init__(self, value):
        self._value = value
        self._hash_value = value if hasattr(value, "__hash__") and value.__hash__ is not None else id(value)
        super(Omitted, self).__init__("omitted(default=%r)" % (value,))

    @property
    def key(self):
        return type(self._value), self._hash_value

    @property
    def value(self):
        return self._value
```



> I do wonder what would happen in objmode where the hash of the key might involve calling hash on an object which inherits its `__hash__` from `object` itself and so ends up calling `id`. I think this lands us back at the same problem but there's now something that can be done in code controlled by the user to fix it (implement the `__hash__`).

@stuartarchibald the changes I'm proposing are fairly simple and I'm happy to submit a PR. If you are worried about this use case could you provide me a example code segment that I could test this behavior on?


Resolving patch:
diff --git a/numba/core/types/abstract.py b/numba/core/types/abstract.py
--- a/numba/core/types/abstract.py
+++ b/numba/core/types/abstract.py
@@ -5,7 +5,7 @@
 
 import numpy as np
 
-from numba.core.utils import cached_property
+from numba.core.utils import cached_property, get_hashable_key
 
 # Types are added to a global registry (_typecache) in order to assign
 # them unique integer codes for fast matching in _dispatcher.c.
@@ -424,12 +424,7 @@ def _literal_init(self, value):
         self._literal_value = value
         # We want to support constants of non-hashable values, therefore
         # fall back on the value's id() if necessary.
-        try:
-            hash(value)
-        except TypeError:
-            self._key = id(value)
-        else:
-            self._key = value
+        self._key = get_hashable_key(value)
 
     @property
     def literal_value(self):
diff --git a/numba/core/types/misc.py b/numba/core/types/misc.py
--- a/numba/core/types/misc.py
+++ b/numba/core/types/misc.py
@@ -3,6 +3,8 @@
                                      SimpleIteratorType)
 from numba.core.typeconv import Conversion
 from numba.core.errors import TypingError, LiteralTypingError
+from numba.core.ir import UndefinedType
+from numba.core.utils import get_hashable_key
 
 
 class PyObject(Dummy):
@@ -87,11 +89,14 @@ class Omitted(Opaque):
 
     def __init__(self, value):
         self._value = value
+        # Use helper function to support both hashable and non-hashable
+        # values. See discussion in gh #6957.
+        self._value_key = get_hashable_key(value)
         super(Omitted, self).__init__("omitted(default=%r)" % (value,))
 
     @property
     def key(self):
-        return type(self._value), id(self._value)
+        return type(self._value), self._value_key
 
     @property
     def value(self):
diff --git a/numba/core/utils.py b/numba/core/utils.py
--- a/numba/core/utils.py
+++ b/numba/core/utils.py
@@ -636,3 +636,19 @@ def __init__(self, old_module_locals, new_module):
     def __reduce__(self):
         args = (self.__old_module_states, self.__new_module)
         return _RedirectSubpackage, args
+
+
+def get_hashable_key(value):
+    """
+        Given a value, returns a key that can be used
+        as a hash. If the value is hashable, we return
+        the value, otherwise we return id(value).
+
+        See discussion in gh #6957
+    """
+    try:
+        hash(value)
+    except TypeError:
+        return id(value)
+    else:
+        return value


Test patch:
diff --git a/numba/tests/test_caching.py b/numba/tests/test_caching.py
--- a/numba/tests/test_caching.py
+++ b/numba/tests/test_caching.py
@@ -1,8 +1,12 @@
+import multiprocessing as mp
+import traceback
 from numba import njit
 from numba.tests.support import (
     TestCase,
     SerialMixin,
-    run_in_new_process_caching
+    override_config,
+    run_in_new_process_caching,
+    temp_directory,
 )
 
 
@@ -43,3 +47,73 @@ def test_constant_unicode_cache(self):
 
     def test_dict_cache(self):
         self.run_test(check_dict_cache)
+
+    def test_omitted(self):
+
+        # Test in a new directory
+        cache_dir = temp_directory(self.__class__.__name__)
+        ctx = mp.get_context()
+        result_queue = ctx.Queue()
+        proc = ctx.Process(
+            target=omitted_child_test_wrapper,
+            args=(result_queue, cache_dir, False),
+        )
+        proc.start()
+        proc.join()
+        success, output = result_queue.get()
+
+        # Ensure the child process is completed before checking its output
+        if not success:
+            self.fail(output)
+
+        self.assertEqual(
+            output,
+            1000,
+            "Omitted function returned an incorrect output"
+        )
+
+        proc = ctx.Process(
+            target=omitted_child_test_wrapper,
+            args=(result_queue, cache_dir, True)
+        )
+        proc.start()
+        proc.join()
+        success, output = result_queue.get()
+
+        # Ensure the child process is completed before checking its output
+        if not success:
+            self.fail(output)
+
+        self.assertEqual(
+            output,
+            1000,
+            "Omitted function returned an incorrect output"
+        )
+
+
+def omitted_child_test_wrapper(result_queue, cache_dir, second_call):
+    with override_config("CACHE_DIR", cache_dir):
+        @njit(cache=True)
+        def test(num=1000):
+            return num
+
+        try:
+            output = test()
+            # If we have a second call, we should have a cache hit.
+            # Otherwise, we expect a cache miss.
+            if second_call:
+                assert test._cache_hits[test.signatures[0]] == 1, \
+                    "Cache did not hit as expected"
+                assert test._cache_misses[test.signatures[0]] == 0, \
+                    "Cache has an unexpected miss"
+            else:
+                assert test._cache_misses[test.signatures[0]] == 1, \
+                    "Cache did not miss as expected"
+                assert test._cache_hits[test.signatures[0]] == 0, \
+                    "Cache has an unexpected hit"
+            success = True
+        # Catch anything raised so it can be propagated
+        except: # noqa: E722
+            output = traceback.format_exc()
+            success = False
+        result_queue.put((success, output))