internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: numba/numba

Issue:
cache=True & generators?
reproducing code:
``` python
import numba
import numpy as np

@numba.njit(cache=True)
def _thisfunc(vals):
    for v in vals:
        yield v

# calling code:
_thisfunc(np.random.rand(500))

```
results in: 
```
C:\Anaconda3\lib\site-packages\numba\caching.py in _dump(self, obj)
    563 
    564     def _dump(self, obj):
--> 565         return pickle.dumps(obj, protocol=-1)
    566 
    567     @contextlib.contextmanager

PicklingError: Can't pickle <function _thisfunc at 0x0000014AF82B4D08>: it's not the same object as __main__._thisfunc
```
but with `cache=False` the function works. Is there any workaround for this?



Discussion:
Thanks for the report.
```python
from numba import njit
@njit(cache=True)
def f():
    yield 10
f()
```
is a minimal reproducer.

Think this is a bug.
Pickle can't deal with generator objects, might have something to do with it.
@Rik-de-Kort It is officially stated that `yield` is supported, but the above code still gives me error.
This issue has been opened relatively long ago, and it looks like a genuine bug.
At the same time, `yield` constructs are now fully supported by numba.
Is there a timescale for fixing this?
> This issue has been opened relatively long ago, and it looks like a genuine bug. At the same time, `yield` constructs are now fully supported by numba. Is there a timescale for fixing this?

@mtazzari thank you for asking about this, I just double checked on current master and the issue still persists. There is currently no timescale or ETA for this being fixed.
Maybe @sklam knows more, I know there has been recent work on integrating cloud-pickle.
cloudpickle should make it work.  let me check

Resolving patch:
diff --git a/numba/core/caching.py b/numba/core/caching.py
--- a/numba/core/caching.py
+++ b/numba/core/caching.py
@@ -560,7 +560,7 @@ def _data_path(self, name):
         return os.path.join(self._cache_path, name)
 
     def _dump(self, obj):
-        return pickle.dumps(obj, protocol=-1)
+        return dumps(obj)
 
     @contextlib.contextmanager
     def _open_for_write(self, filepath):


Test patch:
diff --git a/numba/tests/test_caching.py b/numba/tests/test_caching.py
--- a/numba/tests/test_caching.py
+++ b/numba/tests/test_caching.py
@@ -36,6 +36,19 @@ def check_dict_cache():
     assert exp == got
 
 
+def generator_cache():
+    for v in (1, 2, 3):
+        yield v
+
+
+def check_generator_cache():
+    pyfunc = generator_cache
+    cfunc = njit(cache=True)(pyfunc)
+    exp = list(pyfunc())
+    got = list(cfunc())
+    assert exp == got
+
+
 class TestCaching(SerialMixin, TestCase):
     def run_test(self, func):
         func()
@@ -48,6 +61,9 @@ def test_constant_unicode_cache(self):
     def test_dict_cache(self):
         self.run_test(check_dict_cache)
 
+    def test_generator_cache(self):
+        self.run_test(check_generator_cache)
+
     def test_omitted(self):
 
         # Test in a new directory
Repository: numba/numba

Issue:
Cache temp filename not unique across nodes
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

-->

## Feature request

In numba/core/caching.py, tmpname is supposed to be unique to avoid race conditions.
Please change this to use uuid.uuid1() instead of os.getpid().  I'm using numba in either
a slurm or ray cluster where jobs on different nodes may have the same pid.

Thanks, Mark W

    def _open_for_write(self, filepath):
         tmpname = '%s.tmp.%d' % (filepath, os.getpid().)

should be:

    def _open_for_write(self, filepath):
         tmpname = '%s.tmp.%s' % (filepath, uuid.uuid1())




Discussion:
Thanks for the report. Just to check... Is the issue that within the compute nodes of your cluster some of the processes running Numba have PIDs which collide and they are also all writing to a shared storage resource, e.g. NFS mounted home directory or similar? As a result two or more processes might be writing to e.g. `<shared storage>/some_filepath/something.tmp.<pid>` at the same time?

The code in question is:
https://github.com/numba/numba/blob/637a75023bae889467ebfc0e8489be547b62a209/numba/core/caching.py#L565-L582
Were this to be replaced to be `uuid` based, I think it should probably be `uuid4` as `uuid1` can call `fork(2)` which can lead to other problems (xref https://github.com/numba/numba/issues/7881). I think such a change is reasonable to prevent problems such as that described in the OP.
That is exactly the problem.  I saw this occasionally on our Slurm cluster but I see it all the time with ray autoscaling clusters.
Its exacerbated by the short life of the worker nodes, which start at the same time and have low PIDs for all jobs.
UUID4 would be fine, and you could probably get away with just using socket.gethostname() and PID.
@markweissman Many thanks for confirming, I think this is a bug as writing to a shared cache location is potentially broken in a such an environment (potentially quite common in HPC use cases). A patch to fix would be welcomed.

Resolving patch:
diff --git a/numba/core/caching.py b/numba/core/caching.py
--- a/numba/core/caching.py
+++ b/numba/core/caching.py
@@ -13,6 +13,7 @@
 import pickle
 import sys
 import tempfile
+import uuid
 import warnings
 
 from numba.misc.appdirs import AppDirs
@@ -565,10 +566,10 @@ def _dump(self, obj):
     @contextlib.contextmanager
     def _open_for_write(self, filepath):
         """
-        Open *filepath* for writing in a race condition-free way
-        (hopefully).
+        Open *filepath* for writing in a race condition-free way (hopefully).
+        uuid4 is used to try and avoid name collisions on a shared filesystem.
         """
-        tmpname = '%s.tmp.%d' % (filepath, os.getpid())
+        tmpname = '%s.tmp.%d' % (filepath, uuid.uuid4())
         try:
             with open(tmpname, "wb") as f:
                 yield f


Test patch:
Repository: numba/numba

Issue:
Windows caching tests failing with FileNotFoundError
```
numba.tests.test_extending.TestCachingOverloadObjmode.test_caching_overload_objmode
numba.tests.test_extending.TestCachingOverloadObjmode.test_check_objmode_cache_ndarray
```

These start to fail at merge of #7965 (e0a73c1). Most likely reason is that the path is too long.



```
numba.tests.test_withlifting.TestLiftObjCaching.test_objmode_caching_call_closure_bad
```
also fails in the buildfarm but cannot be replicated outside of the full run yet.


Discussion:

Resolving patch:
diff --git a/numba/core/caching.py b/numba/core/caching.py
--- a/numba/core/caching.py
+++ b/numba/core/caching.py
@@ -555,7 +555,8 @@ def _open_for_write(self, filepath):
         Open *filepath* for writing in a race condition-free way (hopefully).
         uuid4 is used to try and avoid name collisions on a shared filesystem.
         """
-        tmpname = '%s.tmp.%d' % (filepath, uuid.uuid4())
+        uid = uuid.uuid4().hex[:16]  # avoid long paths
+        tmpname = '%s.tmp.%s' % (filepath, uid)
         try:
             with open(tmpname, "wb") as f:
                 yield f
@@ -684,6 +685,8 @@ def _index_key(self, sig, codegen):
         codebytes = self._py_func.__code__.co_code
         if self._py_func.__closure__ is not None:
             cvars = tuple([x.cell_contents for x in self._py_func.__closure__])
+            # Note: cloudpickle serializes a function differently depending
+            #       on how the process is launched; e.g. multiprocessing.Process
             cvarbytes = dumps(cvars)
         else:
             cvarbytes = b''


Test patch:
diff --git a/numba/tests/test_extending.py b/numba/tests/test_extending.py
--- a/numba/tests/test_extending.py
+++ b/numba/tests/test_extending.py
@@ -1922,6 +1922,10 @@ def test_caching():
             raise AssertionError("incorrect result")
         return test_caching
 
+    @classmethod
+    def populate_objmode_cache_ndarray_check_cache(cls):
+        cls.check_objmode_cache_ndarray()
+
     @classmethod
     def check_objmode_cache_ndarray_check_cache(cls):
         disp = cls.check_objmode_cache_ndarray()
@@ -1935,8 +1939,10 @@ def test_check_objmode_cache_ndarray(self):
         # Env is missing after cache load.
         cache_dir = temp_directory(self.__class__.__name__)
         with override_config("CACHE_DIR", cache_dir):
-            # Test in local process to populate the cache.
-            self.check_objmode_cache_ndarray()
+            # Run in new process to populate the cache
+            run_in_new_process_in_cache_dir(
+                self.populate_objmode_cache_ndarray_check_cache, cache_dir
+            )
             # Run in new process to use the cache in a fresh process.
             res = run_in_new_process_in_cache_dir(
                 self.check_objmode_cache_ndarray_check_cache, cache_dir
Repository: numba/numba

Issue:
vectorize docstring example has incorrect identity
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

-->

## Reporting a bug

This is a documentation bug more than a functionality bug, so reproducibility does not exactly apply.

The docstring example in `vectorize` provides a ufunc of `sum` that has `identity=1`.  I believe this should be `0` (additive identity):

https://github.com/numba/numba/blob/15e48788660f5c387d8d1160cc4be5d2363510fc/numba/np/ufunc/decorators.py#L100-L103

The later example for `mul` correctly sets `identity=1`.

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I have tried using the latest released version of Numba (most recent is
 visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
- [ ] ~I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.~ N/A

<!--

Please include details of the bug here, including, if applicable, what you
expected to happen!

-->



Discussion:
@bmcfee thank you for bringing this to our attention. This seems correct as the identity for addition is indeed `0`. 


Resolving patch:
diff --git a/numba/np/ufunc/decorators.py b/numba/np/ufunc/decorators.py
--- a/numba/np/ufunc/decorators.py
+++ b/numba/np/ufunc/decorators.py
@@ -58,7 +58,7 @@ def __new__(cls, func, signature, **kws):
 def vectorize(ftylist_or_function=(), **kws):
     """vectorize(ftylist_or_function=(), target='cpu', identity=None, **kws)
 
-    A decorator that creates a Numpy ufunc object using Numba compiled
+    A decorator that creates a NumPy ufunc object using Numba compiled
     code.  When no arguments or only keyword arguments are given,
     vectorize will return a Numba dynamic ufunc (DUFunc) object, where
     compilation/specialization may occur at call-time.
@@ -98,7 +98,7 @@ def vectorize(ftylist_or_function=(), **kws):
     Examples
     -------
         @vectorize(['float32(float32, float32)',
-                    'float64(float64, float64)'], identity=1)
+                    'float64(float64, float64)'], identity=0)
         def sum(a, b):
             return a + b
 


Test patch:
Repository: numba/numba

Issue:
slow performance observed in np.sort() with nan arrays
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I have tried using the latest released version of Numba (most recent is
 visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
- [x] I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

As title.
```python
from numba import njit
import numpy as np

@njit
def test(a):
    return np.sort(a)

arr = np.full(100_000, np.nan)
# compile it
test(arr[:10])

%%time
test(arr)
# CPU times: user 3.81 s, sys: 40.4 ms, total: 3.85 s
# Wall time: 3.82 s
# array([nan, nan, nan, ..., nan, nan, nan])

%%time
test.py_func(arr)
# CPU times: user 5.55 ms, sys: 0 ns, total: 5.55 ms
# Wall time: 4.17 ms
# array([nan, nan, nan, ..., nan, nan, nan])

```



Discussion:
Thanks for the report and nice reproducer - I can reproduce the observed performance.
The problem is NaN specific. We might need a specialized comparison for float types following what numpy did in https://github.com/numpy/numpy/commit/717c7acf14d04be79d7a1c098d83cac69481e0af

Resolving patch:
diff --git a/numba/np/arrayobj.py b/numba/np/arrayobj.py
--- a/numba/np/arrayobj.py
+++ b/numba/np/arrayobj.py
@@ -5665,7 +5665,11 @@ def impl(ary, indices_or_sections, axis=0):
 
 
 def lt_floats(a, b):
-    return math.isnan(b) or a < b
+    # Adapted from NumPy commit 717c7acf which introduced the behavior of
+    # putting NaNs at the end.
+    # The code is later moved to numpy/core/src/npysort/npysort_common.h
+    # This info is gathered as of NumPy commit d8c09c50
+    return a < b or (np.isnan(b) and not np.isnan(a))
 
 
 def get_sort_func(kind, is_float, is_argsort=False):


Test patch:
diff --git a/numba/tests/test_sort.py b/numba/tests/test_sort.py
--- a/numba/tests/test_sort.py
+++ b/numba/tests/test_sort.py
@@ -831,6 +831,11 @@ def float_arrays(self):
             orig = np.random.random(size=size) * 100
             orig[np.random.random(size=size) < 0.1] = float('nan')
             yield orig
+        # 90% of values are NaNs.
+        for size in (50, 500):
+            orig = np.random.random(size=size) * 100
+            orig[np.random.random(size=size) < 0.9] = float('nan')
+            yield orig
 
     def has_duplicates(self, arr):
         """
Repository: numba/numba

Issue:
CUDA test: `test_reinterpret_array_type` is failing on NumPy 1.23
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I have tried using the latest released version of Numba (most recent is
 visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
- [x] I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

Reproducer is simply install Numba (from 0.56 series or current `main`) alongside NumPy>=1.23 and then run the test:
```
./runtests.py numba.cuda.tests.cudapy.test_array_methods.TestCudaArrayMethod.test_reinterpret_array_type
```
Failure is roughly:
```
<snip>
numba/np/arrayobj.py, line 2540, in array_view
numba/core/base.py, line 559, in get_function
numba/core/base.py, line 561, in get_function

NotImplementedError: No definition for lowering <built-in method impl of ....>(array(uint8, 1d, C), class(int32)) -> none
```

I suspect it's this that's causing the issue:
https://github.com/numba/numba/blob/f4b6fe0124fa7734ba2bb28c7685db5b6306c486/numba/np/arrayobj.py#L2559-L2569
the lowering for `array.view` is being "borrowed" by the CUDA target, but this branch for NumPy 1.23 resolves an `@overload` (following) to determine if it's a compatible view and something goes wrong. I would guess that it either refuses to compile on CUDA (which is probably correct) or the current compilation target gets mixed up leading to similar.
https://github.com/numba/numba/blob/f4b6fe0124fa7734ba2bb28c7685db5b6306c486/numba/np/arrayobj.py#L2498-L2539

The same test runs fine on NumPy 1.22.

CC @gmarkall.


Discussion:
Think this will fix it:
```python
diff --git a/numba/np/arrayobj.py b/numba/np/arrayobj.py
--- a/numba/np/arrayobj.py
+++ b/numba/np/arrayobj.py
@@ -2467,7 +2467,7 @@ def _compatible_view(a, dtype):
     pass
 
 
-@overload(_compatible_view)
+@overload(_compatible_view, target='generic')
 def ol_compatible_view(a, dtype):
     """Determines if the array and dtype are compatible for forming a view."""
     # NOTE: NumPy 1.23+ uses this check.
```

Resolving patch:
diff --git a/numba/np/arrayobj.py b/numba/np/arrayobj.py
--- a/numba/np/arrayobj.py
+++ b/numba/np/arrayobj.py
@@ -2495,7 +2495,7 @@ def _compatible_view(a, dtype):
     pass
 
 
-@overload(_compatible_view)
+@overload(_compatible_view, target='generic')
 def ol_compatible_view(a, dtype):
     """Determines if the array and dtype are compatible for forming a view."""
     # NOTE: NumPy 1.23+ uses this check.


Test patch:
Repository: numba/numba

Issue:
Numba event mechanism is recording seconds as microseconds?
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I have tried using the latest released version of Numba (most recent is
 visible in the change log (https://github.com/numba/numba/blob/main/CHANGE_LOG).
- [x] I have included a self contained code sample to reproduce the problem.
  i.e. it's possible to run as 'python bug.py'.

<!--

Please include details of the bug here, including, if applicable, what you
expected to happen!

-->


I'm using the Numba event mechanism to analyze a program and I notice the report generated by Numba is interpreting seconds as microseconds. So, to test this, I copied the [`ConstAddOne`](https://numba.pydata.org/numba-doc/latest/developer/custom_pipeline.html#compiler-pass-classes) from the docs and put a `time.sleep(6)` in the `run_pass` module. Below is the graph generated on [speedscope.app](speedscope.app) 

![image](https://user-images.githubusercontent.com/2712115/201769197-91f26c77-6ba8-4ef1-88cd-e2a80ffad62e.png)

The code for the reproducer is in a [gist](https://gist.github.com/guilhermeleobas/a7263b571054c3d5c329d05b5aac277b)



Discussion:

Resolving patch:
diff --git a/numba/core/event.py b/numba/core/event.py
--- a/numba/core/event.py
+++ b/numba/core/event.py
@@ -461,11 +461,13 @@ def _prepare_chrome_trace_data(listener: RecordingListener):
     for ts, rec in listener.buffer:
         data = rec.data
         cat = str(rec.kind)
+        ts_scaled = ts * 1_000_000   # scale to microseconds
         ph = 'B' if rec.is_start else 'E'
         name = data['name']
         args = data
         ev = dict(
-            cat=cat, pid=pid, tid=tid, ts=ts, ph=ph, name=name, args=args,
+            cat=cat, pid=pid, tid=tid, ts=ts_scaled, ph=ph, name=name,
+            args=args,
         )
         evs.append(ev)
     return evs


Test patch:
Repository: numba/numba

Issue:
`compile_ptx()` allows compilation of kernel with non-`void` return type
For example:

```python
from numba import cuda, int32

def f(x, y):
    return x[0] + y[0]

ptx, resty = cuda.compile_ptx(f, (int32[::1], int32[::1]))

print(resty)
# int64
```

compiles, and generates PTX where the function body returns nothing (heavily edited for clarity, but the idea is represented:

```assembly
.visible .entry f (
    // args omitted for brevity
)
{
	ret;
}
```

Usually we check that the kernel has a void return type in the `@cuda.jit` decorator, which is why this slips by in `compile_ptx`. The check should probably be pushed a bit deeper to cover both uses.

(cc @brandonwillard )


Discussion:

Resolving patch:
diff --git a/numba/cuda/compiler.py b/numba/cuda/compiler.py
--- a/numba/cuda/compiler.py
+++ b/numba/cuda/compiler.py
@@ -270,6 +270,10 @@ def compile_ptx(pyfunc, args, debug=False, lineinfo=False, device=False,
                         fastmath=fastmath,
                         nvvm_options=nvvm_options)
     resty = cres.signature.return_type
+
+    if resty and not device and resty != types.void:
+        raise TypeError("CUDA kernel must have void return type.")
+
     if device:
         lib = cres.library
     else:


Test patch:
diff --git a/numba/cuda/tests/cudapy/test_compiler.py b/numba/cuda/tests/cudapy/test_compiler.py
--- a/numba/cuda/tests/cudapy/test_compiler.py
+++ b/numba/cuda/tests/cudapy/test_compiler.py
@@ -116,6 +116,13 @@ def f():
         ptx, resty = compile_ptx(f, [], lineinfo=True)
         self.check_line_info(ptx)
 
+    def test_non_void_return_type(self):
+        def f(x, y):
+            return x[0] + y[0]
+
+        with self.assertRaisesRegex(TypeError, 'must have void return type'):
+            compile_ptx(f, (uint32[::1], uint32[::1]))
+
 
 @skip_on_cudasim('Compilation unsupported in the simulator')
 class TestCompileToPTXForCurrentDevice(CUDATestCase):
Repository: numba/numba

Issue:
LLVM14 SVML causes small change in precision in test_logaddexp2_dd_d on osx
Test failed `test_logaddexp2_dd_d (numba.tests.test_ufuncs.TestLoopTypesFloat)`
Error msg: `0.12219341620227443 != 0.1221934162022744 within 2.7132388825073943e-17 delta (2.7755575615628914e-17 difference)`
This only occurred in SVML-enabled test runs.


Discussion:
@stuartarchibald said we can fix this by increasing to 2ulp

Resolving patch:
diff --git a/numba/np/arraymath.py b/numba/np/arraymath.py
--- a/numba/np/arraymath.py
+++ b/numba/np/arraymath.py
@@ -1830,7 +1830,7 @@ def np_argpartition_impl_inner(a, kth_array):
 
     # allocate and fill empty array rather than copy a and mutate in place
     # as the latter approach fails to preserve strides
-    out = np.empty_like(a, dtype=np.int64)
+    out = np.empty_like(a, dtype=np.intp)
 
     idx = np.ndindex(a.shape[:-1])  # Numpy default partition axis is -1
     for s in idx:
@@ -1924,7 +1924,7 @@ def np_argpartition(a, kth):
     def np_argpartition_impl(a, kth):
         a_tmp = _asarray(a)
         if a_tmp.size == 0:
-            return a_tmp.copy().astype('int64')
+            return a_tmp.copy().astype('intp')
         else:
             kth_array = valid_kths(a_tmp, kth)
             return np_argpartition_impl_inner(a_tmp, kth_array)


Test patch:
diff --git a/numba/tests/test_np_functions.py b/numba/tests/test_np_functions.py
--- a/numba/tests/test_np_functions.py
+++ b/numba/tests/test_np_functions.py
@@ -2218,11 +2218,12 @@ def test_argpartition_basic(self):
         cfunc = jit(nopython=True)(pyfunc)
 
         d = np.array([], dtype=np.int64)
+        expected = pyfunc(d, 0)
         got = cfunc(d, 0)
-        self.assertPreciseEqual(d, got)
+        self.assertPreciseEqual(expected, got)
 
         d = np.ones(1, dtype=np.int64)
-        expected = np.zeros(1, dtype=np.int64)
+        expected = pyfunc(d, 0)
         got = cfunc(d, 0)
         self.assertPreciseEqual(expected, got)
 
diff --git a/numba/tests/test_ufuncs.py b/numba/tests/test_ufuncs.py
--- a/numba/tests/test_ufuncs.py
+++ b/numba/tests/test_ufuncs.py
@@ -1347,6 +1347,7 @@ class _LoopTypesTester(TestCase):
              ('log10', 'D'): 5,
              ('tanh', 'F'): 2,
              ('cbrt', 'd'): 2,
+             ('logaddexp2', 'd'): 2,
              }
 
     def _arg_for_type(self, a_letter_type, index=0):