Repository: cupy/cupy

Issue:
JitifyException in cupyx.scipy.ndimage.gaussian_filter with higher dimensions//large arrays? 
### Description



```
_=cupyx.scipy.ndimage.gaussian_filter(cp.random.randn(26,3,52,70,50,dtype='float32'), [1,1,1,1,1], mode='nearest')

```
works fine, but 
```
_=cupyx.scipy.ndimage.gaussian_filter(cp.random.randn(26,3,52,70,50,70,dtype='float32'), [1,1,1,1,1,1], mode='nearest')

```
results  in a  

```
JitifyException                           Traceback (most recent call last)
File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/IPython/core/interactiveshell.py, in run_code:
Line 3441:  exec(code_obj, self.user_global_ns, self.user_ns)

In  [30]:
Line 1:     _=cupyx.scipy.ndimage.gaussian_filter(cp.random.randn(26,3,52,70,50,70,dtype='float32'), [1.,2.,1.,1.,2.,1.], mode='nearest')

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupyx/scipy/ndimage/filters.py, in gaussian_filter:
Line 367:   mode, cval, 0)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupyx/scipy/ndimage/filters.py, in _run_1d_correlates:
Line 207:   input, wghts, output, mode, cval, origin)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupyx/scipy/ndimage/_filters_core.py, in _run_1d_filters:
Line 97:    fltr(input, arg, axis, output, mode, cval, origin)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupyx/scipy/ndimage/filters.py, in correlate1d:
Line 113:   return _correlate_or_convolve(input, weights, output, mode, cval, origins)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupyx/scipy/ndimage/filters.py, in _correlate_or_convolve:
Line 177:   weights_dtype=weights_dtype)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupyx/scipy/ndimage/_filters_core.py, in _call_kernel:
Line 141:   kernel(*args)

File cupy/_core/_kernel.pyx, in cupy._core._kernel.ElementwiseKernel.__call__:
Line 850:   

File cupy/_core/_kernel.pyx, in cupy._core._kernel.ElementwiseKernel._get_elementwise_kernel:
Line 875:   

File cupy/_util.pyx, in cupy._util.memoize.decorator.ret:
Line 59:    

File cupy/_core/_kernel.pyx, in cupy._core._kernel._get_elementwise_kernel:
Line 662:   

File cupy/_core/_kernel.pyx, in cupy._core._kernel._get_simple_elementwise_kernel:
Line 62:    

File cupy/_core/core.pyx, in cupy._core.core.compile_with_cache:
Line 2042:  

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupy/cuda/compiler.py, in compile_with_cache:
Line 463:   cache_in_memory, jitify)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupy/cuda/compiler.py, in _compile_with_cache_cuda:
Line 540:   log_stream, cache_in_memory, jitify)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupy/cuda/compiler.py, in compile_using_nvrtc:
Line 299:   name_expressions, log_stream, jitify)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupy/cuda/compiler.py, in _compile:
Line 268:   source, options, cu_path)

File /fast/work/users/hoffmmax_c/anaconda3/envs/img/lib/python3.7/site-packages/cupy/cuda/compiler.py, in _jitify_prep:
Line 254:   raise JitifyException(str(cex))

JitifyException: Runtime compilation failed
```

Version:



### To Reproduce

```py

import cupy as cp
import cupyx
_=cupyx.scipy.ndimage.gaussian_filter(cp.random.randn(26,3,52,70,50,dtype='float32'), [1,1,1,1,1], mode='nearest')
```


### Installation

Wheel (`pip install cupy-***`)

### Environment

```
OS                           : Linux-3.10.0-1160.41.1.el7.x86_64-x86_64-with-centos-7.9.2009-Core
Python Version               : 3.7.9
CuPy Version                 : 9.5.0
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.19.4
SciPy Version                : 1.6.0
Cython Build Version         : 0.29.22
Cython Runtime Version       : None
CUDA Root                    : /usr/local/cuda-10.2
nvcc PATH                    : /usr/local/cuda-10.2/bin/nvcc
CUDA Build Version           : 10020
CUDA Driver Version          : 11040
CUDA Runtime Version         : 10020
cuBLAS Version               : (available)
cuFFT Version                : 10501
cuRAND Version               : 10205
cuSOLVER Version             : (10, 3, 0)
cuSPARSE Version             : (available)
NVRTC Version                : (10, 2)
Thrust Version               : 100907
CUB Build Version            : 100800
Jitify Build Version         : 60e9e72
cuDNN Build Version          : None
cuDNN Version                : None
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : Tesla V100-SXM2-32GB
Device 0 Compute Capability  : 70
Device 0 PCI Bus ID          : 0000:18:00.0
```

### Additional Information

```
_=cupyx.scipy.ndimage.gaussian_filter(cp.random.randn(26,3,52,70,50,20,dtype='float32'), [1,1,1,1,1,1], mode='nearest')
```

works, too. So it is just related to the size of the array in relation to GPU memory? In this case, it might be cool to have a more informative stack trace




Discussion:
I can reproduce. Looks like a new issue not reported before.

<details>
<summary>Click for full traceback</summary> 

```python
>>> # export CUPY_DUMP_CUDA_SOURCE_ON_ERROR=1
>>> import cupyx.scipy.ndimage
>>> import cupy as cp
>>> _=cupyx.scipy.ndimage.gaussian_filter(cp.random.randn(26,3,52,70,50,70,dtype='float32'), [1,1,1,1,1,1], mode='nearest')
---------------------------------------------------
--- JIT compile log for /tmp/tmpc7kqsqvm/3b64a4e548355fc39eb6c43b955aa2c28ae342f0.cubin.cu ---
---------------------------------------------------
cupy/complex/complex.h(94): warning: __device__ annotation is ignored on a function("complex") that is explicitly defaulted on its first declaration

cupy/complex/complex.h(101): warning: __device__ annotation is ignored on a function("complex") that is explicitly defaulted on its first declaration

/tmp/tmpc7kqsqvm/3b64a4e548355fc39eb6c43b955aa2c28ae342f0.cubin.cu(63): error: more than one instance of overloaded function "max" matches the argument list:
            function "max(int, int)"
__nv_nvrtc_builtin_header.h(152101): here
            function "max(unsigned int, int)"
__nv_nvrtc_builtin_header.h(153439): here
            function "max(long, long)"
__nv_nvrtc_builtin_header.h(153440): here
            function "max(long, unsigned long)"
__nv_nvrtc_builtin_header.h(153442): here
            argument types are: (ptrdiff_t, int)

1 error detected in the compilation of "/tmp/tmpc7kqsqvm/3b64a4e548355fc39eb6c43b955aa2c28ae342f0.cubin.cu".

---------------------------------------------------
JITIFY compilation error: Runtime compilation failed
-----
Name: /tmp/tmpc7kqsqvm/3b64a4e548355fc39eb6c43b955aa2c28ae342f0.cubin.cu
Options: --std=c++11 -DCUPY_USE_JITIFY -I/home/leof/dev/cupy_cuda114/cupy/_core/include -I/home/leof/dev/cupy_cuda114/cupy/_core/include/cupy/_cuda/cuda-11 -I/usr/local/cuda-11.4/include -ftz=true
CUDA source:
01 #include <cupy/complex.cuh>
02 #include <cupy/carray.cuh>
03 #include <cupy/atomics.cuh>
04 
05     typedef float W;
06 typedef float X;
07 typedef float Y;
08 
09 
10 // workaround for HIP: line begins with #include
11 #include <type_traits>  // let Jitify handle this
12 #include <cupy/math_constants.h>
13 
14 // Implements a casting function to make it compatible with scipy
15 // Use like cast<to_type>(value)
16 template<> struct std::is_floating_point<float16> : std::true_type {};
17 template<> struct std::is_signed<float16> : std::true_type {};
18 template<class T> struct std::is_signed<complex<T>> : std::is_signed<T> {};
19 
20 template <class B, class A>
21 __device__ __forceinline__
22 typename std::enable_if<(!std::is_floating_point<A>::value
23                          || std::is_signed<B>::value), B>::type
24 cast(A a) { return (B)a; }
25 
26 template <class B, class A>
27 __device__ __forceinline__
28 typename std::enable_if<(std::is_floating_point<A>::value
29                          && (!std::is_signed<B>::value)), B>::type
30 cast(A a) { return (a >= 0) ? (B)a : -(B)(-a); }
31 
32 template <class T>
33 __device__ __forceinline__ bool nonzero(T x) { return x != static_cast<T>(0); }
34 
35     extern "C" __global__ void cupyx_scipy_ndimage_correlate_6d_nearest_w9_1_1_1_1_1_i64(const CArray<float, 6, 1, 0> x, const CArray<float, 6, 1, 1> w, CArray<float, 6, 1, 0> _raw_y, CIndexer<6> _ind) {
36       ;
37       CUPY_FOR(i, _ind.size()) {
38         _ind.set(i);
39         Y &y = _raw_y[_ind.get()];
40 
41     ptrdiff_t xsize_0=x.shape()[0], ysize_0 = _raw_y.shape()[0], xstride_0=x.strides()[0];
42 ptrdiff_t xsize_1=x.shape()[1], ysize_1 = _raw_y.shape()[1], xstride_1=x.strides()[1];
43 ptrdiff_t xsize_2=x.shape()[2], ysize_2 = _raw_y.shape()[2], xstride_2=x.strides()[2];
44 ptrdiff_t xsize_3=x.shape()[3], ysize_3 = _raw_y.shape()[3], xstride_3=x.strides()[3];
45 ptrdiff_t xsize_4=x.shape()[4], ysize_4 = _raw_y.shape()[4], xstride_4=x.strides()[4];
46 ptrdiff_t xsize_5=x.shape()[5], ysize_5 = _raw_y.shape()[5], xstride_5=x.strides()[5];
47     ptrdiff_t _i = i;
48 ptrdiff_t ind_5 = _i % ysize_5 - 0; _i /= ysize_5;
49 ptrdiff_t ind_4 = _i % ysize_4 - 0; _i /= ysize_4;
50 ptrdiff_t ind_3 = _i % ysize_3 - 0; _i /= ysize_3;
51 ptrdiff_t ind_2 = _i % ysize_2 - 0; _i /= ysize_2;
52 ptrdiff_t ind_1 = _i % ysize_1 - 0; _i /= ysize_1;
53 ptrdiff_t ind_0 = _i - 4;
54     // don't use a CArray for indexing (faster to deal with indexing ourselves)
55     const unsigned char* data = (const unsigned char*)&x[0];
56     int iws = 0;
57     W sum = (W)0;
58 
59     for (int iw_0 = 0; iw_0 < 9; iw_0++)
60     {
61         ptrdiff_t ix_0 = ind_0 + iw_0;
62 
63         ix_0 = min(max(ix_0, 0), xsize_0 - 1);
64         ix_0 *= xstride_0;
65 
66 { ptrdiff_t ix_1 = ind_1 * xstride_1;
67 { ptrdiff_t ix_2 = ind_2 * xstride_2;
68 { ptrdiff_t ix_3 = ind_3 * xstride_3;
69 { ptrdiff_t ix_4 = ind_4 * xstride_4;
70 { ptrdiff_t ix_5 = ind_5 * xstride_5;
71         // inner-most loop
72         W wval = w[iws];
73 if (nonzero(wval)) {
74             sum += cast<W>((*(X*)&data[ix_0 + ix_1 + ix_2 + ix_3 + ix_4 + ix_5])) * wval;
75         }
76         iws++;
77     }}}}}}
78     y = cast<Y>(sum);
79     ;
80       }
81       ;
82     }
83 
-----
Traceback (most recent call last):
  File "/home/leof/dev/cupy_cuda114/cupy/cuda/compiler.py", line 250, in _jitify_prep
    name, options, headers, include_names = jitify(
  File "cupy/cuda/jitify.pyx", line 59, in cupy.cuda.jitify.jitify
    cpdef jitify(str code, tuple opt, dict cached_sources=None):
  File "cupy/cuda/jitify.pyx", line 92, in cupy.cuda.jitify.jitify
    load_program(cuda_source, headers, nullptr, &include_paths,
RuntimeError: Runtime compilation failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/leof/dev/cupy_cuda114/cupyx/scipy/ndimage/filters.py", line 366, in gaussian_filter
    return _run_1d_correlates(input, list(zip(sigmas, orders)), get, output,
  File "/home/leof/dev/cupy_cuda114/cupyx/scipy/ndimage/filters.py", line 205, in _run_1d_correlates
    return _filters_core._run_1d_filters(
  File "/home/leof/dev/cupy_cuda114/cupyx/scipy/ndimage/_filters_core.py", line 97, in _run_1d_filters
    fltr(input, arg, axis, output, mode, cval, origin)
  File "/home/leof/dev/cupy_cuda114/cupyx/scipy/ndimage/filters.py", line 113, in correlate1d
    return _correlate_or_convolve(input, weights, output, mode, cval, origins)
  File "/home/leof/dev/cupy_cuda114/cupyx/scipy/ndimage/filters.py", line 176, in _correlate_or_convolve
    output = _filters_core._call_kernel(kernel, input, weights, output,
  File "/home/leof/dev/cupy_cuda114/cupyx/scipy/ndimage/_filters_core.py", line 141, in _call_kernel
    kernel(*args)
  File "cupy/_core/_kernel.pyx", line 868, in cupy._core._kernel.ElementwiseKernel.__call__
    kern = self._get_elementwise_kernel(dev_id, arginfos, type_map)
  File "cupy/_core/_kernel.pyx", line 893, in cupy._core._kernel.ElementwiseKernel._get_elementwise_kernel
    kern = _get_elementwise_kernel(
  File "cupy/_util.pyx", line 67, in cupy._util.memoize.decorator.ret
    result = f(*args, **kwargs)
  File "cupy/_core/_kernel.pyx", line 680, in cupy._core._kernel._get_elementwise_kernel
    return _get_simple_elementwise_kernel(
  File "cupy/_core/_kernel.pyx", line 69, in cupy._core._kernel._get_simple_elementwise_kernel
    module = compile_with_cache(module_code, options)
  File "cupy/_core/core.pyx", line 2111, in cupy._core.core.compile_with_cache
    return cuda.compiler._compile_module_with_cache(
  File "/home/leof/dev/cupy_cuda114/cupy/cuda/compiler.py", line 487, in _compile_module_with_cache
    return _compile_with_cache_cuda(
  File "/home/leof/dev/cupy_cuda114/cupy/cuda/compiler.py", line 565, in _compile_with_cache_cuda
    ptx, mapping = compile_using_nvrtc(
  File "/home/leof/dev/cupy_cuda114/cupy/cuda/compiler.py", line 316, in compile_using_nvrtc
    return _compile(source, options, cu_path,
  File "/home/leof/dev/cupy_cuda114/cupy/cuda/compiler.py", line 285, in _compile
    options, headers, include_names = _jitify_prep(
  File "/home/leof/dev/cupy_cuda114/cupy/cuda/compiler.py", line 258, in _jitify_prep
    raise JitifyException(str(cex))
cupy.cuda.compiler.JitifyException: Runtime compilation failed
```

</details>
This is something I don't understand:
```
/tmp/tmpc7kqsqvm/3b64a4e548355fc39eb6c43b955aa2c28ae342f0.cubin.cu(63): error: more than one instance of overloaded function "max" matches the argument list:
            function "max(int, int)"
__nv_nvrtc_builtin_header.h(152101): here
            function "max(unsigned int, int)"
__nv_nvrtc_builtin_header.h(153439): here
            function "max(long, long)"
__nv_nvrtc_builtin_header.h(153440): here
            function "max(long, unsigned long)"
__nv_nvrtc_builtin_header.h(153442): here
            argument types are: (ptrdiff_t, int)
```
To me there shouldn't be any ambiguity (`(long, long)` is the only allowed overload)...
> To me there shouldn't be any ambiguity (`(long, long)` is the only allowed overload)...

I see. Even for host compiler it's ambiguous:
```
$ g++ --std=c++11 test_std_max.cpp -o test_std_max
test_std_max.cpp: In function ‘int main()’:
test_std_max.cpp:8:21: error: no matching function for call to ‘max(ptrdiff_t&, int&)’
    8 |     cout << max(a, b) << endl;
      |                     ^
In file included from /usr/include/c++/9/bits/char_traits.h:39,
                 from /usr/include/c++/9/ios:40,
                 from /usr/include/c++/9/ostream:38,
                 from /usr/include/c++/9/iostream:39,
                 from test_std_max.cpp:1:
/usr/include/c++/9/bits/stl_algobase.h:222:5: note: candidate: ‘template<class _Tp> const _Tp& std::max(const _Tp&, const _Tp&)’
  222 |     max(const _Tp& __a, const _Tp& __b)
      |     ^~~
/usr/include/c++/9/bits/stl_algobase.h:222:5: note:   template argument deduction/substitution failed:
test_std_max.cpp:8:21: note:   deduced conflicting types for parameter ‘const _Tp’ (‘long int’ and ‘int’)
    8 |     cout << max(a, b) << endl;
      |                     ^
In file included from /usr/include/c++/9/bits/char_traits.h:39,
                 from /usr/include/c++/9/ios:40,
                 from /usr/include/c++/9/ostream:38,
                 from /usr/include/c++/9/iostream:39,
                 from test_std_max.cpp:1:
/usr/include/c++/9/bits/stl_algobase.h:268:5: note: candidate: ‘template<class _Tp, class _Compare> const _Tp& std::max(const _Tp&, const _Tp&, _Compare)’
  268 |     max(const _Tp& __a, const _Tp& __b, _Compare __comp)
      |     ^~~
/usr/include/c++/9/bits/stl_algobase.h:268:5: note:   template argument deduction/substitution failed:
test_std_max.cpp:8:21: note:   deduced conflicting types for parameter ‘const _Tp’ (‘long int’ and ‘int’)
    8 |     cout << max(a, b) << endl;
      |                     ^
```
@asi1024 I think the overloading ambiguity can be resolved this way:
```diff
diff --git a/cupyx/scipy/ndimage/_filters_core.py b/cupyx/scipy/ndimage/_filters_core.py
index a5f14e293..7dbe5e817 100644
--- a/cupyx/scipy/ndimage/_filters_core.py
+++ b/cupyx/scipy/ndimage/_filters_core.py
@@ -225,7 +225,7 @@ def _generate_nd_kernel(name, pre, found, post, mode, w_shape, int_type,
                          format(j=j, type=int_type))
         else:
             boundary = _util._generate_boundary_condition_ops(
-                mode, 'ix_{}'.format(j), 'xsize_{}'.format(j))
+                mode, 'ix_{}'.format(j), 'xsize_{}'.format(j), int_type)
             # CArray: last line of string becomes inds[{j}] = ix_{j};
             loops.append('''
     for (int iw_{j} = 0; iw_{j} < {wsize}; iw_{j}++)
diff --git a/cupyx/scipy/ndimage/_util.py b/cupyx/scipy/ndimage/_util.py
index 6a763fd35..24953b64f 100644
--- a/cupyx/scipy/ndimage/_util.py
+++ b/cupyx/scipy/ndimage/_util.py
@@ -117,8 +117,10 @@ def _generate_boundary_condition_ops(mode, ix, xsize, int_t="int",
         }}'''.format(ix=ix, xsize=xsize, min=min_func)
     elif mode == 'nearest':
         ops = '''
-        {ix} = {min}({max}({ix}, 0), {xsize} - 1);'''.format(
-            ix=ix, xsize=xsize, min=min_func, max=max_func)
+        {ix} = {min}({max}(({T}){ix}, ({T})0), ({T})({xsize} - 1));'''.format(
+            ix=ix, xsize=xsize, min=min_func, max=max_func,
+            # force using 64-bit signed integer for ptrdiff_t, see cupy/cupy#6048
+            T=('int' if int_t == 'int' else 'long long'))
     elif mode == 'grid-wrap':
         ops = '''
         {ix} %= {xsize};
```

Resolving patch:
diff --git a/cupyx/scipy/ndimage/_filters_core.py b/cupyx/scipy/ndimage/_filters_core.py
--- a/cupyx/scipy/ndimage/_filters_core.py
+++ b/cupyx/scipy/ndimage/_filters_core.py
@@ -225,7 +225,7 @@ def _generate_nd_kernel(name, pre, found, post, mode, w_shape, int_type,
                          format(j=j, type=int_type))
         else:
             boundary = _util._generate_boundary_condition_ops(
-                mode, 'ix_{}'.format(j), 'xsize_{}'.format(j))
+                mode, 'ix_{}'.format(j), 'xsize_{}'.format(j), int_type)
             # CArray: last line of string becomes inds[{j}] = ix_{j};
             loops.append('''
     for (int iw_{j} = 0; iw_{j} < {wsize}; iw_{j}++)
diff --git a/cupyx/scipy/ndimage/_util.py b/cupyx/scipy/ndimage/_util.py
--- a/cupyx/scipy/ndimage/_util.py
+++ b/cupyx/scipy/ndimage/_util.py
@@ -117,8 +117,11 @@ def _generate_boundary_condition_ops(mode, ix, xsize, int_t="int",
         }}'''.format(ix=ix, xsize=xsize, min=min_func)
     elif mode == 'nearest':
         ops = '''
-        {ix} = {min}({max}({ix}, 0), {xsize} - 1);'''.format(
-            ix=ix, xsize=xsize, min=min_func, max=max_func)
+        {ix} = {min}({max}(({T}){ix}, ({T})0), ({T})({xsize} - 1));'''.format(
+            ix=ix, xsize=xsize, min=min_func, max=max_func,
+            # force using 64-bit signed integer for ptrdiff_t,
+            # see cupy/cupy#6048
+            T=('int' if int_t == 'int' else 'long long'))
     elif mode == 'grid-wrap':
         ops = '''
         {ix} %= {xsize};


Test patch:
diff --git a/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py b/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py
--- a/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py
+++ b/tests/cupyx_tests/scipy_tests/ndimage_tests/test_filters.py
@@ -228,6 +228,17 @@ def test_filter(self, xp, scp):
         return self._filter(xp, scp)
 
 
+@testing.with_requires('scipy')
+class TestNearestFilterEdgeCase:
+
+    @testing.numpy_cupy_allclose(atol=1e-5, rtol=1e-5, scipy_name='scp')
+    def test_filter(self, xp, scp):
+        # Use a large input to check any compilation error, see cupy/cupy#6048
+        return scp.ndimage.gaussian_filter(
+            testing.shaped_random((26, 3, 52, 70, 50), xp, xp.float32),
+            [1, 1, 1, 1, 1], mode='nearest')
+
+
 def dummy_deriv_func(input, axis, output, mode, cval, *args, **kwargs):
     # For testing generic_laplace and generic_gradient_magnitude. Doesn't test
     # mode, cval, or extra argument but those are tested indirectly with