Repository: cupy/cupy

Issue:
Fix bug in CUB + support complex numbers using CUB
This is a followup of #2090. The bulk of this PR is completed, but I mark it WIP so as to gather early feedbacks and comments before proceeding to write tests.

This PR does two things:
1. Fix build-time bug: without `#include <stdexcept>` in `cupy/cuda/cupy_cub.cu`, I found that there is a build-time error:
```shell
    building 'cupy.cuda.cub' extension
    gcc -pthread -B /home/leofang/conda_envs/cupy_dev/compiler_compat -Wl,--sysroot=/ -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/home/leofang/.nccl_2.1.15-1+cuda9.1_x86_64/include/ -fPIC -D_FORCE_INLINES=1 -I/usr/local/cuda/include -I/home/leofang/sources/cub-1.8.0/ -I/home/leofang/conda_envs/cupy_dev/include/python3.6m -c cupy/cuda/cub.cpp -o build/temp.linux-x86_64-3.6/cupy/cuda/cub.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    NVCC options: ['--generate-code=arch=compute_30,code=compute_30', '--generate-code=arch=compute_50,code=compute_50', '--generate-code=arch=compute_60,code=sm_60', '--generate-code=arch=compute_61,code=sm_61', '--generate-code=arch=compute_70,code=sm_70', '--generate-code=arch=compute_70,code=compute_70', '-O2', '--compiler-options="-fPIC"', '--std=c++11']
    /usr/local/cuda/bin/nvcc -D_FORCE_INLINES=1 -I/usr/local/cuda/include -I/home/leofang/sources/cub-1.8.0/ -I/home/leofang/conda_envs/cupy_dev/include/python3.6m -c cupy/cuda/cupy_cub.cu -o build/temp.linux-x86_64-3.6/cupy/cuda/cupy_cub.o --generate-code=arch=compute_30,code=compute_30 --generate-code=arch=compute_50,code=compute_50 --generate-code=arch=compute_60,code=sm_60 --generate-code=arch=compute_61,code=sm_61 --generate-code=arch=compute_70,code=sm_70 --generate-code=arch=compute_70,code=compute_70 -O2 --compiler-options="-fPIC" --std=c++11
    cupy/cuda/cupy_cub.cu(28): error: namespace "std" has no member "runtime_error"
```
2.  Support `sum()`/`max()`/`min()` for complex numbers using CUB. This support is compliant with NumPy behavior. 

The current implementation is, admittedly, an ugly and not fully optimized hack for two reasons: (i) CUB does not provide native support for complex numbers (see the CUB source code `cub/util_type.cuh`), and (ii) since the comparison of two complex numbers is ill-defined, NumPy uses a lexical search for `max()` and `min()`, which requires two passes (separately examining the real and imaginary parts) in the worst-case scenario.

However, in my tests this hack still offers substantial speedup (see https://github.com/cupy/cupy/pull/2508#issuecomment-536244675) so that I think it's worth considering. We can revisit this issue once CUB has better support.

Attn: @anaruse 


Discussion:

Resolving patch:
diff --git a/install/build.py b/install/build.py
--- a/install/build.py
+++ b/install/build.py
@@ -143,6 +143,10 @@ def get_compiler_setting(use_cpp11):
 
     cub_path = os.environ.get('CUB_PATH', '')
     if os.path.exists(cub_path):
+        # for <cupy/complex.cuh>
+        cupy_header = os.path.join(os.path.dirname(os.path.realpath(__file__)),
+                                   '../cupy/core/include')
+        include_dirs.append(cupy_header)
         include_dirs.append(cub_path)
 
     return {


Test patch: