Repository: Theano/Theano

Issue:
CUDA headers not included in compilation of GpuMagmaMatrixInverse node
I'm using theano with magma (enabled in theano config) to invert a matrix on the GPU:
```
import theano
import theano.tensor as T
A = T.matrix()
B = T.nlinalg.matrix_inverse(A)
f = theano.function([A], [B])
```

However, the function does not compile:
```
Exception: ('The following error happened while compiling the node', GpuMagmaMatrixInverse{inplace=True}(GpuContiguous.0), '\n', 'Compilation failed (return status=1): In file included from /home/user/.theano/compiledir_Linux-4.4--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-2.7.12-64/tmpQn5ybC/mod.cpp:15:0:. /usr/local/include/gpuarray/ext_cuda.h:4:18: fatal error: cuda.h: No such file or directory. compilation terminated.. ', '[GpuMagmaMatrixInverse{inplace=True}(<GpuArrayType<None>(float32, matrix)>)]')
```

Looking at the compile string:
```
Problem occurred during compilation with the command line below:
/usr/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -march=haswell -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mmovbe -maes -mno-sha -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-pcommit -mno-mwaitx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=haswell -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/usr/local/lib/python2.7/dist-packages/pygpu-0.7.5-py2.7-linux-x86_64.egg/pygpu -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include -I/usr/local/lib/python2.7/dist-packages/theano/gpuarray/c_code -I/usr/local/magma/include -I/usr/local/lib/python2.7/dist-packages/numpy/core/include -I/usr/include/python2.7 -I/usr/local/lib/python2.7/dist-packages/theano/gof/c_code -L/usr/lib -L/usr/local/magma/lib -L/usr/lib -fvisibility=hidden -o /home/user/.theano/compiledir_Linux-4.4--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-2.7.12-64/tmpEwOSsC/328bc07052efa6880d04b71f5ab1c93b229fc94fe91f2ab4f71631da1bc97d76.so /home/user/.theano/compiledir_Linux-4.4--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-2.7.12-64/tmpEwOSsC/mod.cpp -lgpuarray -lmagma -lpython2.7
In file included from /home/user/.theano/compiledir_Linux-4.4--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-2.7.12-64/tmpEwOSsC/mod.cpp:15:0:
/usr/local/include/gpuarray/ext_cuda.h:4:18: fatal error: cuda.h: No such file or directory
compilation terminated.
```
it is apparent that `gpuarray/ext_cuda.h` includes `<cuda.h>` but the CUDA headers are not being provided to the compiler (g++).

As a workaround, I'm pointing the compiler to the CUDA headers by setting:
```
export CPATH=/usr/local/cuda/include
```
However, it would make more sense to do this with a `-I` in the compile string.

Probably, `std_include_dirs()` in `gof/cmodule.py` should include the CUDA headers as well (?)


Discussion:
@notoraptor is looking into it.

Resolving patch:
diff --git a/theano/gpuarray/linalg.py b/theano/gpuarray/linalg.py
--- a/theano/gpuarray/linalg.py
+++ b/theano/gpuarray/linalg.py
@@ -498,7 +498,7 @@ def c_headers(self):
                 'gpuarray_helper.h', 'magma.h']
 
     def c_header_dirs(self):
-        dirs = [gpuarray_helper_inc_dir(), pygpu.get_include()]
+        dirs = [gpuarray_helper_inc_dir(), pygpu.get_include(), config.cuda.include_path]
         if config.magma.include_path:
             dirs.append(config.magma.include_path)
         return dirs


Test patch: