Repository: spack/spack

Issue:
GDAL should have dependency on py-numpy when building with +python
Since the changes to the gdal/package.py on May 26, it builds without the capacity to use numpy arrays in the Python bindings.  

In the previous version of the gdal/package.py, it had an explicit build and run dependency on py-numpy, but this is now missing. I suspect that it is necessary. 

When I add this back in to gdal/package.py as follows, the problem disappears again. (I added the when='+python', which was not in the original)

`depends_on('py-numpy', type=('build', 'run'), when='+python')`

I suspect that the lack of 'build' dependency is the main flaw. When GDAL is building, it thinks that numpy is not available, and quietly builds the numpy-free parts of the Python bindings. 

### Expected Result

GDAL Python bindings should be able to use ReadAsArray() to read raster data into a numpy array.

### Actual Result

If built with +python, the resulting python bindings produce an error when reading raster data with ReadAsArray().

This is true even if py-numpy is already installed and available in the environment. 

### Steps to reproduce the issue

```console
$ spack install gdal+python
$ module load gdal-2.3.0-gcc-6-python2-rsfov4z
$ python
>>> from osgeo import gdal
>>> ds = gdal.Open('image.tif')
>>> b = ds.GetRasterBand(1)
>>> a = b.ReadAsArray()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/scratch/rsc7/floodn/spackInstall2/opt/spack/linux-sles12-x86_64/gcc-6/gdal-2.3.0-mqthuimmblu3uqd2uicgoh7bwjyxq3xf/lib/python2.7/site-packages/osgeo/gdal.py", line 2605, in ReadAsArray
    import gdalnumeric
  File "/scratch/rsc7/floodn/spackInstall2/opt/spack/linux-sles12-x86_64/gcc-6/gdal-2.3.0-mqthuimmblu3uqd2uicgoh7bwjyxq3xf/lib/python2.7/site-packages/osgeo/gdalnumeric.py", line 1, in <module>
    from osgeo.gdal_array import *
  File "/scratch/rsc7/floodn/spackInstall2/opt/spack/linux-sles12-x86_64/gcc-6/gdal-2.3.0-mqthuimmblu3uqd2uicgoh7bwjyxq3xf/lib/python2.7/site-packages/osgeo/gdal_array.py", line 28, in <module>
    _gdal_array = swig_import_helper()
  File "/scratch/rsc7/floodn/spackInstall2/opt/spack/linux-sles12-x86_64/gcc-6/gdal-2.3.0-mqthuimmblu3uqd2uicgoh7bwjyxq3xf/lib/python2.7/site-packages/osgeo/gdal_array.py", line 20, in swig_import_helper
    import _gdal_array
ImportError: No module named _gdal_array
```


### Information on your system

Running on linux-sles12-x86_64, with OS-installed gcc-6. 



Discussion:
This is probably my fault, the documentation didn't list `numpy` as a dependency and I didn't test the Python bindings myself. Would you like to submit a pull request?
Thanks Adam. I am happy for you to just fix it directly, unless you would like me to do it. It is just that one line required (given above). Yes, the dependency on numpy is a bit subtle, and GDAL will happily build Python bindings without it. I think this is a hangover from the old days before numpy existed, and it has the option of reading into a simpler data structure. 
Upon further investigation, it looks like GDAL is installing its Python bindings directly to the Python installation directory, which is a problem. Let me see if I can hack this to work properly.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/gdal/package.py b/var/spack/repos/builtin/packages/gdal/package.py
--- a/var/spack/repos/builtin/packages/gdal/package.py
+++ b/var/spack/repos/builtin/packages/gdal/package.py
@@ -41,6 +41,11 @@ class Gdal(AutotoolsPackage):
     list_url   = "http://download.osgeo.org/gdal/"
     list_depth = 1
 
+    import_modules = [
+        'osgeo', 'osgeo.gdal', 'osgeo.ogr', 'osgeo.osr',
+        'osgeo.gdal_array', 'osgeo.gdalconst'
+    ]
+
     version('2.3.0',  '2fe9d64fcd9dc37645940df020d3e200')
     version('2.1.2',  'ae85b78888514c75e813d658cac9478e')
     version('2.0.2',  '940208e737c87d31a90eaae43d0efd65')
@@ -128,15 +133,14 @@ class Gdal(AutotoolsPackage):
     depends_on('proj', when='+proj @2.3:')
     depends_on('perl', type=('build', 'run'), when='+perl')
     depends_on('python', type=('build', 'run'), when='+python')
+    # swig/python/setup.py
+    depends_on('py-setuptools', type='build', when='+python')
+    depends_on('py-numpy@1.0.0:', type=('build', 'run'), when='+python')
     depends_on('java', type=('build', 'run'), when='+java')
     depends_on('armadillo', when='+armadillo')
     depends_on('cryptopp', when='+cryptopp @2.1:')
     depends_on('openssl', when='+crypto @2.3:')
 
-    # https://trac.osgeo.org/gdal/wiki/SWIG
-    depends_on('swig', type='build', when='+python')
-    depends_on('swig', type='build', when='+perl')
-
     # https://trac.osgeo.org/gdal/wiki/SupportedCompilers
     msg = 'GDAL requires C++11 support'
     conflicts('%gcc@:4.8.0', msg=msg)
@@ -145,6 +149,13 @@ class Gdal(AutotoolsPackage):
     conflicts('%xl@:13.0',   msg=msg)
     conflicts('%xl_r@:13.0', msg=msg)
 
+    def setup_environment(self, spack_env, run_env):
+        # Needed to install Python bindings to GDAL installation
+        # prefix instead of Python installation prefix.
+        # See swig/python/GNUmakefile for more details.
+        spack_env.set('PREFIX', self.prefix)
+        spack_env.set('DESTDIR', '/')
+
     # https://trac.osgeo.org/gdal/wiki/BuildHints
     def configure_args(self):
         spec = self.spec
@@ -425,6 +436,14 @@ def configure_args(self):
 
         return args
 
+    @run_after('install')
+    @on_package_attributes(run_tests=True)
+    def import_module_test(self):
+        if '+python' in self.spec:
+            with working_dir('..'):
+                for module in self.import_modules:
+                    python('-c', 'import {0}'.format(module))
+
     @run_after('install')
     def darwin_fix(self):
         # The shared library is not installed correctly on Darwin; fix this


Test patch: