Repository: spack/spack

Issue:
petsc and zlib linking problem (strange import/linking problem with python + petsc4py + matplotlib)
using spack compiled python+petsc4py+matplotlib:
```
hron@r0d0$ spack find python py-petsc4py py-matplotlib petsc

-- linux-ubuntu16.04-x86_64 / gcc@7.4.0 -------------------------
petsc@3.8.4  py-matplotlib@3.0.0  py-petsc4py@3.8.1  python@3.6.5
```
I see a strange error in linking:

- working order of imports
```
hron@r0d0$ python
Python 3.6.5 (default, Feb  9 2019, 12:35:34) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import matplotlib.pyplot as plt
>>> import petsc4py
>>> petsc4py.init(sys.argv)
>>> 
```

- non-working order
```
hron@r0d0$ python
Python 3.6.5 (default, Feb  9 2019, 12:35:34) 
[GCC 7.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import petsc4py
>>> petsc4py.init(sys.argv)
>>> import matplotlib.pyplot as plt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/py-matplotlib/3.0.0/56bf2zhzyqlj2lpa/lib/python3.6/site-packages/matplotlib/pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/py-matplotlib/3.0.0/56bf2zhzyqlj2lpa/lib/python3.6/site-packages/matplotlib/colorbar.py", line 32, in <module>
    import matplotlib.contour as contour
  File "/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/py-matplotlib/3.0.0/56bf2zhzyqlj2lpa/lib/python3.6/site-packages/matplotlib/contour.py", line 18, in <module>
    import matplotlib.font_manager as font_manager
  File "/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/py-matplotlib/3.0.0/56bf2zhzyqlj2lpa/lib/python3.6/site-packages/matplotlib/font_manager.py", line 48, in <module>
    from matplotlib import afm, cbook, ft2font, rcParams, get_cachedir
ImportError: /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/libpng/1.6.34/uzwwmyb42qhkd7jr/lib/libpng16.so.16)
```

checking the import error - that zlib rquired by libpng16.so.16 is not found - the linking seems to be ok?
```
hron@r0d0$ ldd /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/libpng/1.6.34/uzwwmyb42qhkd7jr/lib/libpng16.so.16
	linux-vdso.so.1 =>  (0x00007f9819e3b000)
	libz.so.1 => /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/lib/libz.so.1 (0x00007f98197d0000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f98194c7000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f98190fd000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9819c18000)
hron@r0d0$ ldd /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/lib/libz.so.1
        linux-vdso.so.1 =>  (0x00007f8af5723000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8af4f20000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8af5500000)
```

Since i don't see which part is doing something wrong - spack/python/petsc4py/matplotlib - my first try is to ask here firs if anyone have seen this problem? Or if anyone has idea how to check whats is wrong here... 



Discussion:
I have tracked down this problem to petsc package.

The `petsc` package depends on `zlib` -- `depends_on('zlib', when='+hdf5')` 
and configures petsc here https://github.com/spack/spack/blob/fae14bff21e871b41c879869898830213ecf03dd/var/spack/repos/builtin/packages/petsc/package.py#L314
```
        # zlib: configuring using '--with-zlib-dir=...' has some issues with
        # SuiteSparse so specify directly the include path and the libraries.
        if 'zlib' in spec:
            options.extend([
                '--with-zlib-include=%s' % spec['zlib'].prefix.include,
                '--with-zlib-lib=%s'     % spec['zlib'].libs.ld_flags,
                '--with-zlib=1'
            ])
        else:
            options.append('--with-zlib=0')
```
however this doesn't work correctly (at least on ubuntu 16.04) since the resulting `libpetsc.so` resolves to system libz.so
```
hron@r0d0:~$ ldd /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/ekuhsnpb64ajtpvq/lib/libpetsc.so.3.10 | grep libz.so
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f913f7b0000)
```

changing the `petsc/package.py` options setup to:
```
        if 'zlib' in spec:
            options.extend([
                '--with-zlib-dir=%s' % spec['zlib'].prefix,
                '--with-zlib=1'
            ])
        else:
            options.append('--with-zlib=0')
```
works fine and links `libpetsc` to the spack provided zlib correctly.

All this was manifesting by the strange linking error when imports were in "wrong" order as described above...

And I don't see any problem with `--with-zlib-dir` and `suitesparse`...???
@v-dobrev , @acfisher , @balay , @scheibelp 
```
        # zlib: configuring using '--with-zlib-dir=...' has some issues with
        # SuiteSparse so specify directly the include path and the libraries.
```
The commit where this handling of zlib was added is here: https://github.com/spack/spack/pull/7423/commits/71aa29133c4089d0375caa2112a556c80e409b1d. It looks like this was a workaround for an issue with building `ceed@1.0.0` on a Mac.

Can you post the PETSc configuration line for the default build (i.e. the build where you see the issue) on your system -- you can find it inside the file `.spack/build.out` which is inside the PETSc install directory -- the configuration call should be the second line in that file. I suspect the zlib options there may be incorrect for some reason.
here it is - the zlib options are at the end
```
==> Executing phase: 'install'
==> '/usr/bin/python' './config/configure.py' '--prefix=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/tqmis3uc3dqj5hmw' '--with-ssl=0' '--download-c2html=0' '--download-sowing=0' '--download-hwloc=0' 'CFLAGS=' 'FFLAGS=' 'CXXFLAGS=' '--with-cc=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/openmpi/3.1.3/gvzek7zv24y46gqf/bin/mpicc' '--with-cxx=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/openmpi/3.1.3/gvzek7zv24y46gqf/bin/mpic++' '--with-fc=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/openmpi/3.1.3/gvzek7zv24y46gqf/bin/mpif90' '--with-precision=double' '--with-scalar-type=real' '--with-shared-libraries=1' '--with-debugging=0' '--with-64-bit-indices=0' 'COPTFLAGS=' 'FOPTFLAGS=' 'CXXOPTFLAGS=' '--with-blas-lapack-lib=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/openblas/0.3.5/imee52wg6ncam2dx/lib/libopenblas.so' '--with-x=0' '--with-cxx-dialect=C++11' '--with-boost=1' '--with-clanguage=C' '--with-scalapack-lib=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/netlib-scalapack/2.0.2/mejmho6bhsoypyeu/lib/libscalapack.so' '--with-scalapack=1' '--with-metis=1' '--with-metis-dir=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5' '--with-hdf5=1' '--with-hdf5-dir=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald' '--with-hypre=1' '--with-hypre-dir=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hypre/2.15.1/krre4s2ubo3tpgoh' '--with-parmetis=1' '--with-parmetis-dir=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/parmetis/4.0.3/4zvk6r5duoshey5g' '--with-mumps=1' '--with-mumps-dir=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/mumps/5.1.2/x36f6hjgy7l27goa' '--with-trilinos=1' '--with-trilinos-dir=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/trilinos/12.12.1/iqwdvv6rosmz4tg7' '--with-cxx-dialect=C++11' '--with-superlu_dist-include=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/superlu-dist/6.1.0/hb7gu2prtdyryuy6/include' '--with-superlu_dist-lib=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/superlu-dist/6.1.0/hb7gu2prtdyryuy6/lib/libsuperlu_dist.a' '--with-superlu_dist=1' '--with-suitesparse-include=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/suite-sparse/5.3.0/w4u5mgyooheondl5/include' '--with-suitesparse-lib=-L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/suite-sparse/5.3.0/w4u5mgyooheondl5/lib -L/usr/lib/x86_64-linux-gnu -lumfpack -lklu -lcholmod -lbtf -lccolamd -lcolamd -lcamd -lamd -lsuitesparseconfig -lrt' '--with-suitesparse=1' '--with-zlib-include=/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/include' '--with-zlib-lib=-L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/lib -lz' '--with-zlib=1'
```
could you post configure.log from this build? You might have to use 'spack install --keep-stage' to save configure.log
attached
petsc [configure.log](https://github.com/spack/spack/files/3372763/configure.log)
spack [build.out](https://github.com/spack/spack/files/3372807/spack-build.out.log)

this is ubuntu 16.04 system with spack compiled gcc 7.4.0

when using 
```


        if 'zlib' in spec:
            options.extend([
                '--with-zlib-include=%s' % spec['zlib'].prefix.include,
                '--with-zlib-lib=%s'     % spec['zlib'].libs.ld_flags,
                '--with-zlib=1'
            ])
        else:
            options.append('--with-zlib=0')
```

the resulting libpetsc.so links to spack build libs except these (including libz.so):
```
$ ldd /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/mw5cpxm7vxii5rhb/lib/libpetsc.so | grep '/lib/x86_64-linux-gnu/'
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f81d6e8d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f81d3e30000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f81d1433000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f81d1219000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f81d0c92000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f81cf596000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f81cd003000)
	libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f81e567f000)
```

if I use `--with-zlib-dir=spec['zlib'].prefix` then its ok


on the other hand, with fresh new spack checkout just doing `spack install petsc` gives correct linking,
so it might be some problem with our setup of spack - anyway it would be good to know what's going on... 

Looking at the following lines from `configure.log`:
```ascii
...
zlib:
  Includes: -I/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/include
  Library:  -L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/lib -lz
hdf5:
  Includes: -I/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/include
  Library:  -Wl,-rpath,/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/lib -L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5
metis:
  Includes: -I/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/include
  Library:  -Wl,-rpath,/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/lib -L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/lib -lmetis
...
```
it looks like `zlib` is missing the rpath option `-Wl,-rpath,...` -- maybe when using `--with-zlib-lib=...` the rpath option has to be explicitly given?
just additional observations:

1)  the resulting `libpetsc.so` shows
```
$ ldd /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/mw5cpxm7vxii5rhb/lib/libpetsc.so | grep libz.so
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6b940b4000)
```

but the correct path for spack's zlib is in the RPATH record of this `libpetsc.so`
```
$ readelf -d /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/mw5cpxm7vxii5rhb/lib/libpetsc.so | grep RPATH
 0x000000000000000f (RPATH)              Library rpath: [/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/lib64:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/trilinos/12.12.1/ualoxpoq3gshro5n/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/mumps/5.1.2/x7x23ns4tknyhz5e/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/netlib-scalapack/2.0.2/g2zttazadrkuysyh/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/superlu-dist/6.1.1/jztdrd3isnoic6ci/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hypre/2.15.1/krre4s2ubo3tpgoh/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/openblas/0.3.5/imee52wg6ncam2dx/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/parmetis/4.0.3/im6yepsywdai3ijk/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hwloc/1.11.11/bglcj467mlyilgep/lib:/opt/slurm/17.11/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/openmpi/3.1.3/gvzek7zv24y46gqf/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/lib/gcc/x86_64-pc-linux-gnu/7.4.0:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/mw5cpxm7vxii5rhb/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/petsc/3.10.5/mw5cpxm7vxii5rhb/lib64:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/libszip/2.1.1/xzjdve4i5qdpkt4z/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/libiconv/1.15/pgd7moayerdl4sqp/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/glm/0.9.7.1/ibknnub7eezjfymk/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/xz/5.2.4/mruja4bb6rsawuzi/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/netcdf/4.6.2/fjtfu2q7qy6yoy7m/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/suite-sparse/5.3.0/w4u5mgyooheondl5/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/libxml2/2.9.8/e4joz6tm44oibymu/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/numactl/2.0.12/fmitlnk5rbo6h3wm/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/matio/1.5.13/vtbdmzixzcoanb4i/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/gdbm/1.18.1/m4ux4qfppfjm7u6l/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/ncurses/6.1/g6j2jjfiqdlxka6v/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/python/3.6.5/b653h6eghcefjj3v/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/bzip2/1.0.6/fyscgkal4u4bi6lh/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/libpciaccess/0.13.5/o7l7x5o35qxb4z23/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/boost/1.65.1/7ofyg5fwvdgj2qf6/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/readline/7.0/2jwwfsa5p5cx2auk/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/scotch/6.0.6/bm25t6s6ojtbc4tx/lib:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/sqlite/3.26.0/a7lqd3oaw4w6aaf3/lib]
```

however in this RPATH there is also (since I use spack compiled gcc@7.4.0)
```
...:/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/lib/gcc/x86_64-pc-linux-gnu/7.4.0:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:...
```
which comes before zlib and leads to the wrong `libz.so` linking...
at the end it might be some problem with using spack compiled gcc?

my compilers.yaml record:
```
- compiler:
    environment: {}
    extra_rpaths: []
    flags: {}
    modules: []
    operating_system: ubuntu16.04
    paths:
      cc: /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/bin/gcc
      cxx: /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/bin/g++
      f77: /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/bin/gfortran
      fc: /usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc5.4.0/gcc/7.4.0/eh46gqvnxtebkdix/bin/gfortran
    spec: gcc@7.4.0
    target: x86_64
```
> Looking at the following lines from `configure.log`:
> 
> ```
> ...
> zlib:
>   Includes: -I/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/include
>   Library:  -L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/zlib/1.2.11/2oxindpgtak27aeq/lib -lz
> hdf5:
>   Includes: -I/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/include
>   Library:  -Wl,-rpath,/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/lib -L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/hdf5/1.10.5/qhtqx5lyvoufgald/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5
> metis:
>   Includes: -I/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/include
>   Library:  -Wl,-rpath,/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/lib -L/usr/local/pkg/Installs/linux-ubuntu16.04-x86_64-gcc7.4.0/metis/5.1.0/rmlr4fafvumbzsm5/lib -lmetis
> ...
> ```
> 
> it looks like `zlib` is missing the rpath option `-Wl,-rpath,...` -- maybe when using `--with-zlib-lib=...` the rpath option has to be explicitly given?

After some tests - when using
```
                '--with-zlib-lib=%s'     % spec['zlib'].libs.joined(),
```

instead of `'--with-zlib-lib=%s'     % spec['zlib'].libs.ld_flags,` then the petsc configure.log has 
```
zlib:
  Includes: -I/usr/local/pkg/DEV/temp/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb/include
  Library:  -Wl,-rpath,/usr/local/pkg/DEV/temp/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb/lib -L/usr/loc\
al/pkg/DEV/temp/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/zlib-1.2.11-5nus6knzumx4ik2yl44jxtgtsl7d54xb/lib -lz
```

it seems that `--with-xxx-lib=` should be a list of libs not the ld_flags...

this fixes my problem by putting the zlib in the `rpath` before the gcc path

however there still might be potential problem in the rpath with spack compiled gcc - see above?? 

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/petsc/package.py b/var/spack/repos/builtin/packages/petsc/package.py
--- a/var/spack/repos/builtin/packages/petsc/package.py
+++ b/var/spack/repos/builtin/packages/petsc/package.py
@@ -316,7 +316,7 @@ def install(self, spec, prefix):
         if 'zlib' in spec:
             options.extend([
                 '--with-zlib-include=%s' % spec['zlib'].prefix.include,
-                '--with-zlib-lib=%s'     % spec['zlib'].libs.ld_flags,
+                '--with-zlib-lib=%s'     % spec['zlib'].libs.joined(),
                 '--with-zlib=1'
             ])
         else:


Test patch: