hashdist/hashstack — pull request #526 by memmett (2014-12-01 20:55:33+00:00)
Title: [WIP] Add 'edison.yaml'.

Edison (@ NERSC) is a Cray XC30 and supports several different "programming environments" (ie, Intel, GNU etc) through their compiler wrappers (cc, CC, and ftn) and the module system. This is my attempt at supporting this environment.

The trouble that I am having is that the compiler wrappers (cc, CC, ans ftn) seem to forcefully make all executables static, so that `patchelf` fails with: `cannot find section .dynamic`.

Consider the following:

$ ssh edison.nersc.gov
$ module load python # this loads 2.7
$ hit build pfmm-stack.yaml
[bzip2] Building bzip2/to7jje7j4eew, follow log with:
[bzip2] tail -f /global/u1/m/memmett/.hashdist/tmp/bzip2-to7jje7j4eew-4/build.log
[ERROR] [u'/global/u1/m/memmett/.hashdist/bld/patchelf/fjpaaza6eztr/bin/patchelf', '--shrink-rpath', u'/global/u1/m/memmett/.hashdist/bld/bzip2/to7jje7j4eew/bin/bzip2recover'] failed: 1
[bzip2|ERROR] hit command failed: Command [u'/global/u1/m/memmett/.hashdist/bld/patchelf/fjpaaza6eztr/bin/patchelf', '--shrink-rpath', u'/global/u1/m/memmett/.hashdist/bld/bzip2/to7jje7j4eew/bin/bzip2recover'] failed with code 1 and stderr:
cannot find section .dynamic

Note that:

$ file /global/u1/m/memmett/.hashdist/tmp/bzip2-to7jje7j4eew/bzip2recover
/global/u1/m/memmett/.hashdist/tmp/bzip2-to7jje7j4eew/bzip2recover: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), for GNU/Linux 2.6.4, statically linked, not stripped

Not sure where to go from here. Am I going about this incorrectly? Should I be avoiding the Cray compiler wrappers? Is there a way to tell HashDist to skip the rpath dance on Edison?

Thanks!