Repository: spack/spack

Issue:
"spack install" aborts when installed software contains dangling symlink
On one of my systems, "spack install gcc" consistently fails with an error

==> Error: FileNotFoundError: [Errno 2] No such file or directory:  '.../spack/opt/spack/linux-ubuntu18.04-sandybridge/gcc-7.4.0/gcc-9.2.0-hazow2qor6gvdcxds2ds7wsvufigvclz/lib/gcc/x86_64-pc-linux-gnu/9.2.0/include-fixed/python3.6'

It turns out that the "fixincludes" process (https://gcc.gnu.org/onlinedocs/gcc/Fixed-Headers.html) inappropriately creates a dangling symlink "python3.6 -> python3.6m" in the include-fixed directory. The system's /usr/include contains this symlink, which points to a directory containing a number of header files which are part of Ubuntu's libpython3.6-dev package.

While I cannot recreate this behaviour on several other test servers, the existence of a dangling symlink in the installed software tree should not cause "spack install" to fail.


### Steps to reproduce the issue

The following is a minimal synthetic example to demonstrate this behaviour. See attached file makedang.txt .

```console
#!/bin/bash
  
set -ex
version=0.0.1
name=dangsym
mkdir $name-$version
cd $name-$version

cat > configure <<'EOF'
#!/bin/bash
echo "this is the dangsym configure script"
echo "args:" "$@"
case "$1" in
--prefix*)
  prefix=${1#--prefix=}
;;
esac

cat > install.sh <<STOP
set -ex
mkdir -p $prefix
cd $prefix
touch file.abc
ln -s b a  # create dangling symlink
ls -l
echo "install complete"
exit 0
STOP

chmod +x install.sh
EOF

chmod +x configure

cat > Makefile <<'EOF'
.SILENT: all
all: ; echo nothing to make
install: ; ./install.sh
EOF

cd ..
tar cvfz $name-$version.tar.gz $name-$version
mv $name-$version.tar.gz /tmp

echo "for spack create just accept the defaults"
spack create file://localhost/tmp/$name-$version.tar.gz
spack install --verbose $name

```

### Error Message

On a pristine Spack (releases/v0.13) installation, copy attached script and execute it

```console
bash ./makedang.txt
[...]
spec : dangsym@0.0.1%gcc@7.4.0 arch=linux-ubuntu18.04-sandybridge
==> Error: FileNotFoundError: [Errno 2] No such file or directory: '/home/user/spack/test1/spack/opt/spack/linux-ubuntu18.04-sandybridge/gcc-7.4.0/dangsym-0.0.1-qhz4ajbcsouinxcejbjtc6kygokp2kwh/a'

/home/user/spack/test1/spack/lib/spack/spack/package.py:1718, in build_process:
       1715                    echo = logger.echo
       1716                    self.log()
       1717
  >>   1718                # Run post install hooks before build stage is removed.
       1719                spack.hooks.post_install(self.spec)
       1720
       1721            # Stop timer.

See build log for details:
  /tmp/user/spack-stage/spack-stage-dangsym-0.0.1-qhz4ajbcsouinxcejbjtc6kygokp2kwh/spack-build-out.txt
```

The log file contains no error messages.

### Information on your system

user@ubuntu18043server:~$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

This is a fresh Spack install (git clone https://github.com/spack/spack.git; git checkout releases/v0.13) with no modifications.


[makedang.txt](https://github.com/spack/spack/files/3824998/makedang.txt)



Discussion:

Resolving patch:
diff --git a/lib/spack/spack/verify.py b/lib/spack/spack/verify.py
--- a/lib/spack/spack/verify.py
+++ b/lib/spack/spack/verify.py
@@ -28,24 +28,26 @@ def compute_hash(path):
 
 def create_manifest_entry(path):
     data = {}
-    stat = os.stat(path)
 
-    data['mode'] = stat.st_mode
-    data['owner'] = stat.st_uid
-    data['group'] = stat.st_gid
+    if os.path.exists(path):
+        stat = os.stat(path)
 
-    if os.path.islink(path):
-        data['type'] = 'link'
-        data['dest'] = os.readlink(path)
+        data['mode'] = stat.st_mode
+        data['owner'] = stat.st_uid
+        data['group'] = stat.st_gid
 
-    elif os.path.isdir(path):
-        data['type'] = 'dir'
+        if os.path.islink(path):
+            data['type'] = 'link'
+            data['dest'] = os.readlink(path)
 
-    else:
-        data['type'] = 'file'
-        data['hash'] = compute_hash(path)
-        data['time'] = stat.st_mtime
-        data['size'] = stat.st_size
+        elif os.path.isdir(path):
+            data['type'] = 'dir'
+
+        else:
+            data['type'] = 'file'
+            data['hash'] = compute_hash(path)
+            data['time'] = stat.st_mtime
+            data['size'] = stat.st_size
 
     return data
 


Test patch: