Repository: spack/spack

Issue:
Julia's package manager certificate not accessible in version 0.5.1
After successfully installing version 0.5.1 non-root users are denied access to Julia's package manager certificate located at `{$ROOT}/etc/curl/cacert.pem`

To my opinion the rights for subdirectory `curl` are not set properly. You might consider adjusting it accordingly. 


Discussion:
@eschnett 
It also looks like we're downloading the cert with `curl` during `install`. This won't work on air-gapped clusters. @eschnett Can you add this file as a checksummed resource and modify the file permissions?
The problem seems to be that `curl` creates a subdirectory with restrictive permissions.

How do I fetch a single file via `resource`? Do you have an example?
The following files contain examples of how to use `resource`:
```
atlas/package.py:    resource(name='lapack',
fenics/package.py:            resource(name=name,
git/package.py:        resource(
llvm/package.py:                resource(name=name,
llvm/package.py:                resource(name=name,
lua/package.py:    resource(
ncl/package.py:    resource(
perl/package.py:    resource(
planck-likelihood/package.py:    resource(
planck-likelihood/package.py:    resource(
planck-likelihood/package.py:    resource(
planck-likelihood/package.py:    resource(
planck-likelihood/package.py:    resource(
rust/package.py:    resource(name='cargo',
```
To fetch a single file instead of a tarball, try adding `expand=False`. If that doesn't work, let me know.
I tried these lines:
```
   resource(name="cacert",
             url="https://curl.haxx.se/ca/cacert-2017-01-18.pem",
             md5="38cd779c9429ab6e2e5ae3437b763238",
             expand=False,
             destination="cacert.pem")
```
This fails with
```
==> Staging archive: /home/eschnett/src/spack/var/spack/stage/julia-0.5.1-ulfd66l4vuuu6osw5med2qsmt4agxztt/julia-0.5.1-full.tar.gz
==> Created stage in /home/eschnett/src/spack/var/spack/stage/julia-0.5.1-ulfd66l4vuuu6osw5med2qsmt4agxztt
==> Already staged resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt in /home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt
==> Moving resource stage
        source : /home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt/
        destination : /home/eschnett/src/spack/var/spack/stage/julia-0.5.1-ulfd66l4vuuu6osw5med2qsmt4agxztt/julia-0.5.1/cacert.pem/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt
==> Error: OSError: [Errno 20] Not a directory: '/home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt/'
/home/eschnett/src/spack/lib/spack/spack/package.py:964, in do_stage:
     957      def do_stage(self, mirror_only=False):
     958          """Unpacks the fetched tarball, then changes into the expanded tarball
     959             directory."""
     960          if not self.spec.concrete:
     961              raise ValueError("Can only stage concrete packages.")
     962
     963          self.do_fetch(mirror_only)
  >> 964          self.stage.expand_archive()
     965          self.stage.chdir_to_source()
```
I notice the directory `/home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt` is empty. I don't think it should be -- shouldn't this be either the downloaded file, or contain the downloaded file?
@tgamblin? Does `resource` not support `expand=False`?
Resource should support ```expand=False```: I've fixed issues with that before e.g. https://github.com/LLNL/spack/pull/1310

I likely won't get a chance to look at this until 3/21. In the meantime: 

If you run ```spack clean julia``` before trying an installation does that help?

If you remove ```destination=cacert.pem``` does it at least get past the staging phase? Certainly it should support setting destination but in case that's the issue.
The original issue can be corrected by a strategically placed `mkdirp` before curl creates the directory.
I usually start with a `spack uninstall`, and a `rm -rf var/spack/stage /tmp/$USER/spack-stage`, since these are often necessary to make things build. Does a `spack clean` do any more than that?
```spack clean``` only removes the staging directory so is a subset of what you did. 
The error message with and without the `destination=` keyword is the same:
```
==> Using cached archive: /home/eschnett/src/spack/var/spack/cache/julia/cacert-0.5.1.None
==> Staging archive: /home/eschnett/src/spack/var/spack/stage/julia-0.5.1-ulfd66l4vuuu6osw5med2qsmt4agxztt/julia-0.5.1-full.tar.gz
==> Created stage in /home/eschnett/src/spack/var/spack/stage/julia-0.5.1-ulfd66l4vuuu6osw5med2qsmt4agxztt
==> Already staged resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt in /home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt
==> Moving resource stage
        source : /home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt/
        destination : /home/eschnett/src/spack/var/spack/stage/julia-0.5.1-ulfd66l4vuuu6osw5med2qsmt4agxztt/julia-0.5.1/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt
==> Error: OSError: [Errno 20] Not a directory: '/home/eschnett/src/spack/var/spack/stage/resource-cacert-ulfd66l4vuuu6osw5med2qsmt4agxztt/'
/home/eschnett/src/spack/lib/spack/spack/package.py:964, in do_stage:
     957      def do_stage(self, mirror_only=False):
     958          """Unpacks the fetched tarball, then changes into the expanded tarball
     959             directory."""
     960          if not self.spec.concrete:
     961              raise ValueError("Can only stage concrete packages.")
     962
     963          self.do_fetch(mirror_only)
  >> 964          self.stage.expand_archive()
     965          self.stage.chdir_to_source()
```


Resolving patch:
diff --git a/var/spack/repos/builtin/packages/julia/package.py b/var/spack/repos/builtin/packages/julia/package.py
--- a/var/spack/repos/builtin/packages/julia/package.py
+++ b/var/spack/repos/builtin/packages/julia/package.py
@@ -150,8 +150,10 @@ def install(self, spec, prefix):
         make("install")
 
         # Julia's package manager needs a certificate
+        cacert_dir = join_path(prefix, "etc", "curl")
+        mkdirp(cacert_dir)
+        cacert_file = join_path(cacert_dir, "cacert.pem")
         curl = which("curl")
-        cacert_file = join_path(prefix, "etc", "curl", "cacert.pem")
         curl("--create-dirs",
              "--output", cacert_file,
              "https://curl.haxx.se/ca/cacert.pem")


Test patch: