Repository: spack/spack

Issue:
Permissions error copying XCode on OSX when trying to install a package
@mathstuf When using Spack on my OSX (10.11.6) laptop with XCode (8.1), and trying to install a package using Spack, it appears to copy the entire XCode directory into Spack and by default it appears I do not have the correct permissions for some folders:

==> Warning: Copying Xcode from /Library to /Users/jrood/spack/var/spack/stage/xcode-select/clang/8.0.0-apple in order to add spack wrappers to it. Please do not interrupt.
==> Error: Error: [('/Library/Application Support/Apple/ParentalControls/Users', '/Users/jrood/spack/var/spack/stage/xcode-select/clang/8.0.0-apple/Application Support/Apple/ParentalControls/Users', "[Errno 13] Permission denied:...............................
.............................
.............................

Is this just a configuration issue on my own laptop? I could probably change the permissions, but is it just possible to copy over just the XCode command line utils to accomplish the compiler wrapping?

jrood-laptop:ParentalControls jrood$ pwd
/Library/Application Support/Apple/ParentalControls
jrood-laptop:ParentalControls jrood$ ls -alh
total 0
drwxr-xr-x   4 root  wheel   136B Aug 27  2015 .
drwxr-xr-x  12 root  wheel   408B Dec  2  2015 ..
drwxr-xr-x   2 root  wheel    68B Aug 27  2015 ALRHelperJobs
drwx------   2 root  wheel    68B Aug 27  2015 Users


Discussion:
Duplicate of #2363. You're not the only one with this problem. @davydden?
I don't have that problem, no idea why 
I was surprised to find that spack now makes a copy of xcode on OSX. 

Is this absolutely necessary?

Since I have several spack instances this wastes quite a bit of space. 
On Thu, Dec 08, 2016 at 13:16:59 -0800, Cyrus Harrison wrote:
> I was surprised to find that spack now makes a copy of xcode on OSX. 
> 
> Is this absolutely necessary?

Yes, some projects only look at Xcode for the compiler (e.g., Qt5), so a
hacked up Xcode is made which replaces the compilers with spack's
wrappers (which are needed to get dependent linker paths and such)..

> Since I have several spack instances this wastes quite a bit of space. 

Work was done to try and minimize what is copied, but there is likely
more to trim if you're interested (it was 12G or so; got it down to
1-2G; at least with the version I had handy).

--Ben

@cyrush: it's really for Qt5 -- this was the only way @mathstuf was able to get it to build.

@mathstuf: it looks like currently makes the copy in the `stage` directory.  Could that be moved to [`misc_cache`](http://spack.readthedocs.io/en/latest/config_yaml.html) so that things can share the same mock XCode and so that it doesn't get purged by the system? (The stage directory can be in a volatile temp dir).

Also, would it be possible to mark packages (e.g., Qt5) as needing this and not do it for others?

would it be possible to only do this for qt5?


I understand the importance of qt5, but my gut tells me trying to mock something as complex as xcode is going to be a long term maintenance nightmare. 

It seems like there should be a way to create qt spec that would point to spack's compiler wrappers.  Obviously I don't know the full details, was there a discussion about what cropped up before the mocked xcode?


@cyrush: So Ben actually tried mocking a Qt spec up first, and ran this up to Qt support.  They basically said that XCode was the only way to build Qt on a mac.  The layer we have to mock up is relatively small -- this basically just gets XCode to point at the OS X compilers. If we could get a thinner mock layer (maybe using more symlinks?) that would be nice but I was relatively convinced when we talked about this that this was the most compatible thing to do.  If you know how to avoid it that would be awesome though.

@mathstuf: is this accurate?
@tgamblin thanks, from that report trying something else could be an endeavor and maybe futile.

I think only doing this mocking when building packages that need it is a good thing to strive for.  It limits the fallout when things go wrong. 
> would it be possible to only do this for qt5?

IIRC, the compiler object is shared for the entire tree, no? So how do we have a package signal that it needs help (which may be on a per-compiler basis) and of what kind? IMO, it's easier just to have the compiler act the same for all packages rather than trying to figure out why you need to do some extra work for package A versus package B, but that's me.

>  If you know how to avoid it that would be awesome though.

Upstream for Qt5 basically said "good luck", so anything we do will need forward ported for Qt5 "forever".

> If we could get a thinner mock layer (maybe using more symlinks?) that would be nice

Possibly. Xcode is really finicky though…it likes piercing your symlinks a lot and then doing relative paths on the resolved paths which get you the naked compilers more often than not (hence the paths which are real in the current setup).

Resolving patch:
diff --git a/lib/spack/spack/compilers/clang.py b/lib/spack/spack/compilers/clang.py
--- a/lib/spack/spack/compilers/clang.py
+++ b/lib/spack/spack/compilers/clang.py
@@ -197,7 +197,32 @@ def setup_custom_environment(self, pkg, env):
             return
 
         xcode_select = Executable('xcode-select')
+
+        # Get the path of the active developer directory
         real_root = xcode_select('--print-path', output=str).strip()
+
+        # The path name can be used to determine whether the full Xcode suite
+        # or just the command-line tools are installed
+        if real_root.endswith('Developer'):
+            # The full Xcode suite is installed
+            pass
+        else:
+            if real_root.endswith('CommandLineTools'):
+                # Only the command-line tools are installed
+                msg  = 'It appears that you have the Xcode command-line tools '
+                msg += 'but not the full Xcode suite installed.\n'
+
+            else:
+                # Xcode is not installed
+                msg  = 'It appears that you do not have Xcode installed.\n'
+
+            msg += 'In order to use Spack to build the requested application, '
+            msg += 'you need the full Xcode suite. It can be installed '
+            msg += 'through the App Store. Make sure you launch the '
+            msg += 'application and accept the license agreement.\n'
+
+            raise OSError(msg)
+
         real_root = os.path.dirname(os.path.dirname(real_root))
         developer_root = os.path.join(spack.stage_path,
                                       'xcode-select',


Test patch: