Repository: spack/spack

Issue:
`use_xcode = True` doesn't work for Xcode 13+
### Steps to reproduce

Packages like `qt` require the full Xcode suite, and create a copy of the Xcode compilers in order to inject Spack's compiler wrappers. However, this seems to no longer work on newer versions of macOS.
```console
$ spack install -v qt
```
This command opens a prompt asking the user to install Xcode command line tools. However, the command line tools, and the full suite, are both already installed. 

### Error message

```
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
Failed to resolve SDK path for 'macosx'
sh: line 1: 32557 Abort trap: 6           /var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/xcode-select/apple-clang/14.0.0/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk '' -find make 2> /dev/null
make: error: sh -c '/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage/xcode-select/apple-clang/14.0.0/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk '' -find make 2> /dev/null' failed with exit code 34304: (null) (errno=No such file or directory)
xcode-select: Failed to locate 'make', requesting installation of command line developer tools.
```

### Information on your system

* **Spack:** 0.20.0.dev0 (58096f44c763542644354065c3164d4987aa0900)
* **Python:** 3.10.8
* **Platform:** darwin-monterey-m1
* **Concretizer:** clingo

It's unclear if this is an issue with newer macOS, newer Xcode, or arm64. Based on https://github.com/spack/spack/pull/32696#discussion_r1028801122, it may be an issue with Xcode 13+.

@mathstuf @davydden @tgamblin @sethrj

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have searched the issues of this repo and believe this is not a duplicate
- [X] I have run the failing commands in debug mode and reported the output


Discussion:
When you delete the build directory (so `/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/ajstewart/spack-stage` and whatever else) and re-run does it tell you that it's copying XCode over again?

Sometimes when I have issues with Qt and spack that seems to do the trick.
The first time I ran `spack install qt`, I do remember it telling me that it's copying Xcode and not to interrupt the process.

I just ran `spack clean && spack install qt` but it's no longer printing this message. Is there somewhere else where this cache is maintained?
It looks like `spack clean` doesn't touch this directory. If I manually remove it and try again, it does copy Xcode again. Is there a different `spack clean` flag that touches this location?
I'm no spack expert, I've just done what you did and delete that directory manually.
But yeah, when I delete that directory and try again, I encounter the same issue. It can't find make and asks me to install CLT.
I wonder if this has to do with relocatable binaries on Apple silicon?
Can anyone attempt to build Qt on x86_64 macOS with recent Xcode/macOS and see if this is limited to arm64?
Any updates on this? Not really sure how to solve this one.
Appears to be an Xcode 13+ problem. I installed Xcode 12.5.1 on an M1/Monterey Mac, removed the staged Xcode.app copy, and was able to install qt. 

With 13.4 or 14.2 I get the same error when I try to run xcodebuild from the copy:
```
2022-12-26 09:48:04.455 xcodebuild[40552:1424876] Requested but did not find extension point with identifier Xcode.IDEFoundation.TypeRegistry
2022-12-26 09:48:05.483 xcodebuild[40552:1424876] Requested but did not find extension point with identifier Xcode.IDEFoundation.IDEResultKitSerializationConverter
xcodebuild: error: Initialization failed.
	Reason: No macOS Platform was found, but Xcode requires a macOS Platform. You may have an incomplete or damaged installation of Xcode.
```
The same command from `/Applications/Xcode.app` works fine.
@haampie would you be interested in working on this? I know you're still working on fixing macOS binary relocation, this may be related.
> Any updates on this? Not really sure how to solve this one.

I removed `use_xode = True` from the qt package, also disabled `qtlocation` for when building with `+opengl` (it didn't build for me on Linux, either), and then building `qt+dbus+opengl` succeeded as well as running a simple GUI application built against it. I didn't try `+webkit`, though. Perhaps Xcode is not required at all?
Could you open a PR so I can test this?

If we can get this working without copying Xcode, I believe this is the only package that required it, so we might even be able to rip out a lot of broken code in Spack.

Resolving patch:
diff --git a/lib/spack/spack/compilers/apple_clang.py b/lib/spack/spack/compilers/apple_clang.py
--- a/lib/spack/spack/compilers/apple_clang.py
+++ b/lib/spack/spack/compilers/apple_clang.py
@@ -2,13 +2,9 @@
 # Spack Project Developers. See the top-level COPYRIGHT file for details.
 #
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
-import os.path
 import re
-import shutil
 
 import llnl.util.lang
-import llnl.util.tty as tty
-from llnl.util.symlink import symlink
 
 import spack.compiler
 import spack.compilers.clang
@@ -119,108 +115,3 @@ def c23_flag(self):
                 self, "the C23 standard", "c23_flag", "< 11.0.3"
             )
         return "-std=c2x"
-
-    def setup_custom_environment(self, pkg, env):
-        """Set the DEVELOPER_DIR environment for the Xcode toolchain.
-
-        On macOS, not all buildsystems support querying CC and CXX for the
-        compilers to use and instead query the Xcode toolchain for what
-        compiler to run. This side-steps the spack wrappers. In order to inject
-        spack into this setup, we need to copy (a subset of) Xcode.app and
-        replace the compiler executables with symlinks to the spack wrapper.
-        Currently, the stage is used to store the Xcode.app copies. We then set
-        the 'DEVELOPER_DIR' environment variables to cause the xcrun and
-        related tools to use this Xcode.app.
-        """
-        super().setup_custom_environment(pkg, env)
-
-        if not pkg.use_xcode:
-            # if we do it for all packages, we get into big troubles with MPI:
-            # filter_compilers(self) will use mockup XCode compilers on macOS
-            # with Clang. Those point to Spack's compiler wrappers and
-            # consequently render MPI non-functional outside of Spack.
-            return
-
-        # Use special XCode versions of compiler wrappers when using XCode
-        # Overwrites build_environment's setting of SPACK_CC and SPACK_CXX
-        xcrun = spack.util.executable.Executable("xcrun")
-        xcode_clang = xcrun("-f", "clang", output=str).strip()
-        xcode_clangpp = xcrun("-f", "clang++", output=str).strip()
-        env.set("SPACK_CC", xcode_clang, force=True)
-        env.set("SPACK_CXX", xcode_clangpp, force=True)
-
-        xcode_select = spack.util.executable.Executable("xcode-select")
-
-        # Get the path of the active developer directory
-        real_root = xcode_select("--print-path", output=str).strip()
-
-        # The path name can be used to determine whether the full Xcode suite
-        # or just the command-line tools are installed
-        if real_root.endswith("Developer"):
-            # The full Xcode suite is installed
-            pass
-        else:
-            if real_root.endswith("CommandLineTools"):
-                # Only the command-line tools are installed
-                msg = "It appears that you have the Xcode command-line tools "
-                msg += "but not the full Xcode suite installed.\n"
-
-            else:
-                # Xcode is not installed
-                msg = "It appears that you do not have Xcode installed.\n"
-
-            msg += "In order to use Spack to build the requested application, "
-            msg += "you need the full Xcode suite. It can be installed "
-            msg += "through the App Store. Make sure you launch the "
-            msg += "application and accept the license agreement.\n"
-
-            raise OSError(msg)
-
-        real_root = os.path.dirname(os.path.dirname(real_root))
-        developer_root = os.path.join(
-            spack.stage.get_stage_root(), "xcode-select", self.name, str(self.version)
-        )
-        xcode_link = os.path.join(developer_root, "Xcode.app")
-
-        if not os.path.exists(developer_root):
-            tty.warn(
-                "Copying Xcode from %s to %s in order to add spack "
-                "wrappers to it. Please do not interrupt." % (real_root, developer_root)
-            )
-
-            # We need to make a new Xcode.app instance, but with symlinks to
-            # the spack wrappers for the compilers it ships. This is necessary
-            # because some projects insist on just asking xcrun and related
-            # tools where the compiler runs. These tools are very hard to trick
-            # as they do realpath and end up ignoring the symlinks in a
-            # "softer" tree of nothing but symlinks in the right places.
-            shutil.copytree(
-                real_root,
-                developer_root,
-                symlinks=True,
-                ignore=shutil.ignore_patterns(
-                    "AppleTV*.platform",
-                    "Watch*.platform",
-                    "iPhone*.platform",
-                    "Documentation",
-                    "swift*",
-                ),
-            )
-
-            real_dirs = ["Toolchains/XcodeDefault.xctoolchain/usr/bin", "usr/bin"]
-
-            bins = ["c++", "c89", "c99", "cc", "clang", "clang++", "cpp"]
-
-            for real_dir in real_dirs:
-                dev_dir = os.path.join(developer_root, "Contents", "Developer", real_dir)
-                for fname in os.listdir(dev_dir):
-                    if fname in bins:
-                        os.unlink(os.path.join(dev_dir, fname))
-                        symlink(
-                            os.path.join(spack.paths.build_env_path, "cc"),
-                            os.path.join(dev_dir, fname),
-                        )
-
-            symlink(developer_root, xcode_link)
-
-        env.set("DEVELOPER_DIR", xcode_link)
diff --git a/lib/spack/spack/package_base.py b/lib/spack/spack/package_base.py
--- a/lib/spack/spack/package_base.py
+++ b/lib/spack/spack/package_base.py
@@ -528,10 +528,6 @@ class PackageBase(WindowsRPath, PackageViewMixin, metaclass=PackageMeta):
     #: By default do not run tests within package's install()
     run_tests = False
 
-    # FIXME: this is a bad object-oriented design, should be moved to Clang.
-    #: By default do not setup mockup XCode on macOS with Clang
-    use_xcode = False
-
     #: Keep -Werror flags, matches config:flags:keep_werror to override config
     # NOTE: should be type Optional[Literal['all', 'specific', 'none']] in 3.8+
     keep_werror: Optional[str] = None


Test patch:
diff --git a/lib/spack/spack/test/compilers/basics.py b/lib/spack/spack/test/compilers/basics.py
--- a/lib/spack/spack/test/compilers/basics.py
+++ b/lib/spack/spack/test/compilers/basics.py
@@ -4,7 +4,6 @@
 # SPDX-License-Identifier: (Apache-2.0 OR MIT)
 """Test basic behavior of compilers in Spack"""
 import os
-import shutil
 import sys
 from copy import copy
 
@@ -832,119 +831,6 @@ def test_compiler_flags_use_real_version(working_env, monkeypatch, tmpdir):
     assert flag == "-std=c++0x"
 
 
-@pytest.mark.skipif(sys.platform == "win32", reason="Apple Clang and XCode unsupported on Windows")
-def test_apple_clang_setup_environment(mock_executable, monkeypatch):
-    """Test a code path that is taken only if the package uses
-    Xcode on MacOS.
-    """
-
-    class MockPackage:
-        use_xcode = False
-
-    apple_clang_cls = spack.compilers.class_for_compiler_name("apple-clang")
-    compiler = apple_clang_cls(
-        spack.spec.CompilerSpec("apple-clang@=11.0.0"),
-        "catalina",
-        "x86_64",
-        ["/usr/bin/clang", "/usr/bin/clang++", None, None],
-    )
-    env = spack.util.environment.EnvironmentModifications()
-    # Check a package that doesn't use xcode and ensure we don't add changes
-    # to the environment
-    pkg = MockPackage()
-    compiler.setup_custom_environment(pkg, env)
-    assert not env
-
-    # Prepare mock executables to fake the Xcode environment
-    xcrun = mock_executable(
-        "xcrun",
-        """
-if [[ "$2" == "clang" ]] ; then
-  echo "/Library/Developer/CommandLineTools/usr/bin/clang"
-fi
-if [[ "$2" == "clang++" ]] ; then
-  echo "/Library/Developer/CommandLineTools/usr/bin/clang++"
-fi
-""",
-    )
-    mock_executable(
-        "xcode-select",
-        """
-echo "/Library/Developer"
-""",
-    )
-    bin_dir = os.path.dirname(xcrun)
-    monkeypatch.setenv("PATH", bin_dir, prepend=os.pathsep)
-
-    def noop(*args, **kwargs):
-        pass
-
-    real_listdir = os.listdir
-
-    def _listdir(path):
-        if not os.path.exists(path):
-            return []
-        return real_listdir(path)
-
-    # Set a few operations to noop
-    monkeypatch.setattr(shutil, "copytree", noop)
-    monkeypatch.setattr(os, "unlink", noop)
-    monkeypatch.setattr(os, "symlink", noop)
-    monkeypatch.setattr(os, "listdir", _listdir)
-
-    # Qt is so far the only package that uses this code path, change
-    # introduced in https://github.com/spack/spack/pull/1832
-    pkg.use_xcode = True
-    compiler.setup_custom_environment(pkg, env)
-    assert len(env) == 3
-    assert env.env_modifications[0].name == "SPACK_CC"
-    assert env.env_modifications[1].name == "SPACK_CXX"
-    assert env.env_modifications[2].name == "DEVELOPER_DIR"
-
-
-@pytest.mark.skipif(sys.platform == "win32", reason="Not supported on Windows (yet)")
-@pytest.mark.parametrize("xcode_select_output", ["", "/Library/Developer/CommandLineTools"])
-def test_xcode_not_available(xcode_select_output, mock_executable, monkeypatch):
-    # Prepare mock executables to fake the Xcode environment
-    xcrun = mock_executable(
-        "xcrun",
-        """
-    if [[ "$2" == "clang" ]] ; then
-      echo "/Library/Developer/CommandLineTools/usr/bin/clang"
-    fi
-    if [[ "$2" == "clang++" ]] ; then
-      echo "/Library/Developer/CommandLineTools/usr/bin/clang++"
-    fi
-    """,
-    )
-    mock_executable(
-        "xcode-select",
-        """
-    echo "{0}"
-    """.format(
-            xcode_select_output
-        ),
-    )
-    bin_dir = os.path.dirname(xcrun)
-    monkeypatch.setenv("PATH", bin_dir, prepend=os.pathsep)
-    # Prepare compiler
-    apple_clang_cls = spack.compilers.class_for_compiler_name("apple-clang")
-    compiler = apple_clang_cls(
-        spack.spec.CompilerSpec("apple-clang@11.0.0"),
-        "catalina",
-        "x86_64",
-        ["/usr/bin/clang", "/usr/bin/clang++", None, None],
-    )
-    env = spack.util.environment.EnvironmentModifications()
-
-    class MockPackage:
-        use_xcode = True
-
-    pkg = MockPackage()
-    with pytest.raises(OSError):
-        compiler.setup_custom_environment(pkg, env)
-
-
 @pytest.mark.enable_compiler_verification
 def test_compiler_executable_verification_raises(tmpdir):
     compiler = MockCompiler()