Repository: spack/spack

Issue:
Invalid flag with apple m2 cpu and apple-clang (clang-1400.0.29.202)
On a system as described in the title, the following flag in not recognized and causes compilation failures:

https://github.com/spack/spack/blob/b2c806f6fcd73ae624582c84a177e465a7877cf7/lib/spack/external/archspec/json/cpu/microarchitectures.json#L2794

Replacing with `-mcpu=apple-m1` seems to fix the issue. There seems to be no flag specific to m2 cpu.



Discussion:
I'm new on AppleM2, and I addressed the same problem.

Just for the records this is the list of supported cpu flags
```
clang --print-supported-cpus
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Available CPUs for this target:

	a64fx
	apple-a10
	apple-a11
	apple-a12
	apple-a13
	apple-a14
	apple-a7
	apple-a8
	apple-a9
	apple-latest
	apple-m1
	apple-s4
	apple-s5
	carmel
	cortex-a34
	cortex-a35
	cortex-a510
	cortex-a53
	cortex-a55
	cortex-a57
	cortex-a65
	cortex-a65ae
	cortex-a72
	cortex-a73
	cortex-a75
	cortex-a76
	cortex-a76ae
	cortex-a77
	cortex-a78
	cortex-a78c
	cortex-r82
	cortex-x1
	cyclone
	exynos-m3
	exynos-m4
	exynos-m5
	falkor
	generic
	kryo
	neoverse-e1
	neoverse-n1
	neoverse-n2
	neoverse-v1
	saphira
	thunderx
	thunderx2t99
	thunderx3t110
	thunderxt81
	thunderxt83
	thunderxt88
	tsv110

Use -mcpu or -mtune to specify the target's processor.
For example, clang --target=aarch64-unknown-linux-gui -mcpu=cortex-a35
```
See also https://github.com/archspec/archspec-json/issues/63
@albestro @v-dobrev Can you check what native resolves to in your case, and post it here?
```bash
➜  ~ echo "void derp() {}" | clang -mcpu=native -c -S -v -x c++ - -o - 2>&1 | egrep -o "\-target-cpu [^ ]+"
-target-cpu apple-a12
```

target-* flags list
```
➜  ~ echo "void derp() {}" | clang -mcpu=native -c -S -v -x c++ - -o - 2>&1 | egrep -o "\-target-[^ ]+ [^ ]+"
-target-sdk-version=13.1 -fvisibility-inlines-hidden-static-local-var
-target-cpu apple-a12
-target-feature +v8.3a
-target-feature +fp-armv8
-target-feature +neon
-target-feature +crc
-target-feature +crypto
-target-feature +fullfp16
-target-feature +ras
-target-feature +lse
-target-feature +rdm
-target-feature +rcpc
-target-feature +zcm
-target-feature +zcz
-target-feature +sha2
-target-feature +aes
-target-abi darwinpcs
-target-linker-version 820.1
```

Extended output
```bash
➜  ~ echo "void derp() {}" | clang -mcpu=native -c -S -v -x c++ - -o - 2>&1 | grep cpu
 "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple arm64-apple-macosx13.0.0 -Wundef-prefix=TARGET_OS_ -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -S -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name - -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -fno-strict-return -fno-rounding-math -funwind-tables=2 -fobjc-msgsend-selector-stubs -target-sdk-version=13.1 -fvisibility-inlines-hidden-static-local-var -target-cpu apple-a12 -target-feature +v8.3a -target-feature +fp-armv8 -target-feature +neon -target-feature +crc -target-feature +crypto -target-feature +fullfp16 -target-feature +ras -target-feature +lse -target-feature +rdm -target-feature +rcpc -target-feature +zcm -target-feature +zcz -target-feature +sha2 -target-feature +aes -target-abi darwinpcs -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 820.1 -v -resource-dir /Library/Developer/CommandLineTools/usr/lib/clang/14.0.0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Library/Developer/CommandLineTools/usr/lib/clang/14.0.0/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -internal-externc-isystem /Library/Developer/CommandLineTools/usr/include -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough -Wno-enum-enum-conversion -Wno-enum-float-conversion -Wno-elaborated-enum-base -Wno-reserved-identifier -Wno-gnu-folding-constant -Wno-cast-function-type -Wno-bitwise-instead-of-logical -fdeprecated-macro -fdebug-compilation-dir=/Users/ialberto -ferror-limit 19 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fno-cxx-modules -fcxx-exceptions -fexceptions -fmax-type-align=16 -fcommon -clang-vendor-feature=+messageToSelfInClassMethodIdReturnType -clang-vendor-feature=+disableInferNewAvailabilityFromInit -clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation -fno-odr-hash-protocols -clang-vendor-feature=+enableAggressiveVLAFolding -clang-vendor-feature=+revert09abecef7bbf -clang-vendor-feature=+thisNoAlignAttr -clang-vendor-feature=+thisNoNullAttr -mllvm -disable-aligned-alloc-awareness=1 -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o - -x c++ -
```

IMHO this is going to be anyway a temporary fix, because `A12` does not look correct.
FIY, using Clang LLVM provided by HomeBrew

```
➜  ~ echo "void derp() {}" | /opt/homebrew/opt/llvm@12/bin/clang -mcpu=native -c -S -v -x c++ - -o - 2>&1 | egrep -o "\-target-[^ ]+ [^ ]+"
-target-cpu cyclone
-target-feature +fp-armv8
-target-feature +neon
-target-feature +crypto
-target-feature +zcm
-target-feature +zcz
-target-feature +sha2
-target-feature +aes
-target-abi darwinpcs
-target-linker-version 820.1
```
I see the same output as @albestro.

I'll just add that without explicit `-mcpu=` flag, the default is `-target-cpu apple-m1` with the following `-target-feature`s:
```
-target-feature +v8.5a
-target-feature +fp-armv8
-target-feature +neon
-target-feature +crc
-target-feature +crypto
-target-feature +dotprod
-target-feature +fp16fml
-target-feature +ras
-target-feature +lse
-target-feature +rdm
-target-feature +rcpc
-target-feature +zcm
-target-feature +zcz
-target-feature +fullfp16
-target-feature +sm4
-target-feature +sha3
-target-feature +sha2
-target-feature +aes
```
This is on a system with M2 Max chip -- I'm sure if the above will be different with other Apple silicon chips or these are the built-in defaults for the compiler on any Apple silicon system.
Just got an update for "Command Line Tools" and now it has `apple-m2` as a supported flag! 🎉 🍾 

```
➜  ~ clang --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
➜  ~ clang --print-supported-cpus
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Available CPUs for this target:

	a64fx
	apple-a10
	apple-a11
	apple-a12
	apple-a13
	apple-a14
	apple-a15
	apple-a16
	apple-a7
	apple-a8
	apple-a9
	apple-latest
	apple-m1
	apple-m2
	apple-s4
	apple-s5
	carmel
	cortex-a34
	cortex-a35
	cortex-a510
	cortex-a53
	cortex-a55
	cortex-a57
	cortex-a65
	cortex-a65ae
	cortex-a710
	cortex-a72
	cortex-a73
	cortex-a75
	cortex-a76
	cortex-a76ae
	cortex-a77
	cortex-a78
	cortex-a78c
	cortex-r82
	cortex-x1
	cortex-x1c
	cortex-x2
	cyclone
	exynos-m3
	exynos-m4
	exynos-m5
	falkor
	generic
	kryo
	neoverse-512tvb
	neoverse-e1
	neoverse-n1
	neoverse-n2
	neoverse-v1
	saphira
	thunderx
	thunderx2t99
	thunderx3t110
	thunderxt81
	thunderxt83
	thunderxt88
	tsv110

Use -mcpu or -mtune to specify the target's processor.
For example, clang --target=aarch64-unknown-linux-gui -mcpu=cortex-a35
```
I'm not sure what changed but in latest `develop` (b524351e) the `SPACK_TARGET_ARGS` variable is now empty in the build environment, regardless of the value here: https://github.com/spack/spack/blob/b524351e102b498e90768cafc2e5db77c0a531b9/lib/spack/external/archspec/json/cpu/microarchitectures.json#L2794
I figured out why `SPACK_TARGET_ARGS` was suddenly empty -- I added `gfortran` to the `apple-clang` compilers.

Removing `gfortran`, I was able to successfully compile with `Apple clang version 14.0.3 (clang-1403.0.22.14.1)`, replacing `-mcpu=vortex` with `-mcpu=apple-m2 -mtune=apple-m2`.
I think https://github.com/archspec/archspec-json/pull/65 would fix the issues here. Can people confirm ? If so, I'll merge it into archspec and port it to Spack.

Resolving patch:
diff --git a/lib/spack/external/__init__.py b/lib/spack/external/__init__.py
--- a/lib/spack/external/__init__.py
+++ b/lib/spack/external/__init__.py
@@ -18,7 +18,7 @@
 
 * Homepage: https://pypi.python.org/pypi/archspec
 * Usage: Labeling, comparison and detection of microarchitectures
-* Version: 0.2.0 (commit e44bad9c7b6defac73696f64078b2fe634719b62)
+* Version: 0.2.0-dev (commit f3667f95030c6573842fb5f6df0d647285597509)
 
 astunparse
 ----------------
diff --git a/lib/spack/external/archspec/cli.py b/lib/spack/external/archspec/cli.py
--- a/lib/spack/external/archspec/cli.py
+++ b/lib/spack/external/archspec/cli.py
@@ -6,19 +6,61 @@
 archspec command line interface
 """
 
-import click
+import argparse
+import typing
 
 import archspec
 import archspec.cpu
 
 
-@click.group(name="archspec")
-@click.version_option(version=archspec.__version__)
-def main():
-    """archspec command line interface"""
+def _make_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        "archspec",
+        description="archspec command line interface",
+        add_help=False,
+    )
+    parser.add_argument(
+        "--version",
+        "-V",
+        help="Show the version and exit.",
+        action="version",
+        version=f"archspec, version {archspec.__version__}",
+    )
+    parser.add_argument("--help", "-h", help="Show the help and exit.", action="help")
 
+    subcommands = parser.add_subparsers(
+        title="command",
+        metavar="COMMAND",
+        dest="command",
+    )
 
-@main.command()
-def cpu():
-    """archspec command line interface for CPU"""
-    click.echo(archspec.cpu.host())
+    cpu_command = subcommands.add_parser(
+        "cpu",
+        help="archspec command line interface for CPU",
+        description="archspec command line interface for CPU",
+    )
+    cpu_command.set_defaults(run=cpu)
+
+    return parser
+
+
+def cpu() -> int:
+    """Run the `archspec cpu` subcommand."""
+    print(archspec.cpu.host())
+    return 0
+
+
+def main(argv: typing.Optional[typing.List[str]] = None) -> int:
+    """Run the `archspec` command line interface."""
+    parser = _make_parser()
+
+    try:
+        args = parser.parse_args(argv)
+    except SystemExit as err:
+        return err.code
+
+    if args.command is None:
+        parser.print_help()
+        return 0
+
+    return args.run()


Test patch: