Repository: pantsbuild/pants

Issue:
Shellcheck fails on M2 (without Rosetta installed)
After running `pants peek ::` in the Pants main repo, I get the following error:

```bash
Error launching process: Os { code: 86, kind: Uncategorized, message: "Bad CPU type in executable" }
```

From the ShellCheck subsystem, I see that the Mac ARM version points to the x86 binaries (there are no Mac ARM binaries in the GH releases):

```python
class Shellcheck(TemplatedExternalTool):
    options_scope = "shellcheck"
    name = "Shellcheck"
    help = "A linter for shell scripts."

    default_version = "v0.8.0"
    default_known_versions = [
        "v0.8.0|macos_arm64 |e065d4afb2620cc8c1d420a9b3e6243c84ff1a693c1ff0e38f279c8f31e86634|4049756",
        "v0.8.0|macos_x86_64|e065d4afb2620cc8c1d420a9b3e6243c84ff1a693c1ff0e38f279c8f31e86634|4049756",
        "v0.8.0|linux_arm64 |9f47bbff5624babfa712eb9d64ece14c6c46327122d0c54983f627ae3a30a4ac|2996468",
        "v0.8.0|linux_x86_64|ab6ee1b178f014d1b86d1e24da20d1139656c8b0ed34d2867fbb834dad02bf0a|1403852",
    ]

    default_url_template = (
        "https://github.com/koalaman/shellcheck/releases/download/{version}/shellcheck-"
        "{version}.{platform}.tar.xz"
    )
    default_url_platform_mapping = {
        "macos_arm64": "darwin.x86_64",
        "macos_x86_64": "darwin.x86_64",
        "linux_arm64": "linux.aarch64",
        "linux_x86_64": "linux.x86_64",
    }
```

## Mitigation

I think that installing Rosetta might fix this...

## Context

- https://github.com/pantsbuild/pants/issues/14416
- https://github.com/koalaman/shellcheck/issues/2680
- https://github.com/koalaman/shellcheck/issues/2714


## Detailed Error

```bash
native_engine.IntrinsicError: Failed to execute: Process {
    argv: [
        "./shellcheck-v0.8.0/shellcheck",
        "--format=json",
        "cargo",
    ],
    env: {},
    working_directory: None,
    input_digests: InputDigests {
        complete: DirectoryDigest {
            digest: Digest {
                hash: Fingerprint<74712eda6b804fa6f81dea6d4b7008be9e369f40d2c7c2fce92bb59ca25a9ba3>,
                size_bytes: 174,
            },
            tree: "Some(..)",
        },
        nailgun: DirectoryDigest {
            digest: Digest {
                hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>,
                size_bytes: 0,
            },
            tree: "Some(..)",
        },
        inputs: DirectoryDigest {
            digest: Digest {
                hash: Fingerprint<74712eda6b804fa6f81dea6d4b7008be9e369f40d2c7c2fce92bb59ca25a9ba3>,
                size_bytes: 174,
            },
            tree: "Some(..)",
        },
        immutable_inputs: {},
        use_nailgun: {},
    },
    output_files: {},
    output_directories: {},
    timeout: None,
    execution_slot_variable: None,
    concurrency_available: 0,
    description: "Detect Shell imports for cargo",
    level: Debug,
    append_only_caches: {},
    jdk_home: None,
    cache_scope: Always,
    execution_environment: ProcessExecutionEnvironment {
        name: None,
        platform: Macos_arm64,
        strategy: Local,
    },
    remote_cache_speculation_delay: 0ns,
}

Error launching process: Os { code: 86, kind: Uncategorized, message: "Bad CPU type in executable" }
```


Discussion:
I still think one big question is why this was run in the first place... I was `peek`ing, so not sure why a lint rule suddenly ran.
Ok, I see it is because we use ShellCheck for dependency inference. That is why the failure to run shellcheck is getting in the way even when we are not linting.

https://github.com/pantsbuild/pants/blob/9382fe26666915be5b9a7cc9d04f222e3a1be2ec/src/python/pants/backend/shell/dependency_inference.py#L96-L111
Shellcheck is available from VSCode: https://github.com/vscode-shellcheck/shellcheck-binaries/releases

and this is the recommendation of shellcheck itself:

https://github.com/koalaman/shellcheck/blob/90d3172dfec30a7569f95b32479ae97af73b8b2e/README.md?plain=1#L236-L237
> There are currently no official binaries for Apple Silicon, but third party builds are available via [ShellCheck for Visual Studio Code](https://github.com/vscode-shellcheck/shellcheck-binaries/releases).

I'll make a PR to add references to this source of binaries.

Resolving patch:
diff --git a/src/python/pants/backend/shell/lint/shellcheck/subsystem.py b/src/python/pants/backend/shell/lint/shellcheck/subsystem.py
--- a/src/python/pants/backend/shell/lint/shellcheck/subsystem.py
+++ b/src/python/pants/backend/shell/lint/shellcheck/subsystem.py
@@ -20,18 +20,21 @@ class Shellcheck(TemplatedExternalTool):
 
     default_version = "v0.8.0"
     default_known_versions = [
-        "v0.8.0|macos_arm64 |e065d4afb2620cc8c1d420a9b3e6243c84ff1a693c1ff0e38f279c8f31e86634|4049756",
-        "v0.8.0|macos_x86_64|e065d4afb2620cc8c1d420a9b3e6243c84ff1a693c1ff0e38f279c8f31e86634|4049756",
-        "v0.8.0|linux_arm64 |9f47bbff5624babfa712eb9d64ece14c6c46327122d0c54983f627ae3a30a4ac|2996468",
-        "v0.8.0|linux_x86_64|ab6ee1b178f014d1b86d1e24da20d1139656c8b0ed34d2867fbb834dad02bf0a|1403852",
+        "v0.8.0|macos_arm64 |36dffd536b801c8bab2e9fa468163037e0c7f7e0a05298e5ad6299b4dde67e31|14525367",
+        "v0.8.0|macos_x86_64|4e93a76ee116b2f08c88e25011830280ad0d61615d8346364a4ea564b29be3f0|6310442",
+        "v0.8.0|linux_arm64 |8f4810485425636eadce2ec23441fd29d5b1b58d239ffda0a5faf8dd499026f5|4884430",
+        "v0.8.0|linux_x86_64|01d181787ffe63ebb0a2293f63bdc8455c5c30d3a6636320664bfa278424638f|2082242",
     ]
 
+    # We use this third party source since it has pre-compiled binaries for both x86 and aarch.
+    # It is recommended by shellcheck
+    # https://github.com/koalaman/shellcheck/blob/90d3172dfec30a7569f95b32479ae97af73b8b2e/README.md?plain=1#L236-L237
     default_url_template = (
-        "https://github.com/koalaman/shellcheck/releases/download/{version}/shellcheck-"
-        "{version}.{platform}.tar.xz"
+        "https://github.com/vscode-shellcheck/shellcheck-binaries/releases/download/{version}/shellcheck-"
+        "{version}.{platform}.tar.gz"
     )
     default_url_platform_mapping = {
-        "macos_arm64": "darwin.x86_64",
+        "macos_arm64": "darwin.aarch64",
         "macos_x86_64": "darwin.x86_64",
         "linux_arm64": "linux.aarch64",
         "linux_x86_64": "linux.x86_64",
@@ -52,7 +55,7 @@ class Shellcheck(TemplatedExternalTool):
     )
 
     def generate_exe(self, _: Platform) -> str:
-        return f"./shellcheck-{self.version}/shellcheck"
+        return "./shellcheck"
 
     def config_request(self, dirs: Iterable[str]) -> ConfigFilesRequest:
         # Refer to https://www.mankier.com/1/shellcheck#RC_Files for how config files are


Test patch: