Repository: saltstack/salt
Pull request: Salt-ssh can deploy a relenv environment
### What does this PR do?
This PR adds a new `--relenv` option to `salt-ssh` that deploys a relenv (onedir) bundled version of Salt and Python on the SSH target. The `--relenv` option works similarly to `--thin` but deploys a bundled Salt environment for cases where it’s preferable to use relenv.

### What issues does this PR fix or reference?
Fixes [#66877](https://github.com/saltstack/salt/issues/66877)

### Previous Behavior
`Salt-ssh` only supported the `--thin` option, which deploys a minimal Salt environment. 

### New Behavior
The new `--relenv` option allows `salt-ssh` to deploy a full Salt environment bundled with Python, offering more flexibility in environments where relenv is preferable. This will eventually become the new default, but for now, it remains an option.

### Merge requirements satisfied?
**[NOTICE] Bug fixes or features added to Salt require tests.**
<!-- Please review the [test documentation](https://docs.saltproject.io/en/master/topics/tutorials/writing_tests.html) for details on how to implement tests into Salt's test suite. -->
- [ ] Docs
- [ ] Changelog - https://docs.saltproject.io/en/master/topics/development/changelog.html
- [ ] Tests written/updated

### Commits signed with GPG?
Yes

Problem statement:
[FEATURE REQUEST] Add `--relenv` Option to `salt-ssh` for Using a Onedir Bundled Version of Salt+Python
**Summary:**

This feature request proposes adding a `--relenv` option to `salt-ssh` that allows downloading and using a bundled version of Salt and Python (relenv) on the SSH target, similar to how the `--thin` option works. This will serve as a transitional feature towards making `relenv` the default method for Salt-SSH deployments in the future.

**Description:**

Currently, `salt-ssh` uses the `--thin` option to copy a minimal version of Salt to the SSH target, along with Python dependencies, and then run Salt remotely. However, with the increasing preference for using `relenv` bundled environments (Salt and Python packaged together), it would be ideal to introduce a `--relenv` option that downloads and uses a one-directory (onedir) version of Salt+Python on the target machine.

The `--relenv` option would follow a similar flow as `--thin`, but instead of copying a minimal installation, it would download and unpack a `relenv` bundle (which includes both Salt and Python). This could be useful for environments where managing Python versions and dependencies on the target machine is difficult or where a fully isolated environment is desired.

### Expected Behavior:

When a user runs `salt-ssh` with the `--relenv` option, Salt should:

1. **Check for `relenv` support on the target**: Ensure that the target system can support the `relenv` bundle.
2. **Download the `relenv` bundle**: Retrieve the pre-packaged Salt+Python bundle (possibly from a pre-configured URL or file server).
3. **Extract and use**: Extract the bundle on the target and use it to run Salt commands.
4. **Handle clean-up**: Optionally clean up the environment after execution (if the user specifies this).
  
This option will allow users to run Salt on machines that may not have Salt or Python pre-installed or where version conflicts exist.

**Proposed CLI Syntax:**

```bash
salt-ssh --relenv <target> test.ping
```

Additional options for managing the `relenv` bundle can include paths to the local or remote onedir package, force re-download, etc.

**Justification:**

The `relenv` bundle represents the future direction for Salt deployments, and adding this as an optional feature in `salt-ssh` will enable users to transition smoothly to the new system. It also simplifies Salt-SSH deployments on targets that have conflicting or missing Python versions and dependencies.

**Current Behavior:**

The current `salt-ssh` approach uses `--thin` to transfer a lightweight Salt environment. This requires Python to be present on the target and can run into compatibility issues depending on the target’s system configuration.

**Additional Context:**

- Once stabilized, `relenv` can eventually become the default for `salt-ssh`.
- This facilitates using salt-ssh to bootstrap `salt` on the targets
- This will facilitate the deployment of ephemeral minions with `salt-ssh`

Hints from issue discussion:
**Implementation Discussion:**

Question: Should we re-use the `--thin` options for the deploy directory when using `--relenv`, or should we create completely separate options for relenv deployment?

- The options will be mutually exclusive, meaning you'll either use `--thin` or `--relenv`, not both simultaneously. Therefore, it should be fine to re-use the `thin_dir` options for relenv deployment as well.

**Consensus:**
- Agreed to re-use the `thin_dir` options for `--relenv` deployments.
- General preference to re-use existing logic where possible.

**Additional Consideration:**

- **Question:** Should `--relenv` check for and replace existing `--thin` deployments, or should it follow the current behavior of doing nothing if the `thin_dir` is already populated?
  
  - **Response:** For now, we'll keep the existing logic and maintain the behavior of doing nothing if the deployment directory (`thin_dir`) is already occupied.

Commit message: Allow fstring linting

Patch to review:
diff --git a/.pylintrc b/.pylintrc
index 5404f6522d0d..1d8f8e512707 100644
--- a/.pylintrc
+++ b/.pylintrc
@@ -452,6 +452,7 @@ disable=R,
   line-too-long,
   locally-disabled,
   logging-format-interpolation,
+  logging-fstring-interpolation,
   missing-docstring,
   no-member,
   protected-access,
diff --git a/changelog/66877.added.md b/changelog/66877.added.md
new file mode 100644
index 000000000000..9988e81b61e0
--- /dev/null
+++ b/changelog/66877.added.md
@@ -0,0 +1 @@
+Add --relenv Option to salt-ssh for Using a Onedir Bundled Salt+Python
diff --git a/salt/client/ssh/__init__.py b/salt/client/ssh/__init__.py
index 6064b7b0064b..140b8bc58cd6 100644
--- a/salt/client/ssh/__init__.py
+++ b/salt/client/ssh/__init__.py
@@ -42,6 +42,7 @@
 import salt.utils.network
 import salt.utils.path
 import salt.utils.platform
+import salt.utils.relenv
 import salt.utils.stringutils
 import salt.utils.thin
 import salt.utils.url
@@ -190,6 +191,59 @@
     ]
 )
 
+
+SSH_SH_SHIM_RELENV = "\n".join(
+    [
+        s.strip()
+        for s in """
+/bin/sh << 'EOF'
+set -e
+set -u
+DEBUG="{DEBUG}"
+if [ -n "$DEBUG" ]; then set -x; fi
+
+SET_PATH="{SET_PATH}"
+if [ -n "$SET_PATH" ]; then export PATH=$SET_PATH; fi
+
+SUDO=""
+if [ -n "{SUDO}" ]; then SUDO="{SUDO} "; fi
+
+SUDO_USER="{SUDO_USER}"
+if [ "$SUDO" ] && [ "$SUDO_USER" ]; then SUDO="$SUDO -u $SUDO_USER"; fi
+
+RELENV_TAR="{THIN_DIR}/salt-relenv.tar.xz"
+mkdir -p "{THIN_DIR}"
+SALT_CALL_BIN="{THIN_DIR}/salt-call"
+
+# Extract relenv tarball if not already extracted
+if [ ! -x "$SALT_CALL_BIN" ]; then
+    if [ ! -f "$RELENV_TAR" ]; then
+        echo deploy
+        echo "ERROR: relenv tarball not found at $RELENV_TAR" >&2
+        exit 11
+    fi
+
+    # Create directory if not exists and extract the tarball
+    tar --strip-components=1 -xf "$RELENV_TAR" -C "{THIN_DIR}"
+fi
+
+# Check if Python binary is executable
+if [ ! -x "$SALT_CALL_BIN" ]; then
+    echo "ERROR: salt-call binary not found or not executable at $SALT_CALL_BIN" >&2
+    exit 1
+fi
+
+echo "{RSTR}"
+echo "{RSTR}" >&2
+
+exec $SUDO "$SALT_CALL_BIN" --retcode-passthrough --local --metadata --out=json -lquiet -c "{THIN_DIR}" {ARGS}
+EOF
+""".split(
+            "\n"
+        )
+    ]
+)
+
 if not salt.utils.platform.is_windows() and not salt.utils.platform.is_junos():
     shim_file = os.path.join(os.path.dirname(__file__), "ssh_py_shim.py")
     if not os.path.exists(shim_file):
@@ -310,15 +364,18 @@ def __init__(self, opts):
             self.opts["ssh_wipe"] = "True"
         self.returners = salt.loader.returners(self.opts, {})
         self.fsclient = salt.fileclient.FSClient(self.opts)
-        self.thin = salt.utils.thin.gen_thin(
-            self.opts["cachedir"],
-            extra_mods=self.opts.get("thin_extra_mods"),
-            overwrite=self.opts["regen_thin"],
-            extended_cfg=self.opts.get("ssh_ext_alternatives"),
-            exclude_saltexts=self.opts.get("thin_exclude_saltexts", False),
-            saltext_allowlist=self.opts.get("thin_saltext_allowlist"),
-            saltext_blocklist=self.opts.get("thin_saltext_blocklist"),
-        )
+        if self.opts.get("relenv"):
+            self.thin = None
+        else:
+            self.thin = salt.utils.thin.gen_thin(
+                self.opts["cachedir"],
+                extra_mods=self.opts.get("thin_extra_mods"),
+                overwrite=self.opts["regen_thin"],
+                extended_cfg=self.opts.get("ssh_ext_alternatives"),
+                exclude_saltexts=self.opts.get("thin_exclude_saltexts", False),
+                saltext_allowlist=self.opts.get("thin_saltext_allowlist"),
+                saltext_blocklist=self.opts.get("thin_saltext_blocklist"),
+            )
         self.mods = mod_data(self.fsclient)
 
     # __setstate__ and __getstate__ are only used on spawning platforms.
@@ -1071,7 +1128,73 @@ def __init__(
             # Determine if Windows client is x86 or AMD64
             arch, _, _ = self.shell.exec_cmd("powershell $ENV:PROCESSOR_ARCHITECTURE")
             self.arch = arch.strip()
-        self.thin = thin if thin else salt.utils.thin.thin_path(opts["cachedir"])
+
+        if self.opts.get("relenv"):
+            kernel, os_arch = self.detect_os_arch()
+            self.thin = salt.utils.relenv.gen_relenv(
+                opts["cachedir"], kernel=kernel, os_arch=os_arch
+            )
+        else:
+            self.thin = thin if thin else salt.utils.thin.thin_path(opts["cachedir"])
+
+    def detect_os_arch(self):
+        """
+        Detect the OS and architecture of the target machine.
+        This is specifically for the purpose of downloading the latest onedir tarball from the Salt repos.
+        Returns a tuple of (kernel, architecture) or raises an error if detection fails.
+        """
+        # Unified command for Unix-based systems (including fallback to OSTYPE and MACHTYPE)
+        unix_cmd = 'uname -s -m || echo "$OSTYPE $MACHTYPE"'
+
+        # Command for Windows systems (PowerShell)
+        windows_cmd = 'echo "$env:PROCESSOR_ARCHITECTURE"'
+
+        # Try Unix command first
+        stdout, stderr, retcode = self.shell.exec_cmd(unix_cmd)
+
+        if retcode == 0 and stdout:
+            # Unix-based detection succeeded
+            stdout = stdout.lower().strip()
+
+            # Determine OS and architecture for Unix
+            if "linux" in stdout:
+                kernel = "linux"
+            elif "darwin" in stdout or "macos" in stdout:
+                kernel = "macos"
+            else:
+                raise ValueError(f"Unsupported Unix-based kernel: {stdout}")
+
+            # Set architecture
+            if "x86" in stdout:
+                os_arch = "x86_64"
+            else:
+                os_arch = "arm64"
+        else:
+            # If Unix detection fails, check for Windows-specific detection
+            stdout, stderr, retcode = self.shell.exec_cmd(windows_cmd)
+
+            if retcode == 0 and stdout:
+                # Windows detection
+                stdout = stdout.lower().strip()
+
+                # Set Windows architecture based on environment variable
+                if "64" in stdout:
+                    os_arch = "amd64"
+                elif "x86" in stdout:
+                    os_arch = "x86"
+                else:
+                    raise ValueError(f"Unsupported architecture for Windows: {stdout}")
+
+                kernel = "windows"
+            else:
+                # Neither Unix nor Windows detection succeeded
+                raise ValueError(
+                    f"Failed to detect OS and architecture. Commands failed with output: {stdout}, {stderr}"
+                )
+
+        log.info(f'Detected kernel "{kernel}" and architecture "{os_arch}" on target')
+
+        return kernel, os_arch
 
     def __arg_comps(self):
         """
@@ -1145,10 +1268,16 @@ def deploy(self):
         """
         Deploy salt-thin
         """
-        self.shell.send(
-            self.thin,
-            os.path.join(self.thin_dir, "salt-thin.tgz"),
-        )
+        if self.opts.get("relenv"):
+            self.shell.send(
+                self.thin,
+                os.path.join(self.thin_dir, "salt-relenv.tar.xz"),
+            )
+        else:
+            self.shell.send(
+                self.thin,
+                os.path.join(self.thin_dir, "salt-thin.tgz"),
+            )
         self.deploy_ext()
         return True
 
@@ -1381,6 +1510,7 @@ def run_wfunc(self):
                 self.args = mine_args
                 self.kwargs = {}
 
+        retcode = 0
         try:
             if self.mine:
                 result = wrapper[mine_fun](*self.args, **self.kwargs)
@@ -1442,12 +1572,24 @@ def _cmd_str(self):
             cachedir = self.opts["_caller_cachedir"]
         else:
             cachedir = self.opts["cachedir"]
-        thin_code_digest, thin_sum = salt.utils.thin.thin_sum(cachedir, "sha1")
         debug = ""
         if not self.opts.get("log_level"):
             self.opts["log_level"] = "info"
         if LOG_LEVELS["debug"] >= LOG_LEVELS[self.opts.get("log_level", "info")]:
             debug = "1"
+
+        if self.opts.get("relenv"):
+            return SSH_SH_SHIM_RELENV.format(
+                DEBUG=debug,
+                SUDO=sudo,
+                SUDO_USER=sudo_user or "",
+                THIN_DIR=self.thin_dir,
+                SET_PATH=self.set_path,
+                RSTR=RSTR,
+                ARGS=" ".join(self.argv),
+            )
+
+        thin_code_digest, thin_sum = salt.utils.thin.thin_sum(cachedir, "sha1")
         arg_str = '''
 OPTIONS.config = \
 """
diff --git a/salt/utils/parsers.py b/salt/utils/parsers.py
index b269745641e4..97b3e4f7667f 100644
--- a/salt/utils/parsers.py
+++ b/salt/utils/parsers.py
@@ -3094,6 +3094,7 @@ class SaltSSHOptionParser(
     ]
 
     def _mixin_setup(self):
+        # pylint: disable=W0106
         self.add_option(
             "-r",
             "--raw",
@@ -3227,6 +3228,16 @@ def _mixin_setup(self):
                 "custom grains/modules/states have been added or updated."
             ),
         )
+        self.add_option(
+            "--relenv",
+            dest="relenv",
+            default=False,
+            action="store_true",
+            help=(
+                "Deploy and use a relenv (Salt+Python bundled) environment on the "
+                "SSH target."
+            ),
+        ),
         self.add_option(
             "--python2-bin",
             default="python2",
@@ -3410,6 +3421,9 @@ def _mixin_after_parsed(self):
                         option.explicit = True
                         break
 
+        if self.options.regen_thin and self.options.relenv:
+            self.error("--thin and --relenv are mutually exclusive")
+
     def setup_config(self):
         return config.master_config(self.get_config_file_path())
 
diff --git a/salt/utils/relenv.py b/salt/utils/relenv.py
new file mode 100644
index 000000000000..ff07b25c97a2
--- /dev/null
+++ b/salt/utils/relenv.py
@@ -0,0 +1,91 @@
+import logging
+import os
+import re
+
+import requests
+
+import salt.utils.files
+import salt.utils.hashutils
+import salt.utils.http
+import salt.utils.thin
+
+log = logging.getLogger(__name__)
+
+
+def gen_relenv(
+    cachedir,
+    kernel,
+    os_arch,
+    overwrite=False,
+):
+    """
+    Deploy salt-relenv.
+    :param cachedir: The cache directory where the downloaded tarball will be stored.
+    :param kernel: The detected OS (e.g., 'linux', 'darwin', 'windows').
+    :param os_arch: The detected architecture (e.g., 'amd64', 'x86_64', 'arm64').
+    :param overwrite: Whether to overwrite the existing cached tarball.
+    :return: The path to the recompressed .tgz file.
+    """
+    # Set up directories
+    relenv_dir = os.path.join(cachedir, "relenv", kernel, os_arch)
+    if not os.path.isdir(relenv_dir):
+        os.makedirs(relenv_dir)
+
+    relenv_url = get_tarball(kernel, os_arch)
+    tarball_path = os.path.join(relenv_dir, "salt-relenv.tar.xz")
+
+    # Download the tarball if it doesn't exist or overwrite is True
+    if overwrite or not os.path.exists(tarball_path):
+        if not download(cachedir, relenv_url, tarball_path):
+            return False
+
+    return tarball_path
+
+
+def get_tarball(kernel, arch):
+    """
+    Get the latest Salt onedir tarball URL for the specified kernel and architecture.
+    :param kernel: The detected OS (e.g., 'linux', 'darwin', 'windows').
+    :param arch: The detected architecture (e.g., 'amd64', 'x86_64', 'arm64').
+    :return: The URL of the latest tarball.
+    """
+    base_url = "https://repo.saltproject.io/salt/py3/onedir/latest/"
+    try:
+        # Request the page listing
+        response = requests.get(base_url, timeout=60)
+        response.raise_for_status()
+    except requests.RequestException as e:
+        log.error(f"Failed to retrieve tarball listing: {e}")
+        raise ValueError("Unable to fetch tarball list from repository")
+
+    # Search for tarball filenames that match the kernel and arch
+    pattern = re.compile(rf'href="(salt-.*-onedir-{kernel}-{arch}\.tar\.xz)"')
+    matches = pattern.findall(response.text)
+    if not matches:
+        log.error(f"No tarballs found for {kernel} and {arch}")
+        raise ValueError(f"No tarball found for {kernel} {arch}")
+
+    # Return the latest tarball URL
+    matches.sort()
+    latest_tarball = matches[-1]
+    return base_url + latest_tarball
+
+
+def download(cachedir, url, destination):
+    """
+    Download the salt artifact from the given destination to the cache.
+    """
+    if not os.path.exists(destination):
+        log.info(f"Downloading from {url} to {destination}")
+        with salt.utils.files.fopen(destination, "wb+") as dest_file:
+            result = salt.utils.http.query(
+                url=url,
+                method="GET",
+                stream=True,
+                streaming_callback=dest_file.write,
+                raise_error=True,
+            )
+            if result.get("status") != 200:
+                log.error(f"Failed to download file from {url}")
+                return False
+    return True
diff --git a/tests/pytests/integration/ssh/test_deploy.py b/tests/pytests/integration/ssh/test_deploy.py
index 1760f00b2ed6..524a2bcc26f9 100644
--- a/tests/pytests/integration/ssh/test_deploy.py
+++ b/tests/pytests/integration/ssh/test_deploy.py
@@ -163,6 +163,28 @@ def test_thin_dir(salt_ssh_cli):
     assert thin_dir.joinpath("running_data").exists()
 
 
+def test_relenv_dir(salt_ssh_cli):
+    """
+    test to make sure thin_dir is created
+    and salt-call file is included
+    """
+    ret = salt_ssh_cli.run("--relenv", "config.get", "thin_dir")
+    assert ret.returncode == 0
+    thin_dir = pathlib.Path(ret.data)
+    assert thin_dir.is_dir()
+    assert thin_dir
+    assert thin_dir.joinpath("salt-call").exists()
+
+
+def test_relenv_ping(salt_ssh_cli):
+    """
+    Test a simple ping
+    """
+    ret = salt_ssh_cli.run("--relenv", "test.ping")
+    assert ret.returncode == 0
+    assert ret.data is True.joinpath("salt-call").exists()
+
+
 def test_wipe(salt_ssh_cli):
     """
     Ensure --wipe is respected by the state module wrapper


Code review comments:
diff_hunk: @@ -163,6 +163,28 @@ def test_thin_dir(salt_ssh_cli):
     assert thin_dir.joinpath("running_data").exists()
 
 
+def test_relenv_dir(salt_ssh_cli):
+    """
+    test to make sure thin_dir is created
+    and salt-call file is included
+    """
+    ret = salt_ssh_cli.run("--relenv", "config.get", "thin_dir")
+    assert ret.returncode == 0
+    thin_dir = pathlib.Path(ret.data)
+    assert thin_dir.is_dir()
+    assert thin_dir
+    assert thin_dir.joinpath("salt-call").exists()
+
+
+def test_relenv_ping(salt_ssh_cli):
+    """
+    Test a simple ping
+    """
+    ret = salt_ssh_cli.run("--relenv", "test.ping")
+    assert ret.returncode == 0
+    assert ret.data is True.joinpath("salt-call").exists()
path: tests/pytests/integration/ssh/test_deploy.py
text: @user1:
Something's not right here... copy/paste error?

@author:
nice catch, thanks