Repository: Sceptre/sceptre

Issue:
Stack outputs in stack groups hide the stack names
### An export format for stack outputs that shows stack name is needed
The main purpose of stack outputs is to consume them in other stacks. Example:

```yaml
VpcId: !stack_output network.yaml::VpcId
```

or in a var file:

```yaml
VpcId: '!stack_output_external my-network-stack::VpcId'
```

In the case of stack groups however, there is no easy way to know which stack exports which outputs without either reading the source, going to the AWS Console, or using AWS CLI commands. Furthermore, even knowing what the stack name of any stack is requires use of Sceptre's `list stacks` command as Sceptre's logic of setting the stack name has never been obvious.

The options we have are:

1. list outputs 
```
% sceptre --output=yaml list outputs
---                       
- direct-connect:                     
  - OutputKey: VPGWId     
    OutputValue: vgw-1fb1e1e1af11da11b
- vpc:                             
  - ExportName: nonprod-DataSubnetCIDRa
    OutputKey: DataSubnetCIDRa     
    OutputValue: 11.11.111.11/11      
  - ExportName: nonprod-PublicSubnetCIDRa
    OutputKey: PublicSubnetCIDRa      
    OutputValue: 11.11.111.111/11
...
```

This command is flawed, in that it does not report true stack names, but names like `direct-connect` and `vpc` that only have meaning inside Sceptre itself.

2. list outputs --export=envvar
```
% sceptre --output=yaml list outputs --export=envvar
export SCEPTRE_VPGWId='vgw-1fb1e1e1af11da11b'
export SCEPTRE_DataSubnetCIDRa='11.11.111.11/11'
export SCEPTRE_PublicSubnetCIDRa='11.11.111.111/11'
export SCEPTRE_DataSubnetCIDRb='11.11.111.11/11'
export SCEPTRE_VPCId='vpc-111111ab111a11e11'
...
```

This command is also flawed, as it does not report the stack name at all.

Note that this same CLI behaviour of listing outputs originates in the initial commit of this project back in 2017. In that original version of the code we had:

```python
@cli.command(name="describe-stack-outputs")
@stack_options
@click.option("--export", type=click.Choice(["envvar"]))
@click.pass_context
@catch_exceptions
def describe_stack_outputs(ctx, environment, stack, export):
    """
    Describes stack outputs.

    Describes ENVIRONMENT/STACK's stack outputs.
    """
    env = get_env(ctx.obj["sceptre_dir"], environment, ctx.obj["options"])
    response = env.stacks[stack].describe_outputs()

    if export == "envvar":
        write("\n".join(
            [
                "export SCEPTRE_{0}={1}".format(
                    output["OutputKey"], output["OutputValue"]
                )
                for output in response
            ]
        ))
    else:
        write(response, ctx.obj["output_format"])
```

### Proposal export=stackoutput and export=stackoutputexternal

In addition to `--export=envvar` options like `--export=stackoutput` and `--export=stackoutputexternal` are needed. They might look like this:

```
% sceptre list outputs --export=stackoutput
!stack_output direct-connect.yaml::VPGWId --- vgw-1fb1e1e1af11da111
!stack_output vpc.yaml::DataSubnetCIDRa --- 11.11.111.11/11
!stack_output vpc.yaml::PublicSubnetCIDRa --- 11.11.111.111/11
!stack_output vpc.yaml::DataSubnetCIDRb --- 11.11.111.11/11
!stack_output vpc.yaml::VPCId --- vpc-111111ab111a11e11
...
```
and

```
% sceptre list outputs --export=stackoutputexternal
!stack_output_external nonprod-direct-connect::VPGWId --- vgw-1fb1e1e1af11da111
!stack_output_external nonprod-vpc::DataSubnetCIDRa --- 11.11.111.11/11
!stack_output_external nonprod-vpc::PublicSubnetCIDRa --- 11.11.111.111/11
!stack_output_external nonprod-vpc::DataSubnetCIDRb --- 11.11.111.11/11
!stack_output_external nonprod-vpc::VPCId --- vpc-111111ab111a11e11
...
```

Comments:

Patch:
diff --git a/docs/_source/docs/hooks.rst b/docs/_source/docs/hooks.rst
index a1eeef6..9a3bc82 100644
--- a/docs/_source/docs/hooks.rst
+++ b/docs/_source/docs/hooks.rst
@@ -44,23 +44,105 @@ Available Hooks
 cmd
 ~~~
 
-Executes the argument string in the shell as a Python subprocess.
-
-For more information about how this works, see the `subprocess documentation`_
+Executes a command through the shell.
 
 Syntax:
 
 .. code-block:: yaml
 
+   # Default shell.
    <hook_point>:
      - !cmd <shell_command>
 
-Example:
+   # Another shell.
+   <hook_point>:
+     - !cmd
+         run: <shell_command>
+         shell: <shell_name_or_path>
+
+Pass the command string as the only argument to use the default shell.
+
+On POSIX the default shell is ``sh``. On Windows it's usually ``cmd.exe``, but ``%ComSpec%`` may
+override that.
+
+Write the command string as you would type it at the shell prompt. This includes quotes and
+backslashes to escape filenames with spaces in them. To minimize escaping, you can use YAML plain
+strings like the following examples.
+
+.. code-block:: yaml
+
+   hooks:
+     before_update:
+       - !cmd echo "Hello, world!"
+
+A successful command prints its standard output messages between status messages from Sceptre.
+
+.. code-block::
+
+   [2023-09-03 01:06:28] - test - Launching Stack
+   [2023-09-03 01:06:29] - test - Stack is in the CREATE_COMPLETE state
+   Hello, world!
+   [2023-09-03 01:06:31] - test - Updating Stack
+   [2023-09-03 01:06:31] - test - No updates to perform.
+
+Pass named arguments to use a different shell. Here the command string is called ``run`` and the
+shell executable is called ``shell``.
+
+Write the executable name as you would type it at an interactive prompt to start the shell. For
+example, if Bash is in the system path, you can write ``bash``; otherwise, you need to write the
+absolute path such as ``/bin/bash``.
+
+.. code-block:: yaml
+
+   hooks:
+     before_update:
+       - !cmd
+           run: echo "Hello, $0!"
+           shell: bash
+
+.. code-block:: text
+
+   [2023-09-04 00:29:42] - test - Launching Stack
+   [2023-09-04 00:29:43] - test - Stack is in the CREATE_COMPLETE state
+   Hello, bash!
+   [2023-09-04 00:29:43] - test - Updating Stack
+   [2023-09-04 00:29:43] - test - No updates to perform.
+
+You can use PowerShell in the same way.
+
+.. code-block:: yaml
+
+   hooks:
+     before_update:
+       - !cmd
+           run: Write-Output "Hello, Posh!"
+           shell: pwsh
+
+.. code-block:: text
+
+   [2023-09-04 00:44:32] - test - Launching Stack
+   [2023-09-04 00:44:33] - test - Stack is in the CREATE_COMPLETE state
+   Hello, Posh!
+   [2023-09-04 00:44:34] - test - Updating Stack
+   [2023-09-04 00:44:34] - test - No updates to perform.
+
+If the shell command fails, so does Sceptre. Its output sits between Sceptre's status
+messages and a Python traceback.
 
 .. code-block:: yaml
 
-   before_create:
-     - !cmd "echo hello"
+   hooks:
+     before_update:
+       - !cmd missing_command
+
+.. code-block:: text
+
+   [2023-09-04 00:46:25] - test - Launching Stack
+   [2023-09-04 00:46:26] - test - Stack is in the CREATE_COMPLETE state
+   /bin/sh: 1: missing_command: not found
+   Traceback (most recent call last):
+   <snip>
+   subprocess.CalledProcessError: Command 'missing_command' returned non-zero exit status 127.
 
 asg_scaling_processes
 ~~~~~~~~~~~~~~~~~~~~~
diff --git a/sceptre/cli/list.py b/sceptre/cli/list.py
index 6f11bd2..c9b7b0c 100644
--- a/sceptre/cli/list.py
+++ b/sceptre/cli/list.py
@@ -5,6 +5,8 @@ from sceptre.context import SceptreContext
 from sceptre.cli.helpers import catch_exceptions, write
 from sceptre.plan.plan import SceptrePlan
 
+from typing import List, Dict
+
 logger = logging.getLogger(__name__)
 
 
@@ -46,12 +48,67 @@ def list_resources(ctx, path):
     write(responses, context.output_format)
 
 
+# flake8: noqa: C901
+def write_outputs(export, responses, plan, context):
+    """
+    Helper function for list outputs.
+    """
+    # Legacy. This option was added in the initial commit of the project,
+    # although its intended use case is unclear. It may relate to a feature
+    # that had been removed prior to the initial commit.
+    if export == "envvar":
+        for response in responses:
+            for stack in response.values():
+                for output in stack:
+                    write(
+                        "export SCEPTRE_{0}='{1}'".format(
+                            output.get("OutputKey"), output.get("OutputValue")
+                        ),
+                        "text",
+                    )
+
+    # Format outputs as !stack_output references.
+    elif export == "stackoutput":
+        for response in responses:
+            for stack_name, stack in response.items():
+                for output in stack:
+                    write(
+                        "!stack_output {0}.yaml::{1} [{2}]".format(
+                            stack_name,
+                            output.get("OutputKey"),
+                            output.get("OutputValue"),
+                        ),
+                        "text",
+                    )
+
+    # Format outputs as !stack_output_external references.
+    elif export == "stackoutputexternal":
+        stack_names = {stack.name: stack.external_name for stack in plan.graph}
+        for response in responses:
+            for stack_name, stack in response.items():
+                for output in stack:
+                    write(
+                        "!stack_output_external {0}::{1} [{2}]".format(
+                            stack_names[stack_name],
+                            output.get("OutputKey"),
+                            output.get("OutputValue"),
+                        ),
+                        "text",
+                    )
+
+    # Legacy. The output here is somewhat confusing in that
+    # outputs are organised in keys that only have meaning inside
+    # Sceptre.
+    else:
+        write(responses, context.output_format)
+
+
 @list_group.command(name="outputs")
 @click.argument("path")
 @click.option(
     "-e",
     "--export",
-    type=click.Choice(["envvar"]),
+    type=click.Choice(["envvar", "stackoutput", "stackoutputexternal"]),
     help="Specify the export formatting.",
 )
 @click.pass_context
@@ -60,7 +117,6 @@ def list_outputs(ctx, path, export):
     """
     List outputs for stack.
     \f
-
     :param path: Path to execute the command on.
     :type path: str
     :param export: Specify the export formatting.
@@ -79,18 +135,7 @@ def list_outputs(ctx, path, export):
     plan = SceptrePlan(context)
     responses = [response for response in plan.describe_outputs().values() if response]
 
-    if export == "envvar":
-        for response in responses:
-            for stack in response.values():
-                for output in stack:
-                    write(
-                        "export SCEPTRE_{0}='{1}'".format(
-                            output.get("OutputKey"), output.get("OutputValue")
-                        ),
-                        "text",
-                    )
-    else:
-        write(responses, context.output_format)
+    write_outputs(export, responses, plan, context)
 
 
 @list_group.command(name="change-sets")
diff --git a/sceptre/hooks/cmd.py b/sceptre/hooks/cmd.py
index 5ec3216..8292e87 100644
--- a/sceptre/hooks/cmd.py
+++ b/sceptre/hooks/cmd.py
@@ -13,16 +13,37 @@ class Cmd(Hook):
 
     def run(self):
         """
-        Runs the argument string in a subprocess.
+        Executes a command through the shell.
 
-        :raises: sceptre.exceptions.InvalidTaskArgumentTypeException
-        :raises: subprocess.CalledProcessError
+        See hooks documentation for details.
+
+        :raises: sceptre.exceptions.InvalidHookArgumentTypeError invalid input
+        :raises: CalledProcessError failed command
+        :raises: FileNotFoundError missing shell
+        :raises: PermissionError non-executable shell
         """
         envs = self.stack.connection_manager.create_session_environment_variables()
-        try:
-            subprocess.check_call(self.argument, shell=True, env=envs)
-        except TypeError:
+
+        if isinstance(self.argument, str) and self.argument != "":
+            command_to_run = self.argument
+            shell = None
+
+        elif (
+            isinstance(self.argument, dict)
+            and set(self.argument) == {"run", "shell"}
+            and isinstance(self.argument["run"], str)
+            and isinstance(self.argument["shell"], str)
+            and self.argument["run"] != ""
+            and self.argument["shell"] != ""
+        ):
+            command_to_run = self.argument["run"]
+            shell = self.argument["shell"]
+
+        else:
             raise InvalidHookArgumentTypeError(
-                'The argument "{0}" is the wrong type - cmd hooks require '
-                "arguments of type string.".format(self.argument)
+                "A cmd hook requires either a string argument or an object with "
+                "`run` and `shell` keys with string values. "
+                f"You gave `{self.argument!r}`."
             )
+
+        subprocess.check_call(command_to_run, shell=True, env=envs, executable=shell)


Test patch:
diff --git a/tests/test_cli/test_cli_commands.py b/tests/test_cli/test_cli_commands.py
index c3b4fd5..b5b8c88 100644
--- a/tests/test_cli/test_cli_commands.py
+++ b/tests/test_cli/test_cli_commands.py
@@ -636,8 +636,8 @@ class TestCli:
         expected_output = "StackOutputKeyOutputValue\n\nStackNameKeyValue\n"
         assert result.output.replace(" ", "") == expected_output
 
-    def test_list_outputs_with_export(self):
-        outputs = {"stack": [{"OutputKey": "Key", "OutputValue": "Value"}]}
+    def test_list_outputs_with_export__envvar(self):
+        outputs = {"mock-stack": [{"OutputKey": "Key", "OutputValue": "Value"}]}
         self.mock_stack_actions.describe_outputs.return_value = outputs
         result = self.runner.invoke(
             cli, ["list", "outputs", "dev/vpc.yaml", "-e", "envvar"]
@@ -645,6 +645,26 @@ class TestCli:
         assert result.exit_code == 0
         assert result.output == "export SCEPTRE_Key='Value'\n"
 
+    def test_list_outputs_with_export__stackoutput(self):
+        outputs = {"mock-stack": [{"OutputKey": "Key", "OutputValue": "Value"}]}
+        self.mock_stack_actions.describe_outputs.return_value = outputs
+        result = self.runner.invoke(
+            cli, ["list", "outputs", "dev/vpc.yaml", "-e", "stackoutput"]
+        )
+        assert result.exit_code == 0
+        assert result.output == "!stack_output mock-stack.yaml::Key [Value]\n"
+
+    def test_list_outputs_with_export__stackoutputexternal(self):
+        outputs = {"mock-stack": [{"OutputKey": "Key", "OutputValue": "Value"}]}
+        self.mock_stack_actions.describe_outputs.return_value = outputs
+        result = self.runner.invoke(
+            cli, ["list", "outputs", "dev/vpc.yaml", "-e", "stackoutputexternal"]
+        )
+        assert result.exit_code == 0
+        assert (
+            result.output == "!stack_output_external mock-stack-external::Key [Value]\n"
+        )
+
     @pytest.mark.parametrize(
         "path,output_format,expected_output",
         [
diff --git a/tests/test_hooks/test_cmd.py b/tests/test_hooks/test_cmd.py
index 3f15a7f..d65b313 100644
--- a/tests/test_hooks/test_cmd.py
+++ b/tests/test_hooks/test_cmd.py
@@ -1,37 +1,174 @@
 # -*- coding: utf-8 -*-
-import subprocess
-from unittest.mock import patch, Mock
-
+from subprocess import CalledProcessError
+from unittest.mock import Mock
 import pytest
 
 from sceptre.exceptions import InvalidHookArgumentTypeError
 from sceptre.hooks.cmd import Cmd
 from sceptre.stack import Stack
 
+ERROR_MESSAGE = (
+    r"^A cmd hook requires either a string argument or an object with `run` and "
+    r"`shell` keys with string values\. You gave `{0}`\.$"
+)
+
+
+@pytest.fixture()
+def stack():
+    stack = Stack(
+        "stack1",
+        "project1",
+        "region1",
+        template_handler_config={"template": "path.yaml"},
+    )
+
+    # Otherwise the test works only when the environment variables already set a
+    # valid AWS session.
+    stack.connection_manager.create_session_environment_variables = Mock(
+        return_value={}
+    )
+
+    return stack
+
+
+def test_null_input_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"None")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd(None, stack).run()
+
+
+def test_empty_string_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"''")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd("", stack).run()
+
+
+def test_list_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\['echo', 'hello'\]")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd(["echo", "hello"], stack).run()
+
+
+def test_dict_without_shell_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'run': 'echo hello'\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": "echo hello"}, stack).run()
+
+
+def test_dict_without_run_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'shell': '/bin/bash'\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"shell": "/bin/bash"}, stack).run()
+
+
+def test_dict_with_list_run_raises_exception(stack):
+    message = ERROR_MESSAGE.format(
+        r"\{'run': \['echo', 'hello'\], 'shell': '/bin/bash'\}"
+    )
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": ["echo", "hello"], "shell": "/bin/bash"}, stack).run()
+
+
+def test_dict_with_empty_run_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'run': '', 'shell': '/bin/bash'\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": "", "shell": "/bin/bash"}, stack).run()
+
+
+def test_dict_with_null_run_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'run': None, 'shell': '/bin/bash'\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": None, "shell": "/bin/bash"}, stack).run()
+
+
+def test_dict_with_list_shell_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'run': 'echo hello', 'shell': \['/bin/bash'\]\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": "echo hello", "shell": ["/bin/bash"]}, stack).run()
+
+
+def test_dict_with_typo_shell_raises_exception(stack):
+    import platform
+
+    if platform.python_version().startswith("3.7."):
+        message = r"^\[Errno 2\] No such file or directory: '/bin/bsah': '/bin/bsah'$"
+    else:
+        message = r"^\[Errno 2\] No such file or directory: '/bin/bsah'$"
+    with pytest.raises(FileNotFoundError, match=message):
+        typo = "/bin/bsah"
+        Cmd({"run": "echo hello", "shell": typo}, stack).run()
+
+
+def test_dict_with_non_executable_shell_raises_exception(stack):
+    message = r"^\[Errno 13\] Permission denied: '/'$"
+    with pytest.raises(PermissionError, match=message):
+        Cmd({"run": "echo hello", "shell": "/"}, stack).run()
+
+
+def test_dict_with_empty_shell_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'run': 'echo hello', 'shell': ''\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": "echo hello", "shell": ""}, stack).run()
+
+
+def test_dict_with_null_shell_raises_exception(stack):
+    message = ERROR_MESSAGE.format(r"\{'run': 'echo hello', 'shell': None\}")
+    with pytest.raises(InvalidHookArgumentTypeError, match=message):
+        Cmd({"run": "echo hello", "shell": None}, stack).run()
+
+
+def test_input_exception_reprs_input(stack):
+    import datetime
+
+    exception_message = ERROR_MESSAGE.format(r"datetime.date\(2023, 8, 31\)")
+    with pytest.raises(InvalidHookArgumentTypeError, match=exception_message):
+        Cmd(datetime.date(2023, 8, 31), stack).run()
+
+
+def test_zero_exit_returns(stack):
+    Cmd("exit 0", stack).run()
+
+
+def test_nonzero_exit_raises_exception(stack):
+    message = r"Command 'exit 1' returned non-zero exit status 1\."
+    with pytest.raises(CalledProcessError, match=message):
+        Cmd("exit 1", stack).run()
+
+
+def test_hook_writes_to_stdout(stack, capfd):
+    Cmd("echo hello", stack).run()
+    cap = capfd.readouterr()
+    assert cap.out.strip() == "hello"
+    assert cap.err.strip() == ""
+
+
+def test_hook_writes_to_stderr(stack, capfd):
+    with pytest.raises(Exception):
+        Cmd("missing_command", stack).run()
+    cap = capfd.readouterr()
+    assert cap.out.strip() == ""
+    assert cap.err.strip() == "/bin/sh: 1: missing_command: not found"
+
+
+def test_default_shell_is_sh(stack, capfd):
+    Cmd("echo $0", stack).run()
+    cap = capfd.readouterr()
+    assert cap.out.strip() == "/bin/sh"
+    assert cap.err.strip() == ""
+
+
+def test_shell_parameter_sets_the_shell(stack, capfd):
+    Cmd({"run": "echo $0", "shell": "/bin/bash"}, stack).run()
+    cap = capfd.readouterr()
+    assert cap.out.strip() == "/bin/bash"
+    assert cap.err.strip() == ""
+
 
-class TestCmd(object):
-    def setup_method(self, test_method):
-        self.stack = Mock(Stack)
-        self.stack.name = "my/stack.yaml"
-        self.cmd = Cmd(stack=self.stack)
-
-    def test_run_with_non_str_argument(self):
-        self.cmd.argument = None
-        with pytest.raises(InvalidHookArgumentTypeError):
-            self.cmd.run()
-
-    @patch("sceptre.hooks.cmd.subprocess.check_call")
-    def test_run_with_str_argument(self, mock_call):
-        self.cmd.argument = "echo hello"
-        self.cmd.run()
-        expected_envs = (
-            self.stack.connection_manager.create_session_environment_variables.return_value
-        )
-        mock_call.assert_called_once_with("echo hello", shell=True, env=expected_envs)
-
-    @patch("sceptre.hooks.cmd.subprocess.check_call")
-    def test_run_with_erroring_command(self, mock_call):
-        mock_call.side_effect = subprocess.CalledProcessError(1, "echo")
-        self.cmd.argument = "echo hello"
-        with pytest.raises(subprocess.CalledProcessError):
-            self.cmd.run()
+def test_shell_has_session_environment_variables(stack, capfd):
+    stack.connection_manager.create_session_environment_variables = Mock(
+        return_value={"AWS_PROFILE": "sceptre_profile"}
+    )
+    Cmd("echo $AWS_PROFILE", stack).run()
+    cap = capfd.readouterr()
+    assert cap.out.strip() == "sceptre_profile"
+    assert cap.err.strip() == ""