Repository: kedro-org/kedro
Pull request: Unhide cookiecutter errors in `kedro new`
## Description
Close [#2564](https://github.com/kedro-org/kedro/issues/2564)

## Development notes

- Unhide cookiecutter errors in `kedro new`, so when `--verbose` flag is not provided short cookiecutter error is in the output; with `--verbose` flag full cookiecutter error is in the output;
- Added unit test to check if original cookiecutter exception is present in the output if no `--verbose` flag is provided;
- Updated the `test_run_with_invalid_config` test that was changing the behaviour of the added test by modifying the global object `KedroCliError`. The reason of it is calling [` _validate_config_file`](https://github.com/kedro-org/kedro/blob/338736e3c82b45f334131f9c0811e42d326e5173/kedro/framework/cli/utils.py#L422) that sets `KedroCliError.VERBOSE_EXISTS=False`.

## Developer Certificate of Origin
We need all contributions to comply with the [Developer Certificate of Origin (DCO)](https://developercertificate.org/). All commits must be signed off by including a `Signed-off-by` line in the commit message. [See our wiki for guidance](https://github.com/kedro-org/kedro/wiki/Guidelines-for-contributing-developers/).

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

## Checklist

- [x] Read the [contributing](https://github.com/kedro-org/kedro/blob/main/CONTRIBUTING.md) guidelines
- [x] Signed off each commit with a [Developer Certificate of Origin (DCO)](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/managing-the-commit-signoff-policy-for-your-repository)
- [x] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
- [ ] Updated the documentation to reflect the code changes
- [x] Added a description of this change in the [`RELEASE.md`](https://github.com/kedro-org/kedro/blob/main/RELEASE.md) file
- [x] Added tests to cover my changes
- [x] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team


Problem statement:
Unhide cookiecutter errors in `kedro new`
## Description
Somehow the opposite of gh-2401: when cookiecutter fails, `kedro new` prompts you to re-run with `--verbose`:

```
> kedro new --starter=spaceflights

Project Name
============
Please enter a human readable name for your new project.
Spaces, hyphens, and underscores are allowed.
 [Spaceflights]: 
kedro.framework.cli.utils.KedroCliError: Failed to generate project when running cookiecutter.
Run with --verbose to see the full exception
Error: Failed to generate project when running cookiecutter.
```

## Context
In my case, the issue was quite silly: the directory already existed.

```
> kedro new --starter=spaceflights --verbose

Project Name
============
Please enter a human readable name for your new project.
Spaces, hyphens, and underscores are allowed.
 [Spaceflights]: 
Traceback (most recent call last):
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/kedro/framework/cli/starters.py", line 351, in _create_project
    result_path = cookiecutter(template=template_path, **cookiecutter_args)
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/cookiecutter/main.py", line 114, in cookiecutter
    result = generate_files(
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/cookiecutter/generate.py", line 291, in generate_files
    project_dir, output_directory_created = render_and_create_dir(
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/cookiecutter/generate.py", line 223, in render_and_create_dir
    raise OutputDirExistsException(msg)
cookiecutter.exceptions.OutputDirExistsException: Error: "/private/tmp/spaceflights" directory already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/kedro/framework/cli/starters.py", line 233, in new
    _create_project(template_path, cookiecutter_args)
  File "/Users/juan_cano/.micromamba/envs/_test310/lib/python3.10/site-packages/kedro/framework/cli/starters.py", line 353, in _create_project
    raise KedroCliError(
kedro.framework.cli.utils.KedroCliError: Failed to generate project when running cookiecutter.
Error: Failed to generate project when running cookiecutter.
```

## Possible Implementation
Capture the error and show it. Something like this:

```
> kedro new --starter=spaceflights --verbose                                                                                                                                (_test310) 

Project Name
============
Please enter a human readable name for your new project.
Spaces, hyphens, and underscores are allowed.
 [Spaceflights]: 
cookiecutter.exceptions.OutputDirExistsException: Error: "/private/tmp/spaceflights" directory already exists
Run with --verbose to see the full exception
Error: Failed to generate project when running cookiecutter.
```

## Possible Alternatives
Just display the full traceback on error. But I don't think it's optimal in its current form, since the actual error is hidden in the middle.

Hints from issue discussion:

Commit message: Fixed VERBOSE_EXISTS redefining

Signed-off-by: Elena Khaustova <ymax70rus@gmail.com>

Patch to review:
diff --git a/RELEASE.md b/RELEASE.md
index c2d3f4c211..d20475cc22 100644
--- a/RELEASE.md
+++ b/RELEASE.md
@@ -1,6 +1,7 @@
 # Upcoming Release 0.19.4
 
 ## Major features and improvements
+* Cookiecutter errors are shown is short format without `--verbose` flag.
 * Kedro commands now work from any subdirectory within a Kedro project.
 * Kedro CLI now provides a better error message when project commands are run outside of a project i.e. `kedro run`
 
diff --git a/kedro/framework/cli/utils.py b/kedro/framework/cli/utils.py
index 46fc5bdc62..194b244b5f 100644
--- a/kedro/framework/cli/utils.py
+++ b/kedro/framework/cli/utils.py
@@ -265,15 +265,21 @@ class KedroCliError(click.exceptions.ClickException):
 
     VERBOSE_ERROR = False
     VERBOSE_EXISTS = True
+    COOKIECUTTER_EXCEPTIONS_PREFIX = "cookiecutter.exceptions"
 
     def show(self, file: IO | None = None) -> None:
         if self.VERBOSE_ERROR:
             click.secho(traceback.format_exc(), nl=False, fg="yellow")
         elif self.VERBOSE_EXISTS:
-            etype, value, _ = sys.exc_info()
+            etype, value, tb = sys.exc_info()
             formatted_exception = "".join(traceback.format_exception_only(etype, value))
+            cookiecutter_exception = ""
+            for ex_line in traceback.format_exception(etype, value, tb):
+                if self.COOKIECUTTER_EXCEPTIONS_PREFIX in ex_line:
+                    cookiecutter_exception = ex_line
+                    break
             click.secho(
-                f"{formatted_exception}Run with --verbose to see the full exception",
+                f"{cookiecutter_exception}{formatted_exception}Run with --verbose to see the full exception",
                 fg="yellow",
             )
         else:
diff --git a/tests/framework/cli/test_starters.py b/tests/framework/cli/test_starters.py
index d89270f862..7480491722 100644
--- a/tests/framework/cli/test_starters.py
+++ b/tests/framework/cli/test_starters.py
@@ -23,6 +23,7 @@
     _parse_yes_no_to_bool,
     _validate_tool_selection,
 )
+from kedro.framework.cli.utils import KedroCliError
 
 FILES_IN_TEMPLATE_WITH_NO_TOOLS = 15
 
@@ -490,6 +491,16 @@ def test_fail_if_dir_exists(self, fake_kedro_cli):
         assert result.exit_code != 0
         assert "directory already exists" in result.output
 
+    def test_cookiecutter_exception_if_no_verbose(self, fake_kedro_cli):
+        """Check if original cookiecutter exception present in the output if no verbose
+        flag provided."""
+        KedroCliError.VERBOSE_EXISTS = True
+        Path("new-kedro-project").mkdir()
+        result = CliRunner().invoke(
+            fake_kedro_cli, ["new"], input=_make_cli_prompt_input()
+        )
+        assert "cookiecutter.exceptions" in result.output
+
     def test_prompt_no_title(self, fake_kedro_cli):
         shutil.copytree(TEMPLATE_PATH, "template")
         _write_yaml(Path("template") / "prompts.yml", {"repo_name": {}})


Code review comments:
diff_hunk: @@ -490,6 +491,16 @@ def test_fail_if_dir_exists(self, fake_kedro_cli):
         assert result.exit_code != 0
         assert "directory already exists" in result.output
 
+    def test_cookiecutter_exception_if_no_verbose(self, fake_kedro_cli):
+        """Check if original cookiecutter exception present in the output if no verbose
+        flag provided."""
+        KedroCliError.VERBOSE_EXISTS = True
path: tests/framework/cli/test_starters.py
text: @user1:
Does that mean to reset the side-effect from the other test? If so I think we make sure this happen after that test so when we add new test in the future or reordering tests won't fail again.

@author:
Makes sense, will move it to the end of the that test.