Repository: beeware/briefcase

Issue:
Improve Error Handling for Invalid Briefcase Template
### What is the problem or limitation you are having?

The error state for using an invalid remote repo can be improved. While the "invalid repo" example is completely manufactured, I encountered the non-existent repo one with a typo.

A non-existent repo

```
❯ briefcase new -t https://github.com/beeware/beeware-template.git --template-branch linux-system --no-input

Generating a new application 'Hello World'
Using app template: https://github.com/beeware/beeware-template.git, branch linux-system

Log saved to /home/russell/tmp/beeware/logs/briefcase.2023_03_05-16_19_56.new.log

Traceback (most recent call last):
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/briefcase", line 8, in <module>
    sys.exit(main())
  File "/home/russell/github/beeware/briefcase/src/briefcase/__main__.py", line 20, in main
    command(**options)
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/new.py", line 528, in __call__
    return self.new_app(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/new.py", line 477, in new_app
    self.generate_template(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/base.py", line 814, in generate_template
    cached_template = self.update_cookiecutter_cache(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/base.py", line 758, in update_cookiecutter_cache
    remote = repo.remote(name="origin")
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/lib/python3.10/site-packages/git/repo/base.py", line 414, in remote
    raise ValueError("Remote named '%s' didn't exist" % name)
ValueError: Remote named 'origin' didn't exist
```

An invalid repo

```
❯ briefcase new -t https://example.com --template-branch linux-system --no-input

Generating a new application 'Hello World'
Using app template: https://example.com, branch linux-system

Log saved to /home/russell/tmp/beeware/logs/briefcase.2023_03_05-16_20_46.new.log

Traceback (most recent call last):
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/briefcase", line 8, in <module>
    sys.exit(main())
  File "/home/russell/github/beeware/briefcase/src/briefcase/__main__.py", line 20, in main
    command(**options)
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/new.py", line 528, in __call__
    return self.new_app(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/new.py", line 477, in new_app
    self.generate_template(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/base.py", line 820, in generate_template
    self.tools.cookiecutter(
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/lib/python3.10/site-packages/cookiecutter/main.py", line 69, in cookiecutter
    repo_dir, cleanup = determine_repo_dir(
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/lib/python3.10/site-packages/cookiecutter/repository.py", line 106, in determine_repo_dir
    cloned_repo = clone(
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/lib/python3.10/site-packages/cookiecutter/vcs.py", line 72, in clone
    repo_type, repo_url = identify_repo(repo_url)
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/lib/python3.10/site-packages/cookiecutter/vcs.py", line 45, in identify_repo
    raise UnknownRepoType
cookiecutter.exceptions.UnknownRepoType
```


### Describe the solution you'd like

Catch these errors and inform the user to verify the template.

### Describe alternatives you've considered

Interpret the errors myself.

### Additional context

[briefcase.2023_03_05-16_19_56.new.log](https://github.com/beeware/briefcase/files/10892997/briefcase.2023_03_05-16_19_56.new.log)
[briefcase.2023_03_05-16_20_46.new.log](https://github.com/beeware/briefcase/files/10892998/briefcase.2023_03_05-16_20_46.new.log)

Discussion:
@Kaushal-Dhungel: HI, Can I work on this ?@freakboy3742: @Kaushal-Dhungel Absolutely! We don't have a formal issue assignment or claim process; if an issue is open, it's available to be worked on. Just drop a comment on the ticket to let others know you're looking at it, so we don't end up with duplicated effort.@rmblau: This will be partially handled by https://github.com/beeware/briefcase/pull/1210@tiffanyxiao: Hi, I will try to tackle this issue for Pycon2024 🐝@rmblau: > Hi, I will try to tackle this issue for Pycon2024 🐝

I will be excited to see what you can do to fully handle it!@tiffanyxiao: After some initial digging around, it looks like the first case no longer triggers the same ValueError but the second still triggers the `cookiecutter.exceptions.UnknownRepoType`. I think its still worthwhile to make the second error message more descriptive, as it took some more digging to understand what a repo "type" is (at least in terms of cookie cutter). Initially while reading this issue, I also wasn't sure what qualified as an invalid repo.

Could I work on writing a more descriptive error message for the UnknownRepoType? What are others thoughts on this? @rmartin16: FWIW, I think I'd be inclined to simply catch any exception that cookiecutter could throw; I'm not sure there's much value in pushing such exceptions to users who are not likely to even know cookiecutter is involved.@sarayourfriend: On the latest version, the output from both commands in the issue body are identical:

```
$ briefcase new -t https://example.com --template-branch linux-system --no-input

[helloworld] Generating a new application 'Hello World'
Using app template: https://example.com, branch linux-system
Cloning template 'https://example.com'...

Unable to clone repository 'https://example.com'.

This may be because your computer is offline, or because the repository URL is incorrect.
```

That looks pretty reasonable and seems to have come from this commit: https://github.com/beeware/briefcase/commit/952daf8d95fdd7c1711c6eb128841d1825d0b595. Did that commit already resolve the problem?@freakboy3742: That would have been an unintended benefit, but it looks like it might have improved the situation.

The only improvement I can see is that it would be desirable to differentiate "repo exists, but branch doesn't" from "repo doesn't exist". As it stands, the first example (referencing a branch that doesn't exist) says it can't clone the *repository*, which isn't *incorrect*, but doesn't point at the actual cause. At a minimum, the error in both cases should warn that the "repository URL or branch is incorrect"; ideally, we'd be able to identify which of the two is actually the culprit.@freakboy3742: To clarify - I don't think it worth doing a bunch of additional work just to do a full diagnosis of whether it's the branch or the repo. If we're not in a situation where the existing code has 2 obvious branches raising the same error, or there's isn't a simple post-clone check we can perform, then I'd be perfectly happy with a tweaked (but generic) error message. @sarayourfriend: Okay, that makes sense. And actually I must apologise, I've realised I misunderstood the original issue. The output is _not_ identical when you use a real template repository reference, and only was identical when I tried it because `beeware/beeware-template` is not a repository at all.

When using a real repository reference, with only an incorrect branch specified, the error message is:

```
$ briefcase new -t git@github.com:beeware/python-macos-template.git --template-branch linux-system --no-input

[helloworld] Generating a new application 'Hello World'
Using app template: git@github.com:beeware/python-macos-template.git, branch linux-system
Cloning template 'git@github.com:beeware/python-macos-template.git'...

Could not find a template branch for Briefcase linux-system.
```

So you do get a distinct message when it's just the branch that's the issue. And that makes sense as looking at the code again, they are distinct code paths (each easily found by searching the error string).

However, to your point, the first error case, where the repository was the issue (rather than the branch) does flatten at least a few separate problems into a single message, and I believe you can tell the difference based on the error's `stderr`. You could either try to clone a host that is not a git server at all, a git server that can't find the referenced repository (which would include one you don't have access rights to), or one that's having SSL errors. I'm sure there are other cases as well (I didn't test what happens if you fail SSH authentication, for example).

- SSL issues: `fatal: unable to access 'https://<broken-ssl-host>': OpenSSL/3.2.2: error:0A000438:SSL routines::tlsv1 alert internal error`
- Git server but no access to the repository (either does not exist or you don't have access): `fatal: Could not read from remote repository.`
- Authentication to the git server failed: `fatal: Authentication failed for 'https://github.com/beeware/beeware-template.git/'`
- Not a git server: `fatal: repository 'https://example.com/' not found`

Like I said, `e.stderr` includes that information, so it could be interpolated into the error message. What do you think, @freakboy3742? I've got a draft PR here that implements something that might work to forward git's error to the Briefcase user: https://github.com/beeware/briefcase/pull/2033@freakboy3742: > Like I said, `e.stderr` includes that information, so it could be interpolated into the error message. What do you think, @freakboy3742? I've got a draft PR here that implements something that might work to forward git's error to the Briefcase user: #2033

Definitely looks like a worthwhile improvement to me; and when combined with the other improvements that have already landed, I'd be happy to call that a fix for this issue. The only (very minor cosmetic) thing I'd flag would be ensuring the error text is title-cased - it looks like git is inconsistent on whether the sentence after the `fatal: ` starts with a capital or not; if we're presenting it to the user in Briefcase livery, we should do what we can to clean it up.
@sarayourfriend: Great! I'll get that PR cleaned up and marked ready-for-review tomorrow.

Solution patch:
diff --git a/changes/1118.feature.rst b/changes/1118.feature.rst
new file mode 100644
index 000000000..1fe3b822b
--- /dev/null
+++ b/changes/1118.feature.rst
@@ -0,0 +1 @@
+Briefcase will now surface git's error messages if an error occurs when cloning a template repository.
diff --git a/src/briefcase/commands/base.py b/src/briefcase/commands/base.py
index cddd3726c..94e34bd74 100644
--- a/src/briefcase/commands/base.py
+++ b/src/briefcase/commands/base.py
@@ -6,6 +6,7 @@
 import inspect
 import os
 import platform
+import re
 import subprocess
 import sys
 from abc import ABC, abstractmethod
@@ -31,10 +32,10 @@
 from briefcase.exceptions import (
     BriefcaseCommandError,
     BriefcaseConfigError,
+    InvalidTemplateBranch,
     InvalidTemplateRepository,
     MissingAppMetadata,
     NetworkFailure,
-    TemplateUnsupportedVersion,
     UnsupportedHostError,
     UnsupportedPythonVersion,
 )
@@ -977,11 +978,23 @@ def update_cookiecutter_cache(self, template: str, branch="master"):
                     # any partial remnants of this initial clone.
                     # If we're getting a GitError, we know the directory must exist.
                     self.tools.shutil.rmtree(cached_template)
+                    git_fatal_message = re.findall(r"(?<=fatal: ).*?$", e.stderr, re.S)
+                    if git_fatal_message:
+                        # GitError captures stderr with single quotes. Because the regex above
+                        # takes everything after git's "fatal" message, we need to strip that final single quote.
+                        hint = git_fatal_message[0].rstrip("'").strip()
+
+                        # git is inconsistent with capitalisation of the first word of the message
+                        # and about periods at the end of the message.
+                        hint = f"{hint[0].upper()}{hint[1:]}{'' if hint[-1] == '.' else '.'}"
+                    else:
+                        hint = (
+                            "This may be because your computer is offline, or "
+                            "because the repository URL is incorrect."
+                        )
+
                     raise BriefcaseCommandError(
-                        f"Unable to clone repository {template!r}.\n"
-                        "\n"
-                        "This may be because your computer is offline, or "
-                        "because the repository URL is incorrect."
+                        f"Unable to clone repository {template!r}.\n\n{hint}"
                     ) from e
 
             try:
@@ -1024,7 +1037,7 @@ def update_cookiecutter_cache(self, template: str, branch="master"):
                     head.checkout()
                 except IndexError as e:
                     # No branch exists for the requested version.
-                    raise TemplateUnsupportedVersion(branch) from e
+                    raise InvalidTemplateBranch(template, branch) from e
             except (ValueError, self.tools.git.exc.GitError) as e:
                 raise BriefcaseCommandError(
                     "Unable to check out template branch.\n"
@@ -1081,7 +1094,7 @@ def _generate_template(self, template, branch, output_path, extra_context):
             raise InvalidTemplateRepository(template) from e
         except cookiecutter_exceptions.RepositoryCloneFailed as e:
             # Branch does not exist.
-            raise TemplateUnsupportedVersion(branch) from e
+            raise InvalidTemplateBranch(template, branch) from e
         except cookiecutter_exceptions.UndefinedVariableInTemplate as e:
             raise BriefcaseConfigError(e.message) from e
 
@@ -1122,7 +1135,7 @@ def generate_template(
                 output_path=output_path,
                 extra_context=extra_context,
             )
-        except TemplateUnsupportedVersion:
+        except InvalidTemplateBranch:
             # Only use the main template if we're on a development branch of briefcase
             # and the user didn't explicitly specify which branch to use.
             if version.dev is None or branch is not None:
diff --git a/src/briefcase/exceptions.py b/src/briefcase/exceptions.py
index c5356a15b..4b67d0e75 100644
--- a/src/briefcase/exceptions.py
+++ b/src/briefcase/exceptions.py
@@ -137,11 +137,12 @@ def __init__(self, error_msg):
         super().__init__(msg=error_msg, skip_logfile=True)
 
 
-class TemplateUnsupportedVersion(BriefcaseCommandError):
-    def __init__(self, briefcase_version):
-        self.briefcase_version = briefcase_version
+class InvalidTemplateBranch(BriefcaseCommandError):
+    def __init__(self, template_repo, branch):
+        self.template_repo = template_repo
+        self.branch = branch
         super().__init__(
-            f"Could not find a template branch for Briefcase {briefcase_version}."
+            f"Could not find a branch named {branch!r} in template repository {template_repo!r}."
         )
 
 


Test patch:
diff --git a/tests/commands/base/test_update_cookiecutter_cache.py b/tests/commands/base/test_update_cookiecutter_cache.py
index 0c6937fdb..052f0f50a 100644
--- a/tests/commands/base/test_update_cookiecutter_cache.py
+++ b/tests/commands/base/test_update_cookiecutter_cache.py
@@ -1,10 +1,11 @@
+import re
 import shutil
 from unittest import mock
 
 import pytest
 from git import exc as git_exceptions
 
-from briefcase.exceptions import BriefcaseCommandError, TemplateUnsupportedVersion
+from briefcase.exceptions import BriefcaseCommandError, InvalidTemplateBranch
 
 from ...utils import create_file
 
@@ -126,6 +127,65 @@ def test_new_repo_invalid_template_url(base_command, mock_git):
     assert not (base_command.data_path / "templates" / "special-template").exists()
 
 
+@pytest.mark.parametrize(
+    ("stderr_string", "error_message"),
+    (
+        pytest.param(
+            "\n    stderr: '\nfatal: could not clone repository 'https://example.com' \n'",
+            "Could not clone repository 'https://example.com'.",
+            id="tailing-whitespace-no-caps-no-period",
+        ),
+        pytest.param(
+            (
+                "\n    stderr: '\nfatal: Could not read from remote repository.\n\n"
+                "Please make sure you have the correct access rights\nand the repository exists. \n'"
+            ),
+            (
+                "Could not read from remote repository.\n\nPlease make sure "
+                "you have the correct access rights\nand the repository exists."
+            ),
+            id="tailing-whitespace-has-caps-has-period",
+        ),
+        pytest.param(
+            (
+                "\n    stderr: '\nfatal: unable to access 'https://example.com/': "
+                "OpenSSL/3.2.2: error:0A000438:SSL routines::tlsv1 alert internal error'"
+            ),
+            (
+                "Unable to access 'https://example.com/': OpenSSL/3.2.2: "
+                "error:0A000438:SSL routines::tlsv1 alert internal error."
+            ),
+            id="no-tailing-whitespace-no-caps-no-period",
+        ),
+        pytest.param(
+            "\n stderr: 'Mysterious git clone edge case with no fatal error.",
+            "This may be because your computer is offline",
+            id="fallback-hint",
+        ),
+    ),
+)
+def test_repo_clone_error(stderr_string, error_message, base_command, mock_git):
+    """If git emits error information when cloning, Briefcase provides that to the user.
+    If git does not emit a 'fatal' error, then fallback to a generic hint."""
+    base_command.tools.git = mock_git
+
+    base_command.tools.git.Repo.clone_from.side_effect = git_exceptions.GitCommandError(
+        "git", 128, stderr_string
+    )
+
+    repository = "https://example.com"
+
+    with pytest.raises(
+        BriefcaseCommandError,
+        # The error message should not retain the "fatal:" prefix; it isn't useful information to the user.
+        match=f"Unable to clone repository '{re.escape(repository)}'.\n\n?(?<!fatal: ){re.escape(error_message)}",
+    ):
+        base_command.update_cookiecutter_cache(
+            template=repository,
+            branch="briefcase-template",
+        )
+
+
 def test_existing_repo_template(base_command, mock_git):
     """If a previously known URL template is specified it is used."""
     base_command.tools.git = mock_git
@@ -334,7 +394,7 @@ def test_cached_missing_branch_template(base_command, mock_git):
     cached_path.mkdir(parents=True)
 
     # Generating the template under there conditions raises an error
-    with pytest.raises(TemplateUnsupportedVersion):
+    with pytest.raises(InvalidTemplateBranch):
         base_command.update_cookiecutter_cache(
             template="https://example.com/magic/special-template.git",
             branch="invalid",
diff --git a/tests/commands/create/test_generate_app_template.py b/tests/commands/create/test_generate_app_template.py
index 9e5c9c281..8c05bae02 100644
--- a/tests/commands/create/test_generate_app_template.py
+++ b/tests/commands/create/test_generate_app_template.py
@@ -12,8 +12,8 @@
 from briefcase.exceptions import (
     BriefcaseCommandError,
     BriefcaseConfigError,
+    InvalidTemplateBranch,
     NetworkFailure,
-    TemplateUnsupportedVersion,
 )
 
 
@@ -243,7 +243,7 @@ def test_default_template_dev_explicit_invalid_branch(
     )
 
     # Generate the template.
-    with pytest.raises(TemplateUnsupportedVersion):
+    with pytest.raises(InvalidTemplateBranch):
         create_command.generate_app_template(myapp)
 
     # Cookiecutter was invoked (once) with the expected template name and context.
@@ -651,7 +651,7 @@ def test_cached_missing_branch_template(monkeypatch, create_command, myapp):
     mock_remote.refs.__getitem__.side_effect = IndexError
 
     # Generating the template under there conditions raises an error
-    with pytest.raises(TemplateUnsupportedVersion):
+    with pytest.raises(InvalidTemplateBranch):
         create_command.generate_app_template(myapp)
 
 
diff --git a/tests/commands/new/test_new_app.py b/tests/commands/new/test_new_app.py
index 1748a6b07..d389d7922 100644
--- a/tests/commands/new/test_new_app.py
+++ b/tests/commands/new/test_new_app.py
@@ -10,8 +10,8 @@
 from briefcase.console import Console, Log
 from briefcase.exceptions import (
     BriefcaseCommandError,
+    InvalidTemplateBranch,
     InvalidTemplateRepository,
-    TemplateUnsupportedVersion,
 )
 
 
@@ -123,7 +123,7 @@ def test_new_app_missing_template(monkeypatch, new_command, tmp_path):
     )
 
     # Create the new app, using the default template. This raises an error.
-    with pytest.raises(TemplateUnsupportedVersion):
+    with pytest.raises(InvalidTemplateBranch):
         new_command.new_app()
 
     # App context is constructed
@@ -397,7 +397,7 @@ def test_new_app_with_invalid_template_branch(monkeypatch, new_command, tmp_path
     )
 
     # Create a new app, with a specific template; this raises an error
-    with pytest.raises(TemplateUnsupportedVersion):
+    with pytest.raises(InvalidTemplateBranch):
         new_command.new_app(template="https://example.com/other.git")
 
     # App context is constructed