Repository: canonical/snapcraft

Issue:
 Fails to parse erroenous build output and freezes indefinitely 
### Bug Description

There are libreoffice snap builds for amd64 [1] and arm64 which are stuck.

Afaict snapcraft crashed while parsing the build output.

[1] https://launchpad.net/~libreoffice/df-libreoffice/+snap/libreoffice-fresh/+build/2350372

Originally reported at https://bugs.launchpad.net/snapcraft/+bug/2046156

### To Reproduce

Seems running a build which outputs e.g. an erroneous utf8 encoded string. In case of libreoffice it is an error message where it seems a byte gets cut off.

### Environment

Runs on Launchpad

### snapcraft.yaml

```shell
https://git.launchpad.net/~libreoffice/+git/libreoffice-snap/tree/snapcraft.yaml?h=24.2
```


### Relevant log output

```shell
:: [build HPX] el/helpcontent2/source/text/simpress/guide
:: Entity: line 1: parser error : Opening and ending tag mismatch: switchinline line 1 and root
Exception in thread Thread-45:
Traceback (most recent call last):
  File "/snap/snapcraft/10279/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/snap/snapcraft/10279/lib/python3.10/site-packages/craft_cli/messages.py", line 285, in run
    self._run_posix()
  File "/snap/snapcraft/10279/lib/python3.10/site-packages/craft_cli/messages.py", line 257, in _run_posix
    self._write(data)
  File "/snap/snapcraft/10279/lib/python3.10/site-packages/craft_cli/messages.py", line 242, in _write
    unicode_line = useful_line.decode("utf8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbc in position 0: invalid start byte
```


### Additional context

The deb build output looks like this which I would expect the snap build to behave as well:

[build HPX] el/helpcontent2/source/text/sbasic/python
HELPFILES=/tmp/gbuild.SMwYPO && S=/<<PKGBUILDDIR>> && I=$S/instdir && W=$S/workdir && POFILES=/tmp/gbuild.i4kgRa && LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}"$I/program:$I/program" $W/LinkTarget/Executable/helpex -l el -mi ${HELPFILES} -m ${POFILES} -o $W/HelpTranslatePartTarget/el/helpcontent2/source/text/sbasic/python && rm -f ${POFILES} && touch /<<PKGBUILDDIR>>/workdir/HelpTranslatePartTarget/el/helpcontent2/source/text/sbasic/python/done && rm -f ${HELPFILES}
[build HPX] el/helpcontent2/source/text/sbasic/shared
Entity: line 1: parser error : Opening and ending tag mismatch: switchinline line 1 and root
�αζί και επιστρέφοντας το άθροισμά τους.</root>
                                                                               ^
Entity: line 1: parser error : Premature end of data in tag root line 1
�αζί και επιστρέφοντας το άθροισμά τους.</root>
                                                                               ^
HELPFILES=/tmp/gbuild.HZ8Z3B && S=/<<PKGBUILDDIR>> && I=$S/instdir && W=$S/workdir && POFILES=/tmp/gbuild.MFTIRb && LD_LIBRARY_PATH=${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}"$I/program:$I/program" $W/LinkTarget/Executable/helpex -l el -mi ${HELPFILES} -m ${POFILES} -o $W/HelpTranslatePartTarget/el/helpcontent2/source/text/sbasic/shared && rm -f ${POFILES} && touch /<<PKGBUILDDIR>>/workdir/HelpTranslatePartTarget/el/helpcontent2/source/text/sbasic/shared/done && rm -f ${HELPFILES}
[build HPX] el/helpcontent2/source/text/sbasic/shared/01


Discussion:

Resolving patch:
diff --git a/snapcraft/remote/git.py b/snapcraft/remote/git.py
--- a/snapcraft/remote/git.py
+++ b/snapcraft/remote/git.py
@@ -24,7 +24,24 @@
 from pathlib import Path
 from typing import Optional
 
-import pygit2
+# Cannot catch the pygit2 error here raised by the global use of
+# pygit2.Settings on import. We would ideally use pygit2.Settings
+# for this
+try:
+    import pygit2
+except Exception:  # pylint: disable=broad-exception-caught
+    # This environment comes from ssl.get_default_verify_paths
+    _old_env = os.getenv("SSL_CERT_DIR")
+    # Needs updating when the base changes for Snapcraft
+    os.environ["SSL_CERT_DIR"] = "/snap/core22/current/etc/ssl/certs"
+    import pygit2
+
+    # Restore the environment in case Snapcraft shells out and the environment
+    # that was setup is required.
+    if _old_env is not None:
+        os.environ["SSL_CERT_DIR"] = _old_env
+    else:
+        del os.environ["SSL_CERT_DIR"]
 
 from .errors import GitError, RemoteBuildInvalidGitRepoError
 


Test patch:
diff --git a/tests/spread/core22/invalid-utf8/snapcraft.yaml b/tests/spread/core22/invalid-utf8/snapcraft.yaml
new file mode 100644
--- /dev/null
+++ b/tests/spread/core22/invalid-utf8/snapcraft.yaml
@@ -0,0 +1,12 @@
+name: invalid-utf8
+base: core22
+summary: a project that echos invalid utf-8 text
+description: a project that echos invalid utf-8 text
+version: "1.0"
+confinement: strict
+
+parts:
+  my-part:
+    plugin: nil
+    override-build: |
+      echo -e "hi \x96 bye"
diff --git a/tests/spread/core22/invalid-utf8/task.yaml b/tests/spread/core22/invalid-utf8/task.yaml
new file mode 100644
--- /dev/null
+++ b/tests/spread/core22/invalid-utf8/task.yaml
@@ -0,0 +1,8 @@
+summary: Handle invalid utf-8 text during the build
+
+restore: |
+  snapcraft clean
+  rm -f ./*.snap
+
+execute: |
+  snapcraft pack -v 2>&1 | MATCH ":: hi � bye"
diff --git a/tests/spread/general/remote-build/task.yaml b/tests/spread/general/remote-build/task.yaml
--- a/tests/spread/general/remote-build/task.yaml
+++ b/tests/spread/general/remote-build/task.yaml
@@ -1,10 +1,6 @@
 summary: Test the remote builder
 manual: true
 kill-timeout: 180m
-systems:
-  - -ubuntu-22.04
-  - -ubuntu-22.04-64
-  - -ubuntu-22.04-amd64
 
 environment:
   LAUNCHPAD_TOKEN: "$(HOST: echo ${LAUNCHPAD_TOKEN})"
@@ -18,12 +14,21 @@ prepare: |
   fi
 
   snapcraft init
+
+  # Commit the project
+  git config --global --add safe.directory "$PWD"
+  git add snap/snapcraft.yaml
+  git commit -m "Initial Commit"
+
+  # Setup launchpad token
   mkdir -p ~/.local/share/snapcraft/provider/launchpad/
   echo -e "$LAUNCHPAD_TOKEN" >> ~/.local/share/snapcraft/provider/launchpad/credentials
 
 restore: |
   rm -f ./*.snap ./*.txt
 
+  rm -rf snap .git
+
 execute: |
   export SNAPCRAFT_REMOTE_BUILD_STRATEGY="$STRATEGY"