Repository: gitguardian/ggshield
Issue: utf-8 encode codec issue on multiscan post

## Environment

- ggshield version: 1.13.6
- Operating system (Linux, macOS, Windows): macOS
- Operating system version: 12.6
- Python version: 13.10.8

## Describe the bug
On Secret scan, this error is happening. The error does not happen on version 1.11.0

A clear and concise description of what the bug is.

Steps to reproduce:
Command
```bash
/usr/local/Cellar/ggshield/1.13.6/bin/ggshield --debug  secret scan --exclude tmo-common --json -b 'Generic Password' -b 'Username Password' -o gg-secret-detection-report-path.json --verbose   path -r -y .
```

I'm unable to share the source code that is being scanned.


1. Setup steps required to reproduce the issue
2. Run command `ggshield scan something`
3. …

Actual result:

The error received

```bash
Scanning Path... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0 files scanned out of 92 -:--:--2022-10-20 12:36:48,802 DEBUG 123145446203392 urllib3.connectionpool:_new_conn:1003 Starting new HTTPS connection (1): api.gitguardian.com:443
Scanning Path... ━━━━━━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  22% 20 files scanned out of 92 -:--:--2022-10-20 12:36:48,908 DEBUG 123145464066048 urllib3.connectionpool:_new_conn:1003 Starting new HTTPS connection (2): api.gitguardian.com:443
Scanning Path... ━━━━━━━━━━━━━━━━━╺━━━━━━━━━━━━━━━━━━━━━━  43% 40 files scanned out of 92 0:00:012022-10-20 12:36:49,136 DEBUG 123145480855552 urllib3.connectionpool:_new_conn:1003 Starting new HTTPS connection (3): api.gitguardian.com:443
--- Logging error ---
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/logging/__init__.py", line 1100, in emit
    msg = self.format(record)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/logging/__init__.py", line 943, in format
    return fmt.format(record)
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/logging/__init__.py", line 678, in format
    record.message = record.getMessage()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/logging/__init__.py", line 368, in getMessage
    msg = msg % self.args
TypeError: %d format: a real number is required, not NoneType
Call stack:
  File "/usr/local/Cellar/ggshield/1.13.6/bin/ggshield", line 8, in <module>
    sys.exit(main())
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/ggshield/cmd/main.py", line 180, in main
    return cli.main(args, prog_name="ggshield", standalone_mode=not show_crash_log)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/ggshield/cmd/secret/scan/path.py", line 50, in path_cmd
    results = files.scan(
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/ggshield/scan/scannable.py", line 294, in scan
    return scanner.scan(self.files, progress_callback, scan_threads)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/ggshield/scan/scannable.py", line 428, in scan
    return self._collect_results(chunks_for_futures)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/ggshield/scan/scannable.py", line 391, in _collect_results
    handle_scan_chunk_error(scan, chunk)
  File "/usr/local/Cellar/ggshield/1.13.6/libexec/lib/python3.10/site-packages/ggshield/scan/scannable.py", line 555, in handle_scan_chunk_error
    logger.error("status_code=%d detail=%s", detail.status_code, detail.detail)
Message: 'status_code=%d detail=%s'
Arguments: (None, "'utf-8' codec can't encode character '\\\\udbde' in position 943: surrogates not allowed")
Scanning Path... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━  87% 80 files scanned out of 92 0:00:012022-10-20 12:36:49,349 DEBUG 123145497645056 urllib3.connectionpool:_new_conn:1003 Starting new HTTPS connection (4): api.gitguardian.com:443
Scanning Path... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 92 files scanned out of 92 0:00:002022-10-20 12:36:49,554 DEBUG 123145446203392 urllib3.connectionpool:_make_request:456 https://api.gitguardian.com:443 "POST /v1/multiscan HTTP/1.1" 200 None
2022-10-20 12:36:49,594 DEBUG 123145464066048 urllib3.connectionpool:_make_request:456 https://api.gitguardian.com:443 "POST /v1/multiscan HTTP/1.1" 200 None
Scanning Path... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 92 files scanned out of 92 0:00:002022-10-20 12:36:49,694 DEBUG 123145480855552 urllib3.connectionpool:_make_request:456 https://api.gitguardian.com:443 "POST /v1/multiscan HTTP/1.1" 200 None
Scanning Path... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 92 files scanned out of 92 0:00:002022-10-20 12:36:49,856 DEBUG 123145497645056 urllib3.connectionpool:_make_request:456 https://api.gitguardian.com:443 "POST /v1/multiscan HTTP/1.1" 200 None
Scanning Path... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 92 files scanned out of 92 0:00:00
Error: Error scanning, network error occurred: 'utf-8' codec can't encode character '\\udbde' in position 943: surrogates not allowed
```

Expected result:

Scan completes normally.

If applicable, add logs or screenshots to help explain your problem.


Changed files:
ggshield/scan/scannable.py

diff --git a/ggshield/scan/scannable.py b/ggshield/scan/scannable.py
index b7f4a76..e0b3066 100644
--- a/ggshield/scan/scannable.py
+++ b/ggshield/scan/scannable.py
@@ -217,7 +217,8 @@ class File:
 
     @staticmethod
     def _decode_bytes(raw_document: bytes, filename: str) -> str:
-        """Low level function to decode bytes, tries hard to find the correct encoding.
+        """Low level function to decode bytes, tries hard to find the correct encoding
+        and converts to UTF-8, as expected by pygitguardian.models.DocumentSchema.
         For now it returns an empty string if the document could not be decoded"""
         result = charset_normalizer.from_bytes(raw_document).best()
         if result is None:
@@ -230,7 +231,11 @@ class File:
         # ourselves
         if result.encoding == "utf_8" and raw_document.startswith(codecs.BOM_UTF8):
             raw_document = raw_document[len(codecs.BOM_UTF8) :]
-        return raw_document.decode(result.encoding, errors="replace")
+        return (
+            raw_document.decode(result.encoding, errors="replace")
+            .encode("utf-8", errors="replace")
+            .decode("utf-8", errors="replace")
+        )
 
     def __repr__(self) -> str:
         return f"<File filename={self.filename} filemode={self.filemode}>"