Repository: pyinstaller/pyinstaller

Issue:
compile_glib_schema_files UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined>
I am encountering an error when running pyinstaller:
`pyinstaller --noconsole --add-data="resource/ui/maingui.glade;resource/ui" --splash "resource/images/DisplayInSplash.png" --icon="resource/images/DisplayInIcon.ico" --onedir displayin.py`

**Error**
`UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined>`

I am not sure why this is happening. Some notable things about the program I am trying to compile:
- I am using the msys2 mingw64 environment
- The program is written with Python GTK 3.0
- The program uses MSVC gstreamer binaries
- I compiled opencv4.7.0 to use those gstreamer binaries

I should note that those factors don't seem to be causing the error as it is failing at compile_glib_schema_files, which seems to be related to GTK. Please help!

**Full Error Log**
```
user@localhost MINGW64 /c/Users/user/git/DisplayIn/src/displayin
$ ./build_win.sh 
1112 INFO: PyInstaller: 5.13.0
1112 INFO: Python: 3.11.4
1112 INFO: Platform: Windows-10-10.0.22621-SP0
1112 INFO: wrote C:/Users/user/git/DisplayIn/src/displayin/displayin.spec
1127 INFO: Extending PYTHONPATH with paths
['C:/Users/user/git/DisplayIn/src/displayin']
2324 INFO: Appending 'datas' from .spec
2325 INFO: checking Analysis
2325 INFO: Building Analysis because Analysis-00.toc is non existent
2325 INFO: Initializing module dependency graph...
2331 INFO: Caching module graph hooks...
2371 INFO: Analyzing base_library.zip ...
3378 INFO: Loading module hook 'hook-encodings.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
3770 INFO: Loading module hook 'hook-heapq.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
4704 INFO: Loading module hook 'hook-pickle.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
6287 INFO: Caching module dependency graph...
6366 INFO: running Analysis Analysis-00.toc
6383 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by C:/tools/msys64/mingw64/bin/python.exe
6403 INFO: Analyzing C:/Users/user/git/DisplayIn/src/displayin/displayin.py
6456 INFO: Loading module hook 'hook-cv2.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
7061 INFO: Loading module hook 'hook-numpy.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/numpy/_pyinstaller'...
7635 INFO: Loading module hook 'hook-difflib.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
7870 INFO: Loading module hook 'hook-multiprocessing.util.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
7970 INFO: Loading module hook 'hook-xml.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
8623 INFO: Loading module hook 'hook-platform.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
8647 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
9753 INFO: Processing pre-safe import module hook gi from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.py'.
9776 INFO: Loading module hook 'hook-gi.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
9870 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gtk.py'.
9872 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
9872 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
9873 INFO: Processing pre-find module path hook pyi_splash from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_find_module_path/hook-pyi_splash.py'.
9874 INFO: Adding pyi_splash module to application dependencies.
9923 INFO: Loading module hook 'hook-sounddevice.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
10866 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
10971 INFO: Processing pre-safe import module hook distutils from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-distutils.py'.
10974 INFO: Processing pre-find module path hook distutils from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
10983 INFO: Loading module hook 'hook-distutils.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
11088 INFO: Loading module hook 'hook-distutils.util.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
11336 INFO: Loading module hook 'hook-OpenGL.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
15132 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
15151 INFO: Processing module hooks...
15329 INFO: Loading module hook 'hook-setuptools.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
17153 INFO: Loading module hook 'hook-pkg_resources.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
17734 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
18287 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
18289 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
18290 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
18291 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/io-wmf.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/io-wmf.dll': The specified procedure could not be found.
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-heif.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-heif.dll': The specified procedure could not be found.
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.dll': The specified procedure could not be found.
g_module_open() failed for C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-tiff.dll: 'C:/tools/msys64/mingw64/bin/../lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-tiff.dll': The specified procedure could not be found.
19661 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
19670 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
20054 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
20066 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
20580 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
20951 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
21363 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
22565 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
22567 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
22959 INFO: Processing pre-safe import module hook gi.repository.HarfBuzz from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.HarfBuzz.py'.
23304 INFO: Loading module hook 'hook-setuptools.msvc.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24127 INFO: Loading module hook 'hook-_tkinter.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24128 INFO: checking Tree
24128 INFO: Building Tree because Tree-00.toc is non existent
24129 INFO: Building Tree Tree-00.toc
24192 INFO: checking Tree
24192 INFO: Building Tree because Tree-01.toc is non existent
24192 INFO: Building Tree Tree-01.toc
24200 INFO: checking Tree
24200 INFO: Building Tree because Tree-02.toc is non existent
24201 INFO: Building Tree Tree-02.toc
24203 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24586 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
24971 INFO: Loading module hook 'hook-gi.repository.HarfBuzz.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
25372 INFO: Processing pre-safe import module hook gi.repository.freetype2 from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.freetype2.py'.
25378 INFO: Loading module hook 'hook-gi.repository.freetype2.py' from 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks'...
25810 INFO: Looking for ctypes DLLs
26107 INFO: Analyzing run-time hooks ...
26118 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
26124 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
26129 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
26134 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
26139 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
26143 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
26147 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
26150 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
26157 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_setuptools.py'
26161 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgres.py'
26169 INFO: Including run-time hook 'C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth__tkinter.py'
26336 INFO: Compiling collected GLib schema files in 'C:/Users/user/git/DisplayIn/src/displayin/build/displayin/_pyi_gschema_compilation'...
Traceback (most recent call last):
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/utils/hooks/gi.py", line 391, in compile_glib_schema_files
    p = subprocess.run(
        ^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/subprocess.py", line 550, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/subprocess.py", line 1196, in communicate
    stdout = self.stdout.read()
             ^^^^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/encodings/cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 44: character maps to <undefined>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:/tools/msys64/mingw64/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/__main__.py", line 194, in _console_script_run
    run()
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/__main__.py", line 180, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/__main__.py", line 61, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1019, in main
    build(specfile, distpath, workpath, clean_build)
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 944, in build
    exec(code, spec_namespace)
  File "C:/Users/user/git/DisplayIn/src/displayin/displayin.spec", line 7, in <module>
    a = Analysis(
        ^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 429, in __init__
    self.__postinit__()
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 664, in assemble
    self.datas = compile_glib_schema_files(self.datas, os.path.join(CONF['workpath'], "_pyi_gschema_compilation"))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:/tools/msys64/mingw64/lib/python3.11/site-packages/PyInstaller/utils/hooks/gi.py", line 402, in compile_glib_schema_files
    logger.warning("Extra output from glib-compile-schemas:\n%s", p.stdout)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'p' where it is not associated with a value
```


Discussion:
Does it help if you add `errors='ignore'` to the list of arguments to the `subprocess.run` in your installed PyInstaller copy: https://github.com/pyinstaller/pyinstaller/blob/58b8235b0ecd474f71c56e119d2889bf09026cb4/PyInstaller/utils/hooks/gi.py#L391-L398

The root cause of this seem to be warning messages that use U+201C and U+201D quotation marks:
```
Warning: Schema “org.freedesktop.ibus” has path “/desktop/ibus/”.  Paths starting with “/apps/”, “/desktop/” or “/system/” are deprecated.
```

Resolving patch:
diff --git a/PyInstaller/utils/hooks/gi.py b/PyInstaller/utils/hooks/gi.py
--- a/PyInstaller/utils/hooks/gi.py
+++ b/PyInstaller/utils/hooks/gi.py
@@ -395,11 +395,17 @@ def compile_glib_schema_files(datas_toc, workdir, collect_source_files=False):
             stderr=subprocess.STDOUT,
             check=True,
             text=True,
+            errors='ignore',
         )
-        logger.debug("Extra output from glib-compile-schemas:\n%s", p.stdout)
+        logger.debug("Output from glib-compile-schemas:\n%s", p.stdout)
+    except subprocess.CalledProcessError as e:
+        # The called glib-compile-schema returned error. Display stdout/stderr, and return original datas TOC to
+        # minimize damage.
+        logger.warning("Failed to recompile GLib schemas! Returning collected files as-is!", exc_info=True)
+        logger.warning("Output from glib-compile-schemas:\n%s", e.stdout)
+        return datas_toc
     except Exception:
         # Compilation failed for whatever reason. Return original datas TOC to minimize damage.
-        logger.warning("Extra output from glib-compile-schemas:\n%s", p.stdout)
         logger.warning("Failed to recompile GLib schemas! Returning collected files as-is!", exc_info=True)
         return datas_toc
 


Test patch: