internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pyinstaller/pyinstaller

Issue:
"No such file" when copying file from bincache - OSX
Hi folks, 

I've just released a new version of mitmproxy, and hit two bugs in pyinstaller (I'll make a separate issue for the second bug). The first is shown by the following traceback when attempting to build either mitmproxy or mitmdump with pyinstaller: 

```
Traceback (most recent call last):
  File "/Users/cortesi/tmp/pyinstaller/pyinstaller.py", line 18, in <module>
    run()
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/main.py", line 88, in run
    run_build(opts, spec_file, pyi_config)
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/main.py", line 46, in run_build
    PyInstaller.build.main(pyi_config, spec_file, **opts.__dict__)
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 1966, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 1915, in build
    execfile(spec)
  File "./release/mitmdump.spec", line 32, in <module>
    console=True )
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 1150, in __init__
    strip_binaries=self.strip, upx_binaries=self.upx,
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 988, in __init__
    self.__postinit__()
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 320, in __postinit__
    self.assemble()
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 1030, in assemble
    dist_nm=inm)
  File "/Users/cortesi/tmp/pyinstaller/PyInstaller/build.py", line 857, in checkCache
    shutil.copy2(fnm, cachedfile)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: '/Users/cortesi/Library/Application Support/pyinstaller/bincache00_py27/cryptography/_Cryptography_cffi_2a871178xb3816a41.so
```

I tracked this down to commit b813174a3067baf15af02c63ae82946dc446187b by @htgoebel - when I revert this, the build succeeds. 

To reproduce this, simply check out the current [mitmproxy master](http://github.com/mitmproxy/mitmproxy), and attempt to package mitmdump. I tested this on OSX Yosemite. 

Thanks,

Aldo



Discussion:
For got to mention that this behaviour was seen on a current checkout of the develop branch.


Resolving patch:
diff --git a/PyInstaller/build.py b/PyInstaller/build.py
--- a/PyInstaller/build.py
+++ b/PyInstaller/build.py
@@ -854,6 +854,8 @@ def checkCache(fnm, strip=False, upx=False, dist_nm=None):
                strip_options = ["-S"]
             cmd = ["strip"] + strip_options + [cachedfile]
 
+    if not os.path.exists(os.path.dirname(cachedfile)):
+        os.makedirs(os.path.dirname(cachedfile))
     shutil.copy2(fnm, cachedfile)
     os.chmod(cachedfile, 0755)
 


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
lensfunpy not working, usage of __file__
A user of my library [lensfunpy](https://pypi.python.org/pypi/lensfunpy) reported that it wasn't working with pyinstaller. Apparently packaged xml files to be loaded weren't found. I haven't tested this myself yet. Note that lensfunpy is a Cython wrapper around a native library (dll), and I guess what fails is the [following code](https://github.com/neothemachine/lensfunpy/blob/master/lensfunpy/__init__.py):

``` python
    _ROOT = os.path.abspath(os.path.dirname(__file__))
    _xml_glob = os.path.join(_ROOT, 'db_files', '*.xml')
...
    paths = glob.glob(_xml_glob)
```

So what is pyinstaller doing in such a case where `__file__` is used? Is that supported? Do I have to change anything?



Discussion:
Have you read this? http://pythonhosted.org/PyInstaller/#adapting-to-being-frozen
Note that presence of the "frozen" attribute of sys applies also to apps bundled by cxfreeze, py2exe, and pyqtdeploy.

You may also want to provide a hook for inclusion into PyInstaller to collect these files when freezing. See https://pythonhosted.org/PyInstaller/#using-hook-files

The hook file is the solution, plus a bugfix in my library (I accidently overwrite `__file__` which then messes up path resolution of data files in pyinstaller's case).

The hook file hook-lensfunpy.py is:

``` python
from hookutils import collect_data_files
# bundle xml DB files, skip other files (like DLL files on Windows)
datas = list(filter(lambda p: p[0].endswith('.xml'), collect_data_files('lensfunpy')))
hiddenimports = ['numpy', 'enum']
```

I let you know once I released a bugfix for my library, then you can include this hook file. Note that I filter the collected data files as it otherwise bundles .dll files twice.


Resolving patch:
diff --git a/PyInstaller/hooks/hook-lensfunpy.py b/PyInstaller/hooks/hook-lensfunpy.py
new file mode 100644
--- /dev/null
+++ b/PyInstaller/hooks/hook-lensfunpy.py
@@ -0,0 +1,13 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2013, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+
+from PyInstaller.hooks.hookutils import collect_data_files
+# bundle xml DB files, skip other files (like DLL files on Windows)
+datas = list(filter(lambda p: p[0].endswith('.xml'), collect_data_files('lensfunpy')))
+hiddenimports = ['numpy', 'enum']


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
be more helpful when the python dynlib is not found
it currently points out that this is a common issue when the python-dev package is missing on debian/ubuntu.

while that's helpful there, it doesn't help on other platforms at all. more helpful would be:
- to say in which pathes it looked
- to say for which filenames it looked
- to say which code file one must hack to fix this

be more helpful when the python dynlib is not found
it currently points out that this is a common issue when the python-dev package is missing on debian/ubuntu.

while that's helpful there, it doesn't help on other platforms at all. more helpful would be:
- to say in which pathes it looked
- to say for which filenames it looked
- to say which code file one must hack to fix this



Discussion:
Pull request are welcome.

Pull request are welcome.


Resolving patch:
diff --git a/PyInstaller/building/build_main.py b/PyInstaller/building/build_main.py
--- a/PyInstaller/building/build_main.py
+++ b/PyInstaller/building/build_main.py
@@ -558,12 +558,13 @@ def _check_python_library(self, binaries):
             binaries.append((os.path.basename(python_lib), python_lib, 'BINARY'))
             logger.info('Using Python library %s', python_lib)
         else:
-            msg = """Python library not found! This usually happens on Debian/Ubuntu
-where you need to install Python library:
-
-  apt-get install python3-dev
-  apt-get install python-dev
+            msg = """Python library not found! This would mean your Python installation doesn't come with proper library files.
+This usually happends by missing development package, or unsuitable build parameters of Python installation.
 
+* On Debian/Ubuntu, you would need to install Python development packages
+  * apt-get install python3-dev
+  * apt-get install python-dev
+* If you're building Python by yourself, please rebuild your Python with `--enable-shared` (or, `--enable-framework` on Darwin)
 """
             raise IOError(msg)
 


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
cherrypy >= 6.1.0 fails tests
From the cherrypy [changelog](https://github.com/cherrypy/cherrypy/blob/master/CHANGES.txt):

```
6.1.0
-----

* Combined wsgiserver2 and wsgiserver3 modules into a
  single module, ``cherrypy.wsgiserver``.
```



Discussion:
@jstewmon, wonderful! Thanks so much. That should make fixing the hook trivial. Thanks so much! Feel free to fix this, or I can when I have a minute...(out of the office right now)...


Resolving patch:
diff --git a/PyInstaller/hooks/hook-cherrypy.py b/PyInstaller/hooks/hook-cherrypy.py
deleted file mode 100644
--- a/PyInstaller/hooks/hook-cherrypy.py
+++ /dev/null
@@ -1,20 +0,0 @@
-#-----------------------------------------------------------------------------
-# Copyright (c) 2015-2016, PyInstaller Development Team.
-#
-# Distributed under the terms of the GNU General Public License with exception
-# for distributing bootloader.
-#
-# The full license is in the file COPYING.txt, distributed with this software.
-#-----------------------------------------------------------------------------
-#
-# CherryPy is a minimalist Python web framework.
-#
-# http://www.cherrypy.org/
-#
-# Tested with CherryPy 5.0.1
-
-
-from PyInstaller.utils.hooks import collect_submodules
-
-
-hiddenimports = collect_submodules('cherrypy.wsgiserver')
\ No newline at end of file


Test patch:
diff --git a/tests/functional/test_libraries.py b/tests/functional/test_libraries.py
--- a/tests/functional/test_libraries.py
+++ b/tests/functional/test_libraries.py
@@ -65,11 +65,10 @@ def test_botocore(pyi_builder):
 
 @importorskip('cherrypy')
 def test_cherrypy(pyi_builder):
-    # import wsgiserver3 on Python 3, or else wsgiserver2 on Python 2.
     pyi_builder.test_source(
         """
-        import cherrypy.wsgiserver.wsgiserver%s
-        """ % sys.version_info[0])
+        import cherrypy.wsgiserver
+        """)
 
 
 @xfail(is_darwin, reason='Issue #1895.')
Repository: pyinstaller/pyinstaller

Issue:
missing hidden import for skimage
When packaging an application that imports skimage.feature (and nothing else), the app would not run due to an ImportError on the "transform" module.  This can be fixed by adding one item to the hiddenimports in hook-skimage.transform.py file (bolded below):

> hiddenimports = ['skimage.draw.draw',
>                  'skimage._shared.geometry',
>                  'skimage.filters.rank.core_cy',
>                  **'skimage._shared.transform'**] 
> 
> datas = collect_data_files('skimage')

PyInstaller 3.2, Windows 7 64 bit, Python 2.7.12, Anaconda 4.1.1 distribution.



Discussion:

Resolving patch:
diff --git a/PyInstaller/hooks/hook-skimage.transform.py b/PyInstaller/hooks/hook-skimage.transform.py
--- a/PyInstaller/hooks/hook-skimage.transform.py
+++ b/PyInstaller/hooks/hook-skimage.transform.py
@@ -12,6 +12,7 @@
 # 64-bit
 hiddenimports = ['skimage.draw.draw',
                  'skimage._shared.geometry',
+                 'skimage._shared.transform',
                  'skimage.filters.rank.core_cy']
 
 datas = collect_data_files('skimage')


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
gi._gobject.option is not part of pygobject
The [GObject hook](https://github.com/pyinstaller/pyinstaller/blob/develop/PyInstaller/hooks/hook-gi.repository.GObject.py) adds a `hiddenimport` for `gi._gobject.option` however `gi/_gobject/option.py` is not part of pygobject.

This leads to the following warning when packaging a Gtk application:
```
4813 INFO: Loading module hook "hook-gi.py"...
4818 INFO: Loading module hook "hook-gi.repository.GObject.py"...
4926 INFO: Processing pre-safe import module hook   gi.repository.GLib
4963 WARNING: Hidden import "gi._gobject.option" not found!
```

Browsing through the [pygobject git history](https://git.gnome.org/browse/pygobject/), I find commit [8afd7e8](https://git.gnome.org/browse/pygobject/commit/gi/_option.py?id=8afd7e880a72a44e6ea46c763bab82146fd75c96) which moved `gi/_glib/option.py` into `gi/_option.py`

Replacing the `hiddenimport` to `hiddenimports += ['gi._option', 'gi._gobject']` silences the issue. However, I do not yet understand enough about pygobject and pyinstaller to know if this is the right thing to do.


Discussion:

Resolving patch:
diff --git a/PyInstaller/hooks/hook-gi.repository.GObject.py b/PyInstaller/hooks/hook-gi.repository.GObject.py
--- a/PyInstaller/hooks/hook-gi.repository.GObject.py
+++ b/PyInstaller/hooks/hook-gi.repository.GObject.py
@@ -19,4 +19,4 @@
 
 binaries, datas, hiddenimports = get_gi_typelibs('GObject', '2.0')
 
-hiddenimports += ['gi._gobject.option', 'gi._gobject']
+hiddenimports += ['gi._gobject']


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
PyInstaller should not check for modules specific for other than the current platform
**Python extensions for Windows**

We should not be checking for these modules in os x and linux. 


Discussion:
This should work in either directions.

Can you please provide a list of modules to specific to each platform and thus to be excluded on other platforms. These should then be added to `excludes` somewhere reasonable - preferable in `depend/analysis.py.

And of course, a pull-request would be fine :-)

Resolving patch:
diff --git a/PyInstaller/hooks/pre_safe_import_module/hook-win32com.py b/PyInstaller/hooks/pre_safe_import_module/hook-win32com.py
--- a/PyInstaller/hooks/pre_safe_import_module/hook-win32com.py
+++ b/PyInstaller/hooks/pre_safe_import_module/hook-win32com.py
@@ -21,11 +21,13 @@
 import os
 
 from PyInstaller.utils.hooks import logger, get_module_file_attribute
+from PyInstaller.compat import is_win
 
 
 def pre_safe_import_module(api):
-    win32com_dir = os.path.dirname(get_module_file_attribute('win32com'))
-    comext_dir = os.path.join(os.path.dirname(win32com_dir), 'win32comext')
-    logger.debug('win32com: extending __path__ with dir %r' % comext_dir)
-    # Append the __path__ where PyInstaller will look for 'win32com' modules.'
-    api.append_package_path(comext_dir)
+    if is_win:
+        win32com_dir = os.path.dirname(get_module_file_attribute('win32com'))
+        comext_dir = os.path.join(os.path.dirname(win32com_dir), 'win32comext')
+        logger.debug('win32com: extending __path__ with dir %r' % comext_dir)
+        # Append the __path__ where PyInstaller will look for 'win32com' modules.'
+        api.append_package_path(comext_dir)


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
Import error with module uvloop
Build log attached. Issue can be replicated with only import uvloop statement.

[build.log](https://github.com/pyinstaller/pyinstaller/files/1325827/build.log)
```
[@localhost]$ ./dist/test 
Traceback (most recent call last):
  File "test.py", line 1, in <module>
    import uvloop
  File "/usr/local/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 631, in exec_module
    exec(bytecode, module.__dict__)
  File "site-packages/uvloop/__init__.py", line 7, in <module>
  File "/usr/local/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 714, in load_module
    module = loader.load_module(fullname)
  File "uvloop/loop.pyx", line 32, in init uvloop.loop (uvloop/loop.c:129043)
ImportError: cannot import name _noop
```


Discussion:
This looks like not all required files are packages. Please have a look at [Make sure everything is packaged correctly](https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs#make-sure-everything-is-packaged-correctly) for tips to find out what is missing.

We would appreciate if you'd submit a hook for this. See https://pyinstaller.readthedocs.org/en/latest/hooks.html for more information about how to implement hooks.

Resolving patch:
diff --git a/PyInstaller/hooks/hook-uvloop.py b/PyInstaller/hooks/hook-uvloop.py
new file mode 100644
--- /dev/null
+++ b/PyInstaller/hooks/hook-uvloop.py
@@ -0,0 +1,16 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2017, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+#
+# Hook for the uvloop package: https://pypi.python.org/pypi/uvloop
+#
+# Tested with uvloop 0.8.1 and Python 3.6.2, on Ubuntu 16.04.1 64bit.
+
+from PyInstaller.utils.hooks import collect_submodules
+
+hiddenimports = collect_submodules('uvloop')


Test patch:
diff --git a/tests/functional/test_libraries.py b/tests/functional/test_libraries.py
--- a/tests/functional/test_libraries.py
+++ b/tests/functional/test_libraries.py
@@ -829,3 +829,9 @@ def test_pinyin(pyi_builder):
     pyi_builder.test_source("""
         import pinyin
         """)
+
+    
+@importorskip('uvloop')
+@skipif(is_win or not is_py35, reason='Windows, or py < 3.5 not supported')
+def test_uvloop(pyi_builder):
+    pyi_builder.test_source("import uvloop")
diff --git a/tests/requirements-libraries.txt b/tests/requirements-libraries.txt
--- a/tests/requirements-libraries.txt
+++ b/tests/requirements-libraries.txt
@@ -80,3 +80,6 @@ pypiwin32==219; python_version <= '3.5' and sys_platform == 'win32' # pyup: igno
 #pefile; sys_platform == 'win32'
 
 h5py==2.7.0
+
+# uvloop requires python 3.5 or higher and does not currently support Windows.
+uvloop==0.8.1; python_version >= '3.5' and sys_platform != 'win32'
\ No newline at end of file
Repository: pyinstaller/pyinstaller

Issue:
pygobject gtk3 linux typelib xlib2 not found
with pyinstaller develop on linux when packaging:
```
import gi
gi.require_version('Gtk', '3.0')
from gi.repository import Gtk

print("tototototo")
```

on another linux machine the result is:
`gi.RepositoryError: Typelib file for namespace 'xlib', version '2.0' not found`

add `xlib-2.0.typelib` in gi_typelibs seems to solve the issue.


Discussion:
well on a virtual machine with debian jessie I have now a segmentation fault
Hm, that typelib should have been picked up. What version of pygobject/gtk is it?
well I don't remember in what environment I made it.
It may be that I build on a system that hasn't gtk installed.

Resolving patch:
diff --git a/PyInstaller/hooks/hook-gi.repository.xlib.py b/PyInstaller/hooks/hook-gi.repository.xlib.py
new file mode 100644
--- /dev/null
+++ b/PyInstaller/hooks/hook-gi.repository.xlib.py
@@ -0,0 +1,15 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2005-2017, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+"""
+Import hook for PyGObject https://wiki.gnome.org/PyGObject
+"""
+
+from PyInstaller.utils.hooks import get_gi_typelibs
+
+binaries, datas, hiddenimports = get_gi_typelibs('xlib', '2.0')
diff --git a/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.xlib.py b/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.xlib.py
new file mode 100644
--- /dev/null
+++ b/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.xlib.py
@@ -0,0 +1,15 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2005-2017, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+
+
+def pre_safe_import_module(api):
+    # PyGObject modules loaded through the gi repository are marked as
+    # MissingModules by modulegraph so we convert them to
+    # RuntimeModules so their hooks are loaded and run.
+    api.add_runtime_module(api.module_name)


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
PyInstaller still says it supports pypiwin32 and suggests installing it.
In commit 085cd85 support for `pypiwin32` was removed in favor of only using `pywin32-ctypes`. But the error message and help text when `pywin32-ctypes` is not detected was not updated and still suggests installing `pypiwin32`.  Specifically at line 218 of `PyInstaller/compat.py`:

            raise SystemExit('PyInstaller cannot check for assembly dependencies.\n'
                                     'Please install PyWin32 or pywin32-ctypes.\n\n'
                                     'pip install pypiwin32\n')

The correct command now seems to be `pip install pywin32-ctypes`.


Discussion:

Resolving patch:
diff --git a/PyInstaller/compat.py b/PyInstaller/compat.py
--- a/PyInstaller/compat.py
+++ b/PyInstaller/compat.py
@@ -204,7 +204,7 @@
     modname_tkinter = 'tkinter'
 
 
-# On Windows we require pypiwin32 or pywin32-ctypes
+# On Windows we require pywin32-ctypes
 # -> all pyinstaller modules should use win32api from PyInstaller.compat to
 #    ensure that it can work on MSYS2 (which requires pywin32-ctypes)
 if is_win:
@@ -216,8 +216,8 @@
         # - It's not an error for pywin32 to not be installed at that point
         if not os.environ.get('PYINSTALLER_NO_PYWIN32_FAILURE'):
             raise SystemExit('PyInstaller cannot check for assembly dependencies.\n'
-                             'Please install PyWin32 or pywin32-ctypes.\n\n'
-                             'pip install pypiwin32\n')
+                             'Please install pywin32-ctypes.\n\n'
+                             'pip install pywin32-ctypes\n')
 
 
 def architecture():


Test patch:
Repository: pyinstaller/pyinstaller

Issue:
Pyinstaller add gtksource lang files
Buenas alguien me puede ayudar estoy haciendo una aplicación que usa gtksourceview y cuando compilo y ejecuto la aplicación no me muestra la sintaxis he intentado agregando los directorios de gtksource donde se encuentran los archivos lang y no funciona 


Discussion:

Resolving patch:
diff --git a/PyInstaller/hooks/hook-gi.repository.GtkSource.py b/PyInstaller/hooks/hook-gi.repository.GtkSource.py
new file mode 100644
--- /dev/null
+++ b/PyInstaller/hooks/hook-gi.repository.GtkSource.py
@@ -0,0 +1,15 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2005-2018, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+
+
+from PyInstaller.utils.hooks import collect_glib_share_files, get_gi_typelibs
+
+binaries, datas, hiddenimports = get_gi_typelibs('GtkSource', '3.0')
+
+datas += collect_glib_share_files('gtksourceview-3.0')
diff --git a/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GtkSource.py b/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GtkSource.py
new file mode 100644
--- /dev/null
+++ b/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GtkSource.py
@@ -0,0 +1,15 @@
+#-----------------------------------------------------------------------------
+# Copyright (c) 2005-2018, PyInstaller Development Team.
+#
+# Distributed under the terms of the GNU General Public License with exception
+# for distributing bootloader.
+#
+# The full license is in the file COPYING.txt, distributed with this software.
+#-----------------------------------------------------------------------------
+
+
+def pre_safe_import_module(api):
+    # PyGObject modules loaded through the gi repository are marked as
+    # MissingModules by modulegraph so we convert them to
+    # RuntimeModules so their hooks are loaded and run.
+    api.add_runtime_module(api.module_name)


Test patch: