Repository: vispy/vispy

Issue:
Bundled six fails to import when frozen
When I freeze a simple vispy program, I can't import vispy because the import logic in `vispy.ext.six` fails when frozen. I have been trying to figure out how to fix this, but I couldn't in a reasonable amount of time so I will post my progress here.

Here is the script I am using to freeze the program (`freeze.py`):

``` python
from cx_Freeze import setup, Executable

setup_kwargs = {
    'scripts': ['program.py'],
    'executables': [Executable("program.py")],
}

setup(**setup_kwargs)

```

Here is the one-line "program" that just imports vispy (`program.py`):

``` python
import vispy
```

To freeze and run, just install `vispy` and `cx_Freeze` and type:

``` sh
$ python freeze.py build_exe
$ ./build/exe.linux-x86_64-3.4/program 
```

(Your build folder is going to be different on a non-Linux machine or if you don't use Python 3.4).

This fails when vispy tries to do `imp.find_module('vispy.ext._bundled.six')`. I tried importing `six` like some of the other bundled packages if `hasattr(sys, "frozen")` is true in `vispy/ext/six.py`...:

``` python
...

if hasattr(sys, "frozen"):
    try:
        from ._bundled.six import *  # noqa
    except ImportError:
        from six import *  # noqa
else:
    _import_six()

...
```

...but that didn't work. I got this error:

```
ImportError: No module named 'vispy.ext.six.moves'; 'vispy.ext.six' is not a package
```

I know @QuLogic did the bundling; do you have any ideas?



Discussion:
Did you try adding `vispy.ext.six` to the list of cx_freeze includes? Althought that one is just a module that is imported in several places. I would, however, expect it to be necessary to add `vispy.ext._bundled.six` to the cx_freeze includes, as its imported dynamically.

I have tried both of those. I manually verified that the `vispy.ext.six` and `vispy.ext._bundled.six` modules were present in the zip file. I was wondering if @QuLogic had input on what is different between the bundled `six` and the other bundled modules in terms of importing functionality. I was under the impression that cx_Freeze uses `imp.find_module` itself, but for some reason in this context it fails.

I also tried just installing `six` separately and including it in the frozen build, but the `imp.find_module` call still fails. I will try to investigate this further in the next couple of weeks.

I think `six` might be different because we need a certain minimum version of `six` for compatibility.

There is no difference between the bundled `six` and the others, but you are having problems with the shim. You can try leaving out the bundled copy and only using a separate copy, but the shim might still give you trouble.

The shim was borrowed from Astropy and uses some crazy magic to get `six` working. I am not familiar with cx_Freeze and why it might be interfering with it. The "not a package" suggests there's a missing `__init__.py` but it should be there (unless cx_Freeze is missing it.) Perhaps it's because `imp` is deprecated in Python 3? Does it work on a Python 2 bundle?

I got an example application to work! Here is a summary of what I did:
- The only way I could get vispy to freeze was to undo the changes @QuLogic made to bundled libraries. More specifically, I removed the shims for `six` and `png` and moved the `six.py` and `png.py` files directly into `vispy/ext`.
- I had to manually add the glsl files to the freeze script, and add a modification to `vispy/glsl/__init__.py`:

``` python
...
if hasattr(sys, "frozen"):
    path = op.join(op.dirname(sys.executable), "vispy", "glsl")
else:
    path = op.dirname(__file__) or '.'
...
```
- I did a similar thing for `vispy.io._data`. Here is the change to `vispy/io/datasets.py`:

``` python
if hasattr(sys, "frozen"):
    DATA_DIR = op.join(op.dirname(sys.executable), 'vispy', 'io', '_data')
else:
    DATA_DIR = op.join(op.dirname(__file__), '_data')
```
- I had to specify that I wanted to include`vispy.app.backends`.

Here is my final `freeze.py` file:

``` python
from cx_Freeze import setup, Executable

import os
import vispy.glsl
import vispy.io

glsl_folder = os.path.dirname(vispy.glsl.__file__)
io_folder = os.path.dirname(vispy.io.__file__)
io_data_folder = os.path.join(io_folder, "_data")
data_files = [
    (glsl_folder, os.path.join("vispy", "glsl")),
    (io_data_folder, os.path.join("vispy", "io", "_data")),
]

setup_kwargs = {
    'scripts': ['program.py'],
    'options': {
        'build_exe': {
            'include_files': data_files,
            'packages': [
                'vispy.app.backends',
            ],
        },
    },
    'executables': [Executable("program.py")],
}

setup(**setup_kwargs)
```

The only real problem I see in integrating these changes is the shims for the bundled packages. I'm afraid I would have to dive deep into cx_Freeze to figure out why they don't work, and I don't have that kind of time :smile:. I guess we have to decide if allowing dependencies to be unbundled is more important than allowing users to freeze code. It would be a shame if vispy couldn't be deployed in a frozen application.

Also, we could use `pkg_resources` to abstract away loading data files. Then, users wouldn't have to manually add the files to their freeze script, and we wouldn't need to add `if hasattr(sys, "frozen")` everywhere we load data.

Lastly, [here](https://github.com/jdreaver/vispy/commit/01dab6b689fd2fe01d7e69560d3205648cbd8715) is a link to the full commit I made for reference.

@QuLogic are the shims we have used by repos other than `astropy` (i.e., are they very common)? Is the purpose to make Debian/Fedora packaging easier?

I should also note that it seems like astropy [can't be frozen either](https://github.com/astropy/astropy/pull/960), so I don't know if they have successfully worked around the shim problems.

I used following dirty workaround with cx_Freeze

``` python
import six
 ....
setup_kwargs = {
    'scripts': ['program.py'],
    'options': {
        'build_exe': {
                 include_files': [six.__file__, ... other included files]
...
```

The problem with this approach is that this includes the six.py file which is later compiled to .pyc (created with the first run of binary release). Not sure if this is any help , definitely not so clean.

The problem with Astropy is unrelated; it's much older than when they included the `six` shim.

If you print the exception [here](https://github.com/vispy/vispy/blob/master/vispy/ext/six.py#L42), you can see that the shim fails to import `vispy`. That's probably a bug in cx_Freeze where it overrides `importlib` but not `imp`. However, even if you switch to `importlib` (which can't be done unilaterally since Vispy supports 2.6), you run into the same "not a package" error above.

I think this can be worked around, but `six` 1.4.1 uses a completely different method to create `six.moves` than later versions. Can we bump the requirement to something more recent?

It should be okay to bump the `six` requirement

Thanks for the info @QuLogic!


Resolving patch:
diff --git a/vispy/ext/_bundled/png.py b/vispy/ext/_bundled/png.py
--- a/vispy/ext/_bundled/png.py
+++ b/vispy/ext/_bundled/png.py
@@ -45,8 +45,8 @@
 import zlib
 # http://www.python.org/doc/2.4.4/lib/module-warnings.html
 import warnings
-from .six.moves import map as imap
-from .six import string_types
+from ..six.moves import map as imap
+from ..six import string_types
 import itertools
 
 
diff --git a/vispy/ext/_bundled/six.py b/vispy/ext/_bundled/six.py
--- a/vispy/ext/_bundled/six.py
+++ b/vispy/ext/_bundled/six.py
@@ -1,6 +1,6 @@
 """Utilities for writing code that runs on Python 2 and 3"""
 
-# Copyright (c) 2010-2013 Benjamin Peterson
+# Copyright (c) 2010-2014 Benjamin Peterson
 #
 # Permission is hereby granted, free of charge, to any person obtaining a copy
 # of this software and associated documentation files (the "Software"), to deal
@@ -20,13 +20,15 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-import io
+from __future__ import absolute_import
+
+import functools
 import operator
 import sys
 import types
 
 __author__ = "Benjamin Peterson <benjamin@python.org>"
-__version__ = "1.4.1"
+__version__ = "1.8.0"
 
 
 # Useful for very coarse version differentiation.
@@ -39,7 +41,6 @@
     class_types = type,
     text_type = str
     binary_type = bytes
-    file_types = (io.TextIOWrapper, io.BufferedRandom)
 
     MAXSIZE = sys.maxsize
 else:
@@ -48,7 +49,6 @@
     class_types = (type, types.ClassType)
     text_type = unicode
     binary_type = str
-    file_types = (file, io.TextIOWrapper, io.BufferedRandom)
 
     if sys.platform.startswith("java"):
         # Jython always uses 32 bits.
@@ -87,9 +87,9 @@ def __init__(self, name):
 
     def __get__(self, obj, tp):
         result = self._resolve()
-        setattr(obj, self.name, result)
+        setattr(obj, self.name, result) # Invokes __set__.
         # This is a bit ugly, but it avoids running this again.
-        delattr(tp, self.name)
+        delattr(obj.__class__, self.name)
         return result
 
 
@@ -107,6 +107,27 @@ def __init__(self, name, old, new=None):
     def _resolve(self):
         return _import_module(self.mod)
 
+    def __getattr__(self, attr):
+        _module = self._resolve()
+        value = getattr(_module, attr)
+        setattr(self, attr, value)
+        return value
+
+
+class _LazyModule(types.ModuleType):
+
+    def __init__(self, name):
+        super(_LazyModule, self).__init__(name)
+        self.__doc__ = self.__class__.__doc__
+
+    def __dir__(self):
+        attrs = ["__doc__", "__name__"]
+        attrs += [attr.name for attr in self._moved_attributes]
+        return attrs
+
+    # Subclasses should override this
+    _moved_attributes = []
+
 
 class MovedAttribute(_LazyDescr):
 
@@ -133,9 +154,72 @@ def _resolve(self):
         return getattr(module, self.attr)
 
 
+class _SixMetaPathImporter(object):
+    """
+    A meta path importer to import six.moves and its submodules.
+
+    This class implements a PEP302 finder and loader. It should be compatible
+    with Python 2.5 and all existing versions of Python3
+    """
+    def __init__(self, six_module_name):
+        self.name = six_module_name
+        self.known_modules = {}
+
+    def _add_module(self, mod, *fullnames):
+        for fullname in fullnames:
+            self.known_modules[self.name + "." + fullname] = mod
+
+    def _get_module(self, fullname):
+        return self.known_modules[self.name + "." + fullname]
+
+    def find_module(self, fullname, path=None):
+        if fullname in self.known_modules:
+            return self
+        return None
+
+    def __get_module(self, fullname):
+        try:
+            return self.known_modules[fullname]
+        except KeyError:
+            raise ImportError("This loader does not know module " + fullname)
+
+    def load_module(self, fullname):
+        try:
+            # in case of a reload
+            return sys.modules[fullname]
+        except KeyError:
+            pass
+        mod = self.__get_module(fullname)
+        if isinstance(mod, MovedModule):
+            mod = mod._resolve()
+        else:
+            mod.__loader__ = self
+        sys.modules[fullname] = mod
+        return mod
+
+    def is_package(self, fullname):
+        """
+        Return true, if the named module is a package.
+
+        We need this method to get correct spec objects with
+        Python 3.4 (see PEP451)
+        """
+        return hasattr(self.__get_module(fullname), "__path__")
 
-class _MovedItems(types.ModuleType):
+    def get_code(self, fullname):
+        """Return None
+
+        Required, if is_package is implemented"""
+        self.__get_module(fullname)  # eventually raises ImportError
+        return None
+    get_source = get_code  # same as get_code
+
+_importer = _SixMetaPathImporter(__name__)
+
+
+class _MovedItems(_LazyModule):
     """Lazy loading of moved objects"""
+    __path__ = []  # mark as package
 
 
 _moved_attributes = [
@@ -143,11 +227,15 @@ class _MovedItems(types.ModuleType):
     MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
     MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
     MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
+    MovedAttribute("intern", "__builtin__", "sys"),
     MovedAttribute("map", "itertools", "builtins", "imap", "map"),
     MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
     MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
     MovedAttribute("reduce", "__builtin__", "functools"),
+    MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
     MovedAttribute("StringIO", "StringIO", "io"),
+    MovedAttribute("UserDict", "UserDict", "collections"),
+    MovedAttribute("UserList", "UserList", "collections"),
     MovedAttribute("UserString", "UserString", "collections"),
     MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
     MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
@@ -156,12 +244,15 @@ class _MovedItems(types.ModuleType):
     MovedModule("builtins", "__builtin__"),
     MovedModule("configparser", "ConfigParser"),
     MovedModule("copyreg", "copy_reg"),
+    MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
+    MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"),
     MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
     MovedModule("http_cookies", "Cookie", "http.cookies"),
     MovedModule("html_entities", "htmlentitydefs", "html.entities"),
     MovedModule("html_parser", "HTMLParser", "html.parser"),
     MovedModule("http_client", "httplib", "http.client"),
     MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
+    MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
     MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
     MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
     MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
@@ -171,12 +262,14 @@ class _MovedItems(types.ModuleType):
     MovedModule("queue", "Queue"),
     MovedModule("reprlib", "repr"),
     MovedModule("socketserver", "SocketServer"),
+    MovedModule("_thread", "thread", "_thread"),
     MovedModule("tkinter", "Tkinter"),
     MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
     MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
     MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
     MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
     MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
+    MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
     MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
     MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
     MovedModule("tkinter_colorchooser", "tkColorChooser",
@@ -192,22 +285,29 @@ class _MovedItems(types.ModuleType):
     MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
     MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
     MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
+    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
+    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
     MovedModule("winreg", "_winreg"),
 ]
 for attr in _moved_attributes:
     setattr(_MovedItems, attr.name, attr)
+    if isinstance(attr, MovedModule):
+        _importer._add_module(attr, "moves." + attr.name)
 del attr
 
-moves = sys.modules[__name__ + ".moves"] = _MovedItems(__name__ + ".moves")
+_MovedItems._moved_attributes = _moved_attributes
 
+moves = _MovedItems(__name__ + ".moves")
+_importer._add_module(moves, "moves")
 
 
-class Module_six_moves_urllib_parse(types.ModuleType):
+class Module_six_moves_urllib_parse(_LazyModule):
     """Lazy loading of moved objects in six.moves.urllib_parse"""
 
 
 _urllib_parse_moved_attributes = [
     MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
+    MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
     MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
     MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
     MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
@@ -221,16 +321,26 @@ class Module_six_moves_urllib_parse(types.ModuleType):
     MovedAttribute("unquote", "urllib", "urllib.parse"),
     MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
     MovedAttribute("urlencode", "urllib", "urllib.parse"),
+    MovedAttribute("splitquery", "urllib", "urllib.parse"),
+    MovedAttribute("splittag", "urllib", "urllib.parse"),
+    MovedAttribute("splituser", "urllib", "urllib.parse"),
+    MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_params", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_query", "urlparse", "urllib.parse"),
+    MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
 ]
 for attr in _urllib_parse_moved_attributes:
     setattr(Module_six_moves_urllib_parse, attr.name, attr)
 del attr
 
-sys.modules[__name__ + ".moves.urllib_parse"] = Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse")
-sys.modules[__name__ + ".moves.urllib.parse"] = Module_six_moves_urllib_parse(__name__ + ".moves.urllib.parse")
+Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"),
+                      "moves.urllib_parse", "moves.urllib.parse")
 
 
-class Module_six_moves_urllib_error(types.ModuleType):
+class Module_six_moves_urllib_error(_LazyModule):
     """Lazy loading of moved objects in six.moves.urllib_error"""
 
 
@@ -243,11 +353,13 @@ class Module_six_moves_urllib_error(types.ModuleType):
     setattr(Module_six_moves_urllib_error, attr.name, attr)
 del attr
 
-sys.modules[__name__ + ".moves.urllib_error"] = Module_six_moves_urllib_error(__name__ + ".moves.urllib_error")
-sys.modules[__name__ + ".moves.urllib.error"] = Module_six_moves_urllib_error(__name__ + ".moves.urllib.error")
+Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"),
+                      "moves.urllib_error", "moves.urllib.error")
 
 
-class Module_six_moves_urllib_request(types.ModuleType):
+class Module_six_moves_urllib_request(_LazyModule):
     """Lazy loading of moved objects in six.moves.urllib_request"""
 
 
@@ -284,16 +396,19 @@ class Module_six_moves_urllib_request(types.ModuleType):
     MovedAttribute("urlcleanup", "urllib", "urllib.request"),
     MovedAttribute("URLopener", "urllib", "urllib.request"),
     MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
+    MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
 ]
 for attr in _urllib_request_moved_attributes:
     setattr(Module_six_moves_urllib_request, attr.name, attr)
 del attr
 
-sys.modules[__name__ + ".moves.urllib_request"] = Module_six_moves_urllib_request(__name__ + ".moves.urllib_request")
-sys.modules[__name__ + ".moves.urllib.request"] = Module_six_moves_urllib_request(__name__ + ".moves.urllib.request")
+Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes
 
+_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"),
+                      "moves.urllib_request", "moves.urllib.request")
 
-class Module_six_moves_urllib_response(types.ModuleType):
+
+class Module_six_moves_urllib_response(_LazyModule):
     """Lazy loading of moved objects in six.moves.urllib_response"""
 
 
@@ -307,11 +422,13 @@ class Module_six_moves_urllib_response(types.ModuleType):
     setattr(Module_six_moves_urllib_response, attr.name, attr)
 del attr
 
-sys.modules[__name__ + ".moves.urllib_response"] = Module_six_moves_urllib_response(__name__ + ".moves.urllib_response")
-sys.modules[__name__ + ".moves.urllib.response"] = Module_six_moves_urllib_response(__name__ + ".moves.urllib.response")
+Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"),
+                      "moves.urllib_response", "moves.urllib.response")
 
 
-class Module_six_moves_urllib_robotparser(types.ModuleType):
+class Module_six_moves_urllib_robotparser(_LazyModule):
     """Lazy loading of moved objects in six.moves.urllib_robotparser"""
 
 
@@ -322,20 +439,26 @@ class Module_six_moves_urllib_robotparser(types.ModuleType):
     setattr(Module_six_moves_urllib_robotparser, attr.name, attr)
 del attr
 
-sys.modules[__name__ + ".moves.urllib_robotparser"] = Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib_robotparser")
-sys.modules[__name__ + ".moves.urllib.robotparser"] = Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser")
+Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes
+
+_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"),
+                      "moves.urllib_robotparser", "moves.urllib.robotparser")
 
 
 class Module_six_moves_urllib(types.ModuleType):
     """Create a six.moves.urllib namespace that resembles the Python 3 namespace"""
-    parse = sys.modules[__name__ + ".moves.urllib_parse"]
-    error = sys.modules[__name__ + ".moves.urllib_error"]
-    request = sys.modules[__name__ + ".moves.urllib_request"]
-    response = sys.modules[__name__ + ".moves.urllib_response"]
-    robotparser = sys.modules[__name__ + ".moves.urllib_robotparser"]
+    __path__ = []  # mark as package
+    parse = _importer._get_module("moves.urllib_parse")
+    error = _importer._get_module("moves.urllib_error")
+    request = _importer._get_module("moves.urllib_request")
+    response = _importer._get_module("moves.urllib_response")
+    robotparser = _importer._get_module("moves.urllib_robotparser")
 
+    def __dir__(self):
+        return ['parse', 'error', 'request', 'response', 'robotparser']
 
-sys.modules[__name__ + ".moves.urllib"] = Module_six_moves_urllib(__name__ + ".moves.urllib")
+_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"),
+                      "moves.urllib")
 
 
 def add_move(move):
@@ -362,11 +485,6 @@ def remove_move(name):
     _func_code = "__code__"
     _func_defaults = "__defaults__"
     _func_globals = "__globals__"
-
-    _iterkeys = "keys"
-    _itervalues = "values"
-    _iteritems = "items"
-    _iterlists = "lists"
 else:
     _meth_func = "im_func"
     _meth_self = "im_self"
@@ -376,11 +494,6 @@ def remove_move(name):
     _func_defaults = "func_defaults"
     _func_globals = "func_globals"
 
-    _iterkeys = "iterkeys"
-    _itervalues = "itervalues"
-    _iteritems = "iteritems"
-    _iterlists = "iterlists"
-
 
 try:
     advance_iterator = next
@@ -429,21 +542,37 @@ def next(self):
 get_function_globals = operator.attrgetter(_func_globals)
 
 
-def iterkeys(d, **kw):
-    """Return an iterator over the keys of a dictionary."""
-    return iter(getattr(d, _iterkeys)(**kw))
+if PY3:
+    def iterkeys(d, **kw):
+        return iter(d.keys(**kw))
+
+    def itervalues(d, **kw):
+        return iter(d.values(**kw))
+
+    def iteritems(d, **kw):
+        return iter(d.items(**kw))
+
+    def iterlists(d, **kw):
+        return iter(d.lists(**kw))
+else:
+    def iterkeys(d, **kw):
+        return iter(d.iterkeys(**kw))
 
-def itervalues(d, **kw):
-    """Return an iterator over the values of a dictionary."""
-    return iter(getattr(d, _itervalues)(**kw))
+    def itervalues(d, **kw):
+        return iter(d.itervalues(**kw))
 
-def iteritems(d, **kw):
-    """Return an iterator over the (key, value) pairs of a dictionary."""
-    return iter(getattr(d, _iteritems)(**kw))
+    def iteritems(d, **kw):
+        return iter(d.iteritems(**kw))
 
-def iterlists(d, **kw):
-    """Return an iterator over the (key, [values]) pairs of a dictionary."""
-    return iter(getattr(d, _iterlists)(**kw))
+    def iterlists(d, **kw):
+        return iter(d.iterlists(**kw))
+
+_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.")
+_add_doc(itervalues, "Return an iterator over the values of a dictionary.")
+_add_doc(iteritems,
+         "Return an iterator over the (key, value) pairs of a dictionary.")
+_add_doc(iterlists,
+         "Return an iterator over the (key, [values]) pairs of a dictionary.")
 
 
 if PY3:
@@ -467,8 +596,9 @@ def int2byte(i):
 else:
     def b(s):
         return s
+    # Workaround for standalone backslash
     def u(s):
-        return unicode(s, "unicode_escape")
+        return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
     unichr = unichr
     int2byte = chr
     def byte2int(bs):
@@ -484,19 +614,16 @@ def iterbytes(buf):
 
 
 if PY3:
-    import builtins
-    exec_ = getattr(builtins, "exec")
+    exec_ = getattr(moves.builtins, "exec")
 
 
     def reraise(tp, value, tb=None):
+        if value is None:
+            value = tp()
         if value.__traceback__ is not tb:
             raise value.with_traceback(tb)
         raise value
 
-
-    print_ = getattr(builtins, "print")
-    del builtins
-
 else:
     def exec_(_code_, _globs_=None, _locs_=None):
         """Execute code in a namespace."""
@@ -516,14 +643,24 @@ def exec_(_code_, _globs_=None, _locs_=None):
 """)
 
 
+print_ = getattr(moves.builtins, "print", None)
+if print_ is None:
     def print_(*args, **kwargs):
-        """The new-style print function."""
+        """The new-style print function for Python 2.4 and 2.5."""
         fp = kwargs.pop("file", sys.stdout)
         if fp is None:
             return
         def write(data):
             if not isinstance(data, basestring):
                 data = str(data)
+            # If the file has an encoding, encode unicode with it.
+            if (isinstance(fp, file) and
+                isinstance(data, unicode) and
+                fp.encoding is not None):
+                errors = getattr(fp, "errors", None)
+                if errors is None:
+                    errors = "strict"
+                data = data.encode(fp.encoding, errors)
             fp.write(data)
         want_unicode = False
         sep = kwargs.pop("sep", None)
@@ -563,18 +700,63 @@ def write(data):
 
 _add_doc(reraise, """Reraise an exception.""")
 
+if sys.version_info[0:2] < (3, 4):
+    def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS,
+              updated=functools.WRAPPER_UPDATES):
+        def wrapper(f):
+            f = functools.wraps(wrapped)(f)
+            f.__wrapped__ = wrapped
+            return f
+        return wrapper
+else:
+    wraps = functools.wraps
 
 def with_metaclass(meta, *bases):
     """Create a base class with a metaclass."""
-    return meta("NewBase", bases, {})
+    # This requires a bit of explanation: the basic idea is to make a dummy
+    # metaclass for one level of class instantiation that replaces itself with
+    # the actual metaclass.
+    class metaclass(meta):
+        def __new__(cls, name, this_bases, d):
+            return meta(name, bases, d)
+    return type.__new__(metaclass, 'temporary_class', (), {})
+
 
 def add_metaclass(metaclass):
     """Class decorator for creating a class with a metaclass."""
     def wrapper(cls):
         orig_vars = cls.__dict__.copy()
+        slots = orig_vars.get('__slots__')
+        if slots is not None:
+            if isinstance(slots, str):
+                slots = [slots]
+            for slots_var in slots:
+                orig_vars.pop(slots_var)
         orig_vars.pop('__dict__', None)
         orig_vars.pop('__weakref__', None)
-        for slots_var in orig_vars.get('__slots__', ()):
-            orig_vars.pop(slots_var)
         return metaclass(cls.__name__, cls.__bases__, orig_vars)
-    return wrapper
\ No newline at end of file
+    return wrapper
+
+# Complete the moves implementation.
+# This code is at the end of this module to speed up module loading.
+# Turn this module into a package.
+__path__ = []  # required for PEP 302 and PEP 451
+__package__ = __name__  # see PEP 366 @ReservedAssignment
+if globals().get("__spec__") is not None:
+    __spec__.submodule_search_locations = []  # PEP 451 @UndefinedVariable
+# Remove other six meta path importers, since they cause problems. This can
+# happen if six is removed from sys.modules and then reloaded. (Setuptools does
+# this for some reason.)
+if sys.meta_path:
+    for i, importer in enumerate(sys.meta_path):
+        # Here's some real nastiness: Another "instance" of the six module might
+        # be floating around. Therefore, we can't use isinstance() to check for
+        # the six meta path importer, since the other six instance will have
+        # inserted an importer with different class.
+        if (type(importer).__name__ == "_SixMetaPathImporter" and
+            importer.name == __name__):
+            del sys.meta_path[i]
+            break
+    del i, importer
+# Finally, add the importer to the meta path import hook.
+sys.meta_path.append(_importer)
diff --git a/vispy/ext/six.py b/vispy/ext/six.py
--- a/vispy/ext/six.py
+++ b/vispy/ext/six.py
@@ -7,13 +7,18 @@
 Handle loading six package from system or from the bundled copy
 """
 
-import imp
+try:
+    import importlib
+except ImportError:
+    importlib = None
+    import imp
 import io
+import sys
 
 from distutils.version import StrictVersion
 
 
-_SIX_MIN_VERSION = StrictVersion('1.4.1')
+_SIX_MIN_VERSION = StrictVersion('1.8.0')
 _SIX_SEARCH_PATH = ['vispy.ext._bundled.six', 'six']
 
 
@@ -37,24 +42,32 @@ def _find_module(name, path=None):
 
 def _import_six(search_path=_SIX_SEARCH_PATH):
     for mod_name in search_path:
-        try:
-            mod_info = _find_module(mod_name)
-        except ImportError:
-            continue
+        if importlib is not None:
+            try:
+                six_mod = importlib.import_module(mod_name)
+            except ImportError:
+                continue
+        else:
+            try:
+                mod_info = _find_module(mod_name)
+            except ImportError:
+                continue
+            else:
+                try:
+                    # Using __name__ causes the import to effectively overwrite
+                    # this shim.
+                    six_mod = imp.load_module(__name__, *mod_info)
+                finally:
+                    if mod_info[0] is not None:
+                        mod_info[0].close()
 
         try:
-            mod = imp.load_module(__name__, *mod_info)
-        finally:
-            if mod_info[0] is not None:
-                mod_info[0].close()
-
-        try:
-            if StrictVersion(mod.__version__) >= _SIX_MIN_VERSION:
+            if StrictVersion(six_mod.__version__) >= _SIX_MIN_VERSION:
                 break
         except (AttributeError, ValueError):
-            # Attribute error if the six module isn't what it should be and doesn't
-            # have a .__version__; ValueError if the version string exists but is
-            # somehow bogus/unparseable
+            # Attribute error if the six module isn't what it should be and
+            # doesn't have a .__version__; ValueError if the version string
+            # exists but is somehow bogus/unparseable
             continue
     else:
         raise ImportError(
@@ -63,6 +76,28 @@ def _import_six(search_path=_SIX_SEARCH_PATH):
             "this warning consult the packager of your Vispy "
             "distribution.".format(_SIX_MIN_VERSION))
 
+    # Using importlib does not overwrite this shim, so do it ourselves.
+    this_module = sys.modules[__name__]
+    if not hasattr(this_module, '_importer'):
+        # Copy all main six attributes.
+        for name, value in six_mod.__dict__.items():
+            if name.startswith('__'):
+                continue
+            this_module.__dict__[name] = value
+
+        # Tell six's importer to accept this shim's name as its own.
+        importer = six_mod._importer
+        known_modules = list(importer.known_modules.items())
+        for name, mod in known_modules:
+            this_name = __name__ + name[len(mod_name):]
+            importer.known_modules[this_name] = mod
+
+        # Turn this shim into a package just like six does.
+        this_module.__path__ = []  # required for PEP 302 and PEP 451
+        this_module.__package__ = __name__  # see PEP 366
+        if this_module.__dict__.get('__spec__') is not None:
+            this_module.__spec__.submodule_search_locations = []  # PEP 451
+
 
 _import_six()
 


Test patch: