Repository: PyFilesystem/pyfilesystem2
Pull request #47 (2017-05-20 16:58:24)

Allow fs to be extended with subpackages
Hi ! This is my last PR, sorry to harass you lately.

I've been developping [fs.sshfs](https://github.com/althonos/fs.sshfs), a Pyfilesystem2 extension that implements the SFTP protocol. So I packaged everything, and that worked fine. However, it was a bummer that I had to import `fs.sshfs` for the `fs.opener.registry` to know about the `ssh` protocol !

### `fs/opener.py` --> `fs/opener/*`

I made the `fs.opener` module a package, containing the following files:

* `fs/opener/_base`, with the base `Opener` class
* `fs/opener/errors`, with the errors specific to `open_fs`, `manage_fs` or `parse`
* `fs/opener/_registry`, with the `Registry` class

Then I added some `pkgutil+importlib` black magic to automatically load files in the `fs/opener` directory. To support this, each `Opener` defined in the main `fs` package was moved to a specific file: `osfs.py`, `ftpfs.py`, `memoryfs.py`, `tarfs.py`, `tempfs.py`, `zipfs.py`. Then, when the `fs` package is first imported, every known protocol will be loaded in the `registry`.

### tidying `fs.opener.Registry`

`manage_fs` was made a `Registry` instance method, `ParseResult` and `_RE_FS_URL` attributes of the `Registry`class, and `parse` a `Registry` class method.


### namespace packages

`fs` and `fs.opener` were made namespace packages, so that files provided by extensions would be installed in the same directory as the Pyfilesystem2 library itself.


### extension framework

Creating an extension for Pyfilesystem2 is made easier, the user doesn't have to know which extensions are installed, and `fs.open_fs` will work out-of-the-box, if the extension if well structured. To support this, I added a HOW-TO to the documentation (pardon the eventual quirks, English is not my mother tongue).


### tl;dr

If you create a python package with the following structure:
```
  └── fs.awesome                           
      ├── fs                                    
      │   ├── awesomefs.py         
      │   └── opener                    
      │       └── awesomefs.py    
      └── setup.py   
```
it will be loaded directly by Pyfilesystem, and `fs.open_fs("awesome://")` will work as intended *without* having to import `fs.awesomefs` first.

You can have a glance of a working extension (I tested it thoroughly) at [the feat-namespace branch of althonos/fs.sshfs](https://github.com/althonos/fs.sshfs/tree/feat-namespace).


coveralls: 
[![Coverage Status](https://coveralls.io/builds/11614726/badge)](https://coveralls.io/builds/11614726)

Coverage increased (+0.002%) to 99.782% when pulling **c4ba4decc3fe3250f8eba6349e80448e3813802b on althonos:feat-namespace** into **9990a1ebeaa0f4a6e35c4098ac1ceef775082abd on PyFilesystem:master**.

willmcgugan: Interesting! I didn't know you could enumerate modules like that. First impressions are this is a good idea, but give me a little time to digest... :-)

You might want to add your sshfs to the docs, in `external.rst`.
willmcgugan: Is `sshfs` essentialy a fs2 version of `https://github.com/PyFilesystem/pyfilesystem/blob/master/fs/sftpfs.py` from the original project?
althonos: @willmcgugan : indeed, I used paramiko as well, but maybe it is not a customizable as the fs1 version (but I'm testing it against a docker `ssh+sftp` server, so I'm sure at least the username/password authentification works). I'll add to the doc, but I just thought this PR was dense enough :smile:



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c7f7bf1..2acb860 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,17 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased]
 
+
+### Fixed
+
+- Stat information was missing from info
+
+### Changed
+
+- More specific error when `validatepath` throws an error about the path
+    argument being the wrong type, and changed from a ValueError to a
+    TypeError.
+
 ## [2.0.3] - 2017-04-22
 
 ### Added
diff --git a/docs/source/extension.rst b/docs/source/extension.rst
new file mode 100644
index 0000000..8c3d78a
--- /dev/null
+++ b/docs/source/extension.rst
@@ -0,0 +1,104 @@
+.. _extension:
+
+Creating an extension
+=====================
+
+Once an new filesystem implemented, it is possible to distribute as a
+subpackage contained in the ``fs`` namespace. Let's say you are trying
+to create an extension for a filesystem called **AwesomeFS**.
+
+
+Name
+----
+
+For the sake of clarity, and to give a clearer sight of the
+Pyfilesystem2 ecosystem, your extension should be called **fs.awesome**
+or **fs.awesomefs**, since PyPI allows packages to be namespaced. Let us
+stick with **fs.awesome** for now.
+
+
+Structure
+---------
+
+The extension must have either of the following structures: ::
+
+  └── fs.awesome                            └── fs.awesome
+      ├── fs                                    ├── fs
+      │   ├── awesomefs.py                      │   ├── awesomefs
+      │   └── opener                            |   |   ├── __init__.py
+      │       └── awesomefs.py                  |   |   ├── some_file.py
+      └── setup.py                              |   |   └── some_other_file.py
+                                                │   └── opener
+                                                │       └── awesomefs.py
+                                                └── setup.py
+
+
+The structure on the left will work fine if you only need a single file
+to implement **AwesomeFS**, but if you end up creating more,
+you should probably use the structure on the right (create a package
+instead of a single file).
+
+.. warning ::
+
+   Do **NOT** create ``fs/__init__.py`` or ``fs/opener/__init__.py`` ! Since
+   those files are vital to the main Pyfilesystem2 package, including them
+   could result in having your extension break the whole Pyfilesystem2
+   package when installing.
+
+
+``setup.py``
+------------
+
+Refer to the `setuptools documentation <https://setuptools.readthedocs.io/>`_
+to see how to write a ``setup.py`` file. There are only a few things that
+should be kept in mind when creating a Pyfilesystem2 extension. Make sure that:
+
+* the name of the package is the *namespaced* name (**fs.awesome** with our
+  example).
+* ``fs``, ``fs.opener`` and ``fs.awesomefs`` packages are included. Since
+  you can't create ``fs/__init__.py`` and ``fs/opener/__init__.py``, setuptools
+  won't be able to find your packages if you use ``setuptools.find_packages``,
+  so you will have to include packages manually.
+* ``fs`` is in the ``install_requires`` list, in order to
+  always have Pyfilesystem2 installed before your extension.
+
+
+Opener
+------
+
+To ensure your new filesystem can be reached through the generic ``fs.open_fs`` method,
+you must declare a :class:`~fs.opener._base.Opener` in the ``fs/opener`` directory. With our example,
+create a file called ``awesomefs.py`` containing the definition of ``AwesomeOpener``
+or ``AwesomeFSOpener`` inside of the ``fs/opener`` directory. This will
+allow your Filesystem to be created directly through ``fs.open_fs``, without
+having to import your extension first !
+
+
+Practices
+---------
+
+* Use relative imports whenever you try to access to a resource in the
+  ``fs`` module or any of its submodules.
+* Keep track of your achievements ! Add ``__version__``, ``__author__``,
+  ``__author_email__`` and ``__license__`` variables to your project
+  (either in ``fs/awesomefs.py`` or ``fs/awesomefs/__init__.py`` depending
+  on the chosen structure), containing:
+
+    ``__version__``
+      the version of the extension (use `Semantic Versioning <http://semver.org/>`_ if possible !)
+
+    ``__author__``
+      your name(s)
+
+    ``__author_email__``
+      your email(s)
+
+    ``__license__``
+      the license of the subpackage
+
+
+Example
+-------
+
+See `fs.sshfs <https://github.com/althonos/fs.sshfs>`_ for a functioning
+PyFilesystem2 extension implementing the SFTP protocol.
diff --git a/docs/source/index.rst b/docs/source/index.rst
index 9c47a84..cd007d7 100644
--- a/docs/source/index.rst
+++ b/docs/source/index.rst
@@ -18,8 +18,9 @@ Contents:
    openers.rst
    walking.rst
    builtin.rst
-   external.rst
    implementers.rst
+   extension.rst
+   external.rst
    reference.rst
 
 
@@ -30,4 +31,3 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
-
diff --git a/docs/source/reference/opener.rst b/docs/source/reference/opener.rst
index e7ded64..2c0f026 100644
--- a/docs/source/reference/opener.rst
+++ b/docs/source/reference/opener.rst
@@ -3,5 +3,11 @@ fs.opener
 
 Open filesystems from a URL.
 
-.. automodule:: fs.opener
-    :members:
\ No newline at end of file
+.. automodule:: fs.opener._base
+    :members:
+
+.. automodule:: fs.opener._registry
+    :members:
+
+.. automodule:: fs.opener._errors
+    :members:
diff --git a/fs/__init__.py b/fs/__init__.py
index a7cf943..a8d44b4 100644
--- a/fs/__init__.py
+++ b/fs/__init__.py
@@ -1,3 +1,6 @@
+from pkgutil import extend_path
+__path__ = extend_path(__path__, __name__)
+
 from ._version import __version__
 from .enums import ResourceType, Seek
-from .opener import open_fs
\ No newline at end of file
+from .opener import open_fs
diff --git a/fs/base.py b/fs/base.py
index 330e615..1715a9f 100644
--- a/fs/base.py
+++ b/fs/base.py
@@ -1120,7 +1120,11 @@ class FS(object):
         self.check()
 
         if isinstance(path, bytes):
-            raise ValueError('path must not be bytes')
+            raise TypeError(
+                'paths must be unicode (not str)'
+                if six.PY2 else
+                'paths must be str (not bytes)'
+            )
 
         meta = self.getmeta()
 
diff --git a/fs/opener.py b/fs/opener.py
deleted file mode 100644
index ae3e04a..0000000
--- a/fs/opener.py
+++ /dev/null
@@ -1,380 +0,0 @@
-from __future__ import absolute_import
-from __future__ import print_function
-from __future__ import unicode_literals
-
-from contextlib import contextmanager
-import os
-import re
-
-from collections import namedtuple
-
-
-ParseResult = namedtuple(
-    'ParseResult',
-    [
-        'protocol',
-        'username',
-        'password',
-        'resource',
-        'path'
-    ]
-)
-
-
-_RE_FS_URL = re.compile(r'''
-^
-(.*?)
-:\/\/
-
-(?:
-(?:(.*?)@(.*?))
-|(.*?)
-)
-
-(?:
-!(.*?)$
-)*$
-''', re.VERBOSE)
-
-
-@contextmanager
-def manage_fs(fs_url, create=False, writeable=True, cwd='.'):
-    '''
-    A context manager opens / closes a filesystem.
-
-    :param fs_url: A FS instance or a FS URL.
-    :type fs_url: str or FS
-    :param bool create: If ``True``, then create the filesytem if it
-        doesn't already exist.
-    :param bool writeable: If ``True``, then the filesystem should be
-        writeable.
-    :param str cwd: The current working directory, if opening a
-        :class:`~fs.osfs.OSFS`.
-
-    Sometimes it is convenient to be able to pass either a FS object
-    *or* an FS URL to a function. This context manager handles the
-    required logic for that.
-
-    Here's an example::
-
-        def print_ls(list_fs):
-            """List a directory."""
-            with manage_fs(list_fs) as fs:
-                print(" ".join(fs.listdir()))
-
-    This function may be used in two ways. You may either pass either a
-    ``str``, as follows::
-
-        print_list('zip://projects.zip')
-
-    Or, an FS instance::
-
-        from fs.osfs import OSFS
-        projects_fs = OSFS('~/')
-        print_list(projects_fs)
-
-    '''
-    from .base import FS
-    if isinstance(fs_url, FS):
-        yield fs_url
-    else:
-        _fs = open_fs(
-            fs_url,
-            create=create,
-            writeable=writeable,
-            cwd=cwd
-        )
-        try:
-            yield _fs
-        except:
-            raise
-        finally:
-            _fs.close()
-
-
-class ParseError(ValueError):
-    """Raised when attempting to parse an invalid FS URL."""
-
-
-class OpenerError(Exception):
-    """Base class for opener related errors."""
-
-
-class Unsupported(OpenerError):
-    """May be raised by opener if the opener fails to open a FS."""
-
-
-def parse(fs_url):
-    """
-    Parse a Filesystem URL and return a :class:`ParseResult`, or raise
-    :class:`ParseError` (subclass of ValueError) if the FS URL is
-    not value.
-
-    :param fs_url: A filesystem URL
-    :type fs_url: str
-    :rtype: :class:`ParseResult`
-
-    """
-    match = _RE_FS_URL.match(fs_url)
-    if match is None:
-        raise ParseError('{!r} is not a fs2 url'.format(fs_url))
-
-    fs_name, credentials, url1, url2, path = match.groups()
-    if credentials:
-        username, _, password = credentials.partition(':')
-        url = url1
-    else:
-        username = None
-        password = None
-        url = url2
-    return ParseResult(
-        fs_name,
-        username,
-        password,
-        url,
-        path
-    )
-
-
-
-
-class Opener(object):
-    """
-    The opener base class.
-
-    An opener is responsible for opening a filesystems from one or more
-    protocols. A list of supported protocols is supplied in a class
-    attribute called `protocols`.
-
-    Openers should be registered with a :class:`~fs.opener.Registry`
-    object, which picks an appropriate opener object for a given FS URL.
-
-    """
-
-    protocols = []
-
-    def __repr__(self):
-        return "<opener {!r}>".format(self.protocols)
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        """
-        Open a filesystem object from a FS URL.
-
-        :param str fs_url: A filesystem URL
-        :param parse_result: A parsed filesystem URL.
-        :type parse_result: :class:`ParseResult`
-        :param bool writeable: True if the filesystem must be writeable.
-        :param bool create: True if the filesystem should be created if
-            it does not exist.
-        :param str cwd: The current working directory (generally only
-            relevant for OS filesystems).
-        :returns: :class:`~fs.base.FS` object
-
-        """
-
-
-class Registry(object):
-    """
-    A registry for `Opener` instances.
-
-    """
-
-    def __init__(self, default_opener='osfs'):
-        """
-        Create a registry object.
-
-        :param default_opener: The protocol to use, if one is not
-            supplied. The default is to use 'osfs', so that the FS URL
-            is treated as a system path if no protocol is given.
-
-        """
-        self.default_opener = default_opener
-        self.protocols = {}
-
-    def install(self, opener):
-        """
-        Install an opener.
-
-        :param opener: An :class:`Opener` instance, or a callable
-            that returns an opener instance.
-
-        May be used as a class decorator. For example::
-
-            registry = Registry()
-
-            @registry.install
-            class ArchiveOpener(Opener):
-                protocols = ['zip', 'tar']
-
-        """
-        if not isinstance(opener, Opener):
-            opener = opener()
-        assert opener.protocols, "must list one or more protocols"
-        for protocol in opener.protocols:
-            self.protocols[protocol] = opener
-
-    def open(self,
-             fs_url,
-             writeable=True,
-             create=False,
-             cwd=".",
-             default_protocol='osfs'):
-        """
-        Open a filesystem from a FS URL. Returns a tuple of a filesystem
-        object and a path. If there is no path in the FS URL, the path
-        value will be ``None``.
-
-        :param str fs_url: A filesystem URL
-        :param bool writeable: True if the filesystem must be writeable.
-        :param bool create: True if the filesystem should be created if
-            it does not exist.
-        :param cwd: The current working directory.
-        :type cwd: str or None
-        :rtype: Tuple of ``(<filesystem>, <path from url>)``
-
-        """
-
-        if '://' not in fs_url:
-            # URL may just be a path
-            fs_url = "{}://{}".format(default_protocol, fs_url)
-
-        parse_result = parse(fs_url)
-        protocol = parse_result.protocol
-        open_path = parse_result.path
-
-        opener = self.protocols.get(protocol, None)
-
-        if not opener:
-            raise Unsupported(
-                "protocol '{}' is not supported".format(protocol)
-            )
-
-        open_fs = opener.open_fs(
-            fs_url,
-            parse_result,
-            writeable,
-            create,
-            cwd
-        )
-        return open_fs, open_path
-
-    def open_fs(self,
-                fs_url,
-                writeable=True,
-                create=False,
-                cwd=".",
-                default_protocol='osfs'):
-        """
-        Open a filesystem object from a FS URL (ignoring the path
-        component).
-
-        :param str fs_url: A filesystem URL
-        :param parse_result: A parsed filesystem URL.
-        :type parse_result: :class:`ParseResult`
-        :param bool writeable: True if the filesystem must be writeable.
-        :param bool create: True if the filesystem should be created if
-            it does not exist.
-        :param str cwd: The current working directory (generally only
-            relevant for OS filesystems).
-        :param str default_protocol: The protocol to use if one is not
-            supplied in the FS URL (defaults to ``"osfs"``).
-        :returns: :class:`~fs.base.FS` object
-
-        """
-        from .base import FS
-        if isinstance(fs_url, FS):
-            _fs = fs_url
-        else:
-            _fs, _path = self.open(
-                fs_url,
-                writeable=writeable,
-                create=create,
-                cwd=cwd,
-                default_protocol=default_protocol
-            )
-        return _fs
-
-
-registry = Registry()
-open_fs = registry.open_fs
-open = registry.open
-
-
-@registry.install
-class OSFSOpener(Opener):
-    protocols = ['file', 'osfs']
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        from .osfs import OSFS
-        _path = os.path.abspath(os.path.join(cwd, parse_result.resource))
-        path = os.path.normpath(_path)
-        osfs = OSFS(path, create=create)
-        return osfs
-
-
-@registry.install
-class TempOpener(Opener):
-    protocols = ['temp']
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        from .tempfs import TempFS
-        temp_fs = TempFS(identifier=parse_result.resource)
-        return temp_fs
-
-
-@registry.install
-class MemOpener(Opener):
-    protocols = ['mem']
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        from .memoryfs import MemoryFS
-        mem_fs = MemoryFS()
-        return mem_fs
-
-
-@registry.install
-class ZipOpener(Opener):
-    protocols = ['zip']
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        from .zipfs import ZipFS
-        zip_fs = ZipFS(
-            parse_result.resource,
-            write=create
-        )
-        return zip_fs
-
-@registry.install
-class TarOpener(Opener):
-    protocols = ['tar']
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        from .tarfs import TarFS
-        tar_fs = TarFS(
-            parse_result.resource,
-            write=create
-        )
-        return tar_fs
-
-
-@registry.install
-class FTPOpener(Opener):
-    protocols = ['ftp']
-
-    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
-        from .ftpfs import FTPFS
-        ftp_host, _, dir_path = parse_result.resource.partition('/')
-        ftp_host, _, ftp_port = ftp_host.partition(':')
-        ftp_port = int(ftp_port) if ftp_port.isdigit() else 21
-        ftp_fs = FTPFS(
-            ftp_host,
-            port=ftp_port,
-            user=parse_result.username,
-            passwd=parse_result.password,
-        )
-        ftp_fs = (
-            ftp_fs.opendir(dir_path)
-            if dir_path else
-            ftp_fs
-        )
-        return ftp_fs
diff --git a/fs/opener/__init__.py b/fs/opener/__init__.py
new file mode 100644
index 0000000..4a6d874
--- /dev/null
+++ b/fs/opener/__init__.py
@@ -0,0 +1,44 @@
+# coding: utf-8
+"""
+fs.opener
+========
+
+Imported at the same time as PyFilesystem2, contains
+various objects and functions to open and manage FS.
+"""
+
+import importlib
+
+# Declares fs.opener as a namespace package
+import pkgutil
+__path__ = pkgutil.extend_path(__path__, __name__)
+
+# Import objects into fs.opener namespace
+from ._registry import registry, Registry
+from ._errors import OpenerError, ParseError, Unsupported
+
+# Create a partial __all__ with imports and aliases
+__all__ = [
+    "registry",
+    "Registry",
+    "OpenerError",
+    "ParseError",
+    "Unsupported",
+    'open_fs',
+    'open',
+    'manage_fs',
+    'parse',
+]
+
+# Alias functions defined as Registry methods
+open_fs = registry.open_fs
+open = registry.open
+manage_fs = registry.manage_fs
+parse = registry.parse
+
+# Import any file in the opener directory not prefixed by an underscore
+# and add its name to the __all__ list when successful
+for _, modname, _ in pkgutil.iter_modules(__path__):
+    if not modname.startswith('_'):
+        importlib.import_module('.'.join([__name__, modname]), package=__name__)
+        __all__.append(modname)
diff --git a/fs/opener/_base.py b/fs/opener/_base.py
new file mode 100644
index 0000000..8a9d2bd
--- /dev/null
+++ b/fs/opener/_base.py
@@ -0,0 +1,47 @@
+# coding: utf-8
+"""
+fs.opener._base
+===============
+
+Defines the Opener abstract base class.
+"""
+
+import six
+import abc
+
+
+@six.add_metaclass(abc.ABCMeta)
+class Opener(object):
+    """
+    The opener base class.
+
+    An opener is responsible for opening a filesystems from one or more
+    protocols. A list of supported protocols is supplied in a class
+    attribute called `protocols`.
+
+    Openers should be registered with a :class:`~fs.opener.Registry`
+    object, which picks an appropriate opener object for a given FS URL.
+
+    """
+
+    protocols = []
+
+    def __repr__(self):
+        return "<opener {!r}>".format(self.protocols)
+
+    @abc.abstractmethod
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        """
+        Open a filesystem object from a FS URL.
+
+        :param str fs_url: A filesystem URL
+        :param parse_result: A parsed filesystem URL.
+        :type parse_result: :class:`ParseResult`
+        :param bool writeable: True if the filesystem must be writeable.
+        :param bool create: True if the filesystem should be created if
+            it does not exist.
+        :param str cwd: The current working directory (generally only
+            relevant for OS filesystems).
+        :returns: :class:`~fs.base.FS` object
+
+        """
diff --git a/fs/opener/_errors.py b/fs/opener/_errors.py
new file mode 100644
index 0000000..69713d5
--- /dev/null
+++ b/fs/opener/_errors.py
@@ -0,0 +1,18 @@
+# coding: utf-8
+"""
+fs.opener._errors
+=================
+
+Errors raised when attempting to open a filesystem
+"""
+
+class ParseError(ValueError):
+    """Raised when attempting to parse an invalid FS URL."""
+
+
+class OpenerError(Exception):
+    """Base class for opener related errors."""
+
+
+class Unsupported(OpenerError):
+    """May be raised by opener if the opener fails to open a FS."""
diff --git a/fs/opener/_registry.py b/fs/opener/_registry.py
new file mode 100644
index 0000000..9dcbf74
--- /dev/null
+++ b/fs/opener/_registry.py
@@ -0,0 +1,253 @@
+# coding: utf-8
+"""
+fs.opener._registry
+===================
+
+Defines the Registry, which maps protocols and FS URLs to their
+respective Opener.
+"""
+
+import re
+import contextlib
+import collections
+
+from ._base import Opener
+from ._errors import OpenerError, ParseError, Unsupported
+
+
+class Registry(object):
+    """
+    A registry for `Opener` instances.
+
+    """
+
+    ParseResult = collections.namedtuple(
+        'ParseResult',
+        [
+            'protocol',
+            'username',
+            'password',
+            'resource',
+            'path'
+        ]
+    )
+
+    _RE_FS_URL = re.compile(r'''
+    ^
+    (.*?)
+    :\/\/
+
+    (?:
+    (?:(.*?)@(.*?))
+    |(.*?)
+    )
+
+    (?:
+    !(.*?)$
+    )*$
+    ''', re.VERBOSE)
+
+    @classmethod
+    def parse(cls, fs_url):
+        """
+        Parse a Filesystem URL and return a :class:`ParseResult`, or raise
+        :class:`ParseError` (subclass of ValueError) if the FS URL is
+        not value.
+
+        :param fs_url: A filesystem URL
+        :type fs_url: str
+        :rtype: :class:`ParseResult`
+
+        """
+        match = cls._RE_FS_URL.match(fs_url)
+        if match is None:
+            raise ParseError('{!r} is not a fs2 url'.format(fs_url))
+
+        fs_name, credentials, url1, url2, path = match.groups()
+        if credentials:
+            username, _, password = credentials.partition(':')
+            url = url1
+        else:
+            username = None
+            password = None
+            url = url2
+        return cls.ParseResult(
+            fs_name,
+            username,
+            password,
+            url,
+            path
+        )
+
+    def __init__(self, default_opener='osfs'):
+        """
+        Create a registry object.
+
+        :param default_opener: The protocol to use, if one is not
+            supplied. The default is to use 'osfs', so that the FS URL
+            is treated as a system path if no protocol is given.
+
+        """
+        self.default_opener = default_opener
+        self.protocols = {}
+
+    def install(self, opener):
+        """
+        Install an opener.
+
+        :param opener: An :class:`Opener` instance, or a callable
+            that returns an opener instance.
+
+        May be used as a class decorator. For example::
+
+            registry = Registry()
+
+            @registry.install
+            class ArchiveOpener(Opener):
+                protocols = ['zip', 'tar']
+
+        """
+        if not isinstance(opener, Opener):
+            opener = opener()
+        assert opener.protocols, "must list one or more protocols"
+        for protocol in opener.protocols:
+            self.protocols[protocol] = opener
+
+    def open(self,
+             fs_url,
+             writeable=True,
+             create=False,
+             cwd=".",
+             default_protocol='osfs'):
+        """
+        Open a filesystem from a FS URL. Returns a tuple of a filesystem
+        object and a path. If there is no path in the FS URL, the path
+        value will be ``None``.
+
+        :param str fs_url: A filesystem URL
+        :param bool writeable: True if the filesystem must be writeable.
+        :param bool create: True if the filesystem should be created if
+            it does not exist.
+        :param cwd: The current working directory.
+        :type cwd: str or None
+        :rtype: Tuple of ``(<filesystem>, <path from url>)``
+
+        """
+
+        if '://' not in fs_url:
+            # URL may just be a path
+            fs_url = "{}://{}".format(default_protocol, fs_url)
+
+        parse_result = self.parse(fs_url)
+        protocol = parse_result.protocol
+        open_path = parse_result.path
+
+        opener = self.protocols.get(protocol, None)
+
+        if not opener:
+            raise Unsupported(
+                "protocol '{}' is not supported".format(protocol)
+            )
+
+        open_fs = opener.open_fs(
+            fs_url,
+            parse_result,
+            writeable,
+            create,
+            cwd
+        )
+        return open_fs, open_path
+
+    def open_fs(self,
+                fs_url,
+                writeable=True,
+                create=False,
+                cwd=".",
+                default_protocol='osfs'):
+        """
+        Open a filesystem object from a FS URL (ignoring the path
+        component).
+
+        :param str fs_url: A filesystem URL
+        :param parse_result: A parsed filesystem URL.
+        :type parse_result: :class:`ParseResult`
+        :param bool writeable: True if the filesystem must be writeable.
+        :param bool create: True if the filesystem should be created if
+            it does not exist.
+        :param str cwd: The current working directory (generally only
+            relevant for OS filesystems).
+        :param str default_protocol: The protocol to use if one is not
+            supplied in the FS URL (defaults to ``"osfs"``).
+        :returns: :class:`~fs.base.FS` object
+
+        """
+        from ..base import FS
+        if isinstance(fs_url, FS):
+            _fs = fs_url
+        else:
+            _fs, _path = self.open(
+                fs_url,
+                writeable=writeable,
+                create=create,
+                cwd=cwd,
+                default_protocol=default_protocol
+            )
+        return _fs
+
+    @contextlib.contextmanager
+    def manage_fs(self, fs_url, create=False, writeable=True, cwd='.'):
+        '''
+        A context manager opens / closes a filesystem.
+
+        :param fs_url: A FS instance or a FS URL.
+        :type fs_url: str or FS
+        :param bool create: If ``True``, then create the filesytem if it
+            doesn't already exist.
+        :param bool writeable: If ``True``, then the filesystem should be
+            writeable.
+        :param str cwd: The current working directory, if opening a
+            :class:`~fs.osfs.OSFS`.
+
+        Sometimes it is convenient to be able to pass either a FS object
+        *or* an FS URL to a function. This context manager handles the
+        required logic for that.
+
+        Here's an example::
+
+            def print_ls(list_fs):
+                """List a directory."""
+                with manage_fs(list_fs) as fs:
+                    print(" ".join(fs.listdir()))
+
+        This function may be used in two ways. You may either pass either a
+        ``str``, as follows::
+
+            print_list('zip://projects.zip')
+
+        Or, an FS instance::
+
+            from fs.osfs import OSFS
+            projects_fs = OSFS('~/')
+            print_list(projects_fs)
+
+        '''
+        from ..base import FS
+        if isinstance(fs_url, FS):
+            yield fs_url
+        else:
+            _fs = self.open_fs(
+                fs_url,
+                create=create,
+                writeable=writeable,
+                cwd=cwd
+            )
+            try:
+                yield _fs
+            except:
+                raise
+            finally:
+                _fs.close()
+
+
+
+registry = Registry()
diff --git a/fs/opener/ftpfs.py b/fs/opener/ftpfs.py
new file mode 100644
index 0000000..9e33b70
--- /dev/null
+++ b/fs/opener/ftpfs.py
@@ -0,0 +1,27 @@
+# coding: utf-8
+"""Defines the FTPOpener."""
+
+from ._base import Opener
+from ._registry import registry
+
+@registry.install
+class FTPOpener(Opener):
+    protocols = ['ftp']
+
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        from ..ftpfs import FTPFS
+        ftp_host, _, dir_path = parse_result.resource.partition('/')
+        ftp_host, _, ftp_port = ftp_host.partition(':')
+        ftp_port = int(ftp_port) if ftp_port.isdigit() else 21
+        ftp_fs = FTPFS(
+            ftp_host,
+            port=ftp_port,
+            user=parse_result.username,
+            passwd=parse_result.password,
+        )
+        ftp_fs = (
+            ftp_fs.opendir(dir_path)
+            if dir_path else
+            ftp_fs
+        )
+        return ftp_fs
diff --git a/fs/opener/memoryfs.py b/fs/opener/memoryfs.py
new file mode 100644
index 0000000..1bb67d9
--- /dev/null
+++ b/fs/opener/memoryfs.py
@@ -0,0 +1,14 @@
+# coding: utf-8
+"""Defines the MemOpener."""
+
+from ._base import Opener
+from ._registry import registry
+
+@registry.install
+class MemOpener(Opener):
+    protocols = ['mem']
+
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        from ..memoryfs import MemoryFS
+        mem_fs = MemoryFS()
+        return mem_fs
diff --git a/fs/opener/osfs.py b/fs/opener/osfs.py
new file mode 100644
index 0000000..b9bfc2e
--- /dev/null
+++ b/fs/opener/osfs.py
@@ -0,0 +1,15 @@
+"""Defines the OSFSOpener."""
+from ._base import Opener
+from ._registry import registry
+
+@registry.install
+class OSFSOpener(Opener):
+    protocols = ['file', 'osfs']
+
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        from ..osfs import OSFS
+        from os.path import abspath, normpath, join
+        _path = abspath(join(cwd, parse_result.resource))
+        path = normpath(_path)
+        osfs = OSFS(path, create=create)
+        return osfs
diff --git a/fs/opener/tarfs.py b/fs/opener/tarfs.py
new file mode 100644
index 0000000..eddd583
--- /dev/null
+++ b/fs/opener/tarfs.py
@@ -0,0 +1,17 @@
+# coding: utf-8
+"""Defines the TarOpener."""
+
+from ._base import Opener
+from ._registry import registry
+
+@registry.install
+class TarOpener(Opener):
+    protocols = ['tar']
+
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        from ..tarfs import TarFS
+        tar_fs = TarFS(
+            parse_result.resource,
+            write=create
+        )
+        return tar_fs
diff --git a/fs/opener/tempfs.py b/fs/opener/tempfs.py
new file mode 100644
index 0000000..f43d215
--- /dev/null
+++ b/fs/opener/tempfs.py
@@ -0,0 +1,14 @@
+# coding: utf-8
+"""Defines the TempOpener."""
+
+from ._base import Opener
+from ._registry import registry
+
+@registry.install
+class TempOpener(Opener):
+    protocols = ['temp']
+
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        from ..tempfs import TempFS
+        temp_fs = TempFS(identifier=parse_result.resource)
+        return temp_fs
diff --git a/fs/opener/zipfs.py b/fs/opener/zipfs.py
new file mode 100644
index 0000000..af6f624
--- /dev/null
+++ b/fs/opener/zipfs.py
@@ -0,0 +1,17 @@
+# coding: utf-8
+"""Defines the ZipOpener."""
+
+from ._base import Opener
+from ._registry import registry
+
+@registry.install
+class ZipOpener(Opener):
+    protocols = ['zip']
+
+    def open_fs(self, fs_url, parse_result, writeable, create, cwd):
+        from ..zipfs import ZipFS
+        zip_fs = ZipFS(
+            parse_result.resource,
+            write=create
+        )
+        return zip_fs
diff --git a/fs/osfs.py b/fs/osfs.py
index 458db44..9ada510 100644
--- a/fs/osfs.py
+++ b/fs/osfs.py
@@ -18,7 +18,7 @@ try:
 except ImportError:
     try:
         from scandir import scandir
-    except ImportError:
+    except ImportError:  # pragma: nocover
         scandir = None
 
 from . import errors
@@ -223,8 +223,8 @@ class OSFS(FS):
             info['details'] = self._make_details_from_stat(_stat)
         if 'stat' in namespaces:
             info['stat'] = {
-                k: getattr(stat, k)
-                for k in dir(stat) if k.startswith('st_')
+                k: getattr(_stat, k)
+                for k in dir(_stat) if k.startswith('st_')
             }
         if 'access' in namespaces:
             info['access'] = self._make_access_from_stat(_stat)
@@ -388,8 +388,8 @@ class OSFS(FS):
                     if 'stat' in namespaces:
                         stat_result = dir_entry.stat()
                         info['stat'] = {
-                            k: getattr(stat, k)
-                            for k in dir(stat) if k.startswith('st_')
+                            k: getattr(stat_result, k)
+                            for k in dir(stat_result) if k.startswith('st_')
                         }
                     if 'access' in namespaces:
                         stat_result = dir_entry.stat()
@@ -420,8 +420,8 @@ class OSFS(FS):
                             self._make_details_from_stat(stat_result)
                     if 'stat' in namespaces:
                         info['stat'] = {
-                            k: getattr(stat, k)
-                            for k in dir(stat) if k.startswith('st_')
+                            k: getattr(stat_result, k)
+                            for k in dir(stat_result) if k.startswith('st_')
                         }
                     if 'access' in namespaces:
                         info['access'] =\
diff --git a/fs/path.py b/fs/path.py
index e890aa1..09ad72b 100644
--- a/fs/path.py
+++ b/fs/path.py
@@ -225,8 +225,9 @@ def combine(path1, path2):
     :param str path2: A PyFilesytem path.
     :rtype: str
 
-    This is faster than :func:`~fs.path.join`, but only works when the second
-    path is relative, and there are no back references in either path.
+    This is faster than :func:`~fs.path.join`, but only works when the
+    second path is relative, and there are no back references in either
+    path.
 
     >>> combine("foo/bar", "baz")
     'foo/bar/baz'


Test patch:
diff --git a/tests/test_base.py b/tests/test_base.py
index 9dce800..d416c22 100644
--- a/tests/test_base.py
+++ b/tests/test_base.py
@@ -44,7 +44,7 @@ class TestBase(unittest.TestCase):
 
     def test_validatepath(self):
         """Test validatepath method."""
-        with self.assertRaises(ValueError):
+        with self.assertRaises(TypeError):
             self.fs.validatepath(b'bytes')
 
         self.fs._meta['invalid_path_chars'] = 'Z'
diff --git a/tests/test_opener.py b/tests/test_opener.py
index b573b3b..0379100 100644
--- a/tests/test_opener.py
+++ b/tests/test_opener.py
@@ -17,7 +17,7 @@ class TestParse(unittest.TestCase):
 
     def test_parse_simple(self):
         parsed = opener.parse('osfs://foo/bar')
-        expected = opener.ParseResult(
+        expected = opener.registry.ParseResult(
             'osfs',
             None,
             None,
@@ -28,7 +28,7 @@ class TestParse(unittest.TestCase):
 
     def test_parse_credentials(self):
         parsed = opener.parse('ftp://user:pass@ftp.example.org')
-        expected = opener.ParseResult(
+        expected = opener.registry.ParseResult(
             'ftp',
             'user',
             'pass',
@@ -38,7 +38,7 @@ class TestParse(unittest.TestCase):
         self.assertEqual(expected, parsed)
 
         parsed = opener.parse('ftp://user@ftp.example.org')
-        expected = opener.ParseResult(
+        expected = opener.registry.ParseResult(
             'ftp',
             'user',
             '',
@@ -49,7 +49,7 @@ class TestParse(unittest.TestCase):
 
     def test_parse_path(self):
         parsed = opener.parse('osfs://foo/bar!example.txt')
-        expected = opener.ParseResult(
+        expected = opener.registry.ParseResult(
             'osfs',
             None,
             None,