Repository: pallets/click
Pull request #868 (2017-10-13 23:12:00)

CliRunner: add stdout / stderr, allow separating standard streams
attempt resolution of #371 (adding stdout / stderr)

This changes the internal API such that a singular out is never returned - it's either `(stdout, stderr)` when not mixed or `(stdout_and_stderr, False)` when mixed

This defaults to mixed to preserve old use cases (and tests); the runner switches to non-mixed via `mix_stderr`


Patch:
diff --git a/.ci/appveyor.yml b/.ci/appveyor.yml
index 1d81f1e..caa509a 100644
--- a/.ci/appveyor.yml
+++ b/.ci/appveyor.yml
@@ -32,6 +32,22 @@ environment:
       PYTHON_VERSION: "3.4.x"
       PYTHON_ARCH: "64"
 
+    - PYTHON: "C:\\Python35"
+      PYTHON_VERSION: "3.5.x"
+      PYTHON_ARCH: "32"
+
+    - PYTHON: "C:\\Python35-x64"
+      PYTHON_VERSION: "3.5.x"
+      PYTHON_ARCH: "64"
+
+    - PYTHON: "C:\\Python36"
+      PYTHON_VERSION: "3.6.x"
+      PYTHON_ARCH: "32"
+
+    - PYTHON: "C:\\Python36-x64"
+      PYTHON_VERSION: "3.6.x"
+      PYTHON_ARCH: "64"
+
 branches:
   only:
     - master
diff --git a/.gitignore b/.gitignore
index 2035180..fe562f2 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,10 +3,12 @@
 *.pyo
 *.egg-ignore
 *.egg-info
+.pytest_cache
 dist
 build
 docs/_build
 click.egg-info
+venv/
 .tox
 .cache
 .ropeproject
diff --git a/.travis.yml b/.travis.yml
index 939edcc..aed7083 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,9 +1,7 @@
 sudo: false
 language: python
 python:
-  - "2.6"
   - "2.7"
-  - "3.3"
   - "3.4"
   - "3.5"
   - "3.6"
@@ -18,7 +16,7 @@ install:
 
 script:
   - |
-    set -x
+    set -ex
     if [[ $TRAVIS_PYTHON_VERSION == pypy ]]; then
       TOX_PY=pypy
     else
diff --git a/CHANGES b/CHANGES
index 0d8e62c..d76faf9 100644
--- a/CHANGES
+++ b/CHANGES
@@ -31,6 +31,14 @@ Version 7.0
   See #790.
 - Allow autocompletion function to determine whether or not to return
   completions that start with the incomplete argument.
+- Subcommands that are named by the function now automatically have the
+  underscore replaced with a dash.  So if you register a function named
+  `my_command` it becomes `my-command` in the command line interface.
+- Stdout is now automatically set to non blocking.
+- Use realpath to convert atomic file internally into its full canonical
+  path so that changing working directories does not harm it.
+- Force stdout/stderr writable.  This works around issues with badly patched
+  standard streams like those from jupyter.
 
 Version 6.8
 -----------
@@ -43,6 +51,9 @@ Version 6.8
 - Fix crash on Windows console, see #744.
 - Fix bashcompletion on chained commands. See #754.
 - Fix option naming routine to match documentation.  See #793
+- Fixed the behavior of click error messages with regards to unicode on 2.x
+  and 3.x respectively.  Message is now always unicode and the str and unicode
+  special methods work as you expect on that platform.
 
 Version 6.7
 -----------
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 69d5594..99b493a 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -41,7 +41,7 @@ Running the testsuite
 ---------------------
 
 You probably want to set up a `virtualenv
-<http://virtualenv.readthedocs.org/en/latest/index.html>`_.
+<https://virtualenv.readthedocs.io/en/latest/index.html>`_.
 
 The minimal requirement for running the testsuite is ``py.test``.  You can
 install it with::
diff --git a/README b/README
deleted file mode 100644
index 02e65f7..0000000
--- a/README
+++ /dev/null
@@ -1,20 +0,0 @@
-$ click_
-
-  Click is a Python package for creating beautiful command line interfaces
-  in a composable way with as little code as necessary.  It's the "Command
-  Line Interface Creation Kit".  It's highly configurable but comes with
-  sensible defaults out of the box.
-
-  It aims to make the process of writing command line tools quick and fun
-  while also preventing any frustration caused by the inability to implement
-  an intended CLI API.
-
-  Click in three points:
-
-  -   arbitrary nesting of commands
-  -   automatic help page generation
-  -   supports lazy loading of subcommands at runtime
-
-  Read the docs at http://click.pocoo.org/
-
- This library is stable and active. Feedback is always welcome!
diff --git a/README.rst b/README.rst
new file mode 100644
index 0000000..07352ad
--- /dev/null
+++ b/README.rst
@@ -0,0 +1,92 @@
+\$ click\_
+==========
+
+What's Click?
+-------------
+
+Click is a Python package for creating beautiful command line interfaces
+in a composable way with as little code as necessary.  It's the "Command
+Line Interface Creation Kit".  It's highly configurable but comes with
+sensible defaults out of the box.
+
+It aims to make the process of writing command line tools quick and fun
+while also preventing any frustration caused by the inability to implement
+an intended CLI API.
+
+Click in three points:
+ -   arbitrary nesting of commands
+ -   automatic help page generation
+ -   supports lazy loading of subcommands at runtime
+
+
+Installing
+----------
+
+Install and update using `pip`_:
+
+.. code-block:: text
+
+    $ pip install click
+
+A Simple Example
+----------------
+
+What does it look like? Here is an example of a simple Click program:
+
+.. code-block:: python
+
+    import click
+    
+    @click.command()
+    @click.option('--count', default=1, help='Number of greetings.')
+    @click.option('--name', prompt='Your name',
+                  help='The person to greet.')
+    def hello(count, name):
+        """Simple program that greets NAME for a total of COUNT times."""
+        for x in range(count):
+            click.echo('Hello %s!' % name)
+    
+    if __name__ == '__main__':
+        hello()
+
+And what it looks like when run:
+
+.. code-block:: text
+
+    $ python hello.py --count=3
+    Your name: John
+    Hello John!
+    Hello John!
+    Hello John!
+
+Donate
+------
+
+The Pallets organization develops and supports Flask and the libraries
+it uses. In order to grow the community of contributors and users, and
+allow the maintainers to devote more time to the projects, `please
+donate today`_.
+
+.. _please donate today: https://psfmember.org/civicrm/contribute/transact?reset=1&id=20
+
+
+Links
+-----
+
+* Website: https://www.palletsprojects.com/p/click/
+* Documentation: http://click.pocoo.org/
+* License: `BSD <https://github.com/pallets/click/blob/master/LICENSE>`_
+* Releases: https://pypi.org/project/click/
+* Code: https://github.com/pallets/click
+* Issue tracker: https://github.com/pallets/click/issues
+* Test status:
+
+  * Linux, Mac: https://travis-ci.org/pallets/click
+  * Windows: https://ci.appveyor.com/project/pallets/click
+
+* Test coverage: https://codecov.io/gh/pallets/click
+
+.. _WSGI: https://wsgi.readthedocs.io
+.. _Werkzeug: https://www.palletsprojects.com/p/werkzeug/
+.. _Jinja: https://www.palletsprojects.com/p/jinja/
+.. _pip: https://pip.pypa.io/en/stable/quickstart/
diff --git a/click/__init__.py b/click/__init__.py
index 58df503..6de314b 100644
--- a/click/__init__.py
+++ b/click/__init__.py
@@ -66,7 +66,7 @@ __all__ = [
 
     # Types
     'ParamType', 'File', 'Path', 'Choice', 'IntRange', 'Tuple', 'STRING',
-    'INT', 'FLOAT', 'BOOL', 'UUID', 'UNPROCESSED', 'FloatRange'
+    'INT', 'FLOAT', 'BOOL', 'UUID', 'UNPROCESSED', 'FloatRange',
 
     # Utilities
     'echo', 'get_binary_stream', 'get_text_stream', 'open_file',
diff --git a/click/_compat.py b/click/_compat.py
index 312e729..eeacc63 100644
--- a/click/_compat.py
+++ b/click/_compat.py
@@ -12,20 +12,23 @@ CYGWIN = sys.platform.startswith('cygwin')
 DEFAULT_COLUMNS = 80
 
 
-_ansi_re = re.compile('\033\[((?:\d|;)*)([a-zA-Z])')
+_ansi_re = re.compile(r'\033\[((?:\d|;)*)([a-zA-Z])')
 
 
 def get_filesystem_encoding():
     return sys.getfilesystemencoding() or sys.getdefaultencoding()
 
 
-def _make_text_stream(stream, encoding, errors):
+def _make_text_stream(stream, encoding, errors,
+                      force_readable=False, force_writable=False):
     if encoding is None:
         encoding = get_best_encoding(stream)
     if errors is None:
         errors = 'replace'
     return _NonClosingTextIOWrapper(stream, encoding, errors,
-                                    line_buffering=True)
+                                    line_buffering=True,
+                                    force_readable=force_readable,
+                                    force_writable=force_writable)
 
 
 def is_ascii_encoding(encoding):
@@ -46,8 +49,10 @@ def get_best_encoding(stream):
 
 class _NonClosingTextIOWrapper(io.TextIOWrapper):
 
-    def __init__(self, stream, encoding, errors, **extra):
-        self._stream = stream = _FixupStream(stream)
+    def __init__(self, stream, encoding, errors,
+                 force_readable=False, force_writable=False, **extra):
+        self._stream = stream = _FixupStream(stream, force_readable,
+                                             force_writable)
         io.TextIOWrapper.__init__(self, stream, encoding, errors, **extra)
 
     # The io module is a place where the Python 3 text behavior
@@ -82,10 +87,16 @@ class _FixupStream(object):
     """The new io interface needs more from streams than streams
     traditionally implement.  As such, this fix-up code is necessary in
     some circumstances.
+
+    The forcing of readable and writable flags are there because some tools
+    put badly patched objects on sys (one such offender are certain version
+    of jupyter notebook).
     """
 
-    def __init__(self, stream):
+    def __init__(self, stream, force_readable=False, force_writable=False):
         self._stream = stream
+        self._force_readable = force_readable
+        self._force_writable = force_writable
 
     def __getattr__(self, name):
         return getattr(self._stream, name)
@@ -102,6 +113,8 @@ class _FixupStream(object):
         return self._stream.read(size)
 
     def readable(self):
+        if self._force_readable:
+            return True
         x = getattr(self._stream, 'readable', None)
         if x is not None:
             return x()
@@ -112,6 +125,8 @@ class _FixupStream(object):
         return True
 
     def writable(self):
+        if self._force_writable:
+            return True
         x = getattr(self._stream, 'writable', None)
         if x is not None:
             return x()
@@ -167,11 +182,12 @@ if PY2:
     # available (which is why we use try-catch instead of the WIN variable
     # here), such as the Google App Engine development server on Windows. In
     # those cases there is just nothing we can do.
+    def set_binary_mode(f):
+        return f
+
     try:
         import msvcrt
-    except ImportError:
-        set_binary_mode = lambda x: x
-    else:
+
         def set_binary_mode(f):
             try:
                 fileno = f.fileno()
@@ -180,6 +196,23 @@ if PY2:
             else:
                 msvcrt.setmode(fileno, os.O_BINARY)
             return f
+    except ImportError:
+        pass
+
+    try:
+        import fcntl
+
+        def set_binary_mode(f):
+            try:
+                fileno = f.fileno()
+            except Exception:
+                pass
+            else:
+                flags = fcntl.fcntl(fileno, fcntl.F_GETFL)
+                fcntl.fcntl(fileno, fcntl.F_SETFL, flags & ~os.O_NONBLOCK)
+            return f
+    except ImportError:
+        pass
 
     def isidentifier(x):
         return _identifier_re.search(x) is not None
@@ -197,19 +230,22 @@ if PY2:
         rv = _get_windows_console_stream(sys.stdin, encoding, errors)
         if rv is not None:
             return rv
-        return _make_text_stream(sys.stdin, encoding, errors)
+        return _make_text_stream(sys.stdin, encoding, errors,
+                                 force_readable=True)
 
     def get_text_stdout(encoding=None, errors=None):
         rv = _get_windows_console_stream(sys.stdout, encoding, errors)
         if rv is not None:
             return rv
-        return _make_text_stream(sys.stdout, encoding, errors)
+        return _make_text_stream(sys.stdout, encoding, errors,
+                                 force_writable=True)
 
     def get_text_stderr(encoding=None, errors=None):
         rv = _get_windows_console_stream(sys.stderr, encoding, errors)
         if rv is not None:
             return rv
-        return _make_text_stream(sys.stderr, encoding, errors)
+        return _make_text_stream(sys.stderr, encoding, errors,
+                                 force_writable=True)
 
     def filename_to_ui(value):
         if isinstance(value, bytes):
@@ -301,7 +337,8 @@ else:
 
         return False
 
-    def _force_correct_text_reader(text_reader, encoding, errors):
+    def _force_correct_text_reader(text_reader, encoding, errors,
+                                   force_readable=False):
         if _is_binary_reader(text_reader, False):
             binary_reader = text_reader
         else:
@@ -327,9 +364,11 @@ else:
         # we're so fundamentally fucked that nothing can repair it.
         if errors is None:
             errors = 'replace'
-        return _make_text_stream(binary_reader, encoding, errors)
+        return _make_text_stream(binary_reader, encoding, errors,
+                                 force_readable=force_readable)
 
-    def _force_correct_text_writer(text_writer, encoding, errors):
+    def _force_correct_text_writer(text_writer, encoding, errors,
+                                   force_writable=False):
         if _is_binary_writer(text_writer, False):
             binary_writer = text_writer
         else:
@@ -355,7 +394,8 @@ else:
         # we're so fundamentally fucked that nothing can repair it.
         if errors is None:
             errors = 'replace'
-        return _make_text_stream(binary_writer, encoding, errors)
+        return _make_text_stream(binary_writer, encoding, errors,
+                                 force_writable=force_writable)
 
     def get_binary_stdin():
         reader = _find_binary_reader(sys.stdin)
@@ -382,19 +422,22 @@ else:
         rv = _get_windows_console_stream(sys.stdin, encoding, errors)
         if rv is not None:
             return rv
-        return _force_correct_text_reader(sys.stdin, encoding, errors)
+        return _force_correct_text_reader(sys.stdin, encoding, errors,
+                                          force_readable=True)
 
     def get_text_stdout(encoding=None, errors=None):
         rv = _get_windows_console_stream(sys.stdout, encoding, errors)
         if rv is not None:
             return rv
-        return _force_correct_text_writer(sys.stdout, encoding, errors)
+        return _force_correct_text_writer(sys.stdout, encoding, errors,
+                                          force_writable=True)
 
     def get_text_stderr(encoding=None, errors=None):
         rv = _get_windows_console_stream(sys.stderr, encoding, errors)
         if rv is not None:
             return rv
-        return _force_correct_text_writer(sys.stderr, encoding, errors)
+        return _force_correct_text_writer(sys.stderr, encoding, errors,
+                                          force_writable=True)
 
     def filename_to_ui(value):
         if isinstance(value, bytes):
@@ -423,7 +466,7 @@ def open_stream(filename, mode='r', encoding=None, errors='strict',
     # Standard streams first.  These are simple because they don't need
     # special handling for the atomic flag.  It's entirely ignored.
     if filename == '-':
-        if 'w' in mode:
+        if any(m in mode for m in ['w', 'a', 'x']):
             if 'b' in mode:
                 return get_binary_stdout(), False
             return get_text_stdout(encoding=encoding, errors=errors), False
@@ -463,7 +506,7 @@ def open_stream(filename, mode='r', encoding=None, errors='strict',
     else:
         f = os.fdopen(fd, mode)
 
-    return _AtomicFile(f, tmp_filename, filename), True
+    return _AtomicFile(f, tmp_filename, os.path.realpath(filename)), True
 
 
 # Used in a destructor call, needs extra protection from interpreter cleanup.
diff --git a/click/_termui_impl.py b/click/_termui_impl.py
index cebf3bb..44b4ae4 100644
--- a/click/_termui_impl.py
+++ b/click/_termui_impl.py
@@ -13,6 +13,7 @@ import os
 import sys
 import time
 import math
+
 from ._compat import _default_text_stdout, range_type, PY2, isatty, \
      open_stream, strip_ansi, term_len, get_best_encoding, WIN, int_types, \
      CYGWIN
@@ -192,43 +193,40 @@ class ProgressBar(object):
 
     def render_progress(self):
         from .termui import get_terminal_size
-        nl = False
 
         if self.is_hidden:
-            buf = [self.label]
-            nl = True
-        else:
-            buf = []
-            # Update width in case the terminal has been resized
-            if self.autowidth:
-                old_width = self.width
-                self.width = 0
-                clutter_length = term_len(self.format_progress_line())
-                new_width = max(0, get_terminal_size()[0] - clutter_length)
-                if new_width < old_width:
-                    buf.append(BEFORE_BAR)
-                    buf.append(' ' * self.max_width)
-                    self.max_width = new_width
-                self.width = new_width
-
-            clear_width = self.width
-            if self.max_width is not None:
-                clear_width = self.max_width
-
-            buf.append(BEFORE_BAR)
-            line = self.format_progress_line()
-            line_len = term_len(line)
-            if self.max_width is None or self.max_width < line_len:
-                self.max_width = line_len
-            buf.append(line)
-
-            buf.append(' ' * (clear_width - line_len))
-        line = ''.join(buf)
+            return
 
+        buf = []
+        # Update width in case the terminal has been resized
+        if self.autowidth:
+            old_width = self.width
+            self.width = 0
+            clutter_length = term_len(self.format_progress_line())
+            new_width = max(0, get_terminal_size()[0] - clutter_length)
+            if new_width < old_width:
+                buf.append(BEFORE_BAR)
+                buf.append(' ' * self.max_width)
+                self.max_width = new_width
+            self.width = new_width
+
+        clear_width = self.width
+        if self.max_width is not None:
+            clear_width = self.max_width
+
+        buf.append(BEFORE_BAR)
+        line = self.format_progress_line()
+        line_len = term_len(line)
+        if self.max_width is None or self.max_width < line_len:
+            self.max_width = line_len
+
+        buf.append(line)
+        buf.append(' ' * (clear_width - line_len))
+        line = ''.join(buf)
         # Render the line only if it changed.
         if line != self._last_line:
             self._last_line = line
-            echo(line, file=self.file, color=self.color, nl=nl)
+            echo(line, file=self.file, color=self.color, nl=True)
             self.file.flush()
 
     def make_step(self, n_steps):
@@ -272,35 +270,35 @@ class ProgressBar(object):
         del next
 
 
-def pager(text, color=None):
+def pager(generator, color=None):
     """Decide what method to use for paging through text."""
     stdout = _default_text_stdout()
     if not isatty(sys.stdin) or not isatty(stdout):
-        return _nullpager(stdout, text, color)
+        return _nullpager(stdout, generator, color)
     pager_cmd = (os.environ.get('PAGER', None) or '').strip()
     if pager_cmd:
         if WIN:
-            return _tempfilepager(text, pager_cmd, color)
-        return _pipepager(text, pager_cmd, color)
+            return _tempfilepager(generator, pager_cmd, color)
+        return _pipepager(generator, pager_cmd, color)
     if os.environ.get('TERM') in ('dumb', 'emacs'):
-        return _nullpager(stdout, text, color)
+        return _nullpager(stdout, generator, color)
     if WIN or sys.platform.startswith('os2'):
-        return _tempfilepager(text, 'more <', color)
+        return _tempfilepager(generator, 'more <', color)
     if hasattr(os, 'system') and os.system('(less) 2>/dev/null') == 0:
-        return _pipepager(text, 'less', color)
+        return _pipepager(generator, 'less', color)
 
     import tempfile
     fd, filename = tempfile.mkstemp()
     os.close(fd)
     try:
         if hasattr(os, 'system') and os.system('more "%s"' % filename) == 0:
-            return _pipepager(text, 'more', color)
-        return _nullpager(stdout, text, color)
+            return _pipepager(generator, 'more', color)
+        return _nullpager(stdout, generator, color)
     finally:
         os.unlink(filename)
 
 
-def _pipepager(text, cmd, color):
+def _pipepager(generator, cmd, color):
     """Page through text by feeding it to another program.  Invoking a
     pager through this might support colors.
     """
@@ -318,17 +316,19 @@ def _pipepager(text, cmd, color):
         elif 'r' in less_flags or 'R' in less_flags:
             color = True
 
-    if not color:
-        text = strip_ansi(text)
-
     c = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE,
                          env=env)
     encoding = get_best_encoding(c.stdin)
     try:
-        c.stdin.write(text.encode(encoding, 'replace'))
-        c.stdin.close()
+        for text in generator:
+            if not color:
+                text = strip_ansi(text)
+
+            c.stdin.write(text.encode(encoding, 'replace'))
     except (IOError, KeyboardInterrupt):
         pass
+    else:
+        c.stdin.close()
 
     # Less doesn't respect ^C, but catches it for its own UI purposes (aborting
     # search or other commands inside less).
@@ -347,10 +347,12 @@ def _pipepager(text, cmd, color):
             break
 
 
-def _tempfilepager(text, cmd, color):
+def _tempfilepager(generator, cmd, color):
     """Page through text by invoking a program on a temporary file."""
     import tempfile
     filename = tempfile.mktemp()
+    # TODO: This never terminates if the passed generator never terminates.
+    text = "".join(generator)
     if not color:
         text = strip_ansi(text)
     encoding = get_best_encoding(sys.stdout)
@@ -362,11 +364,12 @@ def _tempfilepager(text, cmd, color):
         os.unlink(filename)
 
 
-def _nullpager(stream, text, color):
+def _nullpager(stream, generator, color):
     """Simply print unformatted text.  This is the ultimate fallback."""
-    if not color:
-        text = strip_ansi(text)
-    stream.write(text)
+    for text in generator:
+        if not color:
+            text = strip_ansi(text)
+        stream.write(text)
 
 
 class Editor(object):
diff --git a/click/_unicodefun.py b/click/_unicodefun.py
index e09e18b..6383415 100644
--- a/click/_unicodefun.py
+++ b/click/_unicodefun.py
@@ -62,8 +62,11 @@ def _verify_python3_env():
     extra = ''
     if os.name == 'posix':
         import subprocess
-        rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,
-                              stderr=subprocess.PIPE).communicate()[0]
+        try:
+            rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,
+                                  stderr=subprocess.PIPE).communicate()[0]
+        except OSError:
+            rv = b''
         good_locales = set()
         has_c_utf8 = False
 
diff --git a/click/core.py b/click/core.py
index b307407..3392520 100644
--- a/click/core.py
+++ b/click/core.py
@@ -59,9 +59,7 @@ def _check_multicommand(base_command, cmd_name, cmd, register=False):
     raise RuntimeError('%s.  Command "%s" is set to chain and "%s" was '
                        'added as subcommand but it in itself is a '
                        'multi command.  ("%s" is a %s within a chained '
-                       '%s named "%s").  This restriction was supposed to '
-                       'be lifted in 6.0 but the fix was flawed.  This '
-                       'will be fixed in Click 7.0' % (
+                       '%s named "%s").' % (
                            hint, base_command.name, cmd_name,
                            cmd_name, cmd.__class__.__name__,
                            base_command.__class__.__name__,
@@ -381,7 +379,7 @@ class Context(object):
     @property
     def meta(self):
         """This is a dictionary which is shared with all the contexts
-        that are nested.  It exists so that click utiltiies can store some
+        that are nested.  It exists so that click utilities can store some
         state here if they need to.  It is however the responsibility of
         that code to manage this dictionary well.
 
@@ -828,8 +826,6 @@ class Command(BaseCommand):
     def make_parser(self, ctx):
         """Creates the underlying option parser for this command."""
         parser = OptionParser(ctx)
-        parser.allow_interspersed_args = ctx.allow_interspersed_args
-        parser.ignore_unknown_options = ctx.ignore_unknown_options
         for param in self.get_params(ctx):
             param.add_to_parser(parser, ctx)
         return parser
@@ -1230,7 +1226,7 @@ class CommandCollection(MultiCommand):
 
 
 class Parameter(object):
-    """A parameter to a command comes in two versions: they are either
+    r"""A parameter to a command comes in two versions: they are either
     :class:`Option`\s or :class:`Argument`\s.  Other subclasses are currently
     not supported by design as some of the internals for parsing are
     intentionally not finalized.
@@ -1330,12 +1326,13 @@ class Parameter(object):
     def add_to_parser(self, parser, ctx):
         pass
 
+
     def consume_value(self, ctx, opts):
         value = opts.get(self.name)
-        if value is None:
-            value = ctx.lookup_default(self.name)
         if value is None:
             value = self.value_from_envvar(ctx)
+        if value is None:
+            value = ctx.lookup_default(self.name)
         return value
 
     def type_cast_value(self, ctx, value):
@@ -1432,6 +1429,13 @@ class Parameter(object):
     def get_usage_pieces(self, ctx):
         return []
 
+    def get_error_hint(self, ctx):
+        """Get a stringified version of the param for use in error messages to
+        indicate which param caused the error.
+        """
+        hint_list = self.opts or [self.human_readable_name]
+        return ' / '.join('"%s"' % x for x in hint_list)
+
 
 class Option(Parameter):
     """Options are usually optional values on the command line and
@@ -1756,6 +1760,9 @@ class Argument(Parameter):
     def get_usage_pieces(self, ctx):
         return [self.make_metavar()]
 
+    def get_error_hint(self, ctx):
+        return '"%s"' % self.make_metavar()
+
     def add_to_parser(self, parser, ctx):
         parser.add_argument(dest=self.name, nargs=self.nargs,
                             obj=self)
diff --git a/click/decorators.py b/click/decorators.py
index 64af015..bfc7a05 100644
--- a/click/decorators.py
+++ b/click/decorators.py
@@ -85,12 +85,12 @@ def _make_command(f, name, attrs, cls):
         help = inspect.cleandoc(help)
     attrs['help'] = help
     _check_for_unicode_literals()
-    return cls(name=name or f.__name__.lower(),
+    return cls(name=name or f.__name__.lower().replace('_', '-'),
                callback=f, params=params, **attrs)
 
 
 def command(name=None, cls=None, **attrs):
-    """Creates a new :class:`Command` and uses the decorated function as
+    r"""Creates a new :class:`Command` and uses the decorated function as
     callback.  This will also automatically attach all decorated
     :func:`option`\s and :func:`argument`\s as parameters to the command.
 
@@ -105,7 +105,7 @@ def command(name=None, cls=None, **attrs):
     command :class:`Group`.
 
     :param name: the name of the command.  This defaults to the function
-                 name.
+                 name with underscores replaced by dashes.
     :param cls: the command class to instantiate.  This defaults to
                 :class:`Command`.
     """
@@ -164,10 +164,13 @@ def option(*param_decls, **attrs):
                 :class:`Option`.
     """
     def decorator(f):
-        if 'help' in attrs:
-            attrs['help'] = inspect.cleandoc(attrs['help'])
-        OptionClass = attrs.pop('cls', Option)
-        _param_memo(f, OptionClass(param_decls, **attrs))
+        # Issue 926, copy attrs, so pre-defined options can re-use the same cls=
+        option_attrs = attrs.copy()
+
+        if 'help' in option_attrs:
+            option_attrs['help'] = inspect.cleandoc(option_attrs['help'])
+        OptionClass = option_attrs.pop('cls', Option)
+        _param_memo(f, OptionClass(param_decls, **option_attrs))
         return f
     return decorator
 
diff --git a/click/exceptions.py b/click/exceptions.py
index 6f7a536..b564134 100644
--- a/click/exceptions.py
+++ b/click/exceptions.py
@@ -2,6 +2,12 @@ from ._compat import PY2, filename_to_ui, get_text_stderr
 from .utils import echo
 
 
+def _join_param_hints(param_hint):
+    if isinstance(param_hint, (tuple, list)):
+        return ' / '.join('"%s"' % x for x in param_hint)
+    return param_hint
+
+
 class ClickException(Exception):
     """An exception that Click can handle and show to the user."""
 
@@ -19,11 +25,14 @@ class ClickException(Exception):
     def format_message(self):
         return self.message
 
-    def __unicode__(self):
+    def __str__(self):
         return self.message
 
-    def __str__(self):
-        return self.message.encode('utf-8')
+    if PY2:
+        __unicode__ = __str__
+
+        def __str__(self):
+            return self.message.encode('utf-8')
 
     def show(self, file=None):
         if file is None:
@@ -89,11 +98,11 @@ class BadParameter(UsageError):
         if self.param_hint is not None:
             param_hint = self.param_hint
         elif self.param is not None:
-            param_hint = self.param.opts or [self.param.human_readable_name]
+            param_hint = self.param.get_error_hint(self.ctx)
         else:
             return 'Invalid value: %s' % self.message
-        if isinstance(param_hint, (tuple, list)):
-            param_hint = ' / '.join('"%s"' % x for x in param_hint)
+        param_hint = _join_param_hints(param_hint)
+
         return 'Invalid value for %s: %s' % (param_hint, self.message)
 
 
@@ -118,11 +127,10 @@ class MissingParameter(BadParameter):
         if self.param_hint is not None:
             param_hint = self.param_hint
         elif self.param is not None:
-            param_hint = self.param.opts or [self.param.human_readable_name]
+            param_hint = self.param.get_error_hint(self.ctx)
         else:
             param_hint = None
-        if isinstance(param_hint, (tuple, list)):
-            param_hint = ' / '.join('"%s"' % x for x in param_hint)
+        param_hint = _join_param_hints(param_hint)
 
         param_type = self.param_type
         if param_type is None and self.param is not None:
diff --git a/click/termui.py b/click/termui.py
index 9a89afe..6683b89 100644
--- a/click/termui.py
+++ b/click/termui.py
@@ -1,6 +1,8 @@
 import os
 import sys
 import struct
+import inspect
+import itertools
 
 from ._compat import raw_input, text_type, string_types, \
      isatty, strip_ansi, get_winterm_size, DEFAULT_COLUMNS, WIN
@@ -203,22 +205,33 @@ def get_terminal_size():
     return int(cr[1]), int(cr[0])
 
 
-def echo_via_pager(text, color=None):
+def echo_via_pager(text_or_generator, color=None):
     """This function takes a text and shows it via an environment specific
     pager on stdout.
 
     .. versionchanged:: 3.0
        Added the `color` flag.
 
-    :param text: the text to page.
+    :param text_or_generator: the text to page, or alternatively, a
+                              generator emitting the text to page.
     :param color: controls if the pager supports ANSI colors or not.  The
                   default is autodetection.
     """
     color = resolve_color_default(color)
-    if not isinstance(text, string_types):
-        text = text_type(text)
+
+    if inspect.isgeneratorfunction(text_or_generator):
+        i = text_or_generator()
+    elif isinstance(text_or_generator, string_types):
+        i = [text_or_generator]
+    else:
+        i = iter(text_or_generator)
+
+    # convert every element of i to a text type if necessary
+    text_generator = (el if isinstance(el, string_types) else text_type(el)
+                      for el in i)
+
     from ._termui_impl import pager
-    return pager(text + '\n', color)
+    return pager(itertools.chain(text_generator, "\n"), color)
 
 
 def progressbar(iterable=None, length=None, label=None, show_eta=True,
diff --git a/click/types.py b/click/types.py
index fab63e4..f594005 100644
--- a/click/types.py
+++ b/click/types.py
@@ -129,6 +129,9 @@ class Choice(ParamType):
     """The choice type allows a value to be checked against a fixed set of
     supported values.  All of these values have to be strings.
 
+    You should only pass *choices* as list or tuple.  Other iterables (like
+    generators) may lead to surprising results.
+
     See :ref:`choice-opts` for an example.
     """
     name = 'choice'
@@ -274,7 +277,7 @@ class BoolParamType(ParamType):
         if isinstance(value, bool):
             return bool(value)
         value = value.lower()
-        if value in ('true', 't,' '1', 'yes', 'y'):
+        if value in ('true', 't', '1', 'yes', 'y'):
             return True
         elif value in ('false', 'f', '0', 'no', 'n'):
             return False
diff --git a/click/utils.py b/click/utils.py
index fdf4f62..9f175eb 100644
--- a/click/utils.py
+++ b/click/utils.py
@@ -184,7 +184,7 @@ def echo(message=None, file=None, nl=True, err=False, color=None):
     -   hide ANSI codes automatically if the destination file is not a
         terminal.
 
-    .. _colorama: http://pypi.python.org/pypi/colorama
+    .. _colorama: https://pypi.org/project/colorama/
 
     .. versionchanged:: 6.0
        As of Click 6.0 the echo function will properly support unicode
diff --git a/docs/_templates/sidebarintro.html b/docs/_templates/sidebarintro.html
index afc1002..4e49aba 100644
--- a/docs/_templates/sidebarintro.html
+++ b/docs/_templates/sidebarintro.html
@@ -7,7 +7,7 @@
 <h3>Useful Links</h3>
 <ul>
   <li><a href="http://click.pocoo.org/">The Click Website</a></li>
-  <li><a href="http://pypi.python.org/pypi/click">click @ PyPI</a></li>
+  <li><a href="https://pypi.org/project/click/">click @ PyPI</a></li>
   <li><a href="http://github.com/pallets/click">click @ github</a></li>
   <li><a href="http://github.com/pallets/click/issues">Issue Tracker</a></li>
 </ul>
diff --git a/docs/advanced.rst b/docs/advanced.rst
index 17f86da..f8c0ed8 100644
--- a/docs/advanced.rst
+++ b/docs/advanced.rst
@@ -295,8 +295,8 @@ In the end you end up with something like this:
     @click.option('-v', '--verbose', is_flag=True, help='Enables verbose mode')
     @click.argument('timeit_args', nargs=-1, type=click.UNPROCESSED)
     def cli(verbose, timeit_args):
-        """A wrapper around Python's timeit."""
-        cmdline = ['python', '-mtimeit'] + list(timeit_args)
+        """A fake wrapper around Python's timeit."""
+        cmdline = ['echo', 'python', '-mtimeit'] + list(timeit_args)
         if verbose:
             click.echo('Invoking: %s' % ' '.join(cmdline))
         call(cmdline)
@@ -321,7 +321,7 @@ are important to know about how this ignoring of unhandled flag happens:
     generally end up like that.  Note that because the parser cannot know
     if an option will accept an argument or not, the ``bar`` part might be
     handled as an argument.
-*   Unknown short options might be partially handled and reassmebled if
+*   Unknown short options might be partially handled and reassembled if
     necessary.  For instance in the above example there is an option
     called ``-v`` which enables verbose mode.  If the command would be
     ignored with ``-va`` then the ``-v`` part would be handled by Click
diff --git a/docs/clickdoctools.py b/docs/clickdoctools.py
index 36723fa..c3db195 100644
--- a/docs/clickdoctools.py
+++ b/docs/clickdoctools.py
@@ -17,6 +17,13 @@ from docutils.statemachine import ViewList
 from sphinx.domains import Domain
 from sphinx.util.compat import Directive
 
+PY2 = sys.version_info[0] == 2
+
+if PY2:
+    text_type = unicode
+else:
+    text_type = str
+    
 
 class EchoingStdin(object):
 
@@ -70,7 +77,7 @@ def fake_modules():
 
 @contextlib.contextmanager
 def isolation(input=None, env=None):
-    if isinstance(input, unicode):
+    if isinstance(input, text_type):
         input = input.encode('utf-8')
     input = StringIO(input or '')
     output = StringIO()
diff --git a/docs/commands.rst b/docs/commands.rst
index 5a7ef79..2320432 100644
--- a/docs/commands.rst
+++ b/docs/commands.rst
@@ -29,7 +29,7 @@ when an inner command runs:
 
     @cli.command()
     def sync():
-        click.echo('Synching')
+        click.echo('Syncing')
 
 Here is what this looks like:
 
diff --git a/docs/complex.rst b/docs/complex.rst
index 794de2d..9907606 100644
--- a/docs/complex.rst
+++ b/docs/complex.rst
@@ -153,10 +153,10 @@ One obvious way to remedy this is to store a reference to the repo in the
 plugin, but then a command needs to be aware that it's attached below such a
 plugin.
 
-There is a much better system that can built by taking advantage of the linked
-nature of contexts.  We know that the plugin context is linked to the context
-that created our repo.  Because of that, we can start a search for the last
-level where the object stored by the context was a repo.
+There is a much better system that can be built by taking advantage of the
+linked nature of contexts.  We know that the plugin context is linked to the
+context that created our repo.  Because of that, we can start a search for
+the last level where the object stored by the context was a repo.
 
 Built-in support for this is provided by the :func:`make_pass_decorator`
 factory, which will create decorators for us that find objects (it
@@ -210,7 +210,7 @@ As such it runs standalone:
     @click.command()
     @pass_repo
     def cp(repo):
-        click.echo(repo)
+        click.echo(isinstance(repo, Repo))
 
 As you can see:
 
diff --git a/docs/options.rst b/docs/options.rst
index dd61e5b..bb29fcd 100644
--- a/docs/options.rst
+++ b/docs/options.rst
@@ -292,6 +292,11 @@ What it looks like:
     println()
     invoke(digest, args=['--help'])
 
+.. note::
+
+    You should only pass the choices as list or tuple.  Other iterables (like
+    generators) may lead to surprising results.
+
 .. _option-prompting:
 
 Prompting
diff --git a/docs/python3.rst b/docs/python3.rst
index cc84a05..65e1b50 100644
--- a/docs/python3.rst
+++ b/docs/python3.rst
@@ -6,7 +6,7 @@ Python 3 Support
 Click supports Python 3, but like all other command line utility libraries,
 it suffers from the Unicode text model in Python 3.  All examples in the
 documentation were written so that they could run on both Python 2.x and
-Python 3.3 or higher.
+Python 3.4 or higher.
 
 At the moment, it is strongly recommended to use Python 2 for Click
 utilities unless Python 3 is a hard requirement.
diff --git a/docs/quickstart.rst b/docs/quickstart.rst
index 8d02296..234f809 100644
--- a/docs/quickstart.rst
+++ b/docs/quickstart.rst
@@ -51,7 +51,7 @@ If you are on Windows (or none of the above methods worked) you must install
 Once you have it installed, run the ``pip`` command from above, but without
 the `sudo` prefix.
 
-.. _installing pip: http://pip.readthedocs.org/en/latest/installing.html
+.. _installing pip: https://pip.readthedocs.io/en/latest/installing.html
 
 Once you have virtualenv installed, just fire up a shell and create
 your own environment.  I usually create a project folder and a `venv`
diff --git a/docs/utils.rst b/docs/utils.rst
index 9bd98b0..dc25c79 100644
--- a/docs/utils.rst
+++ b/docs/utils.rst
@@ -95,7 +95,7 @@ a single function called :func:`secho`::
     click.secho('ATTENTION', blink=True, bold=True)
 
 
-.. _colorama: https://pypi.python.org/pypi/colorama
+.. _colorama: https://pypi.org/project/colorama/
 
 Pager Support
 -------------
@@ -114,6 +114,17 @@ Example:
         click.echo_via_pager('\n'.join('Line %d' % idx
                                        for idx in range(200)))
 
+If you want to use the pager for a lot of text, especially if generating everything in advance would take a lot of time, you can pass a generator (or generator function) instead of a string:
+
+.. click:example::
+    def _generate_output():
+        for idx in range(50000):
+            yield "Line %d\n" % idx
+
+    @click.command()
+    def less():
+        click.echo_via_pager(_generate_output())
+
 
 Screen Clearing
 ---------------
diff --git a/examples/validation/validation.py b/examples/validation/validation.py
index 4b95091..00fa0a6 100644
--- a/examples/validation/validation.py
+++ b/examples/validation/validation.py
@@ -1,6 +1,6 @@
 import click
 try:
-    from urllib import parser as urlparse
+    from urllib import parse as urlparse
 except ImportError:
     import urlparse
 
diff --git a/setup.py b/setup.py
index 79f7f66..b78403e 100644
--- a/setup.py
+++ b/setup.py
@@ -1,28 +1,44 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+import io
 import re
-import ast
 from setuptools import setup
 
 
 _version_re = re.compile(r'__version__\s+=\s+(.*)')
 
 
-with open('click/__init__.py', 'rb') as f:
-    version = str(ast.literal_eval(_version_re.search(
-        f.read().decode('utf-8')).group(1)))
+with io.open('README.rst', 'rt', encoding='utf8') as f:
+    readme = f.read()
 
+with io.open('click/__init__.py', 'rt', encoding='utf8') as f:
+    version = re.search(r'__version__ = \'(.*?)\'', f.read()).group(1)
 
 setup(
     name='click',
+    version=version,
+    url='https://www.palletsprojects.com/p/click/',
     author='Armin Ronacher',
     author_email='armin.ronacher@active-4.com',
-    version=version,
-    url='http://github.com/pallets/click',
+    maintainer='Pallets team',
+    maintainer_email='contact@palletsprojects.com',
+    long_description=readme,
     packages=['click'],
     description='A simple wrapper around optparse for '
                 'powerful command line utilities.',
+    license='BSD',
     classifiers=[
+        'Development Status :: 5 - Production/Stable',
+        'Intended Audience :: Developers',
         'License :: OSI Approved :: BSD License',
+        'Operating System :: OS Independent'
         'Programming Language :: Python',
+        'Programming Language :: Python :: 2',
+        'Programming Language :: Python :: 2.7',
         'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: 3.4',
+        'Programming Language :: Python :: 3.5',
+        'Programming Language :: Python :: 3.6',
     ],
+    python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*",
 )
diff --git a/tox.ini b/tox.ini
index de1fb38..ef4a703 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,6 @@
 [tox]
-envlist = py{26,27,33,34,35,36,py}
+envlist = py27,py34,py35,py36,pypy
+skip_missing_interpreters = true
 
 [testenv]
 passenv = LANG


Test patch:
diff --git a/click/testing.py b/click/testing.py
index ae36d10..f13d4bf 100644
--- a/click/testing.py
+++ b/click/testing.py
@@ -73,12 +73,14 @@ def make_input_stream(input, charset):
 class Result(object):
     """Holds the captured result of an invoked CLI script."""
 
-    def __init__(self, runner, output_bytes, exit_code, exception,
-                 exc_info=None):
+    def __init__(self, runner, stdout_bytes, stderr_bytes, exit_code,
+                 exception, exc_info=None):
         #: The runner that created the result
         self.runner = runner
-        #: The output as bytes.
-        self.output_bytes = output_bytes
+        #: The standard output as bytes.
+        self.stdout_bytes = stdout_bytes
+        #: The standard error as bytes, or False(y) if not available
+        self.stderr_bytes = stderr_bytes
         #: The exit code as integer.
         self.exit_code = exit_code
         #: The exception that happend if one did.
@@ -88,12 +90,27 @@ class Result(object):
 
     @property
     def output(self):
-        """The output as unicode string."""
-        return self.output_bytes.decode(self.runner.charset, 'replace') \
+        """The (standard) output as unicode string."""
+        return self.stdout
+
+    @property
+    def stdout(self):
+        """The standard output as unicode string."""
+        return self.stdout_bytes.decode(self.runner.charset, 'replace') \
             .replace('\r\n', '\n')
 
+    @property
+    def stderr(self):
+        """The standard error as unicode string."""
+        if not self.stderr_bytes:
+            raise ValueError("stderr not separately captured")
+        return self.stderr_bytes.decode(self.runner.charset, 'replace') \
+            .replace('\r\n', '\n')
+
+
     def __repr__(self):
-        return '<Result %s>' % (
+        return '<%s %s>' % (
+            type(self).__name__,
             self.exception and repr(self.exception) or 'okay',
         )
 
@@ -112,14 +129,21 @@ class CliRunner(object):
                        to stdout.  This is useful for showing examples in
                        some circumstances.  Note that regular prompts
                        will automatically echo the input.
+    :param mix_stderr: if this is set to `False`, then stdout and stderr are
+                       preserved as independent streams.  This is useful for
+                       Unix-philosophy apps that have predictable stdout and
+                       noisy stderr, such that each may be measured
+                       independently
     """
 
-    def __init__(self, charset=None, env=None, echo_stdin=False):
+    def __init__(self, charset=None, env=None, echo_stdin=False,
+                 mix_stderr=True):
         if charset is None:
             charset = 'utf-8'
         self.charset = charset
         self.env = env or {}
         self.echo_stdin = echo_stdin
+        self.mix_stderr = mix_stderr
 
     def get_default_prog_name(self, cli):
         """Given a command object it will return the default program name
@@ -164,16 +188,27 @@ class CliRunner(object):
         env = self.make_env(env)
 
         if PY2:
-            sys.stdout = sys.stderr = bytes_output = StringIO()
+            bytes_output = StringIO()
             if self.echo_stdin:
                 input = EchoingStdin(input, bytes_output)
+            sys.stdout = bytes_output
+            if not self.mix_stderr:
+                bytes_error = StringIO()
+                sys.stderr = bytes_error
         else:
             bytes_output = io.BytesIO()
             if self.echo_stdin:
                 input = EchoingStdin(input, bytes_output)
             input = io.TextIOWrapper(input, encoding=self.charset)
-            sys.stdout = sys.stderr = io.TextIOWrapper(
+            sys.stdout = io.TextIOWrapper(
                 bytes_output, encoding=self.charset)
+            if not self.mix_stderr:
+                bytes_error = io.BytesIO()
+                sys.stderr = io.TextIOWrapper(
+                    bytes_error, encoding=self.charset)
+
+        if self.mix_stderr:
+            sys.stderr = sys.stdout
 
         sys.stdin = input
 
@@ -223,7 +258,7 @@ class CliRunner(object):
                         pass
                 else:
                     os.environ[key] = value
-            yield bytes_output
+            yield (bytes_output, not self.mix_stderr and bytes_error)
         finally:
             for key, value in iteritems(old_env):
                 if value is None:
@@ -243,7 +278,7 @@ class CliRunner(object):
             clickpkg.formatting.FORCED_WIDTH = old_forced_width
 
     def invoke(self, cli, args=None, input=None, env=None,
-               catch_exceptions=True, color=False, **extra):
+               catch_exceptions=True, color=False, mix_stderr=False, **extra):
         """Invokes a command in an isolated environment.  The arguments are
         forwarded directly to the command line script, the `extra` keyword
         arguments are passed to the :meth:`~clickpkg.Command.main` function of
@@ -275,7 +310,7 @@ class CliRunner(object):
                       application can still override this explicitly.
         """
         exc_info = None
-        with self.isolation(input=input, env=env, color=color) as out:
+        with self.isolation(input=input, env=env, color=color) as outstreams:
             exception = None
             exit_code = 0
 
@@ -307,10 +342,12 @@ class CliRunner(object):
                 exc_info = sys.exc_info()
             finally:
                 sys.stdout.flush()
-                output = out.getvalue()
+                stdout = outstreams[0].getvalue()
+                stderr = outstreams[1] and outstreams[1].getvalue()
 
         return Result(runner=self,
-                      output_bytes=output,
+                      stdout_bytes=stdout,
+                      stderr_bytes=stderr,
                       exit_code=exit_code,
                       exception=exception,
                       exc_info=exc_info)
diff --git a/tests/test_arguments.py b/tests/test_arguments.py
index 1972976..a6a3258 100644
--- a/tests/test_arguments.py
+++ b/tests/test_arguments.py
@@ -188,7 +188,7 @@ def test_empty_nargs(runner):
 
     result = runner.invoke(cmd2, [])
     assert result.exit_code == 2
-    assert 'Missing argument "arg"' in result.output
+    assert 'Missing argument "ARG..."' in result.output
 
 
 def test_missing_arg(runner):
@@ -199,7 +199,7 @@ def test_missing_arg(runner):
 
     result = runner.invoke(cmd, [])
     assert result.exit_code == 2
-    assert 'Missing argument "arg".' in result.output
+    assert 'Missing argument "ARG".' in result.output
 
 
 def test_implicit_non_required(runner):
diff --git a/tests/test_basic.py b/tests/test_basic.py
index 573bcbf..8ba251f 100644
--- a/tests/test_basic.py
+++ b/tests/test_basic.py
@@ -187,7 +187,7 @@ def test_boolean_conversion(runner):
         def cli(flag):
             click.echo(flag)
 
-        for value in 'true', 't,' '1', 'yes', 'y':
+        for value in 'true', 't', '1', 'yes', 'y':
             result = runner.invoke(cli, ['--flag', value])
             assert not result.exception
             assert result.output == 'True\n'
@@ -412,7 +412,7 @@ def test_required_option(runner):
 def test_evaluation_order(runner):
     called = []
 
-    def memo(ctx, value):
+    def memo(ctx, param, value):
         called.append(value)
         return value
 
diff --git a/tests/test_commands.py b/tests/test_commands.py
index 9b6a6fb..e8a9535 100644
--- a/tests/test_commands.py
+++ b/tests/test_commands.py
@@ -1,6 +1,8 @@
 # -*- coding: utf-8 -*-
 import re
+
 import click
+import pytest
 
 
 def test_other_command_invoke(runner):
@@ -62,7 +64,7 @@ def test_auto_shorthelp(runner):
         r'Commands:\n\s+'
         r'long\s+This is a long text that is too long to show\.\.\.\n\s+'
         r'short\s+This is a short text\.\n\s+'
-        r'special_chars\s+Login and store the token in ~/.netrc\.\s*',
+        r'special-chars\s+Login and store the token in ~/.netrc\.\s*',
         result.output) is not None
 
 
@@ -206,7 +208,7 @@ def test_other_command_invoke_with_defaults(runner):
     @click.option('--foo', type=click.INT, default=42)
     @click.pass_context
     def other_cmd(ctx, foo):
-        assert ctx.info_name == 'other_cmd'
+        assert ctx.info_name == 'other-cmd'
         click.echo(foo)
 
     result = runner.invoke(cli, [])
@@ -253,3 +255,63 @@ def test_unprocessed_options(runner):
         'Verbosity: 4',
         'Args: -foo|-x|--muhaha|x|y|-x',
     ]
+
+
+def test_subcommand_naming(runner):
+    @click.group()
+    def cli():
+        pass
+
+    @cli.command()
+    def foo_bar():
+        click.echo('foo-bar')
+
+    result = runner.invoke(cli, ['foo-bar'])
+    assert not result.exception
+    assert result.output.splitlines() == ['foo-bar']
+
+
+def test_environment_variables(runner):
+    @click.group()
+    def cli():
+        pass
+
+    @cli.command()
+    @click.option('--name', envvar='CLICK_NAME')
+    def foo(name):
+        click.echo(name)
+
+    result = runner.invoke(cli, ['foo'], env={'CLICK_NAME': 'environment'})
+
+    assert not result.exception
+    assert result.output == 'environment\n'
+
+
+# Ensures the variables are read in the following order:
+# 1. CLI
+# 2. Environment
+# 3. Defaults
+variable_precedence_testdata = [
+    (['foo', '--name=cli'], {'CLICK_NAME': 'environment'}, 'cli\n'),
+    (['foo'], {'CLICK_NAME': 'environment'}, 'environment\n'),
+    (['foo'], None, 'defaults\n'),
+]
+
+
+@pytest.mark.parametrize("command,environment,expected",
+                          variable_precedence_testdata)
+def test_variable_precendence_00(runner, command, environment, expected):
+    @click.group()
+    def cli():
+        pass
+
+    @cli.command()
+    @click.option('--name', envvar='CLICK_NAME')
+    def foo(name):
+        click.echo(name)
+
+    defaults = {'foo': {'name': 'defaults'}}
+    result = runner.invoke(cli, command, default_map=defaults, env=environment)
+
+    assert not result.exception
+    assert result.output == expected
diff --git a/tests/test_compat.py b/tests/test_compat.py
index e4ecdc8..9dacc21 100644
--- a/tests/test_compat.py
+++ b/tests/test_compat.py
@@ -1,4 +1,5 @@
 import click
+import pytest
 
 
 if click.__version__ >= '3.0':
@@ -11,10 +12,10 @@ if click.__version__ >= '3.0':
         def cli(foo):
             click.echo(foo)
 
-        result = runner.invoke(cli, ['--foo', 'wat'])
-        assert result.exit_code == 0
-        assert 'WAT' in result.output
-        assert 'Invoked legacy parameter callback' in result.output
+        with pytest.warns(Warning, match='Invoked legacy parameter callback'):
+            result = runner.invoke(cli, ['--foo', 'wat'])
+            assert result.exit_code == 0
+            assert 'WAT' in result.output
 
 
 def test_bash_func_name():
diff --git a/tests/test_formatting.py b/tests/test_formatting.py
index 4c1c491..d2d54db 100644
--- a/tests/test_formatting.py
+++ b/tests/test_formatting.py
@@ -74,11 +74,11 @@ def test_wrapping_long_options_strings(runner):
 
     # 54 is chosen as a length where the second line is one character
     # longer than the maximum length.
-    result = runner.invoke(cli, ['a_very_long', 'command', '--help'],
+    result = runner.invoke(cli, ['a-very-long', 'command', '--help'],
                            terminal_width=54)
     assert not result.exception
     assert result.output.splitlines() == [
-        'Usage: cli a_very_long command [OPTIONS] FIRST SECOND',
+        'Usage: cli a-very-long command [OPTIONS] FIRST SECOND',
         '                               THIRD FOURTH FIFTH',
         '                               SIXTH',
         '',
@@ -111,11 +111,11 @@ def test_wrapping_long_command_name(runner):
         """A command.
         """
 
-    result = runner.invoke(cli, ['a_very_very_very_long', 'command', '--help'],
+    result = runner.invoke(cli, ['a-very-very-very-long', 'command', '--help'],
                            terminal_width=54)
     assert not result.exception
     assert result.output.splitlines() == [
-        'Usage: cli a_very_very_very_long command ',
+        'Usage: cli a-very-very-very-long command ',
         '           [OPTIONS] FIRST SECOND THIRD FOURTH FIFTH',
         '           SIXTH',
         '',
@@ -159,7 +159,43 @@ def test_formatting_usage_error(runner):
         'Usage: cmd [OPTIONS] ARG',
         'Try "cmd --help" for help.',
         '',
-        'Error: Missing argument "arg".'
+        'Error: Missing argument "ARG".'
+    ]
+
+
+def test_formatting_usage_error_metavar_missing_arg(runner):
+    """
+    :author: @r-m-n
+    Including attribution to #612
+    """
+    @click.command()
+    @click.argument('arg', metavar='metavar')
+    def cmd(arg):
+        pass
+
+    result = runner.invoke(cmd, [])
+    assert result.exit_code == 2
+    assert result.output.splitlines() == [
+        'Usage: cmd [OPTIONS] metavar',
+        'Try "cmd --help" for help.',
+        '',
+        'Error: Missing argument "metavar".'
+    ]
+
+
+def test_formatting_usage_error_metavar_bad_arg(runner):
+    @click.command()
+    @click.argument('arg', type=click.INT, metavar='metavar')
+    def cmd(arg):
+        pass
+
+    result = runner.invoke(cmd, ['3.14'])
+    assert result.exit_code == 2
+    assert result.output.splitlines() == [
+        'Usage: cmd [OPTIONS] metavar',
+        'Try "cmd --help" for help.',
+        '',
+        'Error: Invalid value for "metavar": 3.14 is not a valid integer'
     ]
 
 
@@ -179,7 +215,7 @@ def test_formatting_usage_error_nested(runner):
         'Usage: cmd foo [OPTIONS] BAR',
         'Try "cmd foo --help" for help.',
         '',
-        'Error: Missing argument "bar".'
+        'Error: Missing argument "BAR".'
     ]
 
 
@@ -194,7 +230,7 @@ def test_formatting_usage_error_no_help(runner):
     assert result.output.splitlines() == [
         'Usage: cmd [OPTIONS] ARG',
         '',
-        'Error: Missing argument "arg".'
+        'Error: Missing argument "ARG".'
     ]
 
 
@@ -210,5 +246,5 @@ def test_formatting_usage_custom_help(runner):
         'Usage: cmd [OPTIONS] ARG',
         'Try "cmd --man" for help.',
         '',
-        'Error: Missing argument "arg".'
+        'Error: Missing argument "ARG".'
     ]
diff --git a/tests/test_imports.py b/tests/test_imports.py
index bc54533..f400fa8 100644
--- a/tests/test_imports.py
+++ b/tests/test_imports.py
@@ -32,7 +32,7 @@ click.echo(json.dumps(rv))
 ALLOWED_IMPORTS = set([
     'weakref', 'os', 'struct', 'collections', 'sys', 'contextlib',
     'functools', 'stat', 're', 'codecs', 'inspect', 'itertools', 'io',
-    'threading', 'colorama', 'errno'
+    'threading', 'colorama', 'errno', 'fcntl'
 ])
 
 if WIN:
@@ -48,7 +48,6 @@ def test_light_imports():
     if sys.version_info[0] != 2:
         rv = rv.decode('utf-8')
     imported = json.loads(rv)
-    print(imported)
 
     for module in imported:
         if module == 'click' or module.startswith('click.'):
diff --git a/tests/test_options.py b/tests/test_options.py
index 44a1097..2cf62c8 100644
--- a/tests/test_options.py
+++ b/tests/test_options.py
@@ -84,7 +84,7 @@ def test_counting(runner):
     assert result.output == 'verbosity=0\n'
 
     result = runner.invoke(cli, ['--help'])
-    assert re.search('-v\s+Verbosity', result.output) is not None
+    assert re.search(r'-v\s+Verbosity', result.output) is not None
 
 
 @pytest.mark.parametrize('unknown_flag', ['--foo', '-f'])
@@ -199,7 +199,7 @@ def test_nargs_envvar(runner):
 
 
 def test_custom_validation(runner):
-    def validate_pos_int(ctx, value):
+    def validate_pos_int(ctx, param, value):
         if value < 0:
             raise click.BadParameter('Value needs to be positive')
         return value
@@ -311,6 +311,35 @@ def test_option_custom_class(runner):
     assert 'you wont see me' not in result.output
 
 
+def test_option_custom_class_reusable(runner):
+    """Ensure we can reuse a custom class option. See Issue #926"""
+
+    class CustomOption(click.Option):
+        def get_help_record(self, ctx):
+            '''a dumb override of a help text for testing'''
+            return ('--help', 'I am a help text')
+
+    # Assign to a variable to re-use the decorator.
+    testoption = click.option('--testoption', cls=CustomOption, help='you wont see me')
+
+    @click.command()
+    @testoption
+    def cmd1(testoption):
+        click.echo(testoption)
+
+    @click.command()
+    @testoption
+    def cmd2(testoption):
+        click.echo(testoption)
+
+    # Both of the commands should have the --help option now.
+    for cmd in (cmd1, cmd2):
+
+        result = runner.invoke(cmd, ['--help'])
+        assert 'I am a help text' in result.output
+        assert 'you wont see me' not in result.output
+
+
 def test_aliases_for_flags(runner):
     @click.command()
     @click.option('--warnings/--no-warnings', ' /-W', default=True)
diff --git a/tests/test_termui.py b/tests/test_termui.py
index 7806c40..1d5118c 100644
--- a/tests/test_termui.py
+++ b/tests/test_termui.py
@@ -44,6 +44,19 @@ def test_progressbar_length_hint(runner, monkeypatch):
     assert result.exception is None
 
 
+def test_progressbar_hidden(runner, monkeypatch):
+    label = 'whatever'
+
+    @click.command()
+    def cli():
+        with click.progressbar(tuple(range(10)), label=label) as progress:
+            for thing in progress:
+                pass
+
+    monkeypatch.setattr(click._termui_impl, 'isatty', lambda _: False)
+    assert runner.invoke(cli, []).output == ''
+
+
 def test_choices_list_in_prompt(runner, monkeypatch):
     @click.command()
     @click.option('-g', type=click.Choice(['none', 'day', 'week', 'month']),
@@ -65,7 +78,7 @@ def test_choices_list_in_prompt(runner, monkeypatch):
 
 
 def test_secho(runner):
-    with runner.isolation() as out:
+    with runner.isolation() as outstreams:
         click.secho(None, nl=False)
-        bytes = out.getvalue()
+        bytes = outstreams[0].getvalue()
         assert bytes == b''
diff --git a/tests/test_testing.py b/tests/test_testing.py
index c091095..a74c5de 100644
--- a/tests/test_testing.py
+++ b/tests/test_testing.py
@@ -204,6 +204,30 @@ def test_env():
     assert os.environ == env_orig
 
 
+def test_stderr():
+    @click.command()
+    def cli_stderr():
+        click.echo("stdout")
+        click.echo("stderr", err=True)
+
+    runner = CliRunner(mix_stderr=False)
+
+    result = runner.invoke(cli_stderr)
+
+    assert result.output == 'stdout\n'
+    assert result.stdout == 'stdout\n'
+    assert result.stderr == 'stderr\n'
+
+    runner_mix = CliRunner(mix_stderr=True)
+    result_mix = runner_mix.invoke(cli_stderr)
+
+    assert result_mix.output == 'stdout\nstderr\n'
+    assert result_mix.stdout == 'stdout\nstderr\n'
+
+    with pytest.raises(ValueError):
+        result_mix.stderr
+
+
 @pytest.mark.parametrize('args, expected_output', [
     (None, 'bar\n'),
     ([], 'bar\n'),
diff --git a/tests/test_utils.py b/tests/test_utils.py
index 88923ad..4fd7cbb 100644
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -10,13 +10,13 @@ from click._compat import WIN, PY2
 
 
 def test_echo(runner):
-    with runner.isolation() as out:
+    with runner.isolation() as outstreams:
         click.echo(u'\N{SNOWMAN}')
         click.echo(b'\x44\x44')
         click.echo(42, nl=False)
         click.echo(b'a', nl=False)
         click.echo('\x1b[31mx\x1b[39m', nl=False)
-        bytes = out.getvalue().replace(b'\r\n', b'\n')
+        bytes = outstreams[0].getvalue().replace(b'\r\n', b'\n')
         assert bytes == b'\xe2\x98\x83\nDD\n42ax'
 
     # If we are in Python 2, we expect that writing bytes into a string io
@@ -35,12 +35,12 @@ def test_echo(runner):
     def cli():
         click.echo(b'\xf6')
     result = runner.invoke(cli, [])
-    assert result.output_bytes == b'\xf6\n'
+    assert result.stdout_bytes == b'\xf6\n'
 
     # Ensure we do not strip for bytes.
-    with runner.isolation() as out:
+    with runner.isolation() as outstreams:
         click.echo(bytearray(b'\x1b[31mx\x1b[39m'), nl=False)
-        assert out.getvalue() == b'\x1b[31mx\x1b[39m'
+        assert outstreams[0].getvalue() == b'\x1b[31mx\x1b[39m'
 
 
 def test_echo_custom_file():
@@ -146,14 +146,36 @@ def test_prompts_abort(monkeypatch, capsys):
     assert out == 'Password: \nScrew you.\n'
 
 
+def _test_gen_func():
+    yield 'a'
+    yield 'b'
+    yield 'c'
+    yield 'abc'
+
+
 @pytest.mark.skipif(WIN, reason='Different behavior on windows.')
 @pytest.mark.parametrize('cat', ['cat', 'cat ', 'cat '])
-def test_echo_via_pager(monkeypatch, capfd, cat):
+@pytest.mark.parametrize('test', [
+    # We need lambda here, because pytest will
+    # reuse the parameters, and then the generators
+    # are already used and will not yield anymore
+    ('just text\n', lambda: 'just text'),
+    ('iterable\n', lambda: ["itera", "ble"]),
+    ('abcabc\n', lambda: _test_gen_func),
+    ('abcabc\n', lambda: _test_gen_func()),
+    ('012345\n', lambda: (c for c in range(6))),
+])
+def test_echo_via_pager(monkeypatch, capfd, cat, test):
     monkeypatch.setitem(os.environ, 'PAGER', cat)
     monkeypatch.setattr(click._termui_impl, 'isatty', lambda x: True)
-    click.echo_via_pager('haha')
+
+    expected_output = test[0]
+    test_input = test[1]()
+
+    click.echo_via_pager(test_input)
+
     out, err = capfd.readouterr()
-    assert out == 'haha\n'
+    assert out == expected_output
 
 
 @pytest.mark.skipif(WIN, reason='Test does not make sense on Windows.')
@@ -268,9 +290,9 @@ def test_iter_keepopenfile(tmpdir):
     expected = list(map(str, range(10)))
     p = tmpdir.mkdir('testdir').join('testfile')
     p.write(os.linesep.join(expected))
-    f = p.open()
-    for e_line, a_line in zip(expected, click.utils.KeepOpenFile(f)):
-        assert e_line == a_line.strip()
+    with p.open() as f:
+        for e_line, a_line in zip(expected, click.utils.KeepOpenFile(f)):
+            assert e_line == a_line.strip()
 
 
 @pytest.mark.xfail(WIN and not PY2, reason='God knows ...')
@@ -278,6 +300,7 @@ def test_iter_lazyfile(tmpdir):
     expected = list(map(str, range(10)))
     p = tmpdir.mkdir('testdir').join('testfile')
     p.write(os.linesep.join(expected))
-    f = p.open()
-    for e_line, a_line in zip(expected, click.utils.LazyFile(f.name)):
-        assert e_line == a_line.strip()
+    with p.open() as f:
+        with click.utils.LazyFile(f.name) as lf:
+            for e_line, a_line in zip(expected, lf):
+                assert e_line == a_line.strip()