Repository: certbot/certbot

Issue:
Certbot help showing default arguments improperly
When issuing:
certbot-auto --help certonly

I get output like:
[...]
  --manual              Provide laborious manual instructions for obtaining a
                        cert (default: <certbot.cli._Default object at
                        0x255e410>)
  --webroot             Obtain certs by placing files in a webroot directory.
                        (default: <certbot.cli._Default object at 0x255e490>)

Version numbers:
certbot-auto --version
letsencrypt 0.9.3
python --version
Python 2.7.3

This works properly with the same certbot, but with python 2.7.12.


Discussion:
this also applies to 0.10.1:

/etc/letsencrypt/certbot-auto --version
certbot 0.10.1

/etc/letsencrypt/certbot-auto --help renew
[...]
  -q, --quiet           Silence all output except errors. Useful for
                        automation via cron. Implies --non-interactive.
                        (default: <certbot.cli._Default object at 0x4523b50>)

Thanks for the report @czirkoszoltan 
Can you be a bit more specific about what part of that output is improper?
Also what system are you running these commands on?
The parts like "<certbot.cli._Default object at 0x4523b50>" are incorrect.
The system is Welcome to Ubuntu 12.04.5 LTS, my current certbot version is certbot 0.12.0, with outputs like the above. Python version is Python 2.7.3 (default, Oct 26 2016, 21:01:49).

@bmw any idea why we're spewing these things?
This was introduced in 24cc03d1f6e3d3fb5a89aa489b0bd2f71fc80684. Playing with it, whether or not you get this error seems to depend on your Python/argparse version. It doesn't happen with newer versions of Python.

For whoever works on this, the difference in `argparse`'s behavior is whether or not the `type` parameter is called on a `default` string when `--help` is provided.

EDIT: I'm putting this in 0.13.0 until I track down exactly which Python versions are affected.
The change `argparse`'s behavior I described above happened in 2.7.4 which came out in April of 2013. Versions of Python 2.7 older than this have this problem. Python 2.6 users are unaffected because since `argparse` isn't in the standard library, we use the `argparse` from PyPI which is a newer version of the module.

It appears that Debian 7.0 Wheezy is the only distro that both has a version of Python with this problem and is still supported by its maintainers. We have a lot of users on this platform though so I'll figure out a way to work around this problem.

Resolving patch:
diff --git a/certbot/plugins/standalone.py b/certbot/plugins/standalone.py
--- a/certbot/plugins/standalone.py
+++ b/certbot/plugins/standalone.py
@@ -3,7 +3,6 @@
 import collections
 import logging
 import socket
-import sys
 import threading
 
 import OpenSSL
@@ -13,7 +12,6 @@
 from acme import challenges
 from acme import standalone as acme_standalone
 
-from certbot import cli
 from certbot import errors
 from certbot import interfaces
 
@@ -114,39 +112,57 @@ def running(self):
 SUPPORTED_CHALLENGES = [challenges.TLSSNI01, challenges.HTTP01]
 
 
-def supported_challenges_validator(data):
-    """Supported challenges validator for the `argparse`.
+class SupportedChallengesAction(argparse.Action):
+    """Action class for parsing standalone_supported_challenges."""
 
-    It should be passed as `type` argument to `add_argument`.
+    def __call__(self, parser, namespace, values, option_string=None):
+        logger.warning(
+            "The standalone specific supported challenges flag is "
+            "deprecated. Please use the --preferred-challenges flag "
+            "instead.")
+        converted_values = self._convert_and_validate(values)
+        namespace.standalone_supported_challenges = converted_values
 
-    """
-    if cli.set_by_cli("standalone_supported_challenges"):
-        sys.stderr.write(
-            "WARNING: The standalone specific "
-            "supported challenges flag is deprecated.\n"
-            "Please use the --preferred-challenges flag instead.\n")
-    challs = data.split(",")
-
-    # tls-sni-01 was dvsni during private beta
-    if "dvsni" in challs:
-        logger.info("Updating legacy standalone_supported_challenges value")
-        challs = [challenges.TLSSNI01.typ if chall == "dvsni" else chall
-                  for chall in challs]
-        data = ",".join(challs)
-
-    unrecognized = [name for name in challs
-                    if name not in challenges.Challenge.TYPES]
-    if unrecognized:
-        raise argparse.ArgumentTypeError(
-            "Unrecognized challenges: {0}".format(", ".join(unrecognized)))
-
-    choices = set(chall.typ for chall in SUPPORTED_CHALLENGES)
-    if not set(challs).issubset(choices):
-        raise argparse.ArgumentTypeError(
-            "Plugin does not support the following (valid) "
-            "challenges: {0}".format(", ".join(set(challs) - choices)))
-
-    return data
+    def _convert_and_validate(self, data):
+        """Validate the value of supported challenges provided by the user.
+
+        References to "dvsni" are automatically converted to "tls-sni-01".
+
+        :param str data: comma delimited list of challenge types
+
+        :returns: validated and converted list of challenge types
+        :rtype: str
+
+        """
+        challs = data.split(",")
+
+        # tls-sni-01 was dvsni during private beta
+        if "dvsni" in challs:
+            logger.info(
+                "Updating legacy standalone_supported_challenges value")
+            challs = [challenges.TLSSNI01.typ if chall == "dvsni" else chall
+                      for chall in challs]
+            data = ",".join(challs)
+
+        unrecognized = [name for name in challs
+                        if name not in challenges.Challenge.TYPES]
+
+        # argparse.ArgumentErrors raised out of argparse.Action objects
+        # are caught by argparse which prints usage information and the
+        # error that occurred before calling sys.exit.
+        if unrecognized:
+            raise argparse.ArgumentError(
+                self,
+                "Unrecognized challenges: {0}".format(", ".join(unrecognized)))
+
+        choices = set(chall.typ for chall in SUPPORTED_CHALLENGES)
+        if not set(challs).issubset(choices):
+            raise argparse.ArgumentError(
+                self,
+                "Plugin does not support the following (valid) "
+                "challenges: {0}".format(", ".join(set(challs) - choices)))
+
+        return data
 
 
 @zope.interface.implementer(interfaces.IAuthenticator)
@@ -184,7 +200,7 @@ def __init__(self, *args, **kwargs):
     def add_parser_arguments(cls, add):
         add("supported-challenges",
             help=argparse.SUPPRESS,
-            type=supported_challenges_validator,
+            action=SupportedChallengesAction,
             default=",".join(chall.typ for chall in SUPPORTED_CHALLENGES))
 
     @property


Test patch:
diff --git a/certbot/plugins/standalone_test.py b/certbot/plugins/standalone_test.py
--- a/certbot/plugins/standalone_test.py
+++ b/certbot/plugins/standalone_test.py
@@ -62,28 +62,26 @@ def test_run_bind_error(self):
         self.assertEqual(self.mgr.running(), {})
 
 
-class SupportedChallengesValidatorTest(unittest.TestCase):
-    """Tests for plugins.standalone.supported_challenges_validator."""
+class SupportedChallengesActionTest(unittest.TestCase):
+    """Tests for plugins.standalone.SupportedChallengesAction."""
 
-    def setUp(self):
-        self.set_by_cli_patch = mock.patch(
-            "certbot.plugins.standalone.cli.set_by_cli")
-        self.stderr_patch = mock.patch("certbot.plugins.standalone.sys.stderr")
+    def _call(self, value):
+        with mock.patch("certbot.plugins.standalone.logger") as mock_logger:
+            # stderr is mocked to prevent potential argparse error
+            # output from cluttering test output
+            with mock.patch("sys.stderr"):
+                config = self.parser.parse_args([self.flag, value])
 
-        self.set_by_cli_patch.start().return_value = True
-        self.stderr = self.stderr_patch.start()
+        self.assertTrue(mock_logger.warning.called)
+        return getattr(config, self.dest)
 
-    def tearDown(self):
-        self.set_by_cli_patch.stop()
-        self.stderr_patch.stop()
+    def setUp(self):
+        self.flag = "--standalone-supported-challenges"
+        self.dest = self.flag[2:].replace("-", "_")
+        self.parser = argparse.ArgumentParser()
 
-    def _call(self, data):
-        from certbot.plugins.standalone import (
-            supported_challenges_validator)
-        return_value = supported_challenges_validator(data)
-        self.assertTrue(self.stderr.write.called)  # pylint: disable=no-member
-        self.stderr.write.reset_mock()  # pylint: disable=no-member
-        return return_value
+        from certbot.plugins.standalone import SupportedChallengesAction
+        self.parser.add_argument(self.flag, action=SupportedChallengesAction)
 
     def test_correct(self):
         self.assertEqual("tls-sni-01", self._call("tls-sni-01"))
@@ -93,10 +91,10 @@ def test_correct(self):
 
     def test_unrecognized(self):
         assert "foo" not in challenges.Challenge.TYPES
-        self.assertRaises(argparse.ArgumentTypeError, self._call, "foo")
+        self.assertRaises(SystemExit, self._call, "foo")
 
     def test_not_subset(self):
-        self.assertRaises(argparse.ArgumentTypeError, self._call, "dns")
+        self.assertRaises(SystemExit, self._call, "dns")
 
     def test_dvsni(self):
         self.assertEqual("tls-sni-01", self._call("dvsni"))