Repository: certbot/certbot

Issue:
Renewal conf files from pre-0.1.0 releases contain webroot-path as a string, not a list
I tried to use the new `renew` subcommand, but it first failed with #2468, which I fixed by manually replacing the port, and then with the following error message:

```
2016-02-24 19:23:18,395:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/[redacted].conf produced an unexpected error: The webroot plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('t does not exist or is not a directory',). Skipping.
```

All certificates produced exactly the same error. I looked through the code and it appears it's failing to find the correct webroot path. The path is correct in the config file LE uses to renew the certificate.



Discussion:
Can you provide the config file/log of this problem? The logs are available (by default) in `/var/log/letsencrypt`. Alternatively, if you can recreate the problem you can just run the client with `-vv --debug` and include the output.

These files may contain your domain name/e-mail address which you can redact if you wish.

The errors are the same for every cert, so I'll just post one:

```
Processing /etc/letsencrypt/renewal/[redacted].conf
2016-02-25 00:11:07,786:INFO:letsencrypt.cli:Ancient renewal conf file without webroot-map, restoring webroot-path
2016-02-25 00:11:07,786:INFO:letsencrypt.cli:Auto-renewal forced with --force-renewal...
2016-02-25 00:11:07,788:DEBUG:letsencrypt.cli:Requested authenticator webroot and installer None
2016-02-25 00:11:07,789:DEBUG:letsencrypt.plugins.disco:Other error:(PluginEntryPoint#webroot): t does not exist or is not a directory
Traceback (most recent call last):
  File "/opt/letsencrypt/src/letsencrypt/plugins/disco.py", line 103, in prepare
    self._initialized.prepare()
  File "/opt/letsencrypt/src/letsencrypt/plugins/webroot.py", line 61, in prepare
    raise errors.PluginError(path + " does not exist or is not a directory")
PluginError: t does not exist or is not a directory
2016-02-25 00:11:07,789:DEBUG:letsencrypt.display.ops:No candidate plugin
2016-02-25 00:11:07,789:DEBUG:letsencrypt.cli:Selected authenticator None and installer None
2016-02-25 00:11:07,789:INFO:letsencrypt.cli:Could not choose appropriate plugin: The webroot plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('t does not exist or is not a directory',)
2016-02-25 00:11:07,789:WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/[redacted].conf produced an unexpected error: The webroot plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('t does not exist or is not a directory',). Skipping.
2016-02-25 00:11:07,789:DEBUG:letsencrypt.cli:Traceback was:
Traceback (most recent call last):
  File "/opt/letsencrypt/src/letsencrypt/cli.py", line 1014, in renew
    obtain_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/letsencrypt/src/letsencrypt/cli.py", line 683, in obtain_cert
    installer, authenticator = choose_configurator_plugins(config, plugins, "certonly")
  File "/opt/letsencrypt/src/letsencrypt/cli.py", line 635, in choose_configurator_plugins
    diagnose_configurator_problem("authenticator", req_auth, plugins)
  File "/opt/letsencrypt/src/letsencrypt/cli.py", line 536, in diagnose_configurator_problem
    raise errors.PluginSelectionError(msg)
PluginSelectionError: The webroot plugin is not working; there may be problems with your existing configuration.
The error was: PluginError('t does not exist or is not a directory',)
```

Hm. What about the renewal config in `/etc/letsencrypt/renewal/[redacted].conf`?

Here:

```
cert = /etc/letsencrypt/live/www.[redacted].com/cert.pem
privkey = /etc/letsencrypt/live/www.[redacted].com/privkey.pem
chain = /etc/letsencrypt/live/www.[redacted].com/chain.pem
fullchain = /etc/letsencrypt/live/www.[redacted].com/fullchain.pem

# Options and defaults used in the renewal process
[renewalparams]
apache_enmod = a2enmod
no_verify_ssl = False
ifaces = None
apache_dismod = a2dismod
installer = None
nginx_ctl = nginx
config_dir = /etc/letsencrypt
text_mode = True
func = <function obtaincert at 0x7f5f15a37a28>
prepare = False
work_dir = /var/lib/letsencrypt
tos = True
duplicate = False
http01_port = 80
init = False
nginx_server_root = /etc/nginx
key_path = None
nginx = False
fullchain_path = ./chain.pem
email = lorenz@dolansoft.org
csr = None
agree_dev_preview = True
redirect = None
verbose_count = -3
config_file = /etc/letsencrypt/webroot.ini
renew_by_default = True
authenticator = webroot
domains = www.[redacted].com, [redacted].com
rsa_key_size = 4096
checkpoints = 1
apache = False
cert_path = ./cert.pem
webroot_path = /etc/letsencrypt/challenges-root
strict_permissions = False
apache_server_root = /etc/apache2
account = [redacted]
dvsni_port = 443
chain_path = ./chain.pem
standalone = False
server = https://acme-v01.api.letsencrypt.org/directory
standalone_supported_challenges = "dvsni,http-01"
apache_init_script = /etc/init.d/apache2
apache_ctl = apache2ctl
apache_le_vhost_ext = -le-ssl.conf
debug = False
manual_test_mode = False
logs_dir = /var/log/letsencrypt
configurator = None
```

Btw the renew command tries to renew every certificate, not just the ones with less than 30 days remaining.

Were you a private beta participant? Looking at your config file, `webroot_path` is a string, but since public beta, the value has been a list. If you change the line to this:

```
webroot_path = /etc/letsencrypt/challenges-root,
```

it should fix your problem. I'm leaving this open until we have an actual fix in the client though.

> Btw the renew command tries to renew every certificate, not just the ones with less than 30 days remaining.

Looking at your log, I see `2016-02-25 00:11:07,786:INFO:letsencrypt.cli:Auto-renewal forced with --force-renewal...`. Are you providing `--force-renewal` or `--renew-by-default` on the command line? Alternatively, do you have a [config file](https://letsencrypt.readthedocs.org/en/latest/using.html#configuration-file) with `renew-by-default` set to True?

We should be able to fix this by checking a type [here](https://github.com/letsencrypt/letsencrypt/blob/master/letsencrypt/cli.py#L873) and turning strings into singleton lists?


Resolving patch:
diff --git a/letsencrypt/cli.py b/letsencrypt/cli.py
--- a/letsencrypt/cli.py
+++ b/letsencrypt/cli.py
@@ -872,7 +872,10 @@ def _restore_webroot_config(config, renewalparams):
             setattr(config.namespace, "webroot_map", renewalparams["webroot_map"])
     elif "webroot_path" in renewalparams:
         logger.info("Ancient renewal conf file without webroot-map, restoring webroot-path")
-        setattr(config.namespace, "webroot_path", renewalparams["webroot_path"])
+        wp = renewalparams["webroot_path"]
+        if isinstance(wp, str):  # prior to 0.1.0, webroot_path was a string
+            wp = [wp]
+        setattr(config.namespace, "webroot_path", wp)
 
 
 def _reconstitute(config, full_path):


Test patch:
diff --git a/letsencrypt/tests/cli_test.py b/letsencrypt/tests/cli_test.py
--- a/letsencrypt/tests/cli_test.py
+++ b/letsencrypt/tests/cli_test.py
@@ -20,6 +20,7 @@
 from letsencrypt import crypto_util
 from letsencrypt import errors
 from letsencrypt import le_util
+from letsencrypt import storage
 
 from letsencrypt.plugins import disco
 from letsencrypt.plugins import manual
@@ -630,8 +631,9 @@ def _dump_log(self):
             print "Logs:"
             print lf.read()
 
-    def test_renew_verb(self):
-        with open(test_util.vector_path('sample-renewal.conf')) as src:
+
+    def _make_test_renewal_conf(self, testfile):
+        with open(test_util.vector_path(testfile)) as src:
             # put the correct path for cert.pem, chain.pem etc in the renewal conf
             renewal_conf = src.read().replace("MAGICDIR", test_util.vector_path())
         rd = os.path.join(self.config_dir, "renewal")
@@ -640,9 +642,26 @@ def test_renew_verb(self):
         rc = os.path.join(rd, "sample-renewal.conf")
         with open(rc, "w") as dest:
             dest.write(renewal_conf)
+        return rc
+
+    def test_renew_verb(self):
+        self._make_test_renewal_conf('sample-renewal.conf')
         args = ["renew", "--dry-run", "-tvv"]
         self._test_renewal_common(True, [], args=args, renew=True)
 
+    @mock.patch("letsencrypt.cli._set_by_cli")
+    def test_ancient_webroot_renewal_conf(self, mock_set_by_cli):
+        mock_set_by_cli.return_value = False
+        rc_path = self._make_test_renewal_conf('sample-renewal-ancient.conf')
+        args = mock.MagicMock(account=None, email=None, webroot_path=None)
+        config = configuration.NamespaceConfig(args)
+        lineage = storage.RenewableCert(rc_path,
+            configuration.RenewerConfiguration(config))
+        renewalparams = lineage.configuration["renewalparams"]
+        # pylint: disable=protected-access
+        cli._restore_webroot_config(config, renewalparams)
+        self.assertEqual(config.webroot_path, ["/var/www/"])
+
     def test_renew_verb_empty_config(self):
         rd = os.path.join(self.config_dir, 'renewal')
         if not os.path.exists(rd):
diff --git a/letsencrypt/tests/testdata/sample-renewal-ancient.conf b/letsencrypt/tests/testdata/sample-renewal-ancient.conf
new file mode 100755
--- /dev/null
+++ b/letsencrypt/tests/testdata/sample-renewal-ancient.conf
@@ -0,0 +1,75 @@
+cert = MAGICDIR/live/sample-renewal/cert.pem
+privkey = MAGICDIR/live/sample-renewal/privkey.pem
+chain = MAGICDIR/live/sample-renewal/chain.pem
+fullchain = MAGICDIR/live/sample-renewal/fullchain.pem
+renew_before_expiry = 1 year
+
+# Options and defaults used in the renewal process
+[renewalparams]
+no_self_upgrade = False
+apache_enmod = a2enmod
+no_verify_ssl = False
+ifaces = None
+apache_dismod = a2dismod
+register_unsafely_without_email = False
+apache_handle_modules = True
+uir = None
+installer = none
+nginx_ctl = nginx
+config_dir = MAGICDIR
+text_mode = False
+func = <function obtain_cert at 0x7f093a163c08>
+staging = True
+prepare = False
+work_dir = /var/lib/letsencrypt
+tos = False
+init = False
+http01_port = 80
+duplicate = False
+noninteractive_mode = True
+key_path = None
+nginx = False
+nginx_server_root = /etc/nginx
+fullchain_path = /home/ubuntu/letsencrypt/chain.pem
+email = None
+csr = None
+agree_dev_preview = None
+redirect = None
+verb = certonly
+verbose_count = -3
+config_file = None
+renew_by_default = False
+hsts = False
+apache_handle_sites = True
+authenticator = webroot
+domains = isnot.org,
+rsa_key_size = 2048
+apache_challenge_location = /etc/apache2
+checkpoints = 1
+manual_test_mode = False
+apache = False
+cert_path = /home/ubuntu/letsencrypt/cert.pem
+webroot_path = /var/www/
+reinstall = False
+expand = False
+strict_permissions = False
+apache_server_root = /etc/apache2
+account = None
+dry_run = False
+manual_public_ip_logging_ok = False
+chain_path = /home/ubuntu/letsencrypt/chain.pem
+break_my_certs = False
+standalone = True
+manual = False
+server = https://acme-staging.api.letsencrypt.org/directory
+standalone_supported_challenges = "tls-sni-01,http-01"
+webroot = True
+os_packages_only = False
+apache_init_script = None
+user_agent = None
+apache_le_vhost_ext = -le-ssl.conf
+debug = False
+tls_sni_01_port = 443
+logs_dir = /var/log/letsencrypt
+apache_vhost_root = /etc/apache2/sites-available
+configurator = None