Repository: certbot/certbot

Issue:
Automatically fix ancient Certbot renewal configs
A very verbose description of the problem can be found at https://github.com/certbot/certbot/issues/7978#issuecomment-625442870 under the heading "Why does Certbot have the behavior in the previous post?"

Especially since I think fixing it should be easy, we may as well fix this problem in new versions of Certbot by automatically changing `server` values set to LE's ACMEv1 endpoint in renewal configuration files to ACMEv2 on renewal. We have done this kind of thing before to migrate config options for the `webroot` plugin so I think we can follow a similar pattern.

What I had in mind was updating `certbot._internal.renewal._restore_str` so that if the variable name is `server` and its value is the ACMEv1 URL, return the ACMEv2 URL. This should work, but I haven't tested it and I'm open to other approaches.
Automatically fix ancient Certbot renewal configs
A very verbose description of the problem can be found at https://github.com/certbot/certbot/issues/7978#issuecomment-625442870 under the heading "Why does Certbot have the behavior in the previous post?"

Especially since I think fixing it should be easy, we may as well fix this problem in new versions of Certbot by automatically changing `server` values set to LE's ACMEv1 endpoint in renewal configuration files to ACMEv2 on renewal. We have done this kind of thing before to migrate config options for the `webroot` plugin so I think we can follow a similar pattern.

What I had in mind was updating `certbot._internal.renewal._restore_str` so that if the variable name is `server` and its value is the ACMEv1 URL, return the ACMEv2 URL. This should work, but I haven't tested it and I'm open to other approaches.


Discussion:
A similar but slightly different approach is to always check the `server` value around where it's used (rather than where it's read from the renewal conf), and do the upgrade there like we do for the default.
Asking from the peanut gallery, would there be value in a special "0.31.1" (edit: or "0.27.2") point release, or an easily backportable patch, that could be SRUed by Ubuntu 16.04 or any other particularly affected distros?

Edit: Nothing can be done about systems that don't install updates, but **if** a significant percentage of affected systems are package users who are installing updates, that would help. (But would also take quite a lot of effort.)
> A similar but slightly different approach is to always check the server value around where it's used (rather than where it's read from the renewal conf), and do the upgrade there like we do for the default.

I might be fine with that depending on where exactly you had in mind.

One thing I'm personally a little hesitant to do is to make `certbot --server <ACMEv1 URL>` use the ACMEv2 URL. If you're running Certbot and explicitly requesting ACMEv1, ignoring that and switching to ACMEv2, especially before ACMEv1 is fully shut down next year, seems annoying to me.

We cannot easily get data on this, but I suspect the number of users doing this accidentally is very small so I'd personally rather respect the user's choice than ignoring it and moving it to ACMEv2.

> Asking from the peanut gallery, would there be value in a special "0.31.1" (edit: or "0.27.2") point release, or an easily backportable patch, that could be SRUed by Ubuntu 16.04 or any other particularly affected distros?

The way I expect this to work is once we have fixed the problem in Certbot, we can bring the issue up with our Debian and Ubuntu package maintainers. If they want to fix the problem and will not fully update Certbot, we can help them backport the fix to whatever version they have.
> One thing I'm personally a little hesitant to do is to make certbot --server <ACMEv1 URL> use the ACMEv2 URL. If you're running Certbot and explicitly requesting ACMEv1, ignoring that and switching to ACMEv2, especially before ACMEv1 is fully shut down next year, seems annoying to me.

I actually initially assumed that this was the full ACMEv1 shutdown, which is why I proposed this. Your suggestion makes much more sense in light of that.
I had assumed that as well and implied it in #7978. Thankfully, I was wrong and that's not until next year.
A similar but slightly different approach is to always check the `server` value around where it's used (rather than where it's read from the renewal conf), and do the upgrade there like we do for the default.
Asking from the peanut gallery, would there be value in a special "0.31.1" (edit: or "0.27.2") point release, or an easily backportable patch, that could be SRUed by Ubuntu 16.04 or any other particularly affected distros?

Edit: Nothing can be done about systems that don't install updates, but **if** a significant percentage of affected systems are package users who are installing updates, that would help. (But would also take quite a lot of effort.)
> A similar but slightly different approach is to always check the server value around where it's used (rather than where it's read from the renewal conf), and do the upgrade there like we do for the default.

I might be fine with that depending on where exactly you had in mind.

One thing I'm personally a little hesitant to do is to make `certbot --server <ACMEv1 URL>` use the ACMEv2 URL. If you're running Certbot and explicitly requesting ACMEv1, ignoring that and switching to ACMEv2, especially before ACMEv1 is fully shut down next year, seems annoying to me.

We cannot easily get data on this, but I suspect the number of users doing this accidentally is very small so I'd personally rather respect the user's choice than ignoring it and moving it to ACMEv2.

> Asking from the peanut gallery, would there be value in a special "0.31.1" (edit: or "0.27.2") point release, or an easily backportable patch, that could be SRUed by Ubuntu 16.04 or any other particularly affected distros?

The way I expect this to work is once we have fixed the problem in Certbot, we can bring the issue up with our Debian and Ubuntu package maintainers. If they want to fix the problem and will not fully update Certbot, we can help them backport the fix to whatever version they have.
> One thing I'm personally a little hesitant to do is to make certbot --server <ACMEv1 URL> use the ACMEv2 URL. If you're running Certbot and explicitly requesting ACMEv1, ignoring that and switching to ACMEv2, especially before ACMEv1 is fully shut down next year, seems annoying to me.

I actually initially assumed that this was the full ACMEv1 shutdown, which is why I proposed this. Your suggestion makes much more sense in light of that.
I had assumed that as well and implied it in #7978. Thankfully, I was wrong and that's not until next year.

Resolving patch:
diff --git a/certbot/certbot/_internal/constants.py b/certbot/certbot/_internal/constants.py
--- a/certbot/certbot/_internal/constants.py
+++ b/certbot/certbot/_internal/constants.py
@@ -120,6 +120,8 @@
 )
 STAGING_URI = "https://acme-staging-v02.api.letsencrypt.org/directory"
 
+V1_URI = "https://acme-v01.api.letsencrypt.org/directory"
+
 # The set of reasons for revoking a certificate is defined in RFC 5280 in
 # section 5.3.1. The reasons that users are allowed to submit are restricted to
 # those accepted by the ACME server implementation. They are listed in
diff --git a/certbot/certbot/_internal/renewal.py b/certbot/certbot/_internal/renewal.py
--- a/certbot/certbot/_internal/renewal.py
+++ b/certbot/certbot/_internal/renewal.py
@@ -19,6 +19,7 @@
 from certbot import interfaces
 from certbot import util
 from certbot._internal import cli
+from certbot._internal import constants
 from certbot._internal import hooks
 from certbot._internal import storage
 from certbot._internal import updater
@@ -243,16 +244,28 @@ def _restore_int(name, value):
         raise errors.Error("Expected a numeric value for {0}".format(name))
 
 
-def _restore_str(unused_name, value):
+def _restore_str(name, value):
     """Restores a string key-value pair from a renewal config file.
 
-    :param str unused_name: option name
+    :param str name: option name
     :param str value: option value
 
     :returns: converted option value to be stored in the runtime config
     :rtype: str or None
 
     """
+    # Previous to v0.5.0, Certbot always stored the `server` URL in the renewal config,
+    # resulting in configs which explicitly use the deprecated ACMEv1 URL, today
+    # preventing an automatic transition to the default modern ACME URL.
+    # (https://github.com/certbot/certbot/issues/7978#issuecomment-625442870)
+    # As a mitigation, this function reinterprets the value of the `server` parameter if
+    # necessary, replacing the ACMEv1 URL with the default ACME URL. It is still possible
+    # to override this choice with the explicit `--server` CLI flag.
+    if name == "server" and value == constants.V1_URI:
+        logger.info("Using server %s instead of legacy %s",
+                    constants.CLI_DEFAULTS["server"], value)
+        return constants.CLI_DEFAULTS["server"]
+
     return None if value == "None" else value
 
 


Test patch:
diff --git a/certbot/tests/renewal_test.py b/certbot/tests/renewal_test.py
--- a/certbot/tests/renewal_test.py
+++ b/certbot/tests/renewal_test.py
@@ -110,6 +110,14 @@ def test_must_staple_failure(self, mock_set_by_cli):
         self.assertRaises(
             errors.Error, self._call, self.config, renewalparams)
 
+    @mock.patch('certbot._internal.renewal.cli.set_by_cli')
+    def test_ancient_server_renewal_conf(self, mock_set_by_cli):
+        from certbot._internal import constants
+        self.config.server = None
+        mock_set_by_cli.return_value = False
+        self._call(self.config, {'server': constants.V1_URI})
+        self.assertEqual(self.config.server, constants.CLI_DEFAULTS['server'])
+
 
 if __name__ == "__main__":
     unittest.main()  # pragma: no cover