Repository: certbot/certbot

Issue:
New Let's Encrypt staging hierarchy isn't recognized as fake
## My operating system is (include version):
Ubuntu 20.04 (amd64)

## I installed Certbot with (certbot-auto, OS package manager, pip, etc):
snap

## I ran this command and it produced this output:
```
$ sudo certbot certonly --standalone --staging -d example.mattnordhoff.net
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator standalone, Installer None
Requesting a certificate for example.mattnordhoff.net
Performing the following challenges:
http-01 challenge for example.mattnordhoff.net
Waiting for verification...
Cleaning up challenges

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/example.mattnordhoff.net/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/example.mattnordhoff.net/privkey.pem
   Your certificate will expire on 2021-05-23. To obtain a new or
   tweaked version of this certificate in the future, simply run
   certbot again. To non-interactively renew *all* of your
   certificates, run "certbot renew"
$ sudo certbot renew --cert-name example.mattnordhoff.net --force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/example.mattnordhoff.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate for example.mattnordhoff.net
Failed to renew certificate example.mattnordhoff.net with error: You've asked to renew/replace a seemingly valid certificate with a test certificate (domains: example.mattnordhoff.net). We will not do that unless you use the --break-my-certs flag!

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
All renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/example.mattnordhoff.net/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)
$ sudo certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
  Certificate Name: example.mattnordhoff.net
    Serial Number: fa0e796baf11f5365ad72886817d4a47a177
    Key Type: RSA
    Domains: example.mattnordhoff.net
    Expiry Date: 2021-05-23 15:07:06+00:00 (INVALID: TEST_CERT)
    Certificate Path: /etc/letsencrypt/live/example.mattnordhoff.net/fullchain.pem
    Private Key Path: /etc/letsencrypt/live/example.mattnordhoff.net/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
$ sudo certbot renew --break-my-certs --cert-name example.mattnordhoff.net --force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/example.mattnordhoff.net.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Plugins selected: Authenticator standalone, Installer None
Renewing an existing certificate for example.mattnordhoff.net

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
new certificate deployed without reload, fullchain is
/etc/letsencrypt/live/example.mattnordhoff.net/fullchain.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations, all renewals succeeded:
  /etc/letsencrypt/live/example.mattnordhoff.net/fullchain.pem (success)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
```

## Certbot's behavior differed from what I expected because:
Certbot required `--break-my-certs` to renew a Let's Encrypt staging certificate.

## Here is a Certbot log showing the issue (if available):
###### Logs are stored in `/var/log/letsencrypt` by default. Feel free to redact domains, e-mail and IP addresses as you see fit.
1. `certbot certonly`...: https://mn0.us/pin5jVrpgjgz/letsencrypt.log.3.txt
2. `certbot renew --cert-name example.mattnordhoff.net`...: https://mn0.us/nwb0ynUZfyAt/letsencrypt.log.2.txt
4. `certbot renew --break-my-certs`...: https://mn0.us/04jX8Fya3cuP/letsencrypt.log.txt

## Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:
N/A

---

Let's Encrypt's new staging certificate hierarchy, documented on the [staging environment page](https://letsencrypt.org/docs/staging-environment/), uses "(STAGING)" in the root and intermediate certificate names, but no longer uses the word "Fake".

Certbot has two methods of recognizing staging certificates -- one looks for "staging" in the URL, and the other looks for "fake" in the lowercased issuer name. The latter no longer matches.

https://github.com/certbot/certbot/blob/v1.12.0/certbot/certbot/_internal/renewal.py#L318


Discussion:
I wonder why the issuer name is being checked at all (originally #1860):

https://github.com/certbot/certbot/blob/0f3f07b5cb3bf1d2c3d8834a83022fd11d7566fc/certbot/certbot/_internal/renewal.py#L309-L310

If you do what's described in the above comment, the lineage's `server` will be updated as well. i.e. It should not ordinarily be possible for the lineage's latest certificate and the lineage's `server` to be out of sync?

If that's the case, can `now_valid` just be removed?
That does seem to be true now, based on the testing I just double-checked with. Maybe it wasn't true before? Hm, or maybe there was some idea about switching from one testing server to another testing server....I'm not sure. But either was, I really can't see the value of checking both the renewal conf file and the cert itself. I'm down to just remove that check.

Resolving patch:
diff --git a/certbot/certbot/_internal/renewal.py b/certbot/certbot/_internal/renewal.py
--- a/certbot/certbot/_internal/renewal.py
+++ b/certbot/certbot/_internal/renewal.py
@@ -312,12 +312,9 @@ def _avoid_invalidating_lineage(config, lineage, original_server):
         contents = the_file.read()
     latest_cert = OpenSSL.crypto.load_certificate(
         OpenSSL.crypto.FILETYPE_PEM, contents)
-    # all our test certificates are from happy hacker fake CA, though maybe one day
-    # we should test more methodically
-    now_valid = "fake" not in repr(latest_cert.get_issuer()).lower()
 
     if util.is_staging(config.server):
-        if not util.is_staging(original_server) or now_valid:
+        if not util.is_staging(original_server):
             if not config.break_my_certs:
                 names = ", ".join(lineage.names())
                 raise errors.Error(


Test patch:
diff --git a/certbot/tests/main_test.py b/certbot/tests/main_test.py
--- a/certbot/tests/main_test.py
+++ b/certbot/tests/main_test.py
@@ -1053,7 +1053,7 @@ def write_msg(message, *args, **kwargs):  # pylint: disable=unused-argument
                             mock_get_utility().notification.side_effect = write_msg
                         with mock.patch('certbot._internal.main.renewal.OpenSSL') as mock_ssl:
                             mock_latest = mock.MagicMock()
-                            mock_latest.get_issuer.return_value = "Fake fake"
+                            mock_latest.get_issuer.return_value = "Artificial pretend"
                             mock_ssl.crypto.load_certificate.return_value = mock_latest
                             with mock.patch('certbot._internal.main.renewal.crypto_util') \
                                 as mock_crypto_util: