Repository: certbot/certbot

Issue:
certbot-auto 0.26.0 [Errno 39] Directory not empty: '/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory'
Hello,

I've tried to renew one of my certificates using command `certbot-auto renew` and it updated certbot-auto from version 0.25.1 to 0.26.0, once updated one of my domains needs to renew the certificate but I receive the following error:

`Attempting to renew cert (sub.domain.tld) from /etc/letsencrypt/renewal/sub.domain.tld.conf produced an unexpected error: [Errno 39] Directory not empty: '/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory'. Skipping.`

And yes, /etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory' dir is not empty because I've already an account for api v02 on production.

This is the renewal conf file for my domain:

```
# renew_before_expiry = 30 days
version = 0.24.0
archive_dir = /etc/letsencrypt/archive/sub.domain.tld
cert = /etc/letsencrypt/live/sub.domain.tld/cert.pem
privkey = /etc/letsencrypt/live/sub.domain.tld/privkey.pem
chain = /etc/letsencrypt/live/sub.domain.tld/chain.pem
fullchain = /etc/letsencrypt/live/sub.domain.tld/fullchain.pem

# Options used in the renewal process
[renewalparams]
authenticator = webroot
installer = None
account = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
post_hook = service apache2 reload; service stunnel reload; service dovecot reload
renew_hook = echo "\nHe renovado el certificado localizado en ${RENEWED_LINEAGE} y este certificado contiene los siguientes dominios ${RENEWED_DOMAINS}\n\n" >&2
[[webroot_map]]
sub.domain.tld = /var/www/letsencrypt
```

and this is the log:
```

2018-07-12 08:17:09,574:DEBUG:certbot.main:certbot version: 0.26.0
2018-07-12 08:17:09,575:DEBUG:certbot.main:Arguments: []
2018-07-12 08:17:09,575:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2018-07-12 08:17:09,632:DEBUG:certbot.log:Root logging level set at 20
2018-07-12 08:17:09,633:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2018-07-12 08:17:09,677:DEBUG:certbot.plugins.selection:Requested authenticator <certbot.cli._Default object at 0xb61ed9cc> and installer <certbot.cli._Default object at 0xb61ed9cc>
2018-07-12 08:17:09,717:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,717:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:09,753:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,754:DEBUG:certbot.plugins.selection:Requested authenticator manual and installer None
2018-07-12 08:17:09,790:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,791:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:09,825:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,826:DEBUG:certbot.plugins.selection:Requested authenticator manual and installer None
2018-07-12 08:17:09,861:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,862:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:09,913:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,914:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:09,950:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,951:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:09,986:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:09,987:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,023:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,024:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,060:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,061:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,096:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,096:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,132:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,133:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,168:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,168:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,204:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,205:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,240:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,241:DEBUG:certbot.plugins.selection:Requested authenticator manual and installer None
2018-07-12 08:17:10,276:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,277:DEBUG:certbot.plugins.selection:Requested authenticator manual and installer None
2018-07-12 08:17:10,313:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,314:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,349:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,350:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,385:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,386:DEBUG:certbot.plugins.selection:Requested authenticator manual and installer None
2018-07-12 08:17:10,420:DEBUG:certbot.storage:Should renew, less than 40 days before certificate expiry 2018-08-12 15:17:11 UTC.
2018-07-12 08:17:10,420:INFO:certbot.renewal:Cert is due for renewal, auto-renewing...
2018-07-12 08:17:10,421:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,426:DEBUG:certbot.plugins.selection:Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
Initialized: <certbot.plugins.webroot.Authenticator object at 0xb61c5b6c>
Prep: True
2018-07-12 08:17:10,427:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.webroot.Authenticator object at 0xb61c5b6c> and installer None
2018-07-12 08:17:10,427:INFO:certbot.plugins.selection:Plugins selected: Authenticator webroot, Installer None
2018-07-12 08:17:10,517:WARNING:certbot.renewal:Attempting to renew cert (sub.domain.tld) from /etc/letsencrypt/renewal/sub.domain.tld.conf produced an unexpected error: [Errno 39] Directory not empty: '/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory'. Skipping.
2018-07-12 08:17:10,518:DEBUG:certbot.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/renewal.py", line 430, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py", line 1195, in renew_cert
    le_client = _init_le_client(config, auth, installer)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py", line 641, in _init_le_client
    acc, acme = _determine_account(config)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py", line 508, in _determine_account
    acc = account_storage.load(config.account)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/account.py", line 234, in load
    return self._load_for_server_path(account_id, self.config.server_path)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/account.py", line 210, in _load_for_server_path
    self._symlink_to_accounts_dir(prev_server_path, server_path)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/account.py", line 199, in _symlink_to_accounts_dir
    os.rmdir(accounts_dir)
OSError: [Errno 39] Directory not empty: '/etc/letsencrypt/accounts/acme-v02.api.letsencrypt.org/directory'

2018-07-12 08:17:10,554:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,555:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,590:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,591:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,627:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,628:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,663:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,663:DEBUG:certbot.plugins.selection:Requested authenticator manual and installer None
2018-07-12 08:17:10,699:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,700:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,735:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,736:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,771:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,772:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,808:INFO:certbot.renewal:Cert not yet due for renewal
2018-07-12 08:17:10,809:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
2018-07-12 08:17:10,809:ERROR:certbot.renewal:All renewal attempts failed. The following certs could not be renewed:
2018-07-12 08:17:10,810:ERROR:certbot.renewal:  /etc/letsencrypt/live/sub.domain.tld/fullchain.pem (failure)
2018-07-12 08:17:10,810:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/opt/eff.org/certbot/venv/bin/letsencrypt", line 11, in <module>
    sys.exit(main())
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py", line 1364, in main
    return config.func(config, plugins)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py", line 1276, in renew
    renewal.handle_renewal_request(config)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/renewal.py", line 455, in handle_renewal_request
    len(renew_failures), len(parse_failures)))
Error: 1 renew failure(s), 0 parse failure(s)
```

I've tested it on different installations and the error is the same, if I already have an account for v02 api I receive this error so seems a bug introduced on this new version.

Thank you.

Cheers,
sahsanu



Discussion:
Thanks for the detailed bug report @sahsanu. I'm looking into this now and we'll probably put out a release shortly to fix it.
Unfortunately the fix probably won't be out today, but it'll be out soon. If you need to renew your cert now, you can add `server = https://acme-v01.api.letsencrypt.org/directory` under the `[renewalparams]` line in your renewal configuration file.

Resolving patch:
diff --git a/certbot/account.py b/certbot/account.py
--- a/certbot/account.py
+++ b/certbot/account.py
@@ -1,5 +1,6 @@
 """Creates ACME accounts for server."""
 import datetime
+import functools
 import hashlib
 import logging
 import os
@@ -191,6 +192,11 @@ def _find_all_for_server_path(self, server_path):
     def find_all(self):
         return self._find_all_for_server_path(self.config.server_path)
 
+    def _symlink_to_account_dir(self, prev_server_path, server_path, account_id):
+        prev_account_dir = self._account_dir_path_for_server_path(account_id, prev_server_path)
+        new_account_dir = self._account_dir_path_for_server_path(account_id, server_path)
+        os.symlink(prev_account_dir, new_account_dir)
+
     def _symlink_to_accounts_dir(self, prev_server_path, server_path):
         accounts_dir = self.config.accounts_dir_for_server_path(server_path)
         if os.path.islink(accounts_dir):
@@ -207,7 +213,12 @@ def _load_for_server_path(self, account_id, server_path):
                 prev_server_path = constants.LE_REUSE_SERVERS[server_path]
                 prev_loaded_account = self._load_for_server_path(account_id, prev_server_path)
                 # we didn't error so we found something, so create a symlink to that
-                self._symlink_to_accounts_dir(prev_server_path, server_path)
+                accounts_dir = self.config.accounts_dir_for_server_path(server_path)
+                # If accounts_dir isn't empty, make an account specific symlink
+                if os.listdir(accounts_dir):
+                    self._symlink_to_account_dir(prev_server_path, server_path, account_id)
+                else:
+                    self._symlink_to_accounts_dir(prev_server_path, server_path)
                 return prev_loaded_account
             else:
                 raise errors.AccountNotFound(
@@ -250,49 +261,65 @@ def delete(self, account_id):
         :param account_id: id of account which should be deleted
 
         """
-        # Step 1: remove the account itself
         account_dir_path = self._account_dir_path(account_id)
         if not os.path.isdir(account_dir_path):
             raise errors.AccountNotFound(
                 "Account at %s does not exist" % account_dir_path)
-        shutil.rmtree(account_dir_path)
+        # Step 1: Delete account specific links and the directory
+        self._delete_account_dir_for_server_path(account_id, self.config.server_path)
 
-        # Step 2: remove the directory if it's empty, and linked directories
+        # Step 2: Remove any accounts links and directories that are now empty
         if not os.listdir(self.config.accounts_dir):
             self._delete_accounts_dir_for_server_path(self.config.server_path)
 
+    def _delete_account_dir_for_server_path(self, account_id, server_path):
+        link_func = functools.partial(self._account_dir_path_for_server_path, account_id)
+        nonsymlinked_dir = self._delete_links_and_find_target_dir(server_path, link_func)
+        shutil.rmtree(nonsymlinked_dir)
+
     def _delete_accounts_dir_for_server_path(self, server_path):
-        accounts_dir_path = self.config.accounts_dir_for_server_path(server_path)
+        link_func = self.config.accounts_dir_for_server_path
+        nonsymlinked_dir = self._delete_links_and_find_target_dir(server_path, link_func)
+        os.rmdir(nonsymlinked_dir)
+
+    def _delete_links_and_find_target_dir(self, server_path, link_func):
+        """Delete symlinks and return the nonsymlinked directory path.
+
+        :param str server_path: file path based on server
+        :param callable link_func: callable that returns possible links
+            given a server_path
+
+        :returns: the final, non-symlinked target
+        :rtype: str
+
+        """
+        dir_path = link_func(server_path)
 
         # does an appropriate directory link to me? if so, make sure that's gone
         reused_servers = {}
         for k in constants.LE_REUSE_SERVERS:
             reused_servers[constants.LE_REUSE_SERVERS[k]] = k
 
-        # is there a next one up? call that and be done
-        if server_path in reused_servers:
-            next_server_path = reused_servers[server_path]
-            next_accounts_dir_path = self.config.accounts_dir_for_server_path(next_server_path)
-            if os.path.islink(next_accounts_dir_path) \
-                and os.readlink(next_accounts_dir_path) == accounts_dir_path:
-                self._delete_accounts_dir_for_server_path(next_server_path)
-                return
+        # is there a next one up?
+        possible_next_link = True
+        while possible_next_link:
+            possible_next_link = False
+            if server_path in reused_servers:
+                next_server_path = reused_servers[server_path]
+                next_dir_path = link_func(next_server_path)
+                if os.path.islink(next_dir_path) and os.readlink(next_dir_path) == dir_path:
+                    possible_next_link = True
+                    server_path = next_server_path
+                    dir_path = next_dir_path
 
         # if there's not a next one up to delete, then delete me
-        # and whatever I link to if applicable
-        if os.path.islink(accounts_dir_path):
-            # save my info then delete me
-            target = os.readlink(accounts_dir_path)
-            os.unlink(accounts_dir_path)
-            # then delete whatever I linked to, if appropriate
-            if server_path in constants.LE_REUSE_SERVERS:
-                prev_server_path = constants.LE_REUSE_SERVERS[server_path]
-                prev_accounts_dir_path = self.config.accounts_dir_for_server_path(prev_server_path)
-                if target == prev_accounts_dir_path:
-                    self._delete_accounts_dir_for_server_path(prev_server_path)
-        else:
-            # just delete me
-            os.rmdir(accounts_dir_path)
+        # and whatever I link to
+        while os.path.islink(dir_path):
+            target = os.readlink(dir_path)
+            os.unlink(dir_path)
+            dir_path = target
+
+        return dir_path
 
     def _save(self, account, acme, regr_only):
         account_dir_path = self._account_dir_path(account.id)


Test patch:
diff --git a/certbot/tests/account_test.py b/certbot/tests/account_test.py
--- a/certbot/tests/account_test.py
+++ b/certbot/tests/account_test.py
@@ -249,6 +249,14 @@ def test_upgrade_load(self):
         account = self.storage.load(self.acc.id)
         self.assertEqual(prev_account, account)
 
+    def test_upgrade_load_single_account(self):
+        self._set_server('https://acme-staging.api.letsencrypt.org/directory')
+        self.storage.save(self.acc, self.mock_client)
+        prev_account = self.storage.load(self.acc.id)
+        self._set_server_and_stop_symlink('https://acme-staging-v02.api.letsencrypt.org/directory')
+        account = self.storage.load(self.acc.id)
+        self.assertEqual(prev_account, account)
+
     def test_load_ioerror(self):
         self.storage.save(self.acc, self.mock_client)
         mock_open = mock.mock_open()
@@ -307,6 +315,18 @@ def test_delete_folders_down(self):
         self._test_delete_folders('https://acme-staging-v02.api.letsencrypt.org/directory')
         self._assert_symlinked_account_removed()
 
+    def _set_server_and_stop_symlink(self, server_path):
+        self._set_server(server_path)
+        with open(os.path.join(self.config.accounts_dir, 'foo'), 'w') as f:
+            f.write('bar')
+
+    def test_delete_shared_account_up(self):
+        self._set_server_and_stop_symlink('https://acme-staging-v02.api.letsencrypt.org/directory')
+        self._test_delete_folders('https://acme-staging.api.letsencrypt.org/directory')
+
+    def test_delete_shared_account_down(self):
+        self._set_server_and_stop_symlink('https://acme-staging-v02.api.letsencrypt.org/directory')
+        self._test_delete_folders('https://acme-staging.api.letsencrypt.org/directory')
 
 if __name__ == "__main__":
     unittest.main()  # pragma: no cover