Repository: certbot/certbot

Issue:
client.py broken for python 3.4
## My operating system is (include version):
Gentoo 4.8.17-hardened-r2

## I installed Certbot with (certbot-auto, OS package manager, pip, etc):
https://certbot.eff.org/#gentoo-nginx

## I ran this command and it produced this output:
`certbot renew --dry-run`

```
Exception ignored in: <bound method ClientNetwork.__del__ of <acme.client.ClientNetwork object at 0x7faec2bf1f60>>
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/acme/client.py", line 522, in __del__
    self.session.close()
  File "/usr/lib64/python3.4/site-packages/requests/sessions.py", line 689, in close
    for v in self.adapters.values():
  File "/usr/lib64/python3.4/_collections_abc.py", line 512, in __iter__
    for key in self._mapping:
  File "/usr/lib64/python3.4/collections/__init__.py", line 91, in __iter__
    curr = root.next

ReferenceError: weakly-referenced object no longer exists
```

## Certbot's behavior differed from what I expected because:
There was an exception

## Here is a Certbot log showing the issue (if available):
###### Logs are stored in `/var/log/letsencrypt` by default. Feel free to redact domains, e-mail and IP addresses as you see fit.
The log `/var/log/letsencrypt/letsencrypt.log` does not show any errors.
## Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:
```
user nginx nginx;
worker_processes 1;

error_log /var/log/nginx/error_log info;

events {
        worker_connections 1024;
        use epoll;
}

http {
        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        log_format main
                '$remote_addr - $remote_user [$time_local] '
                '"$request" $status $bytes_sent '
                '"$http_referer" "$http_user_agent" '
                '"$gzip_ratio"';

        client_header_timeout 10m;
        client_body_timeout 10m;
        send_timeout 10m;

        connection_pool_size 256;
        client_header_buffer_size 1k;
        large_client_header_buffers 4 2k;
        request_pool_size 4k;

        gzip off;

        output_buffers 1 32k;
        postpone_output 1460;

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;

        keepalive_timeout 75 20;

        ignore_invalid_headers on;

        index index.html;

        server {
                listen 0.0.0.0;
                server_name <redacted>;

                access_log /var/log/nginx/<redacted>.access_log main;
                error_log /var/log/nginx/<redacted>.error_log info;

                root /var/www/www.<redacted>.com;

                if ($scheme != "https") {
                    return 301 https://$host$request_uri;
                } # managed by Certbot


                # Redirect non-https traffic to https
                # if ($scheme != "https") {
                #     return 301 https://$host$request_uri;
                # } # managed by Certbot

        }

        # SSL example
        server {
                listen 0.0.0.0:443;
                server_name <redacted>;

                ssl on;
                ssl_certificate /etc/letsencrypt/live/<redacted>.com/fullchain.pem; # managed by Certbot
                ssl_certificate_key /etc/letsencrypt/live/<redacted>.com/privkey.pem; # managed by Certbot

                access_log /var/log/nginx/<redacted>.ssl_access_log main;
                error_log /var/log/nginx/<redacted>.ssl_error_log info;

                root /var/www/www.<redacted>.com;

        }
}
```


Discussion:
Hmm I wonder if this is specifically a python 3 issue, maybe @yan12125 can tell if this is a general bug or not?
That looks a Python 3.4+-specific issue. Related issue: https://github.com/requests/requests/issues/2303. People there proposed a workaround that completely skips cleanup for 3.4+, but I'm not sure whether it will cause problems.
```Diff
diff --git a/acme/acme/client.py b/acme/acme/client.py
index 9455159d..15d6a96d 100644
--- a/acme/acme/client.py
+++ b/acme/acme/client.py
@@ -519,7 +519,8 @@ class ClientNetwork(object):  # pylint: disable=too-many-instance-attributes
         self._default_timeout = timeout
 
     def __del__(self):
-        self.session.close()
+        if sys.version_info < (3, 4, 0):
+            self.session.close()
 
     def _wrap_in_jws(self, obj, nonce):
         """Wrap `JSONDeSerializable` object in JWS.
```
I made the change locally, and everything seems to complete as expected.
@seanrich glad to hear it! I'm going to close this issue and try to see if I can get requests to fix it upstream
Oh wait, this is in our code - we should probably fix that
@seanrich, we'll definitely fix this regardless, but does this happen for you on every run of `certbot renew --dry-run` or was it a one time message?
@bmw every time.
Strange. To test this, I've set up 2 Python 3.4 systems in Docker, issued a couple certificates, and ran `certbot renew --dry-run` 100+ times in each system. This includes a hardened Gentoo image. Despite all of these runs, I could never reproduce the error (and the hardened Gentoo image is still running).

With that said, it's very clear from looking at our code and upstream bugs that we have a problem here. I'm glad making the change suggested above locally fixed the problem. We'll include a fix in our next release.

For whoever takes this issue, I'm not sure the best solution here. In my opinion, using `__del__` is extremely error prone and I'm not sure if I've ever personally seen it used properly in non-trivial examples. Right now, I honestly think the best solution is to add a `close` method to `acme.client.ClientNetwork` (and maybe `acme.client.Client` and `certbot.client.Client`) and just call it explicitly when we're finished with the object.

Resolving patch:
diff --git a/acme/acme/client.py b/acme/acme/client.py
--- a/acme/acme/client.py
+++ b/acme/acme/client.py
@@ -519,7 +519,12 @@ def __init__(self, key, alg=jose.RS256, verify_ssl=True,
         self._default_timeout = timeout
 
     def __del__(self):
-        self.session.close()
+        # Try to close the session, but don't show exceptions to the
+        # user if the call to close() fails. See #4840.
+        try:
+            self.session.close()
+        except Exception:  # pylint: disable=broad-except
+            pass
 
     def _wrap_in_jws(self, obj, nonce):
         """Wrap `JSONDeSerializable` object in JWS.


Test patch:
diff --git a/acme/acme/client_test.py b/acme/acme/client_test.py
--- a/acme/acme/client_test.py
+++ b/acme/acme/client_test.py
@@ -600,12 +600,19 @@ def test_send_request_timeout(self):
             mock.ANY, mock.ANY, verify=mock.ANY, headers=mock.ANY,
             timeout=45)
 
-    def test_del(self):
+    def test_del(self, close_exception=None):
         sess = mock.MagicMock()
+
+        if close_exception is not None:
+            sess.close.side_effect = close_exception
+
         self.net.session = sess
         del self.net
         sess.close.assert_called_once_with()
 
+    def test_del_error(self):
+        self.test_del(ReferenceError)
+
     @mock.patch('acme.client.requests')
     def test_requests_error_passthrough(self, mock_requests):
         mock_requests.exceptions = requests.exceptions