Repository: matrix-org/synapse

Issue:
acme config (still) doesn't work on py2
### Description

When I want to enable the acme-feature in `v0.99.1.1` I still get the error which is described in [#4559](https://github.com/matrix-org/synapse/issues/4559) and should be fixed with [#4564](https://github.com/matrix-org/synapse/pull/4564)
```
twisted - 239 - ERROR -  - exceptions.TypeError: expected unicode for text, got 'https://acme-v01.api.letsencrypt.org/directory'
```

### Steps to reproduce

- Trying to start the container with `docker-compose up -d` and having acme enabled in my `homeserver.yaml` (on port **8009** instead of ~80~)


<!-- 
Describe how what happens differs from what you expected.

If you can identify any relevant log snippets from _homeserver.log_, please include
those (please be careful to remove any personal or private data). Please surround them with
``` (three backticks, on a line on their own), so that they are formatted legibly.
-->

### Version information

<!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->

<!-- Was this issue identified on matrix.org or another homeserver? -->
- **Homeserver**: matrix.[myCustomUrl].org

<!-- 
What version of Synapse is running? 
You can find the Synapse version by inspecting the server headers (replace matrix.org with
your own homeserver domain):
$ curl -v https://matrix.org/_matrix/client/versions 2>&1 | grep "Server:"
-->
- **Version**: v0.99.1.1

- **Install method**:  docker-compose using image [avhost/docker-matrix:v0.99.1.1](https://hub.docker.com/r/avhost/docker-matrix/tags)
<!-- examples: package manager/git clone/pip  -->

- **Platform**: Debian GNU/Linux 9 (stretch)
<!--
Tell us about the environment in which your homeserver is operating
distro, hardware, if it's running in a vm/container, etc.
-->

- **Reverse Proxy**: traefik:1.6.5 
- **Synapse Database**: docker.io/postgres:10-alpine


### Acme part in my homeserver.yaml
```yaml
acme:
    # ACME support is disabled by default. Uncomment the following line
    # (and tls_certificate_path and tls_private_key_path above) to enable it.
    #
    enabled: true

    # Endpoint to use to request certificates. If you only want to test,
    # use Let's Encrypt's staging url:
    #     https://acme-staging.api.letsencrypt.org/directory
    #
    url: "https://acme-v01.api.letsencrypt.org/directory"

    # Port number to listen on for the HTTP-01 challenge. Change this if
    # you are forwarding connections through Apache/Nginx/etc.
    #
    port: 8009

    # Local addresses to listen on for incoming connections.
    # Again, you may want to change this if you are forwarding connections
    # through Apache/Nginx/etc.
    #
    bind_addresses: ['::', '0.0.0.0']

    # How many days remaining on a certificate before it is renewed.
    #
    reprovision_threshold: 30
```

### docker-compose.yml
```yaml
version: '3'
services:
  matrix:
    image: avhost/docker-matrix:${TAG}
    container_name: matrix
    restart: always
    ports: 
      - '8448:8448'
      - '8008:8008'
      - '8009:8009'
      - '3478:3478'
      - '3478:3478/udp'
      - '5349:5349/udp'
      - '5349:5349'
      - '49152-49300:49152-49300/udp'
      - '49152-49300:49152-49300'

    volumes: 
      - /opt/synapse:/data
    environment:
      - VIRTUAL_HOST=${MATRIX_DOMAIN}
      - VIRTUAL_NETWORK=nginx-proxy
      - VIRTUAL_PORT=443
      - LETSENCRYPT_HOST=${MATRIX_DOMAIN}
      - LETSENCRYPT_EMAIL=${LETSENCRYPT_EMAIL}
    depends_on:
      - db
    networks:
      - aa_proxy
      - default
    labels:
      - traefik.enable=true
      - traefik.docker.network=aa_proxy
      - traefik.backend=matrix
      - traefik.frontend.rule=${MATRIX_DOMAIN_HOST}
      - traefik.port=8008
      - traefik.protocol=http

  db:
    image: docker.io/postgres:10-alpine
    restart: unless-stopped
    container_name: synapse-db
    environment:
      - POSTGRES_USER=synapse
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - synapse-schemas:/var/lib/postgresql/data

volumes: 
  synapse-schemas:

networks:
  aa_proxy:
    external: true
```

### Error log
```
2019-02-17 12:06:34,060 - twisted - 239 - INFO -  - Site starting on 8009
2019-02-17 12:06:34,061 - synapse.handlers.acme - 105 - INFO -  - Listening for ACME requests on 0.0.0.0:8009
2019-02-17 12:06:34,061 - synapse.app - 47 - WARNING -  - Failed to listen on 0.0.0.0, continuing because listening on [::]
2019-02-17 12:06:34,062 - twisted - 239 - ERROR -  - Error during startup:
2019-02-17 12:06:34,063 - twisted - 239 - ERROR -  - Traceback (most recent call last):
2019-02-17 12:06:34,063 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
2019-02-17 12:06:34,065 - twisted - 239 - ERROR -  -     result = g.send(result)
2019-02-17 12:06:34,066 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/synapse/app/homeserver.py", line 429, in start
2019-02-17 12:06:34,067 - twisted - 239 - ERROR -  -     yield acme.start_listening()
2019-02-17 12:06:34,068 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
2019-02-17 12:06:34,068 - twisted - 239 - ERROR -  -     return _cancellableInlineCallbacks(gen)
2019-02-17 12:06:34,069 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
2019-02-17 12:06:34,069 - twisted - 239 - ERROR -  -     _inlineCallbacks(None, g, status)
2019-02-17 12:06:34,070 - twisted - 239 - ERROR -  - --- <exception caught here> ---
2019-02-17 12:06:34,070 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/synapse/app/homeserver.py", line 429, in start
2019-02-17 12:06:34,071 - twisted - 239 - ERROR -  -     yield acme.start_listening()
2019-02-17 12:06:34,071 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
2019-02-17 12:06:34,072 - twisted - 239 - ERROR -  -     result = g.send(result)
2019-02-17 12:06:34,072 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/synapse/handlers/acme.py", line 121, in start_listening
2019-02-17 12:06:34,073 - twisted - 239 - ERROR -  -     yield self._issuer._ensure_registered()
2019-02-17 12:06:34,075 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/txacme/service.py", line 243, in _ensure_registered
2019-02-17 12:06:34,075 - twisted - 239 - ERROR -  -     return self._with_client(self._register)
2019-02-17 12:06:34,076 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/txacme/service.py", line 184, in _with_client
2019-02-17 12:06:34,077 - twisted - 239 - ERROR -  -     return self._client_creator().addCallback(f, *a, **kw)
2019-02-17 12:06:34,077 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/synapse/handlers/acme.py", line 83, in <lambda>
2019-02-17 12:06:34,078 - twisted - 239 - ERROR -  -     url=URL.from_text(self.hs.config.acme_url),
2019-02-17 12:06:34,078 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/hyperlink/_url.py", line 1098, in from_text
2019-02-17 12:06:34,080 - twisted - 239 - ERROR -  -     um = _URL_RE.match(_textcheck('text', text))
2019-02-17 12:06:34,080 - twisted - 239 - ERROR -  -   File "/usr/local/lib/python2.7/dist-packages/hyperlink/_url.py", line 457, in _textcheck
2019-02-17 12:06:34,081 - twisted - 239 - ERROR -  -     raise TypeError('expected %s for %s, got %r' % (exp, name, value))
2019-02-17 12:06:34,081 - twisted - 239 - ERROR -  - exceptions.TypeError: expected unicode for text, got 'https://acme-v01.api.letsencrypt.org/directory'
2019-02-17 12:06:34,086 - synapse.metrics - 374 - INFO -  - Collecting gc 0
2019-02-17 12:06:34,126 - twisted - 239 - CRITICAL -  - Unhandled error in Deferred:
2019-02-17 12:06:34,127 - twisted - 239 - CRITICAL -  -
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1416, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 491, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/local/lib/python2.7/dist-packages/synapse/app/homeserver.py", line 451, in start
    sys.exit(1)
SystemExit: 1
2019-02-17 12:06:34,129 - twisted - 239 - INFO -  - (TCP Port 8009 Closed)
```


Discussion:
this will only affect people who specify the url explicitly, so a workaround (since you're using the default) is to remove the `url` setting.

Also your periodic reminder that we're dropping python 2 support altogether in 5 weeks...

Resolving patch:
diff --git a/synapse/config/tls.py b/synapse/config/tls.py
--- a/synapse/config/tls.py
+++ b/synapse/config/tls.py
@@ -19,6 +19,8 @@
 from datetime import datetime
 from hashlib import sha256
 
+import six
+
 from unpaddedbase64 import encode_base64
 
 from OpenSSL import crypto
@@ -36,9 +38,11 @@ def read_config(self, config):
             acme_config = {}
 
         self.acme_enabled = acme_config.get("enabled", False)
-        self.acme_url = acme_config.get(
+
+        # hyperlink complains on py2 if this is not a Unicode
+        self.acme_url = six.text_type(acme_config.get(
             "url", u"https://acme-v01.api.letsencrypt.org/directory"
-        )
+        ))
         self.acme_port = acme_config.get("port", 80)
         self.acme_bind_addresses = acme_config.get("bind_addresses", ['::', '0.0.0.0'])
         self.acme_reprovision_threshold = acme_config.get("reprovision_threshold", 30)
@@ -55,7 +59,7 @@ def read_config(self, config):
                 )
             if not self.tls_private_key_file:
                 raise ConfigError(
-                    "tls_certificate_path must be specified if TLS-enabled listeners are "
+                    "tls_private_key_path must be specified if TLS-enabled listeners are "
                     "configured."
                 )
 


Test patch: