Repository: ansible/ansible

Issue:
letsencrypt v2.4 http-01 certificate validation false positive error - No 'Not after' date found
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
letsencrypt

##### ANSIBLE VERSION
```
ansible-doc 2.4.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/morel/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-doc
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]

```

##### CONFIGURATION
logging on - doesn't make a difference

##### OS / ENVIRONMENT
Both are:
Ubuntu 16.04.3 LTS x64

##### SUMMARY

I get this error in version 2.4.0 when trying to create a letsencrypt certificate which I confirmed does not occur in 2.3.2.

`FAILED! => {"changed": false, "failed": true, "msg": "No 'Not after' date found in /etc/letsencrypt/certs/********.crt"}`

##### STEPS TO REPRODUCE
```
--- # Install Letsencrypt Certificate

- name: Create ssl file directories
  file:
    path: "{{ ssl_root }}/{{ item }}"
    state: directory
  with_items:
    - csr
    - keys
    - certs

- name: Generate account key
  command: "openssl genrsa -out {{ ssl_root }}/account.key"

- name: Generate certificate private key
  command: "openssl genrsa -out {{ ssl_root }}/keys/{{ inventory_hostname }}.pem"

- name: Generate certificate signing request
  command: "openssl req -new -sha256 -key {{ ssl_root }}/keys/{{ inventory_hostname }}.pem -out {{ ssl_root }}/csr/{{ inventory_hostname }}.pem -subj /CN={{ inventory_hostname }}"

- name: Initiate letsencrypt challenge
  letsencrypt:
    account_key: "{{ ssl_root }}/account.key"
    csr: "{{ ssl_root }}/csr/{{ inventory_hostname }}.pem"
    acme_directory: "https://acme-v01.api.letsencrypt.org/directory"
    dest: "{{ ssl_root }}/certs/{{ inventory_hostname }}.crt"
    account_email: "{{ admin_email }}"
    challenge: "http-01"
  register: letsencrypt_challenge

- name: Create directory to fulfill letsencrypt challenge
  file:
    dest: /var/www/html/{{ letsencrypt_challenge['challenge_data'][inventory_hostname]['http-01']['resource']|dirname }}
    state: directory

- name: Create file to fulfill letsencrypt challenge
  copy:
    dest: /var/www/html/{{ letsencrypt_challenge ['challenge_data'][inventory_hostname]['http-01']['resource'] }}
    content: "{{ letsencrypt_challenge ['challenge_data'][inventory_hostname]['http-01']['resource_value'] }}"
    owner: www-data
    group: www-data
    mode: 0644
  when: letsencrypt_challenge|changed

- name: Write signed letsencrypt certificate
  letsencrypt:
    account_key: "{{ ssl_root }}/account.key"
    csr: "{{ ssl_root }}/csr/{{ inventory_hostname }}.pem"
    dest: "{{ ssl_root }}/certs/{{ inventory_hostname }}.crt"
    acme_directory: "https://acme-v01.api.letsencrypt.org/directory"
    account_email: "{{ admin_email }}"
    challenge: "http-01"
    data: "{{ letsencrypt_challenge }}"
```

##### EXPECTED RESULTS

Success.

##### ACTUAL RESULTS

```
The full traceback is:
  File "/tmp/ansible_1eahLV/ansible_module_letsencrypt.py", line 218, in get_cert_days
    not_after = datetime.datetime.fromtimestamp(time.mktime(time.strptime(not_after_str, '%b %d %H:%M:%S %Y %Z')))

fatal: [ilios.refactored.ca]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "account_email": "OMITTED", 
            "account_key": "/etc/letsencrypt/account.key", 
            "acme_directory": "https://acme-v01.api.letsencrypt.org/directory", 
            "agreement": "https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf", 
            "challenge": "http-01", 
            "csr": "/etc/letsencrypt/csr/ilios.refactored.ca.pem", 
            "data": null, 
            "dest": "/etc/letsencrypt/certs/ilios.refactored.ca.crt", 
            "remaining_days": 10
        }
    }, 
    "msg": "No 'Not after' date found in /etc/letsencrypt/certs/ilios.refactored.ca.crt"
}
```



Discussion:
cc @mgruener
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
ping @jimi-c you mentioned a similar issue in the past. Is this related?
cc @resmo
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Finding the same issue in:
```
$ openssl version
OpenSSL 1.0.2l  25 May 2017

$ ansible --version
ansible 2.4.0.0
  config file = None
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/ubuntu/.conda/envs/showit/lib/python3.6/site-packages/ansible
  executable location = /home/ubuntu/.conda/envs/showit/bin/ansible
  python version = 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]

$  lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial
```

Linking https://github.com/ansible/ansible-modules-extras/issues/3155 which supposedly fixed the issue.
I was able to make the letsencrypt module work with Ansible 2.4.2 on Python 2.7. 

I couldn't get the module to work on Python 3.
Workaround: Run it with 
```
LC_ALL=C ansible-playbook ...
```
The plugin seemed to be not compatible with python3. As soon as I switched the environment back to python 2.7 - the plugin started to work for me.

Best wishes,
Siva on 6+

> On Dec 31, 2017, at 6:30 AM, Bernhard Strähle <notifications@github.com> wrote:
> 
> Workaround: Run it with
> 
> LC_ALL=C ansible-playbook ...
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 

I've just also encountered this problem. After debugging the module a bit, it turns out that the problem are in the following lines:
```
    _, out, _ = module.run_command(openssl_cert_cmd, check_rc=True)
    try:
        not_after_str = re.search(r"\s+Not After\s*:\s+(.*)", out.decode('utf8')).group(1)
        not_after = datetime.fromtimestamp(time.mktime(time.strptime(not_after_str, '%b %d %H:%M:%S %Y %Z')))
    except AttributeError:
        module.fail_json(msg="No 'Not after' date found in {0}".format(cert_file))
```
For Python 3, `out` is a unicode string (`str`), while the code expects it to be a bytes string (`bytes`). Since byte strings have a `decode()` method and unicode strings do not, the call `out.decode()` fails with an `AttributeError`, which produces this (incorrect) error message.

From the documentation of `run_command` (in lib/ansible/module_utils/basic.py):
```
        :returns: A 3-tuple of return code (integer), stdout (native string),
            and stderr (native string).  On python2, stdout and stderr are both
            byte strings.  On python3, stdout and stderr are text strings converted
            according to the encoding and errors parameters.  If you want byte
            strings on python3, use encoding=None to turn decoding to text off.
```
So passing `encoding=None` to all `run_command` invocations should fix this problem once and for all.

Resolving patch:
diff --git a/lib/ansible/modules/web_infrastructure/letsencrypt.py b/lib/ansible/modules/web_infrastructure/letsencrypt.py
--- a/lib/ansible/modules/web_infrastructure/letsencrypt.py
+++ b/lib/ansible/modules/web_infrastructure/letsencrypt.py
@@ -245,7 +245,7 @@ def get_cert_days(module, cert_file):
 
     openssl_bin = module.get_bin_path('openssl', True)
     openssl_cert_cmd = [openssl_bin, "x509", "-in", cert_file, "-noout", "-text"]
-    _, out, _ = module.run_command(openssl_cert_cmd, check_rc=True)
+    _, out, _ = module.run_command(openssl_cert_cmd, check_rc=True, encoding=None)
     try:
         not_after_str = re.search(r"\s+Not After\s*:\s+(.*)", out.decode('utf8')).group(1)
         not_after = datetime.fromtimestamp(time.mktime(time.strptime(not_after_str, '%b %d %H:%M:%S %Y %Z')))
@@ -367,7 +367,7 @@ def __init__(self, module):
             _, tmpsrc = tempfile.mkstemp()
             f = open(tmpsrc, 'wb')
             try:
-                f.write(self.key_content)
+                f.write(self.key_content.encode('utf-8'))
                 self.key = tmpsrc
             except Exception as err:
                 os.remove(tmpsrc)
@@ -759,7 +759,7 @@ def _new_cert(self):
 
         new_cert = {
             "resource": "new-cert",
-            "csr": nopad_b64(out),
+            "csr": nopad_b64(to_bytes(out)),
         }
         result, info = self.account.send_signed_request(self.directory['new-cert'], new_cert)
 
@@ -833,7 +833,7 @@ def get_certificate(self):
             if chain and self.module.params['fullchain']:
                 pem_cert += "\n".join(chain)
 
-            if write_file(self.module, self.dest, pem_cert):
+            if write_file(self.module, self.dest, pem_cert.encode('utf8')):
                 self.cert_days = get_cert_days(self.module, self.dest)
                 self.changed = True
 


Test patch: