Repository: ansible/ansible

Issue:
letsencrypt module does not create any cert when using DNS-based validation
##### ISSUE TYPE
Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
letsencrypt

##### ANSIBLE VERSION

Ansible version is `2.2.1.0`

##### SUMMARY

It looks like the Let's Encrypt Ansible module doesn't create any cert when running with the DNS-based validation. 

##### STEPS TO REPRODUCE

```
# vars
---
aws:
   access_key: "some_access_key"
   secret_key: "some_secret_key"
letsencrypt_account_email: "some_email@example.com"
letsencrypt_key_filename: "some_filename"
letsencrypt_dir: "some/path"
letsencrypt_domains:
   - "foo"
   - "bar"

# tasks
---
- name: Install Route53 dependencies
  pip:
    name: boto

- name: Create folder
  file:
    path: "{{ letsencrypt_dir }}"
    owner: root
    group: root
    mode: 0755
    state: directory

- name: Create key
  shell: 'ssh-keygen -t rsa -b 2048 -C "{{ letsencrypt_account_email }}" -f ~/.ssh/{{ letsencrypt_key_filename }} -q -N ""'
  args:
    creates: "~/.ssh/{{ letsencrypt_key_filename }}"

- name: Create domain key
  shell: 'ssh-keygen -t rsa -b 2048 -C "{{ letsencrypt_account_email }}" -f ~/.ssh/id_rsa.{{ item }} -q -N ""'
  args:
    creates: '~/.ssh/id_rsa.{{ item }}'
  with_items: "{{ letsencrypt_domains }}"

- name: Create CSR (Certificate Signing Request)
  shell: 'openssl req -new -nodes -key ~/.ssh/{{ letsencrypt_key_filename }} -out {{ letsencrypt_dir }}/{{ item }}.csr -subj "/CN={{ item }}"'
  args:
    creates: '{{ letsencrypt_dir }}/{{ item }}.csr'
  with_items: "{{ letsencrypt_domains }}"

- name: Create challenge
  letsencrypt:
    account_key: '~/.ssh/id_rsa.{{ item }}'
    challenge: dns-01
    csr: '{{ letsencrypt_dir }}/{{ item }}.csr'
    dest: '{{ letsencrypt_dir }}/{{ item }}.crt'
    remaining_days: 20
  register: letsencrypt_challenge
  with_items: "{{ letsencrypt_domains }}"

- name: Create Route53 TXT record for DNS-based certificate validation
  route53:
    command: create
    aws_access_key: "{{ aws.access_key_id }}"
    aws_secret_key: "{{ aws.secret_access_key }}"
    zone: 'some_hosted_zone.com'
    record: "{{ item.1.challenge_data[item.0]['dns-01']['resource'] }}.some_hosted_zone.com"
    retry_interval: 300
    type: TXT
    ttl: 7200
    value: '"{{ item.1.challenge_data[item.0]["dns-01"]["resource_value"] }}"'
    wait: yes
  when: "letsencrypt_challenge|changed"
  with_together:
    - "{{ letsencrypt_domains }}"
    - "{{ letsencrypt_challenge['results'] }}"
  ignore_errors: yes

- name: Validate challenge
  letsencrypt:
    account_key: '~/.ssh/id_rsa.{{ item }}'
    challenge: dns-01
    csr: '{{ letsencrypt_dir }}/{{ item }}.csr'
    dest: '{{ letsencrypt_dir }}/{{ item }}.crt'
    data: '{{ letsencrypt_challenge }}'
    remaining_days: 20
  when: '{{ letsencrypt_challenge|changed }}'
  with_items: "{{ letsencrypt_domains }}"

- name: Delete Route53 TXT record for DNS-based certificate validation
  route53:
    command: delete
    aws_access_key: "{{ aws.access_key_id }}"
    aws_secret_key: "{{ aws.secret_access_key }}"
    zone: 'some_hosted_zone.com'
    record: "{{ item.1.challenge_data[item.0]['dns-01']['resource'] }}.some_hosted_zone.com"
    retry_interval: 300
    type: TXT
    ttl: 7200
    value: '"{{ item.1.challenge_data[item.0]["dns-01"]["resource_value"] }}"'
    wait: yes
  when: "letsencrypt_challenge|changed"
  with_together:
    - "{{ letsencrypt_domains }}"
    - "{{ letsencrypt_challenge['results'] }}"
  ignore_errors: yes

```

##### ACTUAL RESULTS

No errors are being thrown or anything, it provisions just fine

##### EXPECTED RESULTS
 
There should be a cert generated



Discussion:
You shouldn't set 'when: "letsencrypt_challenge|changed"' on the second call to the module. I'm pretty sure the first result can be OK (e.g. if the requested domain has already been validated for this LetsEncrypt account) even if the certificate still needs to be created.
@walterdolce can you try the suggestion and let us know if the issue persists?
Also, please use the PR template for providing info in future PR submissions.
cc @mgruener
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
@flowerysong @rcarrillocruz tried again. No certificate created in `{{ letsencrypt_dir }}`.
Note I'm running this from a Vagrant box.

How do I reveal the values being output as `VALUE_SPECIFIED_IN_NO_LOG_PARAMETER` ?
Perhaps that would help me troubleshoot this. 
@walterdolce the problem here is that `letsencrypt_challenge` is filled with an array of challenges. 
when you loop through the domains when validating the challenge you should split the challenges as well as the letsencrypt module cannot handle the array of challenges directly.

I suggest to try in the `Validate challenge` task:
```
 with_together:
    - "{{ letsencrypt_domains }}"
    - "{{ letsencrypt_challenge['results'] }}"
```
and use `data: "{{ item.1 }}"` 

@walterdolce I tried the same yesterday and got the same results, digging into it looks like the problem wasn't the DNS challenge. It seems to me that you are using `{{ letsencrypt_key_filename }}` to create domain CSR instead of `id_rsa.{{ item }}`. That leads to [https://github.com/ansible/ansible/issues/20638](url) and as result no certificate will be created.
I would suggest anyway to augment the error reporting for this, if possible.

I hope this is understandable, since English is not my main language.
cc @resmo
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/web_infrastructure/letsencrypt.py b/lib/ansible/modules/web_infrastructure/letsencrypt.py
--- a/lib/ansible/modules/web_infrastructure/letsencrypt.py
+++ b/lib/ansible/modules/web_infrastructure/letsencrypt.py
@@ -88,6 +88,9 @@
       - "The CSR may contain multiple Subject Alternate Names, but each one
          will lead to an individual challenge that must be fulfilled for the
          CSR to be signed."
+      - "Note: the private key used to create the CSR I(must not) be the the
+         account key. This is a bad idea from a security point of view, and
+         Let's Encrypt will not accept the CSR."
     required: true
     aliases: ['src']
   data:


Test patch: