Repository: ansible/ansible

Issue:
openssl_certificate returns incorrect serial_number if cert exists
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
`openssl_certificate`

##### ANSIBLE VERSION
```
ansible 2.4.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/gross/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.14 (default, Sep 20 2017, 01:25:59) [GCC 7.2.0]
```

##### CONFIGURATION
```
ANSIBLE_PIPELINING(/home/gross/-screened-/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/home/gross/-screened-/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey
CACHE_PLUGIN(/home/gross/-screened-/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/home/gross/-screened-/ansible.cfg) = .facts-cache
CACHE_PLUGIN_TIMEOUT(/home/gross/-screened-/ansible.cfg) = 28800
DEFAULT_GATHERING(/home/gross/-screened-/ansible.cfg) = smart
DEFAULT_GATHER_SUBSET(/home/gross/-screened-/ansible.cfg) = network
DEFAULT_HOST_LIST(/home/gross/-screened-/ansible.cfg) = [u'/home/gross/-screened-/servers-dev']
DEFAULT_MODULE_NAME(/home/gross/-screened-/ansible.cfg) = shell
DEFAULT_REMOTE_USER(/home/gross/-screened-/ansible.cfg) = root
DEFAULT_VAULT_PASSWORD_FILE(/home/gross/-screened-/ansible.cfg) = /home/gross/-screened-/.vault-pass
RETRY_FILES_SAVE_PATH(/home/gross/-screened-/ansible.cfg) = /home/gross/-screened-/.retry
```

##### OS / ENVIRONMENT
Host: ArchLinux, python 3.6.3-1
Target: CentOS 7.4, python 2.7.5, openssl 1.0.2k, pyOpenSSL 0.15.1

pyOpenSSL is built from [rpm spec](https://src.fedoraproject.org/rpms/pyOpenSSL/blob/f23/f/pyOpenSSL.spec) originally for Fedora 23 since last pyOpenSSL in centos 7.4 repos is 0.13.1.

##### SUMMARY
`openssl_certificate` returns new `serial_number` on each invocation.

##### STEPS TO REPRODUCE
```
ansible host -m openssl_privatekey -a 'path=/root/test.pem size=1024'
ansible host -m openssl_csr -a 'path=/root/test.csr privatekey_path=/root/test.pem common_name=host.tld'
ansible host -m openssl_certificate -a 'path=/root/test.crt csr_path=/root/test.csr privatekey_path=/root/test.pem provider=selfsigned'
ansible host -m openssl_certificate -a 'path=/root/test.crt csr_path=/root/test.csr privatekey_path=/root/test.pem provider=selfsigned'
ansible host -m shell -a 'openssl x509 -noout -text -in /root/test.crt'
```

##### EXPECTED RESULTS
Same `serial_number` in all invocations.

##### ACTUAL RESULTS
`serial_number` from first invocation is correct (see results from `openssl x509 ...` command below. Subsequent `openssl_certificate` invocations return new `serial_number` each time.

```
-> % ansible all -i servers-test -m openssl_certificate -a 'path=/root/test.crt csr_path=/root/test.csr privatekey_path=/root/test.pem provider=selfsigned' -vvv
ansible 2.4.2.0
  config file = /home/gross/-screened-/ansible.cfg
  configured module search path = [u'/home/gross/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.14 (default, Sep 20 2017, 01:25:59) [GCC 7.2.0]
Using /home/gross/-screened-/ansible.cfg as config file
Parsed /home/gross/-screened-/servers-test inventory source with ini plugin
META: ran handlers
Using module file /usr/lib/python2.7/site-packages/ansible/modules/crypto/openssl_certificate.py
<host> ESTABLISH SSH CONNECTION FOR USER: root
<host> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/home/gross/.ansible/cp/d3eea4c148 host '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''                                                                                    
<host> (0, '\n{"notBefore": null, "changed": true, "privatekey": "/root/test.pem", "filename": "/root/test.crt", "invocation": {"module_args": {"privatekey_passphrase": null, "directory_mode": null, "force": false, "extendedKeyUsage": null, "csr_path": "/root/test.csr", "remote_src": null, "signature_algorithms": null, "regexp": null, "owner": null, "follow": false, "issuer": null, "group": null, "subjectAltName": null, "notBefore": null, "unsafe_writes": null, "subjectAltName_strict": false, "has_expired": false, "notAfter": null, "content": null, "selfsigned_notAfter": null, "state": "present", "acme_accountkey_path": null, "version": null, "provider": "selfsigned", "selfsigned_digest": "sha256", "subject": null, "setype": null, "extendedKeyUsage_strict": false, "keyUsage_strict": false, "selevel": null, "acme_challenge_path": null, "privatekey_path": "/root/test.pem", "path": "/root/test.crt", "keyUsage": null, "src": null, "serole": null, "backup": null, "seuser": null, "invalid_at": null, "delimiter": null, "mode": null, "attributes": null, "selfsigned_notBefore": null, "valid_in": null, "valid_at": null}}, "serial_number": 42202, "notAfter": null, "csr": "/root/test.csr"}\n', "Warning: Permanently added 'host,192.168.123.243' (ED25519) to the list of known hosts.\r\n")                                                                                                                   
host | SUCCESS => {
    "changed": true, 
    "csr": "/root/test.csr", 
    "filename": "/root/test.crt", 
    "invocation": {
        "module_args": {
            "acme_accountkey_path": null, 
            "acme_challenge_path": null, 
            "attributes": null, 
            "backup": null, 
            "content": null, 
            "csr_path": "/root/test.csr", 
            "delimiter": null, 
            "directory_mode": null, 
            "extendedKeyUsage": null, 
            "extendedKeyUsage_strict": false, 
            "follow": false, 
            "force": false, 
            "group": null, 
            "has_expired": false, 
            "invalid_at": null, 
            "issuer": null, 
            "keyUsage": null, 
            "keyUsage_strict": false, 
            "mode": null, 
            "notAfter": null, 
            "notBefore": null, 
            "owner": null, 
            "path": "/root/test.crt", 
            "privatekey_passphrase": null, 
            "privatekey_path": "/root/test.pem", 
            "provider": "selfsigned", 
            "regexp": null, 
            "remote_src": null, 
            "selevel": null, 
            "selfsigned_digest": "sha256", 
            "selfsigned_notAfter": null, 
            "selfsigned_notBefore": null, 
            "serole": null, 
            "setype": null, 
            "seuser": null, 
            "signature_algorithms": null, 
            "src": null, 
            "state": "present", 
            "subject": null, 
            "subjectAltName": null, 
            "subjectAltName_strict": false, 
            "unsafe_writes": null, 
            "valid_at": null, 
            "valid_in": null, 
            "version": null
        }
    }, 
    "notAfter": null, 
    "notBefore": null, 
    "privatekey": "/root/test.pem", 
    "serial_number": 42202
}
META: ran handlers
META: ran handlers
gross@wahnsinn [04:04:57] [~/-screened-] 
-> % ansible all -i servers-test -m openssl_certificate -a 'path=/root/test.crt csr_path=/root/test.csr privatekey_path=/root/test.pem provider=selfsigned' -vvv
ansible 2.4.2.0
  config file = /home/gross/-screened-/ansible.cfg
  configured module search path = [u'/home/gross/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.14 (default, Sep 20 2017, 01:25:59) [GCC 7.2.0]
Using /home/gross/-screened-/ansible.cfg as config file
Parsed /home/gross/-screened-/servers-test inventory source with ini plugin
META: ran handlers
Using module file /usr/lib/python2.7/site-packages/ansible/modules/crypto/openssl_certificate.py
<host> ESTABLISH SSH CONNECTION FOR USER: root
<host> SSH: EXEC ssh -o ControlMaster=auto -o ControlPersist=60s -o PreferredAuthentications=publickey -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/home/gross/.ansible/cp/d3eea4c148 host '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''                                                                                    
<host> (0, '\n{"notBefore": null, "changed": false, "privatekey": "/root/test.pem", "filename": "/root/test.crt", "invocation": {"module_args": {"privatekey_passphrase": null, "directory_mode": null, "force": false, "extendedKeyUsage": null, "csr_path": "/root/test.csr", "remote_src": null, "signature_algorithms": null, "regexp": null, "owner": null, "follow": false, "issuer": null, "group": null, "subjectAltName": null, "notBefore": null, "unsafe_writes": null, "subjectAltName_strict": false, "has_expired": false, "notAfter": null, "content": null, "selfsigned_notAfter": null, "state": "present", "acme_accountkey_path": null, "version": null, "provider": "selfsigned", "selfsigned_digest": "sha256", "subject": null, "setype": null, "extendedKeyUsage_strict": false, "keyUsage_strict": false, "selevel": null, "acme_challenge_path": null, "privatekey_path": "/root/test.pem", "path": "/root/test.crt", "keyUsage": null, "src": null, "serole": null, "backup": null, "seuser": null, "invalid_at": null, "delimiter": null, "mode": null, "attributes": null, "selfsigned_notBefore": null, "valid_in": null, "valid_at": null}}, "serial_number": 26117, "notAfter": null, "csr": "/root/test.csr"}\n', '')                                                                                                                                                                                                                
host | SUCCESS => {
    "changed": false, 
    "csr": "/root/test.csr", 
    "filename": "/root/test.crt", 
    "invocation": {
        "module_args": {
            "acme_accountkey_path": null, 
            "acme_challenge_path": null, 
            "attributes": null, 
            "backup": null, 
            "content": null, 
            "csr_path": "/root/test.csr", 
            "delimiter": null, 
            "directory_mode": null, 
            "extendedKeyUsage": null, 
            "extendedKeyUsage_strict": false, 
            "follow": false, 
            "force": false, 
            "group": null, 
            "has_expired": false, 
            "invalid_at": null, 
            "issuer": null, 
            "keyUsage": null, 
            "keyUsage_strict": false, 
            "mode": null, 
            "notAfter": null, 
            "notBefore": null, 
            "owner": null, 
            "path": "/root/test.crt", 
            "privatekey_passphrase": null, 
            "privatekey_path": "/root/test.pem", 
            "provider": "selfsigned", 
            "regexp": null, 
            "remote_src": null, 
            "selevel": null, 
            "selfsigned_digest": "sha256", 
            "selfsigned_notAfter": null, 
            "selfsigned_notBefore": null, 
            "serole": null, 
            "setype": null, 
            "seuser": null, 
            "signature_algorithms": null, 
            "src": null, 
            "state": "present", 
            "subject": null, 
            "subjectAltName": null, 
            "subjectAltName_strict": false, 
            "unsafe_writes": null, 
            "valid_at": null, 
            "valid_in": null, 
            "version": null
        }
    }, 
    "notAfter": null, 
    "notBefore": null, 
    "privatekey": "/root/test.pem", 
    "serial_number": 26117
}
META: ran handlers
META: ran handlers
-> % ansible host -m shell -a 'openssl x509 -noout -text -in /root/test.crt' 
host | SUCCESS | rc=0 >>
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 42202 (0xa4da)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: 
        Validity
            Not Before: Dec 15 01:04:57 2017 GMT
            Not After : Dec 13 01:04:57 2027 GMT
        Subject: CN=host.tld
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:e8:a7:05:3e:c2:57:d6:31:82:74:a6:9f:e3:2d:
                    ce:d0:2e:af:f1:9b:6a:70:04:9c:de:cf:04:64:87:
                    2f:22:c3:58:9f:3e:93:d3:f3:26:f3:d8:c1:9e:fc:
                    c4:da:1a:4f:53:54:47:2e:88:af:5a:f4:99:07:45:
                    db:90:7d:ab:19:72:ca:0c:a2:93:d1:66:b4:f7:3f:
                    92:04:33:ba:c4:73:dd:f5:58:d1:b7:a8:ad:ff:2c:
                    05:b0:b1:14:3b:46:39:63:4b:d2:c4:08:97:d9:d4:
                    70:23:09:f4:fa:01:33:4d:60:45:cd:8a:ef:3d:74:
                    4f:e4:09:59:0b:26:f6:d0:33
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:host.tld
    Signature Algorithm: sha256WithRSAEncryption
         90:c7:2b:e1:79:e2:f0:1a:d3:65:5a:cc:10:0b:8c:92:49:a0:
         22:60:ad:ba:6f:74:33:e8:f8:69:be:3e:1a:e0:b3:f2:fb:f8:
         fb:fb:4e:ea:1a:45:c0:74:32:74:7e:70:24:27:40:63:e4:e2:
         a9:ca:3c:72:32:c5:04:38:ca:b9:58:c3:c5:3a:9b:78:29:9b:
         ce:58:e7:64:56:81:5c:ca:de:fe:37:65:42:50:23:eb:99:30:
         f0:90:db:c1:53:75:8f:d0:c5:7f:4f:22:0d:23:61:02:57:2b:
         b3:19:2d:68:b7:c3:f0:53:d8:41:ba:7f:d5:1e:30:e6:85:89:
         ec:e4
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/crypto/openssl_certificate.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/crypto/openssl_certificate.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @MarkusTeufelberger @Spredzy
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/crypto/openssl_certificate.py b/lib/ansible/modules/crypto/openssl_certificate.py
--- a/lib/ansible/modules/crypto/openssl_certificate.py
+++ b/lib/ansible/modules/crypto/openssl_certificate.py
@@ -378,7 +378,6 @@ class SelfSignedCertificate(Certificate):
 
     def __init__(self, module):
         super(SelfSignedCertificate, self).__init__(module)
-        self.serial_number = randint(1000, 99999)
         self.notBefore = module.params['selfsigned_notBefore']
         self.notAfter = module.params['selfsigned_notAfter']
         self.digest = module.params['selfsigned_digest']
@@ -387,7 +386,6 @@ def __init__(self, module):
         self.privatekey = crypto_utils.load_privatekey(
             self.privatekey_path, self.privatekey_passphrase
         )
-        self.cert = None
 
     def generate(self, module):
 
@@ -403,7 +401,7 @@ def generate(self, module):
 
         if not self.check(module, perms_required=False) or self.force:
             cert = crypto.X509()
-            cert.set_serial_number(self.serial_number)
+            cert.set_serial_number(randint(1000, 99999))
             if self.notBefore:
                 cert.set_notBefore(self.notBefore)
             else:
@@ -420,11 +418,11 @@ def generate(self, module):
             cert.add_extensions(self.csr.get_extensions())
 
             cert.sign(self.privatekey, self.digest)
-            self.certificate = cert
+            self.cert = cert
 
             try:
                 with open(self.path, 'wb') as cert_file:
-                    cert_file.write(crypto.dump_certificate(crypto.FILETYPE_PEM, self.certificate))
+                    cert_file.write(crypto.dump_certificate(crypto.FILETYPE_PEM, self.cert))
             except EnvironmentError as exc:
                 raise CertificateError(exc)
 
@@ -441,9 +439,9 @@ def dump(self):
             'filename': self.path,
             'privatekey': self.privatekey_path,
             'csr': self.csr_path,
-            'notBefore': self.notBefore,
-            'notAfter': self.notAfter,
-            'serial_number': self.serial_number,
+            'notBefore': self.cert.get_notBefore(),
+            'notAfter': self.cert.get_notAfter(),
+            'serial_number': self.cert.get_serial_number(),
         }
 
         return result


Test patch:
diff --git a/test/integration/targets/openssl_certificate/tasks/main.yml b/test/integration/targets/openssl_certificate/tasks/main.yml
--- a/test/integration/targets/openssl_certificate/tasks/main.yml
+++ b/test/integration/targets/openssl_certificate/tasks/main.yml
@@ -17,6 +17,16 @@
         privatekey_path: '{{ output_dir }}/privatekey.pem'
         provider: selfsigned
         selfsigned_digest: sha256
+      register: selfsigned_certificate
+
+    - name: Generate selfsigned certificate
+      openssl_certificate:
+        path: '{{ output_dir }}/cert.pem'
+        csr_path: '{{ output_dir }}/csr.csr'
+        privatekey_path: '{{ output_dir }}/privatekey.pem'
+        provider: selfsigned
+        selfsigned_digest: sha256
+      register: selfsigned_certificate_idempotence
 
     - name: Check selfsigned certificate
       openssl_certificate:
diff --git a/test/integration/targets/openssl_certificate/tests/validate.yml b/test/integration/targets/openssl_certificate/tests/validate.yml
--- a/test/integration/targets/openssl_certificate/tests/validate.yml
+++ b/test/integration/targets/openssl_certificate/tests/validate.yml
@@ -16,6 +16,13 @@
       - cert_modulus.stdout == privatekey_modulus.stdout
       - cert_version.stdout == '3'
 
+- name: Validate certificate idempotence
+  assert:
+    that:
+      - selfsigned_certificate.serial_number == selfsigned_certificate_idempotence.serial_number
+      - selfsigned_certificate.notBefore == selfsigned_certificate_idempotence.notBefore
+      - selfsigned_certificate.notAfter == selfsigned_certificate_idempotence.notAfter
+
 - name: Validate certificate v2 (test - certificate version == 2)
   shell: 'openssl x509 -noout  -in {{ output_dir}}/cert_v2.pem -text | grep "Version" | sed "s/.*: \(.*\) .*/\1/g"'
   register: cert_v2_version