Repository: ansible-collections/community.general

Issue:
Error creating keystore when extracting public certificate from pkcs12
### Summary

When I recently updated to Ansible 2.10.8, the Playbook I'm using in order to create a keystore with the VMs certificate in it, the `java_cert` module fails to put that certificate into the keystore with the message: `"Internal module failure, cannot extract public certificate from pkcs12, error: "`.

With Ansible 2.10.6, I don't get that error. I suspect this to be related to the change regarding how the keystore password is handled (I've seen `Internal module failure, cannot extract public certificate from pkcs12, error: Enter keystore password:`, but not with the task provided here).

### Issue Type

Bug Report

### Component Name

java_cert

### Ansible Version

```console (paste below)
$ ansible --version
ansible 2.10.8
  config file = /Users/splieth/projects/deployment/ansible.cfg
  configured module search path = ['/Users/splieth/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/splieth/.local/share/virtualenvs/deployment-054P1BD-/lib/python3.9/site-packages/ansible
  executable location = /Users/splieth/.local/share/virtualenvs/deployment-054P1BD-/bin/ansible
  python version = 3.9.4 (default, Apr  5 2021, 01:50:46) [Clang 12.0.0 (clang-1200.0.32.29)]
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed

ANSIBLE_SSH_ARGS(/Users/splieth/projects/deployment/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=60s -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=60 -o StrictHostKeyChecking=no
ANSIBLE_SSH_CONTROL_PATH(/Users/splieth/projects/deployment/ansible.cfg) = /tmp/%%h-%%p
ANSIBLE_SSH_RETRIES(/Users/splieth/projects/deployment/ansible.cfg) = 10
CACHE_PLUGIN(/Users/splieth/projects/deployment/ansible.cfg) = yaml
CACHE_PLUGIN_CONNECTION(/Users/splieth/projects/deployment/ansible.cfg) = ~/ansible_cache
CACHE_PLUGIN_PREFIX(/Users/splieth/projects/deployment/ansible.cfg) = _ansible_cache_
CACHE_PLUGIN_TIMEOUT(/Users/splieth/projects/deployment/ansible.cfg) = 43200
DEFAULT_FORKS(/Users/splieth/projects/deployment/ansible.cfg) = 20
DEFAULT_LOG_PATH(/Users/splieth/projects/deployment/ansible.cfg) = /Users/splieth/projects/deployment/logs/ansible.log
DEFAULT_ROLES_PATH(/Users/splieth/projects/deployment/ansible.cfg) = ['/Users/splieth/projects/deployment/roles', '/etc/ansible/roles']
DEFAULT_TIMEOUT(/Users/splieth/projects/deployment/ansible.cfg) = 900
DEFAULT_VAULT_PASSWORD_FILE(/Users/splieth/projects/deployment/ansible.cfg) = /Users/splieth/.ansible_vault_pw
DISPLAY_SKIPPED_HOSTS(/Users/splieth/projects/deployment/ansible.cfg) = True
HOST_KEY_CHECKING(/Users/splieth/projects/deployment/ansible.cfg) = False
INVENTORY_CACHE_ENABLED(/Users/splieth/projects/deployment/ansible.cfg) = True
INVENTORY_CACHE_PLUGIN(/Users/splieth/projects/deployment/ansible.cfg) = yaml
INVENTORY_CACHE_TIMEOUT(/Users/splieth/projects/deployment/ansible.cfg) = 43200
RETRY_FILES_ENABLED(/Users/splieth/projects/deployment/ansible.cfg) = False
```


### OS / Environment

macOS, CentOS 8.3

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
---
- name: create new pkcs
  when: not ansible_check_mode
  openssl_pkcs12:
    action: export
    path: "/tmp/{{ inventory_hostname }}.p12"
    privatekey_path: "/etc/pki/tls/private/{{ inventory_hostname }}.key"
    certificate_path: "/etc/pki/tls/certs/{{ inventory_hostname }}.crt"
    passphrase: "changeit"
    friendly_name: "{{ inventory_hostname_short }}"
    owner: "someUser"
    group: "someUser"

- name: create new keystore keystore.p12
  become: yes
  become_user: "someUser"
  become_flags: "-i"
  java_cert:
    pkcs12_path: "/tmp/{{ inventory_hostname }}.p12"
    pkcs12_password: "changeit"
    pkcs12_alias: "{{ inventory_hostname_short }}"
    cert_alias: "{{ item }}"
    keystore_path: "/tmp/{{ inventory_hostname_short }}.jks"
    keystore_pass: "changeit"
    keystore_create: yes
    state: present
  loop:
    - "{{ inventory_hostname }}"
    - "{{ inventory_hostname_short }}"
```


### Expected Results

No error when creating the keystore.

### Actual Results

```console (paste below)
TASK [deploy_role : create new pkcs] ***********************************
task path: /Users/splieth/projects/deployment/roles/deploy_role/tasks/cert.yml:2
redirecting (type: modules) ansible.builtin.openssl_pkcs12 to community.crypto.openssl_pkcs12
changed: [hostname.dev.some-domain.com] => {"changed": true, "filename": "/tmp/hostname.p12", "mode": "0400", "privatekey_path": "/etc/pki/tls/private/hostname.dev.some-domain.com.key"}

TASK [deploy_role : create new keystore keystore.p12] ******************
task path: /Users/splieth/projects/deployment/roles/deploy_role/tasks/cert.yml:14
redirecting (type: modules) ansible.builtin.java_cert to community.general.java_cert
failed: [hostname.dev.some-domain.com] (item=hostname.dev.some-domain.com) => {"ansible_loop_var": "item", "changed": false, "item": "hostname.dev.some-domain.com", "msg": "Internal module failure, cannot extract public certificate from pkcs12, error: ", "rc": 1}
redirecting (type: modules) ansible.builtin.java_cert to community.general.java_cert
failed: [hostname.dev.some-domain.com] (item=hostname) => {"ansible_loop_var": "item", "changed": false, "item": "hostname", "msg": "Internal module failure, cannot extract public certificate from pkcs12, error: ", "rc": 1}
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/system/java_cert.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/system/java_cert.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @absynth76 @haad
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
It's indeed really easy to reproduce, sorry about that. The integration test is using a basic password "changeit" and so it succeeded. However as soon as you introduce some special character in it; I suppose wrongly interpreted by the shell; the module fails with the mentioned error.

This change was introduce to fix a security issue where password could be exposed because passed as a command argument. 

If you have any suggestion to solve it, please shoot. 
"module.run_command" related issue, appears several time in the module, like here:
```
def _export_public_cert_from_pkcs12(module, executable, pkcs_file, alias, password, dest):
    """ Runs keytools to extract the public cert from a PKCS12 archive and write it to a file. """
    export_cmd = [
        executable,
        "-list",
        "-keystore",
        pkcs_file,
        "-alias",
        alias,
        "-storetype",
        "pkcs12",
        "-rfc"
    ]
    (export_rc, export_stdout, export_err) = module.run_command(export_cmd, data=password, check_rc=False)
```

[edit] I was incorrectly testing when I posted that comment
The way the module uses run_command, no shell is involved. Do you have an example what kind of special characters cause problems?
I only managed to reproduce this with non-ASCII characters, but not with regular special characters (like $, punctuation, ...). Mishandling of non-ASCII characters could be a encoding problem.
To test I duplicated the embedded keystore, 
so far I tested with:
changeit
changeit!
changeit$
changeit\
changeit/
changeit.

and no issue at all, I can only reproduce the failure in case I provide an incorrect keystore password to the task.

In the meantime, reading the code I saw I made a mistake at another place, 
"Unused variable 'pkcs12_alias'pylint(unused-variable)"
```
            _export_public_cert_from_pkcs12(module, executable, pkcs12_path, cert_alias, pkcs12_pass, new_certificate)
```
it should be
```
            _export_public_cert_from_pkcs12(module, executable, pkcs12_path, pkcs12_alias, pkcs12_pass, new_certificate)
```
sorry for the noise, this one should deserve another ticket I guess

@absynth76 for the unrelated part, just create a small PR fixing it.
> I only managed to reproduce this with non-ASCII characters, but not with regular special characters (like $, punctuation, ...). Mishandling of non-ASCII characters could be a encoding problem.

In fact I didn't even reproduce it with this module, but with java_keystore. I tried to reproduce it using the java_cert integration tests, but the task it failed was a java_keystore task... I guess the problem there is that `openssl` (password via stdin) and `keytool` (password via env variable) handle encodings differently...

@splieth I have a question about your bug report. You are stating that you are using the `java_cert` module to **create** a keystore. The documentation does not state that it creates keystores, it only states that it **modifies** existing keystores. It could be that older versions of the module would create a keystore if none exists.
To answer to @felixfontein, even if the main description does not talk about "creation" of a keystore, the module is able to do so. There is even a (default to False) flag to make it happen "keystore_create". It could (maybe) deserve a small update and mention that fact in the main description ("eventually create").



> @splieth I have a question about your bug report. You are stating that you are using the `java_cert` module to **create** a keystore. The documentation does not state that it creates keystores, it only states that it **modifies** existing keystores. It could be that older versions of the module would create a keystore if none exists.

Sorry, I was actually referring to modifying a keystore rather than creating a new one since I'm running it on a already provisioned VM. But I'll cross check that later, thanks for the hint.

I just checked and I'm actually already using `keystore_create`, but the nevertheless it's only an update. The password does not contain any special characters (it's just `changeit`). So I guess encoding shouldn't be a problem.

I also just ran the playbook again (with 2.10.9, because 2.10.8 can't be found on pypi.org right now), the error also still occurs: `"Internal module failure, cannot extract public certificate from pkcs12, error: Enter keystore password:  "`.

Given this error message, it seems to me that no password is passed at all. That's also what happens when I try to do that manually on the shell.

When I run `keytool -list -keystore keystore.jks`, I get `Enter keystore password:`.
On the other hand, when I run `keytool -list -keystore keystore.jks -storepass invalidPassword`, I can see an error message: `keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect`.

But works like charm if I pass the correct password.
I think I found why it does that, I was focusing on an incorrect password, or on the sudo '-i' modifier.
But it's coming from the mistake I mention above, 
>In the meantime, reading the code I saw I made a mistake at another place,
>"Unused variable 'pkcs12_alias'pylint(unused-variable)"

or it's clear cert_alias and pkcs12_alias are different here, so it fails.

Resolving patch:
diff --git a/plugins/modules/system/java_cert.py b/plugins/modules/system/java_cert.py
--- a/plugins/modules/system/java_cert.py
+++ b/plugins/modules/system/java_cert.py
@@ -278,7 +278,7 @@ def _export_public_cert_from_pkcs12(module, executable, pkcs_file, alias, passwo
     (export_rc, export_stdout, export_err) = module.run_command(export_cmd, data=password, check_rc=False)
 
     if export_rc != 0:
-        module.fail_json(msg="Internal module failure, cannot extract public certificate from pkcs12, error: %s" % export_err,
+        module.fail_json(msg="Internal module failure, cannot extract public certificate from pkcs12, error: %s" % export_stdout,
                          rc=export_rc)
 
     with open(dest, 'w') as f:
@@ -498,7 +498,7 @@ def main():
 
         if pkcs12_path:
             # Extracting certificate with openssl
-            _export_public_cert_from_pkcs12(module, executable, pkcs12_path, cert_alias, pkcs12_pass, new_certificate)
+            _export_public_cert_from_pkcs12(module, executable, pkcs12_path, pkcs12_alias, pkcs12_pass, new_certificate)
 
         elif path:
             # Extracting the X509 digest is a bit easier. Keytool will print the PEM


Test patch:
diff --git a/tests/integration/targets/java_cert/tasks/state_change.yml b/tests/integration/targets/java_cert/tasks/state_change.yml
--- a/tests/integration/targets/java_cert/tasks/state_change.yml
+++ b/tests/integration/targets/java_cert/tasks/state_change.yml
@@ -4,19 +4,89 @@
   args:
     creates: "{{ test_key_path }}"
 
-- name: Create the test keystore
-  java_keystore:
-    name: placeholder
-    dest: "{{ test_keystore2_path }}"
-    password: "{{ test_keystore2_password }}"
-    private_key: "{{ lookup('file', '{{ test_key_path }}') }}"
-    certificate: "{{ lookup('file', '{{ test_cert_path }}') }}"
-
 - name: Generate the self signed cert we will use for testing
   command: openssl req -x509 -newkey rsa:4096 -keyout '{{ test_key2_path }}' -out '{{ test_cert2_path }}' -days 365 -nodes -subj '/CN=localhost'
   args:
     creates: "{{ test_key2_path }}"
 
+- name: Create the pkcs12 archive from the test x509 cert
+  command: >
+    openssl pkcs12
+    -in {{ test_cert_path }}
+    -inkey {{ test_key_path }}
+    -export
+    -name test_pkcs12_cert
+    -out {{ test_pkcs_path }}
+    -passout pass:"{{ test_keystore2_password }}"
+
+- name: Create the pkcs12 archive from the certificate we will be trying to add to the keystore
+  command: >
+    openssl pkcs12
+    -in {{ test_cert2_path }}
+    -inkey {{ test_key2_path }}
+    -export
+    -name test_pkcs12_cert
+    -out {{ test_pkcs2_path }}
+    -passout pass:"{{ test_keystore2_password }}"
+
+- name: try to create the test keystore based on the just created pkcs12, keystore_create flag not enabled
+  java_cert:
+    cert_alias: test_pkcs12_cert
+    pkcs12_alias: test_pkcs12_cert
+    pkcs12_path: "{{ test_pkcs_path }}"
+    pkcs12_password: "{{ test_keystore2_password }}"
+    keystore_path: "{{ test_keystore2_path }}"
+    keystore_pass: "{{ test_keystore2_password }}"
+  ignore_errors: true
+  register: result_x509_changed
+
+- name: Verify the x509 status is failed
+  assert:
+    that:
+      - result_x509_changed is failed
+
+- name: Create the test keystore based on the just created pkcs12
+  java_cert:
+    cert_alias: test_pkcs12_cert
+    pkcs12_alias: test_pkcs12_cert
+    pkcs12_path: "{{ test_pkcs_path }}"
+    pkcs12_password: "{{ test_keystore2_password }}"
+    keystore_path: "{{ test_keystore2_path }}"
+    keystore_pass: "{{ test_keystore2_password }}"
+    keystore_create: yes
+
+- name: try to import from pkcs12 a non existing alias
+  java_cert:
+    cert_alias: test_pkcs12_cert
+    pkcs12_alias: non_existing_alias
+    pkcs12_path: "{{ test_pkcs_path }}"
+    pkcs12_password: "{{ test_keystore2_password }}"
+    keystore_path: "{{ test_keystore2_path }}"
+    keystore_pass: "{{ test_keystore2_password }}"
+    keystore_create: yes
+  ignore_errors: yes
+  register: result_x509_changed
+
+- name: Verify the x509 status is failed
+  assert:
+    that:
+      - result_x509_changed is failed
+
+- name:  import initial test certificate from file path
+  java_cert:
+    cert_alias: test_cert
+    cert_path: "{{ test_cert_path }}"
+    keystore_path: "{{ test_keystore2_path }}"
+    keystore_pass: "{{ test_keystore2_password }}"
+    keystore_create: yes
+    state: present
+  register: result_x509_changed
+
+- name: Verify the x509 status is changed
+  assert:
+    that:
+      - result_x509_changed is changed
+
 - name: |
     Import the newly created certificate. This is our main test.
     If the java_cert has been updated properly, then this task will report changed each time
@@ -30,7 +100,7 @@
     state: present
   register: result_x509_changed
 
-- name: Verify the x509 status has changed
+- name: Verify the x509 status is changed
   assert:
     that:
       - result_x509_changed is changed
@@ -50,26 +120,6 @@
     that:
       - result_x509_succeeded is succeeded
 
-- name: Create the pkcs12 archive from the test x509 cert
-  command: >
-    openssl pkcs12
-    -in {{ test_cert_path }}
-    -inkey {{ test_key_path }}
-    -export
-    -name test_pkcs12_cert
-    -out {{ test_pkcs_path }}
-    -passout pass:"{{ test_keystore2_password }}"
-
-- name: Create the pkcs12 archive from the certificate we will be trying to add to the keystore
-  command: >
-    openssl pkcs12
-    -in {{ test_cert2_path }}
-    -inkey {{ test_key2_path }}
-    -export
-    -name test_pkcs12_cert
-    -out {{ test_pkcs2_path }}
-    -passout pass:"{{ test_keystore2_password }}"
-
 - name: >
     Ensure the original pkcs12 cert is in the keystore
   java_cert:
@@ -83,7 +133,7 @@
 
 - name: |
     Perform the same test, but we will now be testing the pkcs12 functionality
-    If we add a different pkcs12 cert with the same alias, we should have a chnaged result, NOT the same
+    If we add a different pkcs12 cert with the same alias, we should have a changed result, NOT the same
   java_cert:
     cert_alias: test_pkcs12_cert
     pkcs12_alias: test_pkcs12_cert
@@ -94,7 +144,7 @@
     state: present
   register: result_pkcs12_changed
 
-- name: Verify the pkcs12 status has changed
+- name: Verify the pkcs12 status is changed
   assert:
     that:
       - result_pkcs12_changed is changed
@@ -155,7 +205,7 @@
     that:
       - result_x509_absent is changed
 
-- name: Ensure we can remove the pkcs12 archive
+- name: Ensure we can remove the certificate imported from pkcs12 archive
   java_cert:
     cert_alias: test_pkcs12_cert
     keystore_path: "{{ test_keystore2_path }}"