Repository: ansible/ansible
Pull request: Add to k8s_raw docs re: vault-encrypted files
I didn't read the examples far enough and maybe would not have tried all the examples so I ended up creating https://github.com/ansible/ansible/issues/47259 and learned that definition key with lookup works well and we agreed the docs should say something. :)

+label: docsite_pr

##### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->

<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
Fixes #47259 

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
- Docs Pull Request

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below -->
k8s_raw

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
2.7
```

##### ADDITIONAL INFORMATION
<!--- Include additional information to help people understand the change here -->
<!--- A step-by-step reproduction of the problem is helpful if there is no related issue -->

<!--- Paste verbatim command output below, e.g. before and after your change -->
```paste below

```


Problem statement:
k8s_raw cannot handle Ansible Vault encrypted files
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Explain the problem briefly below -->
k8s_raw works fine with raw .yaml files but does not work with Ansible Vault encrypted files, but fails apparently trying to parse them. It does not warn if no vault password is provided like normal modules would.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below -->
k8s_raw

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.7.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jul 13 2018, 13:06:57) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
ANSIBLE_SSH_ARGS(/etc/ansible/ansible.cfg) = -o ForwardAgent=yes
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
CentOS 7 trying to hit Minikube VM

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->
Generate a .yml file for use with k8s_raw, encrypt with `ansible-vault`, and try to use as `src:`.

<!--- Paste example playbooks or commands between quotes below -->
playbook:
```yaml
#!/usr/bin/env ansible-playbook

- hosts: localhost
  become: true
  become_user: root
  gather_facts: false
  tasks:
    - include_role:
        name: ../../roles/k8s-has-resource
      vars:
        secret_path: resources/test.yml
        kubeconfig_path: "{{ kubectl_config_host_path }}"
```
role:
```
- name: Resource is present
  k8s:
    state: present
    src: "{{ secret_path }}"
    kubeconfig: "{{ kubeconfig_path }}"
    context: "{{ kubectl_context }}"
    namespace: "{{ namespace }}"
```
resources/test.yml:
```
---
apiVersion: v1
kind: Namespace
metadata:
  name: metrics
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
I expect the resource to be successfully verified being present or to be applied.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
Module fails with `'str' object has no attribute 'get'`.
<!--- Paste verbatim command output between quotes -->
```paste below
TASK [../../roles/k8s-has-resource : Resource is present] *********************************************************************************************
task path: /w/roles/k8s-has-resource/tasks/main.yml:2
<192.168.99.100> ESTABLISH LOCAL CONNECTION FOR USER: root
<192.168.99.100> EXEC /bin/sh -c 'echo ~root && sleep 0'
<192.168.99.100> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155 `" && echo ansible-tmp-1539844152.01-140706276929155="` echo /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155 `" ) && sleep 0'
Using module file /usr/lib/python2.7/site-packages/ansible/modules/clustering/k8s/k8s.py
<192.168.99.100> PUT /root/.ansible/tmp/ansible-local-5195h_dFx/tmpLoKpuc TO /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py
<192.168.99.100> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/ /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py && sleep 0'
<192.168.99.100> EXEC /bin/sh -c '/usr/bin/python2 /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py && sleep 0'
<192.168.99.100> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py", line 113, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py", line 105, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py", line 48, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/tmp/ansible_k8s_payload_J6uVkn/__main__.py", line 171, in <module>
  File "/tmp/ansible_k8s_payload_J6uVkn/__main__.py", line 167, in main
  File "/tmp/ansible_k8s_payload_J6uVkn/ansible_k8s_payload.zip/ansible/module_utils/k8s/raw.py", line 91, in execute_module
AttributeError: 'str' object has no attribute 'get'

fatal: [localhost]: FAILED! => {
    "changed": false, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1539844152.01-140706276929155/AnsiballZ_k8s.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_k8s_payload_J6uVkn/__main__.py\", line 171, in <module>\n  File \"/tmp/ansible_k8s_payload_J6uVkn/__main__.py\", line 167, in main\n  File \"/tmp/ansible_k8s_payload_J6uVkn/ansible_k8s_payload.zip/ansible/module_utils/k8s/raw.py\", line 91, in execute_module\nAttributeError: 'str' object has no attribute 'get'\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", 
    "rc": 1
}
```

Hints from issue discussion:
Hi @r4j4h, thank you for submitting this issue!

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: thank_submitter --->
Files identified in the description:
* [lib/ansible/modules/clustering/k8s/_k8s_raw.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/clustering/k8s/_k8s_raw.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @chouseknecht @fabianvf @flaper87 @maxamillion @willthames
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Use `definition` rather than `src`

`definition: {{ lookup('file', 'path/to/vaulted/file') }}"`

should work with vaulted files
I've closed this but one could argue that it could be addressed in the docs. PRs appreciated if anyone feels that way.
Thanks @willthames, that worked perfectly! 

I see now it is actually already demonstrated in some of the latter examples, but it would still be good to call it out especially closer to `src`. I will hopefully send a PR after dinner to add something to the docs. :)

Commit message: Remove trailing whitespace

Patch to review:
diff --git a/lib/ansible/modules/clustering/k8s/k8s.py b/lib/ansible/modules/clustering/k8s/k8s.py
index 999d8ba0a15294..1829ce93fb3644 100644
--- a/lib/ansible/modules/clustering/k8s/k8s.py
+++ b/lib/ansible/modules/clustering/k8s/k8s.py
@@ -28,7 +28,7 @@
 description:
   - Use the OpenShift Python client to perform CRUD operations on K8s objects.
   - Pass the object definition from a source file or inline. See examples for reading
-    files and using Jinja templates.
+    files and using Jinja templates and vault-encrypted files.
   - Access to the full range of K8s APIs.
   - Use the M(k8s_facts) module to obtain a list of items about an object of type C(kind)
   - Authenticate using either a config file, certificates, password or token.
@@ -116,7 +116,9 @@
     state: present
     src: /testing/deployment.yml
 
-- name: Read definition file from the Ansible controller file system
+- name: >-
+    Read definition file from the Ansible controller file system
+    or by reading the definition from a file encrypted with Ansible Vault.
   k8s:
     state: present
     definition: "{{ lookup('file', '/testing/deployment.yml') }}"
diff --git a/lib/ansible/utils/module_docs_fragments/k8s_resource_options.py b/lib/ansible/utils/module_docs_fragments/k8s_resource_options.py
index 833e6dd18fcbf5..97cb053746814b 100644
--- a/lib/ansible/utils/module_docs_fragments/k8s_resource_options.py
+++ b/lib/ansible/utils/module_docs_fragments/k8s_resource_options.py
@@ -38,4 +38,5 @@ class ModuleDocFragment(object):
     - Reads from the local file system. To read from the Ansible controller's file system, use the file lookup
       plugin or template lookup plugin, combined with the from_yaml filter, and pass the result to
       I(resource_definition). See Examples below.
+    - To read Ansible Vault-encrypted files, use the file lookup plugin. See Examples below.
 '''


Code review comments:
diff_hunk: @@ -116,7 +116,9 @@
     state: present
     src: /testing/deployment.yml
 
-- name: Read definition file from the Ansible controller file system
+- name: >-
+    Read definition file from the Ansible controller file system
+    or by reading the definition from a file encrypted with Ansible Vault.
path: lib/ansible/modules/clustering/k8s/k8s.py
text: @user1:
I would make this a separate sentence that notes it will be handled properly if it's vault encrypted, ie

```
Read definition file from the Ansible controller file system. 
If the definition file has been encrypted with Ansible Vault it will automatically be decrypted.
```

diff_hunk: @@ -38,4 +38,5 @@ class ModuleDocFragment(object):
     - Reads from the local file system. To read from the Ansible controller's file system, use the file lookup
       plugin or template lookup plugin, combined with the from_yaml filter, and pass the result to
       I(resource_definition). See Examples below.
+    - To read Ansible Vault-encrypted files, use the file lookup plugin. See Examples below.
path: lib/ansible/utils/module_docs_fragments/k8s_resource_options.py
text: @user1:
Sorry, my notes on this got lost. I suggest going ahead and mashing the two together. Importantly, you can use the template lookup plugin too.

I'd probably go with something along the lines of:

```
To read from the Ansible controller's file system, including vaulted files, use the file lookup plugin or template lookup plugin,...
```

diff_hunk: @@ -28,7 +28,7 @@
 description:
   - Use the OpenShift Python client to perform CRUD operations on K8s objects.
   - Pass the object definition from a source file or inline. See examples for reading
-    files and using Jinja templates.
+    files and using Jinja templates and vault-encrypted files.
path: lib/ansible/modules/clustering/k8s/k8s.py
text: @user1:
Minor nit, I would use `or` rather than `and`.