Repository: ansible-collections/community.general

Issue:
xfconf not setting return values as stated in documentation
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Explain the problem briefly below -->
The xfconf documentations that it will return several values related to the operation (channel, previous_name, property, value, value_type). None of these are getting set ever.

This looks like it might have got missed during the recent refactor in #1322 by @russoz. There is a method named  `update_xfconf_output()` in the `XFConfProperty` class that looks like it would be the right thing to use for this, it just isn't called anywhere.

I think I see what needs to be fixed for this and will take a stab at a PR.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
plugins/modules/system/xfconf.py

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.10.3
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
Not relevant
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
Not relevant

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->
Attempting to get or set any value doesn't set the correct return values. In the case below, you would expect the return to contain a value named `value` with the current value of `/panels/panel-0/plugin-ids`, but it is missing.

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: Get current panel layout
  xfconf:
    channel: xfce4-panel
    property: /panels/panel-0/plugin-ids
    state: get
  register: panel_order
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
Expect to see the values set per the documentation in the return.

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
None of the expected values are there in the return. Note, the values do appear in the stdout from running the command, but this is a side effect and not following the docs.

<!--- Paste verbatim command output between quotes -->
```paste below
ok: [localhost] => {
    "changed": false,
    "force_lang": "C",
    "invocation": {
        "module_args": {
            "channel": "xfce4-panel",
            "force_array": false,
            "property": "/panels/panel-0/plugin-ids",
            "state": "get",
            "value": null,
            "value_type": null
        }
    },
    "rc": 0,
    "stderr": "",
    "stderr_lines": [],
    "stdout": "Value is an array with 13 items:\n\n1\n2\n3\n4\n15\n5\n6\n7\n8\n9\n10\n11\n12\n",
    "stdout_lines": [
        "Value is an array with 13 items:",
        "",
        "1",
        "2",
        "3",
        "4",
        "15",
        "5",
        "6",
        "7",
        "8",
        "9",
        "10",
        "11",
        "12"
    ]
}
```



Discussion:
Files identified in the description:
* [`plugins/modules/system/xfconf.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/system/xfconf.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @jbenden
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
After looking into it a bit more, I think this got missed due to a bug in the test logic. This code will fail to asset for the presence of `value` and others in the output. This logic will skip the asserts if it is missing in the output which is not really what I we want I think.
https://github.com/ansible-collections/community.general/blob/main/tests/unit/plugins/modules/system/test_xfconf.py#L338
```py
    if 'msg' in results:
        assert results.get('msg') == testcase['msg']
    if 'value' in results:
        assert results['value'] == testcase['value']
    if 'previous_value' in results:
        assert results['previous_value'] == testcase['previous_value']
```

I think the logic wants to be flipped to run the assert when the key is in the testcase, not just the output. This way it will properly catch the case where the key was expected but is missing in the output.
```py
    if 'msg' in testcase:
        assert results.get('msg') == testcase['msg']
    if 'value' in testcase:
        assert results['value'] == testcase['value']
    if 'previous_value' in testcase:
        assert results['previous_value'] == testcase['previous_value']
```

Still working on a PR for this.
Random question that someone watching the PR might have an answer to. I've used pytest in the past and it does re-writing of asserts such that you get a useful output showing you have various variables expand making debugging much much eaiser. This seems to not happen when using `ansible-test`. I _think_ this might be related to this Ansible PR (ansible/ansible#62120), but I really don't know enough about ansible-test or pytest to make a call here. Anyone know if there is an easy way to turn that back on? I've resorted to manually writing out assert messages by hand where I need them.
@lifeisafractal thanks for spotting this! My first response is "oopsie!". I wrote that method but apparently I forgot to put it to use. 

The underlying problem here is that, historically, the modules *never returned values*, instead it set `ansible_facts`, which would be akin to a function setting values to global variables. I wrote that methos to have the values being added both to the facts (to maintain backward compatibility) _and_ to the regular output.


@russoz thanks for the info and no problem, I've had my fair share of "oopsies" in the past too!

I'm pretty new to Ansible and am not yet super clear on best practices. My use case was checking the panel config of XFCE to see if I had added a pager to it or not yet which is a little more advance than just a simple check. I figured I could register the output and then run a second task that does the checking and manipulation for adding the page. Are facts a more appropriate way of accomplishing this?
@lifeisafractal I would not describe myself as "seasoned" in Ansible, even though it's been a year or so since I started putting my hands on it. 

Responding to your question though, in theory, registering the output should be preferred to _facts_ (they are pretty much like global variables in that sense), but also in theory we should have two separate modules, one would be something like `xfconf_info` and it's purpose would be to read information from `xfconf-query`, while the other one remained as an idempotent _setter_ of state. For the time being, I believe your approach is sound - we should deprecate the _facts_ outcome at some point but in the meanwhile we're good.
I would really prevent returning facts if possible, especially adding new ones. Adding new facts always increases the chances of randomly breaking things, if someone else already used that name somewhere. (I guess there are some people who think differently of facts, but I really, really dislike them in general - but then I also prefer explicit over implicit in general. There are some cases where facts are really good and helpful, but random modules returning them is definitely *not* helpful IMO.)

One really bad example is the `community.general.docker_compose` module, which returned a fact named after every service it creates. So if you start a service called `busybox`, you suddenly have a facts called `busybox`, and if you had a variable called `busybox` before, it's now replaced by the service definition. This led to multiple bug reports from users who were totally surprised by this behavior (very understandably). The main problem is that removing facts is a very tedious job, since it can break existing roles/playbooks in random places which are very hard to detect (and it is not possible to show any specific warnings which places could break).

Resolving patch:
diff --git a/plugins/modules/system/xfconf.py b/plugins/modules/system/xfconf.py
--- a/plugins/modules/system/xfconf.py
+++ b/plugins/modules/system/xfconf.py
@@ -95,20 +95,28 @@
     type: str
     sample: "/Xft/DPI"
   value_type:
-    description: The type of the value that was changed
+    description:
+    - The type of the value that was changed (C(none) for C(get) and C(reset)
+      state). Either a single string value or a list of strings for array
+      types.
     returned: success
-    type: str
-    sample: "int"
+    type: string or list of strings
+    sample: '"int" or ["str", "str", "str"]'
   value:
-    description: The value of the preference key after executing the module
+    description:
+    - The value of the preference key after executing the module. Either a
+      single string value or a list of strings for array types.
     returned: success
-    type: str
-    sample: "192"
+    type: string or list of strings
+    sample: '"192" or ["orange", "yellow", "violet"]'
   previous_value:
-    description: The value of the preference key before executing the module (None for "get" state).
+    description:
+    - The value of the preference key before executing the module (C(none) for
+      C(get) state). Either a single string value or a list of strings for array
+      types.
     returned: success
-    type: str
-    sample: "96"
+    type: string or list of strings
+    sample: '"96" or ["red", "blue", "green"]'
 '''
 
 from ansible_collections.community.general.plugins.module_utils.module_helper import (
@@ -176,6 +184,9 @@ def __init_module__(self):
         self.does_not = 'Property "{0}" does not exist on channel "{1}".'.format(self.module.params['property'],
                                                                                  self.module.params['channel'])
         self.vars.previous_value = self._get()
+        self.update_xfconf_output(property=self.module.params['property'],
+                                  channel=self.module.params['channel'],
+                                  previous_value=None)
 
     def process_command_output(self, rc, out, err):
         if err.rstrip() == self.does_not:
@@ -205,10 +216,13 @@ def _get(self):
 
     def state_get(self):
         self.vars.value = self.vars.previous_value
+        self.update_xfconf_output(value=self.vars.value)
 
     def state_absent(self):
         self.vars.value = None
         self.run_command(params=('channel', 'property', 'reset'), extra_params={"reset": True})
+        self.update_xfconf_output(previous_value=self.vars.previous_value,
+                                  value=None)
 
     def state_present(self):
         # stringify all values - in the CLI they will all be happy strings anyway
@@ -249,6 +263,11 @@ def state_present(self):
 
         if not self.vars.is_array:
             self.vars.value = self.vars.value[0]
+            value_type = value_type[0]
+
+        self.update_xfconf_output(previous_value=self.vars.previous_value,
+                                  value=self.vars.value,
+                                  type=value_type)
 
 
 def main():


Test patch:
diff --git a/tests/sanity/ignore-2.10.txt b/tests/sanity/ignore-2.10.txt
--- a/tests/sanity/ignore-2.10.txt
+++ b/tests/sanity/ignore-2.10.txt
@@ -552,6 +552,7 @@ plugins/modules/system/runit.py validate-modules:parameter-type-not-in-doc
 plugins/modules/system/runit.py validate-modules:undocumented-parameter
 plugins/modules/system/timezone.py pylint:blacklisted-name
 plugins/modules/system/xfconf.py validate-modules:parameter-state-invalid-choice
+plugins/modules/system/xfconf.py validate-modules:return-syntax-error
 plugins/modules/web_infrastructure/jenkins_plugin.py use-argspec-type-path
 plugins/modules/web_infrastructure/rundeck_acl_policy.py pylint:blacklisted-name
 scripts/inventory/gce.py pylint:blacklisted-name
diff --git a/tests/sanity/ignore-2.11.txt b/tests/sanity/ignore-2.11.txt
--- a/tests/sanity/ignore-2.11.txt
+++ b/tests/sanity/ignore-2.11.txt
@@ -552,6 +552,7 @@ plugins/modules/system/runit.py validate-modules:parameter-type-not-in-doc
 plugins/modules/system/runit.py validate-modules:undocumented-parameter
 plugins/modules/system/timezone.py pylint:blacklisted-name
 plugins/modules/system/xfconf.py validate-modules:parameter-state-invalid-choice
+plugins/modules/system/xfconf.py validate-modules:return-syntax-error
 plugins/modules/web_infrastructure/jenkins_plugin.py use-argspec-type-path
 plugins/modules/web_infrastructure/rundeck_acl_policy.py pylint:blacklisted-name
 scripts/inventory/gce.py pylint:blacklisted-name
diff --git a/tests/sanity/ignore-2.9.txt b/tests/sanity/ignore-2.9.txt
--- a/tests/sanity/ignore-2.9.txt
+++ b/tests/sanity/ignore-2.9.txt
@@ -480,6 +480,7 @@ plugins/modules/system/runit.py validate-modules:doc-default-does-not-match-spec
 plugins/modules/system/runit.py validate-modules:parameter-type-not-in-doc
 plugins/modules/system/runit.py validate-modules:undocumented-parameter
 plugins/modules/system/timezone.py pylint:blacklisted-name
+plugins/modules/system/xfconf.py validate-modules:return-syntax-error
 plugins/modules/web_infrastructure/jenkins_plugin.py use-argspec-type-path
 plugins/modules/web_infrastructure/nginx_status_facts.py validate-modules:deprecation-mismatch
 plugins/modules/web_infrastructure/nginx_status_facts.py validate-modules:invalid-documentation
diff --git a/tests/unit/plugins/modules/system/test_xfconf.py b/tests/unit/plugins/modules/system/test_xfconf.py
--- a/tests/unit/plugins/modules/system/test_xfconf.py
+++ b/tests/unit/plugins/modules/system/test_xfconf.py
@@ -55,7 +55,8 @@ def test_without_required_parameters(capfd, patch_xfconf):
                 ),
             ],
             'changed': False,
-            'previous_value': '100',
+            'previous_value': None,
+            'value_type': None,
             'value': '100',
         }
     ],
@@ -75,6 +76,7 @@ def test_without_required_parameters(capfd, patch_xfconf):
             ],
             'changed': False,
             'previous_value': None,
+            'value_type': None,
             'value': None,
         }
     ],
@@ -93,7 +95,8 @@ def test_without_required_parameters(capfd, patch_xfconf):
                 ),
             ],
             'changed': False,
-            'previous_value': ['Main', 'Work', 'Tmp'],
+            'previous_value': None,
+            'value_type': None,
             'value': ['Main', 'Work', 'Tmp'],
         },
     ],
@@ -112,7 +115,8 @@ def test_without_required_parameters(capfd, patch_xfconf):
                 ),
             ],
             'changed': False,
-            'previous_value': 'true',
+            'previous_value': None,
+            'value_type': None,
             'value': 'true',
         },
     ],
@@ -131,7 +135,8 @@ def test_without_required_parameters(capfd, patch_xfconf):
                 ),
             ],
             'changed': False,
-            'previous_value': 'false',
+            'previous_value': None,
+            'value_type': None,
             'value': 'false',
         },
     ],
@@ -166,6 +171,7 @@ def test_without_required_parameters(capfd, patch_xfconf):
             ],
             'changed': True,
             'previous_value': '100',
+            'value_type': 'int',
             'value': '90',
         },
     ],
@@ -200,6 +206,7 @@ def test_without_required_parameters(capfd, patch_xfconf):
             ],
             'changed': False,
             'previous_value': '90',
+            'value_type': 'int',
             'value': '90',
         },
     ],
@@ -235,6 +242,7 @@ def test_without_required_parameters(capfd, patch_xfconf):
             ],
             'changed': True,
             'previous_value': ['Main', 'Work', 'Tmp'],
+            'value_type': ['str', 'str', 'str'],
             'value': ['A', 'B', 'C'],
         },
     ],
@@ -270,6 +278,7 @@ def test_without_required_parameters(capfd, patch_xfconf):
             ],
             'changed': False,
             'previous_value': ['A', 'B', 'C'],
+            'value_type': ['str', 'str', 'str'],
             'value': ['A', 'B', 'C'],
         },
     ],
@@ -302,6 +311,7 @@ def test_without_required_parameters(capfd, patch_xfconf):
             ],
             'changed': True,
             'previous_value': ['A', 'B', 'C'],
+            'value_type': None,
             'value': None,
         },
     ],
@@ -333,14 +343,20 @@ def test_xfconf(mocker, capfd, patch_xfconf, testcase):
     results = json.loads(out)
     print("testcase =\n%s" % testcase)
     print("results =\n%s" % results)
+
     assert 'changed' in results
     assert results['changed'] == testcase['changed']
-    if 'msg' in results:
-        assert results.get('msg') == testcase['msg']
-    if 'value' in results:
-        assert results['value'] == testcase['value']
-    if 'previous_value' in results:
-        assert results['previous_value'] == testcase['previous_value']
+
+    for test_result in ('channel', 'property'):
+        assert test_result in results, "'{0}' not found in {1}".format(test_result, results)
+        assert results[test_result] == results['invocation']['module_args'][test_result], \
+            "'{0}': '{1}' != '{2}'".format(test_result, results[test_result], results['invocation']['module_args'][test_result])
+
+    for conditional_test_result in ('msg', 'value', 'previous_value'):
+        if conditional_test_result in testcase:
+            assert conditional_test_result in results, "'{0}' not found in {1}".format(conditional_test_result, results)
+            assert results[conditional_test_result] == testcase[conditional_test_result], \
+                "'{0}': '{1}' != '{2}'".format(conditional_test_result, results[conditional_test_result], testcase[conditional_test_result])
 
     assert mock_run_command.call_count == len(testcase['run_command.calls'])
     if mock_run_command.call_count: