Repository: ansible-collections/community.general

Issue:
using YAML callback plugin changes multiline behavior of to_yaml filter
### Summary

Using the `to_yaml` or `to_nice_yaml` filter returns differences in multi-line styling whether ANSIBLE_STDOUT_CALLBACK=community.general.yaml is used or not

### Issue Type

Bug Report

### Component Name

to_yaml
to_nice_yaml
community.general.yaml
plugins/callback/yaml.py
plugins/callback/yaml

### Ansible Version

```console (paste below)
$ ansible --version
ansible [core 2.11.1]
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.8/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.8.10 (default, May  6 2021, 00:05:59) [GCC 10.2.1 20201203]
  jinja version = 3.0.1
  libyaml = False
```


### Community.general Version

```console (paste below)
$ ansible-galaxy collection list community.general
# /root/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 3.7.0

# /usr/lib/python3.8/site-packages/ansible_collections
Collection        Version
----------------- -------
community.general 3.2.0
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed
# output is empty
```


### OS / Environment

```
# I believe this is irrelevant
$ uname -a
Linux bip.ly.lan 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
$ docker pull cytopia/ansible                                                                                                                                                                                                                                                                                                 Using default tag: latest
latest: Pulling from cytopia/ansible
Digest: sha256:133e030551e1c453422b68b1fc24bfdfab501f19331363b359d97f8c18796744
Status: Image is up to date for cytopia/ansible:latest
docker.io/cytopia/ansible:latest
```

### Steps to Reproduce



<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
# file test.yml
- hosts: localhost
  connection: local
  gather_facts: no
  vars:
    example:
      mykey: |
        line 1
        line 2
        line 3

  tasks:

    - copy:
        dest: /tmp/myfile.yaml
        content: "{{ example | to_yaml }}"
```

```bash
/data # ansible-playbook test.yml -D
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does
not match 'all'

PLAY [localhost] **********************************************************************************************

TASK [copy] ***************************************************************************************************
--- before
+++ after: /root/.ansible/tmp/ansible-local-1225mn3i25/tmpk5lu3cuj
@@ -0,0 +1,7 @@
+{mykey: 'line 1
+
+    line 2
+
+    line 3
+
+    '}

changed: [localhost]

PLAY RECAP ****************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

/data # ANSIBLE_STDOUT_CALLBACK=yaml ansible-playbook test.yml -D
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does
not match 'all'

PLAY [localhost] **********************************************************************************************

TASK [copy] ***************************************************************************************************
--- before: /tmp/myfile.yaml
+++ after: /root/.ansible/tmp/ansible-local-45ygiut1yl/tmpnp9d0nqa
@@ -1,7 +1,4 @@
-{mykey: 'line 1
-
-    line 2
-
-    line 3
-
-    '}
+mykey: |-
+  line 1
+  line 2
+  line 3

changed: [localhost]

PLAY RECAP ****************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```

### Expected Results

changing call back should have no effect on actual behavior of a task / filter

### Actual Results

switching stdout call back causes the content of the generated file to be modified

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/cache/yaml.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/cache/yaml.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @bcoca
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Files identified in the description:
* [`plugins/cache/yaml.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/cache/yaml.py)
* [`plugins/callback/yaml.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/callback/yaml.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
as an external viewer, this seems close to #1519
!component =plugins/callback/yaml.py
I think this is mainly related to https://github.com/ansible-collections/community.general/blob/main/plugins/callback/yaml.py#L82 - the callback modifies the way PyYAML does formatting, which seeps through to all other uses in PyYAML in Ansible controller.
Files identified in the description:
* [`plugins/callback/yaml.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/callback/yaml.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Resolving patch:
diff --git a/plugins/callback/yaml.py b/plugins/callback/yaml.py
--- a/plugins/callback/yaml.py
+++ b/plugins/callback/yaml.py
@@ -42,28 +42,29 @@ def should_use_block(value):
     return False
 
 
-def my_represent_scalar(self, tag, value, style=None):
-    """Uses block style for multi-line strings"""
-    if style is None:
-        if should_use_block(value):
-            style = '|'
-            # we care more about readable than accuracy, so...
-            # ...no trailing space
-            value = value.rstrip()
-            # ...and non-printable characters
-            value = ''.join(x for x in value if x in string.printable or ord(x) >= 0xA0)
-            # ...tabs prevent blocks from expanding
-            value = value.expandtabs()
-            # ...and odd bits of whitespace
-            value = re.sub(r'[\x0b\x0c\r]', '', value)
-            # ...as does trailing space
-            value = re.sub(r' +\n', '\n', value)
-        else:
-            style = self.default_style
-    node = yaml.representer.ScalarNode(tag, value, style=style)
-    if self.alias_key is not None:
-        self.represented_objects[self.alias_key] = node
-    return node
+class MyDumper(AnsibleDumper):
+    def represent_scalar(self, tag, value, style=None):
+        """Uses block style for multi-line strings"""
+        if style is None:
+            if should_use_block(value):
+                style = '|'
+                # we care more about readable than accuracy, so...
+                # ...no trailing space
+                value = value.rstrip()
+                # ...and non-printable characters
+                value = ''.join(x for x in value if x in string.printable or ord(x) >= 0xA0)
+                # ...tabs prevent blocks from expanding
+                value = value.expandtabs()
+                # ...and odd bits of whitespace
+                value = re.sub(r'[\x0b\x0c\r]', '', value)
+                # ...as does trailing space
+                value = re.sub(r' +\n', '\n', value)
+            else:
+                style = self.default_style
+        node = yaml.representer.ScalarNode(tag, value, style=style)
+        if self.alias_key is not None:
+            self.represented_objects[self.alias_key] = node
+        return node
 
 
 class CallbackModule(Default):
@@ -79,7 +80,6 @@ class CallbackModule(Default):
 
     def __init__(self):
         super(CallbackModule, self).__init__()
-        yaml.representer.BaseRepresenter.represent_scalar = my_represent_scalar
 
     def _dump_results(self, result, indent=None, sort_keys=True, keep_invocation=False):
         if result.get('_ansible_no_log', False):
@@ -121,7 +121,7 @@ def _dump_results(self, result, indent=None, sort_keys=True, keep_invocation=Fal
 
         if abridged_result:
             dumped += '\n'
-            dumped += to_text(yaml.dump(abridged_result, allow_unicode=True, width=1000, Dumper=AnsibleDumper, default_flow_style=False))
+            dumped += to_text(yaml.dump(abridged_result, allow_unicode=True, width=1000, Dumper=MyDumper, default_flow_style=False))
 
         # indent by a couple of spaces
         dumped = '\n  '.join(dumped.split('\n')).rstrip()


Test patch:
diff --git a/tests/integration/targets/callback_yaml/tasks/main.yml b/tests/integration/targets/callback_yaml/tasks/main.yml
--- a/tests/integration/targets/callback_yaml/tasks/main.yml
+++ b/tests/integration/targets/callback_yaml/tasks/main.yml
@@ -58,3 +58,40 @@
           "PLAY RECAP *********************************************************************",
           "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
+      - name: Test to_yaml
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.yaml
+        playbook: |
+          - hosts: testhost
+            gather_facts: false
+            vars:
+              data: |
+                line 1
+                line 2
+                line 3
+            tasks:
+              - name: Test to_yaml
+                debug:
+                  msg: "{{ '{{' }}'{{ '{{' }}'{{ '}}' }} data | to_yaml {{ '{{' }}'{{ '}}' }}'{{ '}}' }}"
+        # The above should be:   msg: "{{ data | to_yaml }}"
+        # Unfortunately, the way Ansible handles templating, we need to do some funny 'escaping' tricks...
+        expected_output: [
+          "",
+          "PLAY [testhost] ****************************************************************",
+          "",
+          "TASK [Test to_yaml] ************************************************************",
+          "ok: [testhost] => ",
+          "  msg: |-",
+          "    'line 1",
+          "  ",
+          "      line 2",
+          "  ",
+          "      line 3",
+          "  ",
+          "      '",
+          "",
+          "PLAY RECAP *********************************************************************",
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+        ]