Repository: ansible-collections/community.general

Issue:
Issue with mutlliline yaml variables
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
It seems to me that the following issues are still presnet in Ansible 2.10 (tested on 2.10.4)
- https://github.com/ansible/ansible/issues/62841
- https://github.com/ansible/ansible/issues/65843

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
YAML callback

##### ANSIBLE VERSION
```
ansible 2.10.4
  config file = $HOME/.ansible.cfg
  configured module search path = ['$HOME/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible
  executable location = /Library/Frameworks/Python.framework/Versions/3.8/bin/ansible
  python version = 3.8.3 (v3.8.3:6f8c8320e9, May 13 2020, 16:29:34) [Clang 6.0 (clang-600.0.57)]
```

##### CONFIGURATION
```
DEFAULT_STDOUT_CALLBACK($HOME/.ansible.cfg) = yaml
DEFAULT_TIMEOUT($HOME/.ansible.cfg) = 60
DEFAULT_VAULT_PASSWORD_FILE($HOME/.ansible.cfg) = /some/path/to/My-Keystore
```

##### OS / ENVIRONMENT
MacOS

##### STEPS TO REPRODUCE

Sample to reproduce :

`inventories/dev/group_vars/all.yml`:
```yaml
accent_issue:
  ko: 'Cette chaîne avec un saut de ligne ne contient pas d''accents (éèê)

  Quel dommage'
  ko_2: "Cette chaîne avec un saut de ligne ne contient pas d'accents (éèê)\nQuel dommage"
  ko_3: |-
    Cette chaîne avec un saut de ligne ne contient pas d'accents (éèê)`
    Quel dommage
  ok: 'Cette chaîne contient bien des accents (éèê)'
```

Sample playbook:
```yaml
---
- hosts: 127.0.0.1
  tasks:
  - debug:
      var: accent_issue
```

##### EXPECTED RESULTS
From given steps, I expect accents to be present in each variable.

##### ACTUAL RESULTS
Output - we can see that for each `ko*`param, accents are removed

```
ansible-playbook -i environments/dev/ monitor-debug.yml
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

PLAY [127.0.0.1] ********************************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1]

TASK [debug] ************************************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] =>
  accent_issue:
    ko: |-
      Cette chane avec un saut de ligne ne contient pas d'accents ()
      Quel dommage
    ko_2: |-
      Cette chane avec un saut de ligne ne contient pas d'accents ()
      Quel dommage
    ko_3: |-
      Cette chane avec un saut de ligne ne contient pas d'accents ()`
      Quel dommage
    ok: Cette chaîne contient bien des accents (éèê)

PLAY RECAP **************************************************************************************************************************************************************************************************************************************************
127.0.0.1                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
I don't think this is related to the community.general collection at all. You need to create an issue in https://github.com/ansible/ansible/.
Thank you for you feedback @felixfontein. Mentionned issues have been closed from ansible/ansible, pointing to this repo.
I will create a new one though.
@geoffroya you mentioned nowhere in this issue that this is about the YAML callback, and not about Ansible itself. If it is indeed about the YAML callback, this is the right repository. You should mention that in the issue though.
@felixfontein, well it's true I didn't mention it, but isn't it weird to have to know which piece of code is responsible for a given bug, to be able to fill this bug?
Files identified in the description:
* [`plugins/callback/yaml.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/callback/yaml.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
It's pretty clear that the fault is somehow with the YAML callback plugin, or at least related to it, since it works fine with the standard callback plugin. The accents are only missing with the YAML callback plugin.
I just played with it a bit, this is the line that removes the accents: https://github.com/ansible-collections/community.general/blob/main/plugins/callback/yaml.py#L53

Replacing the line with
```.py
value = ''.join(x for x in value if x in string.printable or ord(x) >= 0xA0)
```
seems to fix the problem. I guess the original author did not knew that `string.printable` restricts to ASCII, and not to printable Unicode. Simply accepting all characters with code 0xA0 or higher that are not in `string.printable` should be fine (see https://en.wikipedia.org/wiki/Unicode_control_characters).
resolved_by_pr #1522

Resolving patch:
diff --git a/plugins/callback/yaml.py b/plugins/callback/yaml.py
--- a/plugins/callback/yaml.py
+++ b/plugins/callback/yaml.py
@@ -50,7 +50,7 @@ def my_represent_scalar(self, tag, value, style=None):
             # ...no trailing space
             value = value.rstrip()
             # ...and non-printable characters
-            value = ''.join(x for x in value if x in string.printable)
+            value = ''.join(x for x in value if x in string.printable or ord(x) >= 0xA0)
             # ...tabs prevent blocks from expanding
             value = value.expandtabs()
             # ...and odd bits of whitespace


Test patch:
diff --git a/tests/integration/targets/callback_diy/inventory.yml b/tests/integration/targets/callback/inventory.yml
similarity index 100%
rename from tests/integration/targets/callback_diy/inventory.yml
rename to tests/integration/targets/callback/inventory.yml
diff --git a/tests/integration/targets/callback/tasks/main.yml b/tests/integration/targets/callback/tasks/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/callback/tasks/main.yml
@@ -0,0 +1,96 @@
+---
+####################################################################
+# WARNING: These are designed specifically for Ansible tests       #
+# and should not be used as examples of how to write Ansible roles #
+####################################################################
+
+- block:
+  - name: Create temporary playbook files
+    tempfile:
+      state: file
+      suffix: temp
+    loop: "{{ tests }}"
+    loop_control:
+      loop_var: test
+      label: "{{ test.name }}"
+    register: temporary_playbook_files
+
+  - name: Set temporary playbook file content
+    copy:
+      content: "{{ test.playbook }}"
+      dest: "{{ temporary_playbook_files.results[test_idx].path }}"
+    loop: "{{ tests }}"
+    loop_control:
+      loop_var: test
+      index_var: test_idx
+      label: "{{ test.name }}"
+
+  - name: Collect outputs
+    command: "ansible-playbook -i {{ inventory }} {{ playbook }}"
+    environment: "{{ test.environment }}"
+    loop: "{{ tests }}"
+    loop_control:
+      loop_var: test
+      label: "{{ test.name }}"
+    register: outputs
+    changed_when: false
+    vars:
+      inventory: "{{ role_path }}/inventory.yml"
+      playbook: "
+        {%- for result in temporary_playbook_files.results -%}
+        {%-   if result.test.name == test.name -%}
+        {{-     result.path -}}
+        {%-   endif -%}
+        {%- endfor -%}"
+
+  - name: Assert test output equals expected output
+    assert:
+      that: result.output.differences | length == 0
+    loop: "{{ results }}"
+    loop_control:
+      loop_var: result
+      label: "{{ result.name }}"
+    register: assertions
+    vars:
+      results: >-
+        {%- set results = []                                                                           -%}
+        {%- for result in outputs.results                                                              -%}
+        {%-   set differences = []                                                                     -%}
+        {%-   for i in range([result.test.expected_output | count, result.stdout_lines | count] | max) -%}
+        {%-     set line = "line_%s" | format(i+1)                                                     -%}
+        {%-     set test_line = result.stdout_lines[i] | default(none)                                 -%}
+        {%-     set expected_lines = result.test.expected_output[i] | default(none)                    -%}
+        {%-     if expected_lines is not string and expected_lines is not none                         -%}
+        {%-       if test_line not in expected_lines                                                   -%}
+        {{-         differences.append({
+                      line: {
+                        'expected_one_of': expected_lines,
+                        'got': test_line }})                                                           -}}
+        {%-       endif                                                                                -%}
+        {%-     else                                                                                   -%}
+        {%-       if expected_lines != test_line                                                       -%}
+        {{-         differences.append({
+                      line: {
+                        'expected': expected_lines,
+                        'got': test_line }})                                                           -}}
+        {%-       endif                                                                                -%}
+        {%-     endif                                                                                  -%}
+        {%-   endfor                                                                                   -%}
+        {{-   results.append({
+                'name': result.test.name,
+                'output': {
+                    'differences': differences,
+                    'expected': result.test.expected_output,
+                    'got': result.stdout_lines }})                                                     -}}
+        {%- endfor                                                                                     -%}
+        {{- results                                                                                    -}}
+
+  always:
+  - name: Remove temporary playbooks
+    file:
+      path: "{{ temporary_file.path }}"
+      state: absent
+    loop: "{{ temporary_playbook_files.results }}"
+    loop_control:
+      loop_var: temporary_file
+      label: "{{ temporary_file.test.name }}: {{ temporary_file.path }}"
diff --git a/tests/integration/targets/callback_diy/aliases b/tests/integration/targets/callback_diy/aliases
--- a/tests/integration/targets/callback_diy/aliases
+++ b/tests/integration/targets/callback_diy/aliases
@@ -1,2 +1,2 @@
 shippable/posix/group3
-skip/aix
+needs/target/callback
diff --git a/tests/integration/targets/callback_diy/handlers/main.yml b/tests/integration/targets/callback_diy/handlers/main.yml
deleted file mode 100644
--- a/tests/integration/targets/callback_diy/handlers/main.yml
+++ /dev/null
@@ -1,9 +0,0 @@
-  
-- name: Remove temporary playbooks
-  file:
-    path: "{{ temporary_file.path }}"
-    state: absent
-  loop: "{{ temporary_playbook_files.results }}"
-  loop_control:
-    loop_var: temporary_file
-    label: "{{ temporary_file.test.name }}: {{ temporary_file.path }}"
diff --git a/tests/integration/targets/callback_diy/tasks/main.yml b/tests/integration/targets/callback_diy/tasks/main.yml
--- a/tests/integration/targets/callback_diy/tasks/main.yml
+++ b/tests/integration/targets/callback_diy/tasks/main.yml
@@ -3,13 +3,16 @@
 # and should not be used as examples of how to write Ansible roles #
 ####################################################################
 
-- name: Set tests
-  set_fact:
+- name: Run tests
+  include_role:
+    name: callback
+  vars:
     tests:
       - name: Not using diy callback options
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
         playbook: |
           - hosts: testhost
             gather_facts: false
@@ -22,19 +25,20 @@
           "PLAY [testhost] ****************************************************************",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set playbook_on_start_msg callback using environment variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-          ANSIBLE_CALLBACK_DIY_PLAYBOOK_ON_START_MSG="Sample output Sample playbook message"
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+          ANSIBLE_CALLBACK_DIY_PLAYBOOK_ON_START_MSG: "Sample output Sample playbook message"
         playbook: |
           - hosts: testhost
             gather_facts: false
@@ -48,24 +52,25 @@
           "PLAY [testhost] ****************************************************************",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set playbook_on_play_start_msg callback using play variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - name: Sample play name
             hosts: testhost
             gather_facts: false
             vars:
-              ansible_callback_diy_playbook_on_play_start_msg: Sample output \{\{ ansible_callback_diy.play.name \}\}
+              ansible_callback_diy_playbook_on_play_start_msg: Sample output {{ ansible_callback_diy.play.name }}
             tasks:
               - name: Sample task name
                 debug:
@@ -74,23 +79,24 @@
           "Sample output Sample play name",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set playbook_on_task_start_msg callback using play variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             vars:
-              ansible_callback_diy_playbook_on_task_start_msg: Sample output \{\{ ansible_callback_diy.task.name \}\}
+              ansible_callback_diy_playbook_on_task_start_msg: Sample output {{ ansible_callback_diy.task.name }}
             tasks:
               - name: Sample task name
                 debug:
@@ -99,19 +105,20 @@
           "",
           "PLAY [testhost] ****************************************************************",
           "Sample output Sample task name",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set playbook_on_task_start_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
@@ -119,24 +126,25 @@
                 debug:
                   msg: sample debug msg
                 vars:
-                  ansible_callback_diy_playbook_on_task_start_msg: Sample output \{\{ ansible_callback_diy.task.name \}\}
+                  ansible_callback_diy_playbook_on_task_start_msg: Sample output {{ ansible_callback_diy.task.name }}
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
           "Sample output Sample task name",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set runner_on_ok_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
@@ -144,7 +152,7 @@
                 debug:
                   msg: sample debug msg
                 vars:
-                  ansible_callback_diy_runner_on_ok_msg: Sample output \{\{ ansible_callback_diy.result.output.msg \}\}
+                  ansible_callback_diy_runner_on_ok_msg: Sample output {{ ansible_callback_diy.result.output.msg }}
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
@@ -153,13 +161,14 @@
           "Sample output sample debug msg",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set runner_on_failed_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
         playbook: |
           - hosts: testhost
             gather_facts: false
@@ -179,14 +188,15 @@
           "Sample output Sample failure message",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    \u001b[1;35mignored=1   \u001b[0m"
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1   "
         ]
 
       - name: Set runner_on_skipped_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
@@ -195,7 +205,7 @@
                   msg: sample debug msg
                 when: false
                 vars:
-                  ansible_callback_diy_runner_on_skipped_msg: Sample output Skipped \{\{ ansible_callback_diy.task.name \}\}
+                  ansible_callback_diy_runner_on_skipped_msg: Sample output Skipped {{ ansible_callback_diy.task.name }}
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
@@ -204,26 +214,27 @@
           "Sample output Skipped Sample task name",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : ok=0    changed=0    unreachable=0    failed=0    \u001b[0;36mskipped=1   \u001b[0m rescued=0    ignored=0   "
+          "testhost                   : ok=0    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   "
         ]
 
       - name: Set runner_item_on_ok_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
               - name: Sample task name
                 debug:
-                  msg: sample debug msg \{\{ item \}\}
+                  msg: sample debug msg {{ item }}
                 loop:
                   - sample item 1
                   - sample item 2
                   - sample item 3
                 vars:
-                  ansible_callback_diy_runner_item_on_ok_msg: Sample output Looping \{\{ ansible_callback_diy.result.output.msg \}\}
+                  ansible_callback_diy_runner_item_on_ok_msg: Sample output Looping {{ ansible_callback_diy.result.output.msg }}
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
@@ -234,20 +245,21 @@
           "Sample output Looping sample debug msg sample item 3",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set runner_item_on_failed_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
               - name: Sample task name
                 debug:
-                  msg: sample debug msg \{\{ item \}\}
+                  msg: sample debug msg {{ item }}
                 loop:
                   - sample item 1
                   - sample item 2
@@ -261,93 +273,95 @@
           "PLAY [testhost] ****************************************************************",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;32mok: [testhost] => (item=sample item 1) => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg sample item 1\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => (item=sample item 1) => {",
+          "    \"msg\": \"sample debug msg sample item 1\"",
+          "}",
           "Sample output Looping sample failure message",
-          "\u001b[0;32mok: [testhost] => (item=sample item 3) => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg sample item 3\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => (item=sample item 3) => {",
+          "    \"msg\": \"sample debug msg sample item 3\"",
+          "}",
           [
             # Apparently a bug was fixed in Ansible, as before it ran through with "All items completed"
-            "\u001b[0;31mfatal: [testhost]: FAILED! => {\"msg\": \"All items completed\"}\u001b[0m",
-            "\u001b[0;31mfatal: [testhost]: FAILED! => {\"msg\": \"One or more items failed\"}\u001b[0m",
+            "fatal: [testhost]: FAILED! => {\"msg\": \"All items completed\"}",
+            "fatal: [testhost]: FAILED! => {\"msg\": \"One or more items failed\"}",
           ],
-          "\u001b[0;36m...ignoring\u001b[0m",
+          "...ignoring",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    \u001b[1;35mignored=1   \u001b[0m"
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=1   "
         ]
 
       - name: Set runner_item_on_skipped_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
               - name: Sample task name
                 debug:
-                  msg: sample debug msg \{\{ item \}\}
+                  msg: sample debug msg {{ item }}
                 loop:
                   - sample item 1
                   - sample item 2
                   - sample item 3
                 when: item != 'sample item 2'
                 vars:
-                  ansible_callback_diy_runner_item_on_skipped_msg: Sample output Looping Skipped \{\{ ansible_callback_diy.result.output.item \}\}
+                  ansible_callback_diy_runner_item_on_skipped_msg: Sample output Looping Skipped {{ ansible_callback_diy.result.output.item }}
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;32mok: [testhost] => (item=sample item 1) => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg sample item 1\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => (item=sample item 1) => {",
+          "    \"msg\": \"sample debug msg sample item 1\"",
+          "}",
           "Sample output Looping Skipped sample item 2",
-          "\u001b[0;32mok: [testhost] => (item=sample item 3) => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg sample item 3\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => (item=sample item 3) => {",
+          "    \"msg\": \"sample debug msg sample item 3\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set playbook_on_stats_msg callback using play variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             vars:
               ansible_callback_diy_playbook_on_stats_msg: |+2
                       Sample output stats
                 ===============================
-                {\% for key in ansible_callback_diy.stats | sort %\}
-                {\% set color_one = "" %\}
-                {\% set color_two = "" %\}
-                {\% if ansible_callback_diy.stats[key] %\}
-                {\% if key == 'ok' %\}
-                {\% set prefix = '      ' %\}
-                {\% set suffix = '     ' %\}
-                {\% elif key == 'changed' %\}
-                {\% set prefix = '   ' %\}
-                {\% set suffix = '   ' %\}
-                {\% elif key == 'processed' %\}
-                {\% set prefix = '  ' %\}
-                {\% set suffix = '  ' %\}
-                {\% elif key == 'skipped' %\}
-                {\% set prefix = '   ' %\}
-                {\% set suffix = '   ' %\}
-                {\% else %\}
-                {\% set prefix = "" %\}
-                {\% set suffix = "" %\}
-                {\% endif %\}
-                \{\{ color_one \}\}\{\{ "%s%s%s" | format(prefix,key,suffix) \}\}\{\{ color_two \}\}: \{\{ ansible_callback_diy.stats[key] | to_nice_yaml \}\}
-                {\% endif %\}
-                {\% endfor %\}
+                {% for key in ansible_callback_diy.stats | sort %}
+                {% set color_one = "" %}
+                {% set color_two = "" %}
+                {% if ansible_callback_diy.stats[key] %}
+                {% if key == 'ok' %}
+                {% set prefix = '      ' %}
+                {% set suffix = '     ' %}
+                {% elif key == 'changed' %}
+                {% set prefix = '   ' %}
+                {% set suffix = '   ' %}
+                {% elif key == 'processed' %}
+                {% set prefix = '  ' %}
+                {% set suffix = '  ' %}
+                {% elif key == 'skipped' %}
+                {% set prefix = '   ' %}
+                {% set suffix = '   ' %}
+                {% else %}
+                {% set prefix = "" %}
+                {% set suffix = "" %}
+                {% endif %}
+                {{ color_one }}{{ "%s%s%s" | format(prefix,key,suffix) }}{{ color_two }}: {{ ansible_callback_diy.stats[key] | to_nice_yaml }}
+                {% endif %}
+                {% endfor %}
             tasks:
               - name: Sample task name
                 debug:
@@ -357,9 +371,9 @@
           "PLAY [testhost] ****************************************************************",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "      Sample output stats",
           "===============================",
           "      ok     : testhost: 1",
@@ -368,9 +382,10 @@
         ]
 
       - name: Suppress output on playbook_on_task_start_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
         playbook: |
           - hosts: testhost
             gather_facts: false
@@ -383,18 +398,19 @@
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
-          "\u001b[0;32mok: [testhost] => {\u001b[0m",
-          "\u001b[0;32m    \"msg\": \"sample debug msg\"\u001b[0m",
-          "\u001b[0;32m}\u001b[0m",
+          "ok: [testhost] => {",
+          "    \"msg\": \"sample debug msg\"",
+          "}",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Suppress output on runner_on_ok_msg callback using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
         playbook: |
           - hosts: testhost
             gather_facts: false
@@ -411,14 +427,15 @@
           "TASK [Sample task name] ********************************************************",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
 
       - name: Set runner_on_ok_msg_color using task variable
-        environment: >-
-          ANSIBLE_FORCE_COLOR=True
-          ANSIBLE_STDOUT_CALLBACK=community.general.diy
-        playbook: |
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.diy
+        playbook: !unsafe |
           - hosts: testhost
             gather_facts: false
             tasks:
@@ -426,116 +443,15 @@
                 debug:
                   msg: sample debug msg
                 vars:
-                  ansible_callback_diy_runner_on_ok_msg: Sample output \{\{ ansible_callback_diy.result.output.msg \}\}
+                  ansible_callback_diy_runner_on_ok_msg: Sample output {{ ansible_callback_diy.result.output.msg }}
                   ansible_callback_diy_runner_on_ok_msg_color: blue
         expected_output: [
           "",
           "PLAY [testhost] ****************************************************************",
           "",
           "TASK [Sample task name] ********************************************************",
-          "\u001b[0;34mSample output sample debug msg\u001b[0m",
+          "Sample output sample debug msg",
           "",
           "PLAY RECAP *********************************************************************",
-          "\u001b[0;32mtesthost\u001b[0m                   : \u001b[0;32mok=1   \u001b[0m changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
         ]
-
-- when: ansible_distribution == 'FreeBSD' and ansible_distribution_version in ['11.1', '12.1']
-  name: Create temporary playbooks
-  block:
-    - name: Create temporary playbook files
-      tempfile:
-        state: file
-        suffix: temp
-      loop: "{{ tests }}"
-      loop_control:
-        loop_var: test
-        label: "{{ test.name }}"
-      register: temporary_playbook_files
-      notify: Remove temporary playbooks
-
-    - name: Set temporary playbook file content
-      copy:
-        content: "{{ playbook }}"
-        dest: "{{ playbook_file }}"
-      loop: "{{ tests }}"
-      loop_control:
-        loop_var: test
-        label: "{{ test.name }}"
-      vars:
-        playbook_file: "{{ (temporary_playbook_files.results | selectattr('test.name', 'equalto', test.name) | list())[0].path }}"
-        playbook: "{{
-          test.playbook
-          | regex_replace('(\\\\{){2}(.*?)(\\\\}){2}', '{{\\2}}')
-          | regex_replace('({\\\\%)(.*?)(%\\\\})', '{%\\2%}')
-        }}"
-
-- name: Collect outputs
-  shell: "{{ env }} ansible-playbook -i {{ inventory }} {{ playbook }}"
-  loop: "{{ tests }}"
-  loop_control:
-    loop_var: test
-    label: "{{ test.name }}"
-  register: outputs
-  changed_when: false
-  vars:
-    env: "{{ test.environment | regex_replace('(\\\\{){2}(.*?)(\\\\}){2}', '{{\\2}}') }}"
-    inventory: "{{ role_path }}/inventory.yml"
-    playbook: "
-      {%- if temporary_playbook_files is changed                                                                  -%}
-      {%-   for result in temporary_playbook_files.results                                                        -%}
-      {%-     if result.test.name == test.name                                                                    -%}
-      {{-       result.path                                                                                       -}}
-      {%-     endif                                                                                               -%}
-      {%-   endfor                                                                                                -%}
-      {%- else                                                                                                    -%}
-      {{-   [
-              '/dev/fd/0 <<EOF',
-              (
-                test.playbook
-                | regex_replace('(\\\\{){2}(.*?)(\\\\}){2}', '{{\\2}}')
-                | regex_replace('({\\\\%)(.*?)(%\\\\})', '{%\\2%}')
-              )
-            ] | join('\n')                                                                                        -}}
-      {%- endif                                                                                                   -%}"
-
-- name: Assert test output equals expected output
-  assert:
-    that: result.output.differences | length == 0
-  loop: "{{ results }}"
-  loop_control:
-    loop_var: result
-    label: "{{ result.name }}"
-  register: assertions
-  vars:
-    results: >-
-      {%- set results = []                                                                           -%}
-      {%- for result in outputs.results                                                              -%}
-      {%-   set differences = []                                                                     -%}
-      {%-   for i in range([result.test.expected_output | count, result.stdout_lines | count] | max) -%}
-      {%-     set line = "line_%s" | format(i+1)                                                     -%}
-      {%-     set test_line = result.stdout_lines[i] | default(none)                                 -%}
-      {%-     set expected_lines = result.test.expected_output[i] | default(none)                    -%}
-      {%-     if expected_lines is not string                                                        -%}
-      {%-       if test_line not in expected_lines                                                   -%}
-      {{-         differences.append({
-                    line: {
-                      'expected_one_of': expected_lines,
-                      'got': test_line }})                                                           -}}
-      {%-       endif                                                                                -%}
-      {%-     else                                                                                   -%}
-      {%-       if expected_lines != test_line                                                       -%}
-      {{-         differences.append({
-                    line: {
-                      'expected': expected_line,
-                      'got': test_line }})                                                           -}}
-      {%-       endif                                                                                -%}
-      {%-     endif                                                                                  -%}
-      {%-   endfor                                                                                   -%}
-      {{-   results.append({
-              'name': result.test.name,
-              'output': {
-                  'differences': differences,
-                  'expected': result.test.expected_output,
-                  'got': result.stdout_lines }})                                                     -}}
-      {%- endfor                                                                                     -%}
-      {{- results                                                                                    -}}
diff --git a/tests/integration/targets/callback_yaml/aliases b/tests/integration/targets/callback_yaml/aliases
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/callback_yaml/aliases
@@ -0,0 +1,2 @@
+shippable/posix/group1
+needs/target/callback
diff --git a/tests/integration/targets/callback_yaml/tasks/main.yml b/tests/integration/targets/callback_yaml/tasks/main.yml
new file mode 100644
--- /dev/null
+++ b/tests/integration/targets/callback_yaml/tasks/main.yml
@@ -0,0 +1,60 @@
+---
+####################################################################
+# WARNING: These are designed specifically for Ansible tests       #
+# and should not be used as examples of how to write Ansible roles #
+####################################################################
+
+- name: Run tests
+  include_role:
+    name: callback
+  vars:
+    tests:
+      - name: Basic run
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.yaml
+        playbook: |
+          - hosts: testhost
+            gather_facts: false
+            tasks:
+              - name: Sample task name
+                debug:
+                  msg: sample debug msg
+        expected_output: [
+          "",
+          "PLAY [testhost] ****************************************************************",
+          "",
+          "TASK [Sample task name] ********************************************************",
+          "ok: [testhost] => ",
+          "  msg: sample debug msg",
+          "",
+          "PLAY RECAP *********************************************************************",
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+        ]
+      - name: Test umlauts in multiline
+        environment:
+          ANSIBLE_NOCOLOR: 'true'
+          ANSIBLE_FORCE_COLOR: 'false'
+          ANSIBLE_STDOUT_CALLBACK: community.general.yaml
+        playbook: |
+          - hosts: testhost
+            gather_facts: false
+            tasks:
+              - name: Umlaut output
+                debug:
+                  msg: "äöü\néêè\nßï☺"
+        expected_output: [
+          "",
+          "PLAY [testhost] ****************************************************************",
+          "",
+          "TASK [Umlaut output] ***********************************************************",
+          "ok: [testhost] => ",
+          "  msg: |-",
+          "    äöü",
+          "    éêè",
+          "    ßï☺",
+          "",
+          "PLAY RECAP *********************************************************************",
+          "testhost                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   "
+        ]