Repository: ansible/ansible
Pull request: Add support for unicode in ansible-inventory CLI
##### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
Fixes #57378
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
- Bugfix Pull Request

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below -->
`lib/ansible/cli/inventory.py`

Problem statement:
ansible-inventory doesn’t use unicode
##### SUMMARY
Unicode content is output in a cryptic escaped form.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
ansible-inventory

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.8.0
  config file = /var/home/frank/Projekte/Ansible/ansible.cfg
  configured module search path = ['/var/home/frank/Projekte/Ansible/library']
  ansible python module location = /usr/lib/python3.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.7.3 (default, May 11 2019, 00:38:04) [GCC 9.1.1 20190503 (Red Hat 9.1.1-1)]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
DEFAULT_HOST_LIST(/var/home/frank/Projekte/Ansible/ansible.cfg) = ['/var/home/frank/Projekte/Ansible/hosts']
DEFAULT_MODULE_PATH(/var/home/frank/Projekte/Ansible/ansible.cfg) = ['/var/home/frank/Projekte/Ansible/library']
```

##### OS / ENVIRONMENT
Fedora Silveblue 30


##### STEPS TO REPRODUCE

Put the example line `aäoöuüsß ansible_host=<FQDN>` in the hosts file and a variable like `ssh_öff_schlüssel: "/var/home/<USER>/.ssh/id_ed25519.pub"` in `group_ars/all/something.yaml`.


##### EXPECTED RESULTS
I’d expect correct a unicode presentation.

##### ACTUAL RESULTS
```yaml
ansible-inventory --list
{
    "_meta": {
        "hostvars": {
            "a\u00e4o\u00f6u\u00fcs\u00df": {
                "ansible_host": "<FQDN>",
                "nutzerkonto": "<USER>",
                "ssh_\u00f6ff_schl\u00fcssel": "/var/home/<USER>/.ssh/id_ed25519.pub"
            },
        }
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    },
    "ungrouped": {
        "hosts": [
            "a\u00e4o\u00f6u\u00fcs\u00df"
        ]
    }
}
```

Hints from issue discussion:
Files identified in the description:
* [`lib/ansible/cli/inventory.py`](https://github.com/ansible/ansible/blob/devel/lib/ansible/cli/inventory.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Commit message: First test fix

Patch to review:
diff --git a/changelogs/fragments/57378-inventory-cli-unicode.yml b/changelogs/fragments/57378-inventory-cli-unicode.yml
new file mode 100644
index 00000000000000..50d1d50345956f
--- /dev/null
+++ b/changelogs/fragments/57378-inventory-cli-unicode.yml
@@ -0,0 +1,2 @@
+bugfixes:
+  - Add unicode support to ``ansible-inventory`` CLI (https://github.com/ansible/ansible/issues/57378)
diff --git a/lib/ansible/cli/inventory.py b/lib/ansible/cli/inventory.py
index 135fe1fa974482..c5fdea1afa896a 100644
--- a/lib/ansible/cli/inventory.py
+++ b/lib/ansible/cli/inventory.py
@@ -15,7 +15,7 @@
 from ansible.cli import CLI
 from ansible.cli.arguments import option_helpers as opt_help
 from ansible.errors import AnsibleError, AnsibleOptionsError
-from ansible.module_utils._text import to_bytes, to_native
+from ansible.module_utils._text import to_bytes, to_native, to_text
 from ansible.utils.vars import combine_vars
 from ansible.utils.display import Display
 from ansible.vars.plugins import get_vars_from_inventory_sources, get_vars_from_path
@@ -172,7 +172,7 @@ def dump(stuff):
         if context.CLIARGS['yaml']:
             import yaml
             from ansible.parsing.yaml.dumper import AnsibleDumper
-            results = yaml.dump(stuff, Dumper=AnsibleDumper, default_flow_style=False)
+            results = to_text(yaml.dump(stuff, Dumper=AnsibleDumper, default_flow_style=False, allow_unicode=True))
         elif context.CLIARGS['toml']:
             from ansible.plugins.inventory.toml import toml_dumps, HAS_TOML
             if not HAS_TOML:
@@ -192,9 +192,9 @@ def dump(stuff):
             import json
             from ansible.parsing.ajson import AnsibleJSONEncoder
             try:
-                results = json.dumps(stuff, cls=AnsibleJSONEncoder, sort_keys=True, indent=4, preprocess_unsafe=True)
+                results = json.dumps(stuff, cls=AnsibleJSONEncoder, sort_keys=True, indent=4, preprocess_unsafe=True, ensure_ascii=False)
             except TypeError as e:
-                results = json.dumps(stuff, cls=AnsibleJSONEncoder, sort_keys=False, indent=4, preprocess_unsafe=True)
+                results = json.dumps(stuff, cls=AnsibleJSONEncoder, sort_keys=False, indent=4, preprocess_unsafe=True, ensure_ascii=False)
                 display.warning("Could not sort JSON output due to issues while sorting keys: %s" % to_native(e))
 
         return results
diff --git a/test/integration/targets/ansible-inventory/files/unicode.yml b/test/integration/targets/ansible-inventory/files/unicode.yml
new file mode 100644
index 00000000000000..ff95db0d4cf4ab
--- /dev/null
+++ b/test/integration/targets/ansible-inventory/files/unicode.yml
@@ -0,0 +1,3 @@
+all:
+  hosts:
+    příbor:
diff --git a/test/integration/targets/ansible-inventory/tasks/main.yml b/test/integration/targets/ansible-inventory/tasks/main.yml
index 7ec924f21fea02..0178ae85096e75 100644
--- a/test/integration/targets/ansible-inventory/tasks/main.yml
+++ b/test/integration/targets/ansible-inventory/tasks/main.yml
@@ -105,3 +105,30 @@
     that:
         - result is failed
         - '"ERROR! The source inventory contains a non-string key" in result.stderr'
+
+- name: "test json output with unicode characters"
+  command: ansible-inventory --list -i {{ role_path }}/files/unicode.yml
+  register: result
+
+- assert:
+    that:
+        - result is succeeded
+        - result.stdout is contains('příbor')
+
+- name: "test yaml output with unicode characters"
+  command: ansible-inventory --list --yaml -i {{ role_path }}/files/unicode.yml
+  register: result
+
+- assert:
+    that:
+        - result is succeeded
+        - result.stdout is contains('příbor')
+
+- name: "test toml output with unicode characters"
+  command: ansible-inventory --list --toml -i {{ role_path }}/files/unicode.yml
+  register: result
+
+- assert:
+    that:
+        - result is succeeded
+        - result.stdout is contains('příbor')


Code review comments:
diff_hunk: @@ -105,3 +105,30 @@
     that:
         - result is failed
         - '"ERROR! The source inventory contains a non-string key" in result.stderr'
+
+- name: "test json output with unicode characters"
+  command: ansible-inventory --list -i {{ role_path }}/files/unicode.yml
+  register: result
+
+- assert:
+    that:
+        - result is succeeded
+        - result.stdout is contains('příbor')
+
+- name: "test yaml output with unicode characters"
+  command: ansible-inventory --list --yaml -i {{ role_path }}/files/unicode.yml
+  register: result
+
+- assert:
+    that:
+        - result is succeeded
+        - result.stdout is contains('příbor')
+
+- name: "test toml output with unicode characters"
+  command: ansible-inventory --list --toml -i {{ role_path }}/files/unicode.yml
+  register: result
+
+- assert:
+    that:
+        - result is succeeded
+        - result.stdout is contains('příbor')
path: test/integration/targets/ansible-inventory/tasks/main.yml
text: @user1:
I suppose just restrict this assertion to py3.