Repository: ansible/ansible

Issue:
UnicodeError ios_command
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.4.0 (devel aed35aba70) last updated 2017/07/20 09:27:04 (GMT -400)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/dnewswan/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/dnewswan/code/ansible/lib/ansible
  executable location = /Users/dnewswan/code/ansible/bin/ansible
  python version = 2.7.13 (default, Apr  4 2017, 08:47:57) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->

##### SUMMARY
<!--- Explain the problem briefly -->

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
`ansible-playbook -i inventory.network  iosxr.yaml -vvvv -u root -k -e limit_to=iosxr_command`

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
TASK [iosxr_command : get output for multiple commands] **********************************************************************************************************************************************************************************************************************
task path: /Users/dnewswan/code/ansible/test/integration/targets/iosxr_command/tests/cli/output.yaml:14
<iosxr01> using connection plugin network_cli
<iosxr01> socket_path: /Users/dnewswan/.ansible/pc/c519def7ce
Using module file /Users/dnewswan/code/ansible/lib/ansible/modules/network/iosxr/iosxr_command.py
<iosxr01> ESTABLISH LOCAL CONNECTION FOR USER: dnewswan
<iosxr01> EXEC /bin/sh -c 'echo ~ && sleep 0'
<iosxr01> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151 `" && echo ansible-tmp-1500564512.81-129367623740151="` echo /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151 `" ) && sleep 0'
<iosxr01> PUT /var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/tmpsvhXiZ TO /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/iosxr_command.py
<iosxr01> EXEC /bin/sh -c 'chmod u+x /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/ /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/iosxr_command.py && sleep 0'
<iosxr01> EXEC /bin/sh -c 'python /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/iosxr_command.py; rm -rf "/Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/" > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py", line 228, in <module>
    main()
  File "/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py", line 222, in main
    'stdout_lines': list(to_lines(responses))
  File "/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py", line 144, in to_lines
    item = str(item).split('\n')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe6' in position 5055: ordinal not in range(128)

fatal: [iosxr01]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py\", line 228, in <module>\n    main()\n  File \"/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py\", line 222, in main\n    'stdout_lines': list(to_lines(responses))\n  File \"/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py\", line 144, in to_lines\n    item = str(item).split('\\n')\nUnicodeEncodeError: 'ascii' codec can't encode character u'\\xe6' in position 5055: ordinal not in range(128)\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}

```

UnicodeError ios_command
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.4.0 (devel aed35aba70) last updated 2017/07/20 09:27:04 (GMT -400)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/dnewswan/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/dnewswan/code/ansible/lib/ansible
  executable location = /Users/dnewswan/code/ansible/bin/ansible
  python version = 2.7.13 (default, Apr  4 2017, 08:47:57) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]
```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->

##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say “N/A” for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->

##### SUMMARY
<!--- Explain the problem briefly -->

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
`ansible-playbook -i inventory.network  iosxr.yaml -vvvv -u root -k -e limit_to=iosxr_command`

<!--- Paste example playbooks or commands between quotes below -->
```yaml

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
TASK [iosxr_command : get output for multiple commands] **********************************************************************************************************************************************************************************************************************
task path: /Users/dnewswan/code/ansible/test/integration/targets/iosxr_command/tests/cli/output.yaml:14
<iosxr01> using connection plugin network_cli
<iosxr01> socket_path: /Users/dnewswan/.ansible/pc/c519def7ce
Using module file /Users/dnewswan/code/ansible/lib/ansible/modules/network/iosxr/iosxr_command.py
<iosxr01> ESTABLISH LOCAL CONNECTION FOR USER: dnewswan
<iosxr01> EXEC /bin/sh -c 'echo ~ && sleep 0'
<iosxr01> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151 `" && echo ansible-tmp-1500564512.81-129367623740151="` echo /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151 `" ) && sleep 0'
<iosxr01> PUT /var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/tmpsvhXiZ TO /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/iosxr_command.py
<iosxr01> EXEC /bin/sh -c 'chmod u+x /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/ /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/iosxr_command.py && sleep 0'
<iosxr01> EXEC /bin/sh -c 'python /Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/iosxr_command.py; rm -rf "/Users/dnewswan/.ansible/tmp/ansible-tmp-1500564512.81-129367623740151/" > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py", line 228, in <module>
    main()
  File "/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py", line 222, in main
    'stdout_lines': list(to_lines(responses))
  File "/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py", line 144, in to_lines
    item = str(item).split('\n')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe6' in position 5055: ordinal not in range(128)

fatal: [iosxr01]: FAILED! => {
    "changed": false,
    "failed": true,
    "module_stderr": "Traceback (most recent call last):\n  File \"/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py\", line 228, in <module>\n    main()\n  File \"/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py\", line 222, in main\n    'stdout_lines': list(to_lines(responses))\n  File \"/var/folders/ql/d9_087wj4cd10grvwzxmk4hc0000gn/T/ansible_qtA4lm/ansible_module_iosxr_command.py\", line 144, in to_lines\n    item = str(item).split('\\n')\nUnicodeEncodeError: 'ascii' codec can't encode character u'\\xe6' in position 5055: ordinal not in range(128)\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 0
}

```



Discussion:

Resolving patch:
diff --git a/lib/ansible/modules/network/iosxr/iosxr_command.py b/lib/ansible/modules/network/iosxr/iosxr_command.py
--- a/lib/ansible/modules/network/iosxr/iosxr_command.py
+++ b/lib/ansible/modules/network/iosxr/iosxr_command.py
@@ -137,11 +137,12 @@
 from ansible.module_utils.netcli import Conditional
 from ansible.module_utils.six import string_types
 from ansible.module_utils.iosxr import iosxr_argument_spec, check_args
+from ansible.module_utils._text import to_native
 
 def to_lines(stdout):
     for item in stdout:
         if isinstance(item, string_types):
-            item = str(item).split('\n')
+            item = to_native(item, errors='surrogate_or_strict').split('\n')
         yield item
 
 def parse_commands(module, warnings):


Test patch: