Repository: ansible-collections/community.general

Issue:
docker_container: UnicodeDecodeError on probe to set docker ENV variable with non ASCII chars
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Explain the problem briefly below -->
If i put any Cyrillic chars in docker ENV then running `ansible-playbook myapp.yml` fails with Python error:
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 17: ordinal not in range(128)

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
docker_container

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.9.6
ansible python module location = /usr/lib/python3.6/site-packages/ansible
python version = 3.6.10 (default, Jan 16 2020, 09:12:04) [GCC]
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below

```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->

- CentOS Linux 7
- Python 2.7.5
- Docker version 1.13.1, build 64e9980/1.13.1

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->

1. create docker container with ansible
<!--- Paste example playbooks or commands between quotes below -->
```yaml
---
 - hosts:
    - virtualbox
  tasks:
    - name: pull image and restart container
      docker_container:
         image: busybox
        name: test_non_ascii
        env:
          MY_ENV_TO: "Абвг «Д»"
        command: "echo $MY_ENV_TO"
```
2. run this playbook again

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
no error

##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes -->
```paste below
MODULE FAILURE
File \"/tmp/ansible_docker_container_payload_LJPmWa/ansible_docker_container_payload.zip/ansible/modules/cloud/docker/docker_container.py\", line 2174, in has_different_configuration\r\nUnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 17: ordinal not in range(128)\r\n"
```
```paste
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1594371809.4944637-14716-155893099858081/AnsiballZ_docker_container.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1594371809.4944637-14716-155893099858081/AnsiballZ_docker_container.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1594371809.4944637-14716-155893099858081/AnsiballZ_docker_container.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible.modules.cloud.docker.docker_container', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib64/python2.7/runpy.py", line 176, in run_module
    fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 82, in _run_module_code
    mod_name, mod_fname, mod_loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/tmp/ansible_docker_container_payload_etxiiI/ansible_docker_container_payload.zip/ansible/modules/cloud/docker/docker_container.py", line 3369, in <module>
  File "/tmp/ansible_docker_container_payload_etxiiI/ansible_docker_container_payload.zip/ansible/modules/cloud/docker/docker_container.py", line 3360, in main
  File "/tmp/ansible_docker_container_payload_etxiiI/ansible_docker_container_payload.zip/ansible/modules/cloud/docker/docker_container.py", line 2568, in __init__
  File "/tmp/ansible_docker_container_payload_etxiiI/ansible_docker_container_payload.zip/ansible/modules/cloud/docker/docker_container.py", line 2639, in present
  File "/tmp/ansible_docker_container_payload_etxiiI/ansible_docker_container_payload.zip/ansible/modules/cloud/docker/docker_container.py", line 2174, in has_different_configuration
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 10: ordinal not in range(128)```


Discussion:
This seems to be a Python 2 problem; with Python 3 I cannot reproduce it. I'll look into it more closely the next days...

Resolving patch:
diff --git a/plugins/modules/cloud/docker/docker_container.py b/plugins/modules/cloud/docker/docker_container.py
--- a/plugins/modules/cloud/docker/docker_container.py
+++ b/plugins/modules/cloud/docker/docker_container.py
@@ -1104,6 +1104,9 @@
 from time import sleep
 
 from ansible.module_utils.common.text.formatters import human_to_bytes
+from ansible.module_utils.six import string_types
+from ansible.module_utils._text import to_native, to_text
+
 from ansible_collections.community.general.plugins.module_utils.docker.common import (
     AnsibleDockerClient,
     DifferenceTracker,
@@ -1117,7 +1120,6 @@
     DOCKER_COMMON_ARGS,
     RequestException,
 )
-from ansible.module_utils.six import string_types
 
 try:
     from docker import utils
@@ -1311,7 +1313,7 @@ def __init__(self, client):
         if self.groups:
             # In case integers are passed as groups, we need to convert them to
             # strings as docker internally treats them as strings.
-            self.groups = [str(g) for g in self.groups]
+            self.groups = [to_text(g, errors='surrogate_or_strict') for g in self.groups]
 
         for param_name in REQUIRES_CONVERSION_TO_BYTES:
             if client.module.params.get(param_name):
@@ -1343,7 +1345,7 @@ def __init__(self, client):
         try:
             self.healthcheck, self.disable_healthcheck = parse_healthcheck(self.healthcheck)
         except ValueError as e:
-            self.fail(str(e))
+            self.fail(to_native(e))
 
         self.exp_links = None
         self.volume_binds = self._get_volume_binds(self.volumes)
@@ -1370,12 +1372,12 @@ def __init__(self, client):
 
         if self.entrypoint:
             # convert from list to str.
-            self.entrypoint = ' '.join([str(x) for x in self.entrypoint])
+            self.entrypoint = ' '.join([to_text(x, errors='surrogate_or_strict') for x in self.entrypoint])
 
         if self.command:
             # convert from list to str
             if isinstance(self.command, list):
-                self.command = ' '.join([str(x) for x in self.command])
+                self.command = ' '.join([to_text(x, errors='surrogate_or_strict') for x in self.command])
 
         self.mounts_opt, self.expected_mounts = self._process_mounts()
 
@@ -1628,7 +1630,7 @@ def _parse_publish_ports(self):
 
         binds = {}
         for port in self.published_ports:
-            parts = split_colon_ipv6(str(port), self.client)
+            parts = split_colon_ipv6(to_text(port, errors='surrogate_or_strict'), self.client)
             container_port = parts[-1]
             protocol = ''
             if '/' in container_port:
@@ -1704,7 +1706,7 @@ def _parse_exposed_ports(self, published_ports):
         exposed = []
         if self.exposed_ports:
             for port in self.exposed_ports:
-                port = str(port).strip()
+                port = to_text(port, errors='surrogate_or_strict').strip()
                 protocol = 'tcp'
                 match = re.search(r'(/.+$)', port)
                 if match:
@@ -1798,9 +1800,10 @@ def _parse_log_config(self):
                 if not isinstance(v, string_types):
                     self.client.module.warn(
                         "Non-string value found for log_options option '%s'. The value is automatically converted to '%s'. "
-                        "If this is not correct, or you want to avoid such warnings, please quote the value." % (k, str(v))
+                        "If this is not correct, or you want to avoid such warnings, please quote the value." % (
+                            k, to_text(v, errors='surrogate_or_strict'))
                     )
-                v = str(v)
+                v = to_text(v, errors='surrogate_or_strict')
                 self.log_options[k] = v
                 options['Config'][k] = v
 
@@ -1834,13 +1837,13 @@ def _get_environment(self):
         if self.env_file:
             parsed_env_file = utils.parse_env_file(self.env_file)
             for name, value in parsed_env_file.items():
-                final_env[name] = str(value)
+                final_env[name] = to_text(value, errors='surrogate_or_strict')
         if self.env:
             for name, value in self.env.items():
                 if not isinstance(value, string_types):
                     self.fail("Non-string value found for env option. Ambiguous env options must be "
                               "wrapped in quotes to avoid them being interpreted. Key: %s" % (name, ))
-                final_env[name] = str(value)
+                final_env[name] = to_text(value, errors='surrogate_or_strict')
         return final_env
 
     def _get_network_id(self, network_name):
@@ -1851,7 +1854,7 @@ def _get_network_id(self, network_name):
                     network_id = network['Id']
                     break
         except Exception as exc:
-            self.fail("Error getting network id for %s - %s" % (network_name, str(exc)))
+            self.fail("Error getting network id for %s - %s" % (network_name, to_native(exc)))
         return network_id
 
     def _process_mounts(self):
@@ -2208,7 +2211,7 @@ def has_different_configuration(self, image):
             if not minimal_version.get('supported', True):
                 continue
             compare = self.parameters.client.comparisons[self.parameters_map.get(key, key)]
-            self.log('check differences %s %s vs %s (%s)' % (key, getattr(self.parameters, key), str(value), compare))
+            self.log('check differences %s %s vs %s (%s)' % (key, getattr(self.parameters, key), to_text(value, errors='surrogate_or_strict'), compare))
             if getattr(self.parameters, key, None) is not None:
                 match = self._compare(getattr(self.parameters, key), value, compare)
 
@@ -2231,7 +2234,7 @@ def sort_key_fn(x):
                         else:
                             # We sort the list of dictionaries by using the sorted items of a dict as its key.
                             def sort_key_fn(x):
-                                return sorted((a, str(b)) for a, b in x.items())
+                                return sorted((a, to_text(b, errors='surrogate_or_strict')) for a, b in x.items())
                         if p is not None:
                             p = sorted(p, key=sort_key_fn)
                         if c is not None:
@@ -2411,9 +2414,9 @@ def _get_expected_ports(self):
             elif isinstance(config[0], tuple):
                 expected_bound_ports[container_port] = []
                 for host_ip, host_port in config:
-                    expected_bound_ports[container_port].append({'HostIp': host_ip, 'HostPort': str(host_port)})
+                    expected_bound_ports[container_port].append({'HostIp': host_ip, 'HostPort': to_text(host_port, errors='surrogate_or_strict')})
             else:
-                expected_bound_ports[container_port] = [{'HostIp': config[0], 'HostPort': str(config[1])}]
+                expected_bound_ports[container_port] = [{'HostIp': config[0], 'HostPort': to_text(config[1], errors='surrogate_or_strict')}]
         return expected_bound_ports
 
     def _get_expected_links(self):
@@ -2523,7 +2526,7 @@ def _get_expected_exposed(self, image):
             image_ports = [self._normalize_port(p) for p in image_exposed_ports.keys()]
         param_ports = []
         if self.parameters.ports:
-            param_ports = [str(p[0]) + '/' + p[1] for p in self.parameters.ports]
+            param_ports = [to_text(p[0], errors='surrogate_or_strict') + '/' + p[1] for p in self.parameters.ports]
         result = list(set(image_ports + param_ports))
         self.log(result, pretty_print=True)
         return result
@@ -2547,7 +2550,7 @@ def _get_expected_sysctls(self, config_sysctls):
             return None
         result = dict()
         for key, value in config_sysctls.items():
-            result[key] = str(value)
+            result[key] = to_text(value, errors='surrogate_or_strict')
         return result
 
     def _get_expected_cmd(self):
@@ -2739,7 +2742,7 @@ def present(self, state):
                             self.client.unpause(container=container.Id)
                     except Exception as exc:
                         self.fail("Error %s container %s: %s" % (
-                            "pausing" if self.parameters.paused else "unpausing", container.Id, str(exc)
+                            "pausing" if self.parameters.paused else "unpausing", container.Id, to_native(exc)
                         ))
                     container = self._get_container(container.Id)
                 self.results['changed'] = True
@@ -2861,7 +2864,7 @@ def _add_networks(self, container, differences):
                         self.client.disconnect_container_from_network(container.Id, diff['parameter']['id'])
                     except Exception as exc:
                         self.fail("Error disconnecting container from network %s - %s" % (diff['parameter']['name'],
-                                                                                          str(exc)))
+                                                                                          to_native(exc)))
             # connect to the network
             params = dict()
             for para in ('ipv4_address', 'ipv6_address', 'links', 'aliases'):
@@ -2874,7 +2877,7 @@ def _add_networks(self, container, differences):
                     self.log(params, pretty_print=True)
                     self.client.connect_container_to_network(container.Id, diff['parameter']['id'], **params)
                 except Exception as exc:
-                    self.fail("Error connecting container to network %s - %s" % (diff['parameter']['name'], str(exc)))
+                    self.fail("Error connecting container to network %s - %s" % (diff['parameter']['name'], to_native(exc)))
         return self._get_container(container.Id)
 
     def _purge_networks(self, container, networks):
@@ -2885,7 +2888,7 @@ def _purge_networks(self, container, networks):
                     self.client.disconnect_container_from_network(container.Id, network['name'])
                 except Exception as exc:
                     self.fail("Error disconnecting container from network %s - %s" % (network['name'],
-                                                                                      str(exc)))
+                                                                                      to_native(exc)))
         return self._get_container(container.Id)
 
     def container_create(self, image, create_parameters):
@@ -2900,7 +2903,7 @@ def container_create(self, image, create_parameters):
                 new_container = self.client.create_container(image, **create_parameters)
                 self.client.report_warnings(new_container)
             except Exception as exc:
-                self.fail("Error creating container: %s" % str(exc))
+                self.fail("Error creating container: %s" % to_native(exc))
             return self._get_container(new_container['Id'])
         return new_container
 
@@ -2912,7 +2915,7 @@ def container_start(self, container_id):
             try:
                 self.client.start(container=container_id)
             except Exception as exc:
-                self.fail("Error starting container %s: %s" % (container_id, str(exc)))
+                self.fail("Error starting container %s: %s" % (container_id, to_native(exc)))
 
             if self.parameters.detach is False:
                 if self.client.docker_py_version >= LooseVersion('3.0'):
@@ -2964,21 +2967,21 @@ def container_remove(self, container_id, link=False, force=False):
                         # New docker daemon versions do not allow containers to be removed
                         # if they are paused. Make sure we don't end up in an infinite loop.
                         if count == 3:
-                            self.fail("Error removing container %s (tried to unpause three times): %s" % (container_id, str(exc)))
+                            self.fail("Error removing container %s (tried to unpause three times): %s" % (container_id, to_native(exc)))
                         count += 1
                         # Unpause
                         try:
                             self.client.unpause(container=container_id)
                         except Exception as exc2:
-                            self.fail("Error unpausing container %s for removal: %s" % (container_id, str(exc2)))
+                            self.fail("Error unpausing container %s for removal: %s" % (container_id, to_native(exc2)))
                         # Now try again
                         continue
                     if 'removal of container ' in exc.explanation and ' is already in progress' in exc.explanation:
                         pass
                     else:
-                        self.fail("Error removing container %s: %s" % (container_id, str(exc)))
+                        self.fail("Error removing container %s: %s" % (container_id, to_native(exc)))
                 except Exception as exc:
-                    self.fail("Error removing container %s: %s" % (container_id, str(exc)))
+                    self.fail("Error removing container %s: %s" % (container_id, to_native(exc)))
                 # We only loop when explicitly requested by 'continue'
                 break
         return response
@@ -2994,7 +2997,7 @@ def container_update(self, container_id, update_parameters):
                     result = self.client.update_container(container_id, **update_parameters)
                     self.client.report_warnings(result)
                 except Exception as exc:
-                    self.fail("Error updating container %s: %s" % (container_id, str(exc)))
+                    self.fail("Error updating container %s: %s" % (container_id, to_native(exc)))
         return self._get_container(container_id)
 
     def container_kill(self, container_id):
@@ -3021,7 +3024,7 @@ def container_restart(self, container_id):
                 else:
                     dummy = self.client.restart(container_id)
             except Exception as exc:
-                self.fail("Error restarting container %s: %s" % (container_id, str(exc)))
+                self.fail("Error restarting container %s: %s" % (container_id, to_native(exc)))
         return self._get_container(container_id)
 
     def container_stop(self, container_id):
@@ -3044,18 +3047,18 @@ def container_stop(self, container_id):
                         # New docker daemon versions do not allow containers to be removed
                         # if they are paused. Make sure we don't end up in an infinite loop.
                         if count == 3:
-                            self.fail("Error removing container %s (tried to unpause three times): %s" % (container_id, str(exc)))
+                            self.fail("Error removing container %s (tried to unpause three times): %s" % (container_id, to_native(exc)))
                         count += 1
                         # Unpause
                         try:
                             self.client.unpause(container=container_id)
                         except Exception as exc2:
-                            self.fail("Error unpausing container %s for removal: %s" % (container_id, str(exc2)))
+                            self.fail("Error unpausing container %s for removal: %s" % (container_id, to_native(exc2)))
                         # Now try again
                         continue
-                    self.fail("Error stopping container %s: %s" % (container_id, str(exc)))
+                    self.fail("Error stopping container %s: %s" % (container_id, to_native(exc)))
                 except Exception as exc:
-                    self.fail("Error stopping container %s: %s" % (container_id, str(exc)))
+                    self.fail("Error stopping container %s: %s" % (container_id, to_native(exc)))
                 # We only loop when explicitly requested by 'continue'
                 break
         return response


Test patch: