Repository: ansible/ansible

Issue:
crash when using docker_container with a ulimit in check mode
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
!component -lib/ansible/modules/cloud/docker/docker_container.py
!component +lib/ansible/module_utils/basic.py


##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.2.0
  config file = /Users/emario/Documents/src/deployment/ansible/ansible.cfg
  configured module search path = [u'/Users/emario/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/Cellar/ansible/2.4.2.0/libexec/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.14 (default, Oct  2 2017, 15:18:06) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.37)]
```

##### CONFIGURATION
ANSIBLE_PIPELINING(/Users/emario/Documents/src/deployment/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/Users/emario/Documents/src/deployment/ansible/ansible.cfg) = -o ControlMaster=auto -o ControlPersist=30m
DEFAULT_LOG_PATH(/Users/emario/Documents/src/deployment/ansible/ansible.cfg) = /tmp/ansible.log
DEFAULT_LOOKUP_PLUGIN_PATH(/Users/emario/Documents/src/deployment/ansible/ansible.cfg) = [u'/Users/emario/Documents/src/deployment/ansible/lookup_plugins']
DEFAULT_MODULE_LANG(/Users/emario/Documents/src/deployment/ansible/ansible.cfg) = C
DEFAULT_MODULE_SET_LOCALE(/Users/emario/Documents/src/deployment/ansible/ansible.cfg) = False

##### OS / ENVIRONMENT
OS we are managing: coreos
Bug happens from linux and mac os x. 

##### SUMMARY
<!--- Explain the problem briefly -->
The last step of the ansible/module_utils/basic.py is to clean up the results before returning them. During cleanup copies are made of all values inside the results dictionary. Since 2.4.0 this fails if the value is of type [Ulimit](https://github.com/docker/docker-py/blob/master/docker/types/containers.py#L61). The breaking change is found [here](https://github.com/ansible/ansible/commit/be4e7a05faac288634d5ec270d40fcc55bb5d4a8#diff-90085fdcec6ed8b273ba885eaee60328L436). Line 472 uses `type(value)()` to create a new instance. As the value is of type Ulimit, this fails, because of missing init parameters. 

This only happens, if either debug or check mode are active, as the Ulimits instance is put into the result.actions list. It also only happnes, if the container does not, yet, exist on the target machine. 

##### STEPS TO REPRODUCE

Assuming a target machine that has docker installed, add a docker_container task with a `ulimits:` parameter. 

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: eventstore - run docker container
  docker_container:
    docker_api_version: auto
    name: "eventstore"
    image: "eventstore/eventstore:release-4.0.2"
    state: started
    restart_policy: always
    ulimits: nofile:16384:16384
```

Then run ansible-playbook with the `--check` flag: 
`ansible-playbook --diff --check -i some_inventory site.yml`

##### EXPECTED RESULTS
I expect an output of `Changed` for the task eventstore. 

##### ACTUAL RESULTS
Ansible tasks exits with a python exception. Error: `ValueError: Ulimit.name must be a string`

```
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_cJg98M/ansible_module_docker_container.py", line 2124, in <module>
    main()
  File "/tmp/ansible_cJg98M/ansible_module_docker_container.py", line 2120, in main
    client.module.exit_json(**cm.results)
  File "/tmp/ansible_cJg98M/ansible_modlib.zip/ansible/module_utils/basic.py", line 2279, in exit_json
  File "/tmp/ansible_cJg98M/ansible_modlib.zip/ansible/module_utils/basic.py", line 2272, in _return_formatted
  File "/tmp/ansible_cJg98M/ansible_modlib.zip/ansible/module_utils/basic.py", line 518, in remove_values
  File "/tmp/ansible_cJg98M/ansible_modlib.zip/ansible/module_utils/basic.py", line 480, in _remove_values_conditions
  File "/opt/bin/active_python/lib/python2.7/site-packages/docker/types/containers.py", line 61, in __init__
    raise ValueError("Ulimit.name must be a string was " + str(name.__class__) + " " + str(kwargs))
ValueError: Ulimit.name must be a string was <type 'NoneType'> {}
```


Discussion:
Files identified in the description:
* [lib/ansible/modules/cloud/docker/docker_container.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/docker/docker_container.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @ThomasSteinbach @chouseknecht @cove @dusdanig @jctanner @joshuaconner @kassiansun @softzilla @zfil
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
!component -lib/ansible/modules/cloud/docker/docker_container.py
!component +lib/ansible/module_utils/basic.py

Files identified in the description:
* [lib/ansible/module_utils/basic.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/basic.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
This actually happens with as simple `docker_container` calls as:

```
    - docker_container:
        name: mytest
        image: alpine:3.7
```

In `basic.py` [here](https://github.com/ansible/ansible/blob/284625f18c5da6012a72155826ffc88defd860a1/lib/ansible/module_utils/basic.py#L480):

```
    elif isinstance(value, Mapping):
        if isinstance(value, MutableMapping):
            new_value = type(value)()
        else:
            new_value = {}  # Need a mutable value
```
constructor for `MutableMapping` is called without arguments, but the type in question is `docker.types.containers.HostConfig`, and [here](https://github.com/docker/docker-py/blob/d7bc8ac2b8507b93097f164ab994a138f9d6bab1/docker/types/containers.py#L104):

```
class HostConfig(dict):
    def __init__(self, version, binds=None, port_bindings=None,
                 lxc_conf=None, publish_all_ports=False, links=None,
                 ...
```
constructor for `HostConfig` expects `version` to be passed.


I just ran into this with Ansible 2.7. This happens because `container_create()` inserts `self.parameters.create_parameters` (which contains the `HostConfig` object) into `results['actions']`:
``` .py
self.results['actions'].append(dict(created="Created container", create_parameters=create_parameters))
```
There's no problem without check mode (and without debug mode, in fact) because the module removes `actions` from `self.results` in that case:
``` .py
        if not self.check_mode and not self.parameters.debug:
            self.results.pop('actions')
```
I guess this can be fixed by inserting a "censored" version of `create_parameters` into the actions list, or by just leaving it away. Since it doesn't really work anyway, there's no reason to leave it in (in my opinion).

Resolving patch:
diff --git a/lib/ansible/module_utils/docker_common.py b/lib/ansible/module_utils/docker_common.py
--- a/lib/ansible/module_utils/docker_common.py
+++ b/lib/ansible/module_utils/docker_common.py
@@ -121,6 +121,24 @@ class APIError(Exception):
         pass
 
 
+def sanitize_result(data):
+    """Sanitize data object for return to Ansible.
+
+    When the data object contains types such as docker.types.containers.HostConfig,
+    Ansible will fail when these are returned via exit_json or fail_json.
+    HostConfig is derived from dict, but its constructor requires additional
+    arguments. This function sanitizes data structures by recursively converting
+    everything derived from dict to dict and everything derived from list (and tuple)
+    to a list.
+    """
+    if isinstance(data, dict):
+        return dict((k, sanitize_result(v)) for k, v in data.items())
+    elif isinstance(data, (list, tuple)):
+        return [sanitize_result(v) for v in data]
+    else:
+        return data
+
+
 class DockerBaseClass(object):
 
     def __init__(self):
diff --git a/lib/ansible/modules/cloud/docker/docker_container.py b/lib/ansible/modules/cloud/docker/docker_container.py
--- a/lib/ansible/modules/cloud/docker/docker_container.py
+++ b/lib/ansible/modules/cloud/docker/docker_container.py
@@ -607,7 +607,7 @@
 import shlex
 
 from ansible.module_utils.basic import human_to_bytes
-from ansible.module_utils.docker_common import HAS_DOCKER_PY_2, HAS_DOCKER_PY_3, AnsibleDockerClient, DockerBaseClass
+from ansible.module_utils.docker_common import HAS_DOCKER_PY_2, HAS_DOCKER_PY_3, AnsibleDockerClient, DockerBaseClass, sanitize_result
 from ansible.module_utils.six import string_types
 
 try:
@@ -1780,7 +1780,7 @@ def absent(self):
             self.container_remove(container.Id)
 
     def fail(self, msg, **kwargs):
-        self.client.module.fail_json(msg=msg, **kwargs)
+        self.client.module.fail_json(msg=msg, **sanitize_result(kwargs))
 
     def _output_logs(self, msg):
         self.client.module.log(msg=msg)
@@ -2095,7 +2095,7 @@ def main():
         client.module.fail_json(msg="'auto_remove' is not compatible with the 'docker-py' Python package. It requires the newer 'docker' Python package.")
 
     cm = ContainerManager(client)
-    client.module.exit_json(**cm.results)
+    client.module.exit_json(**sanitize_result(cm.results))
 
 
 if __name__ == '__main__':


Test patch: