Repository: ansible/ansible

Issue:
Exception when looping include_role using with_subelements
<!---
Please do not report issues/requests related to Ansible modules here !!

Report them to the appropriate modules-core or modules-extras project:
  - https://github.com/ansible/ansible-modules-core/issues
  - https://github.com/ansible/ansible-modules-extras/issues

Also verify first that your issue/request is not already reported in GitHub
-->


##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report
 
##### COMPONENT NAME
include_role module

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```
ansible 2.2.0.0
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
N/A

##### SUMMARY
`with_subelements` creates an `item` variable which is a tuple, where the first item (at least) is a dict.

Parameters passed through the `include_role` module are hashed when creating the role. The `hash_params` function only recurses over dicts and lists, which means that the `item` tuple is inserted into the set of params unmodified, causing a `TypeError: unhashable type: 'dict'` to be thrown.

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```
- hosts: localhost
  tasks:
    - include_role: name=test
      with_subelements:
        - - name: foo
            values: ['bar']
        - values
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
The `test` role is included once and is passed an `item` tuple equal to `({'name': 'foo'}, 'bar')`

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
The play fails with an unexpected exception.
<!--- Paste verbatim command output between quotes below -->
```
TASK [include_role] ************************************************************
task path: /Users/ahaines/work/iot/infrastructure/test.yml:3
ERROR! Unexpected Exception: unhashable type: 'dict'
the full traceback was:

Traceback (most recent call last):
  File "/usr/local/bin/ansible-playbook", line 103, in <module>
    exit_code = cli.run()
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/cli/playbook.py", line 159, in run
    results = pbex.run()
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/executor/playbook_executor.py", line 154, in run
    result = self._tqm.run(play=play)
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 282, in run
    play_return = strategy.run(iterator, play_context)
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/plugins/strategy/linear.py", line 304, in run
    all_role_blocks.extend(new_ir.get_block_list(play=iterator._play, variable_manager=self._variable_manager, loader=self._loader))
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/playbook/role_include.py", line 76, in get_block_list
    actual_role = Role.load(ri, myplay, parent_role=self._parent_role, from_files=self._from_files)
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/playbook/role/__init__.py", line 118, in load
    hashed_params = hash_params(params)
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/playbook/role/__init__.py", line 54, in hash_params
    s.update((k, hash_params(v)))
  File "/usr/local/Cellar/ansible/2.2.0.0_1/libexec/lib/python2.7/site-packages/ansible/playbook/role/__init__.py", line 61, in hash_params
    s.update((k, v))
TypeError: unhashable type: 'dict'
```



Discussion:

Resolving patch:
diff --git a/lib/ansible/playbook/role/__init__.py b/lib/ansible/playbook/role/__init__.py
--- a/lib/ansible/playbook/role/__init__.py
+++ b/lib/ansible/playbook/role/__init__.py
@@ -19,10 +19,10 @@
 from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
-from ansible.compat.six import iteritems
-
+import collections
 import os
 
+from ansible.compat.six import iteritems, binary_type, text_type
 from ansible.errors import AnsibleError, AnsibleParserError
 from ansible.playbook.attribute import FieldAttribute
 from ansible.playbook.base import Base
@@ -41,25 +41,54 @@
 #       the role due to the fact that it would require the use of self
 #       in a static method. This is also used in the base class for
 #       strategies (ansible/plugins/strategy/__init__.py)
+
 def hash_params(params):
-    if not isinstance(params, dict):
-        if isinstance(params, list):
-            return frozenset(params)
+    """
+    Construct a data structure of parameters that is hashable.
+
+    This requires changing any mutable data structures into immutable ones.
+    We chose a frozenset because role parameters have to be unique.
+
+    .. warning::  this does not handle unhashable scalars.  Two things
+        mitigate that limitation:
+
+        1) There shouldn't be any unhashable scalars specified in the yaml
+        2) Our only choice would be to return an error anyway.
+    """
+    # Any container is unhashable if it contains unhashable items (for
+    # instance, tuple() is a Hashable subclass but if it contains a dict, it
+    # cannot be hashed)
+    if isinstance(params, collections.Container) and not isinstance(params, (text_type, binary_type)):
+        if isinstance(params, collections.Mapping):
+            try:
+                # Optimistically hope the contents are all hashable
+                new_params = frozenset(params.items())
+            except TypeError:
+                new_params = set()
+                for k, v in params.items():
+                    # Hash each entry individually
+                    new_params.update((k, hash_params(v)))
+                new_params = frozenset(new_params)
+
+        elif isinstance(params, (collections.Set, collections.Sequence)):
+            try:
+                # Optimistically hope the contents are all hashable
+                new_params = frozenset(params)
+            except TypeError:
+                new_params = set()
+                for v in params:
+                    # Hash each entry individually
+                    new_params.update(hash_params(v))
+                new_params = frozenset(new_params)
         else:
-            return params
-    else:
-        s = set()
-        for k,v in iteritems(params):
-            if isinstance(v, dict):
-                s.update((k, hash_params(v)))
-            elif isinstance(v, list):
-                things = []
-                for item in v:
-                    things.append(hash_params(item))
-                s.update((k, tuple(things)))
-            else:
-                s.update((k, v))
-        return frozenset(s)
+            # This is just a guess.
+            new_params = frozenset(params)
+        return new_params
+
+    # Note: We do not handle unhashable scalars but our only choice would be
+    # to raise an error there anyway.
+    return frozenset((params,))
+
 
 class Role(Base, Become, Conditional, Taggable):
 


Test patch:
diff --git a/test/units/playbook/role/test_role.py b/test/units/playbook/role/test_role.py
--- a/test/units/playbook/role/test_role.py
+++ b/test/units/playbook/role/test_role.py
@@ -19,18 +19,107 @@
 from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
+import collections
+
 from ansible.compat.tests import unittest
 from ansible.compat.tests.mock import patch, MagicMock
 
 from ansible.errors import AnsibleError, AnsibleParserError
 from ansible.playbook.block import Block
-from ansible.playbook.role import Role
-from ansible.playbook.role.include import RoleInclude
 from ansible.playbook.task import Task
 
 from units.mock.loader import DictDataLoader
 from units.mock.path import mock_unfrackpath_noop
 
+from ansible.playbook.role import Role
+from ansible.playbook.role.include import RoleInclude
+from ansible.playbook.role import hash_params
+
+
+class TestHashParams(unittest.TestCase):
+    def test(self):
+        params = {'foo': 'bar'}
+        res = hash_params(params)
+        self._assert_set(res)
+        self._assert_hashable(res)
+
+    def _assert_hashable(self, res):
+        a_dict = {}
+        try:
+            a_dict[res] = res
+        except TypeError as e:
+            self.fail('%s is not hashable: %s' % (res, e))
+
+    def _assert_set(self, res):
+        self.assertIsInstance(res, frozenset)
+
+    def test_dict_tuple(self):
+        params = {'foo': (1, 'bar',)}
+        res = hash_params(params)
+        self._assert_set(res)
+
+    def test_tuple(self):
+        params = (1, None, 'foo')
+        res = hash_params(params)
+        self._assert_hashable(res)
+
+    def test_tuple_dict(self):
+        params = ({'foo': 'bar'}, 37)
+        res = hash_params(params)
+        self._assert_hashable(res)
+
+    def test_list(self):
+        params = ['foo', 'bar', 1, 37, None]
+        res = hash_params(params)
+        self._assert_set(res)
+        self._assert_hashable(res)
+
+    def test_dict_with_list_value(self):
+        params = {'foo': [1, 4, 'bar']}
+        res = hash_params(params)
+        self._assert_set(res)
+        self._assert_hashable(res)
+
+    def test_empty_set(self):
+        params = set([])
+        res = hash_params(params)
+        self._assert_hashable(res)
+        self._assert_set(res)
+
+    def test_generator(self):
+        def my_generator():
+            for i in ['a', 1, None, {}]:
+                yield i
+
+        params = my_generator()
+        res = hash_params(params)
+        self._assert_hashable(res)
+
+    def test_container_but_not_iterable(self):
+        # This is a Container that is not iterable, which is unlikely but...
+        class MyContainer(collections.Container):
+            def __init__(self, some_thing):
+                self.data = []
+                self.data.append(some_thing)
+
+            def __contains__(self, item):
+                return item in self.data
+
+            def __hash__(self):
+                return hash(self.data)
+
+            def __len__(self):
+                return len(self.data)
+
+            def __call__(self):
+                return False
+
+        foo = MyContainer('foo bar')
+        params = foo
+
+        self.assertRaises(TypeError, hash_params, params)
+
+
 class TestRole(unittest.TestCase):
 
     def setUp(self):