Repository: ansible/ansible

Issue:
Speedup inventory breaks include_role over with_items
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
include_role

##### ANSIBLE VERSION
```
ansible 2.5.0 (devel cb5f2c7ac3) last updated 2017/09/26 11:45:15 (GMT +1000)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/will/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/will/src/ansible/lib/ansible
  executable location = /home/will/src/ansible/bin/ansible
  python version = 2.7.13 (default, Sep  5 2017, 08:53:59) [GCC 7.1.1 20170622 (Red Hat 7.1.1-3)]
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
N/A

##### SUMMARY
include_role combined with with_items no longer works after the inventory speedup #30587 

##### STEPS TO REPRODUCE

playbook:
```yaml
- hosts: host1
  connection: local
  gather_facts: false

  tasks:
    - name: apply role whatever over list of items
      include_role:
        name: whatever
      with_items:
        - a
```

inventory file:
```
[group4]
host1

[group2]
host2

[group3]
host1
host2

[group3]
host3

[group2]
host3
```

To see the working version (it's not needed for the failure)
```
mkdir -p roles/whatever/tasks && echo "- debug: msg=hello" > roles/whatever/tasks/main.yml
```

##### EXPECTED RESULTS
With Ansible 2.4.0, I get:
```
$ ansible-playbook playbook.yml -i inventory

PLAY [host1] *************************************************************************************************************************************************

TASK [apply role whatever over list of items] ****************************************************************************************************************

TASK [whatever : debug] **************************************************************************************************************************************
ok: [host1] => {
    "msg": "hello"
}

PLAY RECAP ***************************************************************************************************************************************************
host1                      : ok=1    changed=0    unreachable=0    failed=0   
```

##### ACTUAL RESULTS
```
$ ansible-playbook playbook.yml -i inventory

PLAY [host1] *************************************************************************************************************************************************

TASK [apply role whatever over list of items] ****************************************************************************************************************
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/threading.py", line 757, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/will/src/ansible/lib/ansible/plugins/strategy/__init__.py", line 84, in results_thread_main
    result = strategy._final_q.get()
  File "/usr/lib64/python2.7/multiprocessing/queues.py", line 117, in get
    res = self._recv()
  File "/home/will/src/ansible/lib/ansible/inventory/host.py", line 37, in __setstate__
    return self.deserialize(data)
  File "/home/will/src/ansible/lib/ansible/inventory/host.py", line 82, in deserialize
    g.deserialize(group_data)
  File "/home/will/src/ansible/lib/ansible/inventory/group.py", line 77, in deserialize
    self._hosts = set(self.hosts)
  File "/home/will/src/ansible/lib/ansible/inventory/host.py", line 48, in __hash__
    return hash(self.name)
AttributeError: 'Host' object has no attribute 'name'
```

At which point the playbook just hangs.


Discussion:
 ... wrong deserialization, should be host names ..
@willthames Thanks for the repro, it helped me isolate the issue faster than running it through CI.

The timeout issue we're seeing in CI is intermittent, and only on Python 3, but this repro triggers it and yields the same error. It occurs while deserializing a Group which has multiple Host references.

During deserialization of Group, it tries to add all the Host objects to a set. However, not all of the Host objects have been deserialized yet, so they are missing attributes, causing the AttributeError.
@willthames Can you test the fix in https://github.com/ansible/ansible/pull/31033 to see if it resolves your issue?

Resolving patch:
diff --git a/lib/ansible/inventory/group.py b/lib/ansible/inventory/group.py
--- a/lib/ansible/inventory/group.py
+++ b/lib/ansible/inventory/group.py
@@ -18,7 +18,6 @@
 __metaclass__ = type
 
 from ansible.errors import AnsibleError
-from ansible.utils.vars import combine_vars
 
 
 class Group:
@@ -31,7 +30,7 @@ def __init__(self, name=None):
         self.depth = 0
         self.name = name
         self.hosts = []
-        self._hosts = set()
+        self._hosts = None
         self.vars = {}
         self.child_groups = []
         self.parent_groups = []
@@ -72,9 +71,8 @@ def deserialize(self, data):
         self.name = data.get('name')
         self.vars = data.get('vars', dict())
         self.depth = data.get('depth', 0)
-        self.hosts = data.get('hosts', {})
-
-        self._hosts = set(self.hosts)
+        self.hosts = data.get('hosts', [])
+        self._hosts = None
 
         parent_groups = data.get('parent_groups', [])
         for parent_data in parent_groups:
@@ -82,6 +80,12 @@ def deserialize(self, data):
             g.deserialize(parent_data)
             self.parent_groups.append(g)
 
+    @property
+    def host_names(self):
+        if self._hosts is None:
+            self._hosts = set(self.hosts)
+        return self._hosts
+
     def get_name(self):
         return self.name
 
@@ -119,7 +123,7 @@ def _check_children_depth(self):
             raise AnsibleError("The group named '%s' has a recursive dependency loop." % self.name)
 
     def add_host(self, host):
-        if host.name not in self._hosts:
+        if host.name not in self.host_names:
             self.hosts.append(host)
             self._hosts.add(host.name)
             host.add_group(self)
@@ -127,7 +131,7 @@ def add_host(self, host):
 
     def remove_host(self, host):
 
-        if host.name in self._hosts:
+        if host.name in self.host_names:
             self.hosts.remove(host)
             self._hosts.remove(host.name)
             host.remove_group(self)


Test patch: