Repository: ansible/ansible

Issue:
'all' group has no hosts with a YAML inventory file
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
YAML inventory

##### ANSIBLE VERSION
both 2.2.0 and devel

```
$ ansible --version
ansible 2.2.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

```

```
$ ansible --version
ansible 2.3.0 (devel b598575213) last updated 2017/01/14 00:40:53 (GMT +900)
  config file =
  configured module search path = Default w/o overrides

```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
N/A

##### SUMMARY
All of the hosts in an inventory file are included in the 'all' group when it is written in INI.
However when an inventory file is written in YAML, the 'all' group has no hosts.

##### STEPS TO REPRODUCE
* launch `ansible localhost -i test-hosts.yml -m debug -a var=groups.all`

The test-hosts.yml is the smallest inventory file in YAML.

```yaml
group1:
  hosts:
    host1:
```

##### EXPECTED RESULTS
```
$ ansible localhost -i test-hosts.yml -m debug -a var=groups.all -vvvv
No config file found; using defaults
Loading callback plugin minimal of type stdout, v2.0 from /home/vagrant/ansible/lib/ansible/plugins/callback/__init__.pyc
localhost | SUCCESS => {
    "groups.all": [
        "host1"
    ]
}
```

##### ACTUAL RESULTS
The groups.all is empty with a YAML inventory file.

```
$ ansible localhost -i test-hosts.yml -m debug -a var=groups.all -vvvv
No config file found; using defaults
Loading callback plugin minimal of type stdout, v2.0 from /home/vagrant/ansible/lib/ansible/plugins/callback/__init__.pyc
localhost | SUCCESS => {
    "groups.all": []
}
```



Discussion:
Reproduced with devel:

```
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ansible --version
ansible 2.3.0 (devel 90d94bd6db) last updated 2017/01/16 10:19:58 (GMT -400)
  config file = /home/adrian/.ansible.cfg
  configured module search path = [u'/home/adrian/ansible/my-modules', u'/home/adrian/ansible/lib/modules/']
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ cat run-test-yaml.sh 
#!/bin/bash
ansible localhost -i hosts.yml -m debug -a var=groups.all -vvvv
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ./run-test-yaml.sh 
Using /home/adrian/.ansible.cfg as config file
Loading callback plugin minimal of type stdout, v2.0 from /home/adrian/src/ansible/lib/ansible/plugins/callback/__init__.pyc
localhost | SUCCESS => {
    "groups.all": []
}
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ansible-playbook -vvv -i hosts.yml test.yml 
Using /home/adrian/.ansible.cfg as config file

PLAYBOOK: test.yml *************************************************************************************************************************************************************************************************
1 plays in test.yml

PLAY [all] *********************************************************************************************************************************************************************************************************

TASK [debug all group] *********************************************************************************************************************************************************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:6
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups.all": [], 
    "invocation": {
        "module_args": {
            "var": "groups.all"
        }, 
        "module_name": "debug"
    }
}

TASK [debug groups] ************************************************************************************************************************************************************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:10
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups": {
        "all": [], 
        "group1": [
            "host1"
        ], 
        "ungrouped": []
    }, 
    "invocation": {
        "module_args": {
            "var": "groups"
        }, 
        "module_name": "debug"
    }
}

PLAY RECAP *********************************************************************************************************************************************************************************************************
host1                      : ok=2    changed=0    unreachable=0    failed=0   

[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ./run-test-ini.sh 
Using /home/adrian/.ansible.cfg as config file
Loading callback plugin minimal of type stdout, v2.0 from /home/adrian/src/ansible/lib/ansible/plugins/callback/__init__.pyc
localhost | SUCCESS => {
    "groups.all": [
        "host1"
    ]
}
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ansible-playbook -vvv -i hosts.ini test.yml 
Using /home/adrian/.ansible.cfg as config file

PLAYBOOK: test.yml *************************************************************************************************************************************************************************************************
1 plays in test.yml

PLAY [all] *********************************************************************************************************************************************************************************************************

TASK [debug all group] *********************************************************************************************************************************************************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:6
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups.all": [
        "host1"
    ], 
    "invocation": {
        "module_args": {
            "var": "groups.all"
        }, 
        "module_name": "debug"
    }
}

TASK [debug groups] ************************************************************************************************************************************************************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:10
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups": {
        "all": [
            "host1"
        ], 
        "group1": [
            "host1"
        ], 
        "ungrouped": []
    }, 
    "invocation": {
        "module_args": {
            "var": "groups"
        }, 
        "module_name": "debug"
    }
}

PLAY RECAP *********************************************************************************************************************************************************************************************************
host1                      : ok=2    changed=0    unreachable=0    failed=0   
```
And with stable-2.2

```
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ansible --version
ansible 2.2.1.0 (stable-2.2 0c153146e3) last updated 2017/01/16 10:05:51 (GMT -400)
  lib/ansible/modules/core: (detached HEAD b387c9c1e2) last updated 2017/01/16 10:07:45 (GMT -400)
  lib/ansible/modules/extras: (detached HEAD e9bf9fd844) last updated 2017/01/16 10:07:46 (GMT -400)
  config file = /home/adrian/.ansible.cfg
  configured module search path = ['/home/adrian/ansible/my-modules', '/home/adrian/ansible/lib/modules/']
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ./run-test-yaml.sh 
Using /home/adrian/.ansible.cfg as config file
Loading callback plugin minimal of type stdout, v2.0 from /home/adrian/src/ansible/lib/ansible/plugins/callback/__init__.pyc
localhost | SUCCESS => {
    "groups.all": []
}
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ./run-test-ini.sh 
Using /home/adrian/.ansible.cfg as config file
Loading callback plugin minimal of type stdout, v2.0 from /home/adrian/src/ansible/lib/ansible/plugins/callback/__init__.pyc
localhost | SUCCESS => {
    "groups.all": [
        "host1"
    ]
}
[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ansible-playbook -vvv -i hosts.yml test.yml 
Using /home/adrian/.ansible.cfg as config file

PLAYBOOK: test.yml *************************************************************
1 plays in test.yml

PLAY [all] *********************************************************************

TASK [debug all group] *********************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:6
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups.all": [], 
    "invocation": {
        "module_args": {
            "var": "groups.all"
        }, 
        "module_name": "debug"
    }
}

TASK [debug groups] ************************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:10
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups": {
        "all": [], 
        "group1": [
            "host1"
        ], 
        "ungrouped": []
    }, 
    "invocation": {
        "module_args": {
            "var": "groups"
        }, 
        "module_name": "debug"
    }
}

PLAY RECAP *********************************************************************
host1                      : ok=2    changed=0    unreachable=0    failed=0   

[fedora-25:20252_yaml_hosts_not_setting_all (master %)]$ ansible-playbook -vvv -i hosts.ini test.yml 
Using /home/adrian/.ansible.cfg as config file

PLAYBOOK: test.yml *************************************************************
1 plays in test.yml

PLAY [all] *********************************************************************

TASK [debug all group] *********************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:6
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups.all": [
        "host1"
    ], 
    "invocation": {
        "module_args": {
            "var": "groups.all"
        }, 
        "module_name": "debug"
    }
}

TASK [debug groups] ************************************************************
task path: /home/adrian/src/ansible-bug-repro/20252_yaml_hosts_not_setting_all/test.yml:10
ok: [host1] => {
    "_ansible_no_log": false, 
    "_ansible_verbose_always": true, 
    "changed": false, 
    "groups": {
        "all": [
            "host1"
        ], 
        "group1": [
            "host1"
        ], 
        "ungrouped": []
    }, 
    "invocation": {
        "module_args": {
            "var": "groups"
        }, 
        "module_name": "debug"
    }
}

PLAY RECAP *********************************************************************
host1                      : ok=2    changed=0    unreachable=0    failed=0   
```
repro example at https://github.com/alikins/ansible-bug-repro/tree/master/20252_yaml_hosts_not_setting_all
related #20304

Resolving patch:
diff --git a/lib/ansible/inventory/__init__.py b/lib/ansible/inventory/__init__.py
--- a/lib/ansible/inventory/__init__.py
+++ b/lib/ansible/inventory/__init__.py
@@ -140,7 +140,6 @@ def parse_inventory(self, host_list):
                         display.warning("A duplicate localhost-like entry was found (%s). First found localhost was %s" % (h, self.localhost.name))
                     display.vvvv("Set default localhost to %s" % h)
                     self.localhost = new_host
-                all.add_host(new_host)
         elif self._loader.path_exists(host_list):
             # TODO: switch this to a plugin loader and a 'condition' per plugin on which it should be tried, restoring 'inventory pllugins'
             if self.is_directory(host_list):
@@ -170,8 +169,13 @@ def parse_inventory(self, host_list):
             host.vars = combine_vars(host.vars, self.get_host_variables(host.name))
             self.get_host_vars(host)
 
-            # clear ungrouped of any incorrectly stored by parser
             mygroups = host.get_groups()
+
+            # ensure hosts are always in 'all'
+            if all not in mygroups:
+                all.add_host(host)
+
+            # clear ungrouped of any incorrectly stored by parser
             if len(mygroups) > 2 and ungrouped in mygroups:
                 host.remove_group(ungrouped)
 


Test patch: