Repository: ansible/ansible

Issue:
Ansible 2.4 group_vars not parsed lexicographically
##### ISSUE TYPE
 - ~Bug  Report~
- Feature Idea

##### COMPONENT NAME
group_vars

##### ANSIBLE VERSION
```
ansible 2.4.0.0
  config file = /opt/ansible/ansible.cfg
  configured module search path = [u'/usr/lib/pymodules/python2.7/ansible/module_utils']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.14 (default, Sep 20 2017, 01:25:59) [GCC 7.2.0]
```

##### CONFIGURATION
```
[root@local test]# ansible-config dump --only-changed
[root@local test]# 
```
##### OS / ENVIRONMENT
```
[root@local test]# uname -a
Linux local 4.13.4-1-ARCH #1 SMP PREEMPT Thu Sep 28 08:39:52 CEST 2017 x86_64 GNU/Linux
```

##### SUMMARY
group variable files of higher lexicographical order do not override lower ones

##### STEPS TO REPRODUCE
```
[root@local test]# for file in $(find -type f); do echo "#File: ${file}"; cat $file; echo; done
#File: ./playbooks/debug.yml
- hosts: localhost
  gather_facts: yes
  tasks:
  - debug: var=myvar

#File: ./playbooks/group_vars/all/02_vars
myvar: "02"

#File: ./playbooks/group_vars/all/00_vars
myvar: "00"

[root@local test]# ansible-playbook -i "127.0.0.1," -c local playbooks/debug.yml 
```

##### EXPECTED RESULTS
ok: [127.0.0.1] => {
    "myvar": "02"
}
##### ACTUAL RESULTS
```
PLAY [localhost] *************************************************************************************************************************************

TASK [Gathering Facts] *******************************************************************************************************************************
ok: [127.0.0.1]

TASK [debug] *****************************************************************************************************************************************
ok: [127.0.0.1] => {
    "myvar": "00"
}

PLAY RECAP *******************************************************************************************************************************************
127.0.0.1                  : ok=2    changed=0    unreachable=0    failed=0   


```



Discussion:
It seems the group_vars/host_vars files are found in the function `_get_dir_files` in `plugins/vars/host_group_vars.py`. This function uses `os.listdir`. According to the python documentation:
> The list is in arbitrary order.

So one can't rely on any ordering. This behaviour should at least be documented.

I think it would be better to load the files in sorted order. This can be benificial for multi-stage environments to specify defaults for all environments as shown in the article [How to Manage Multistage Environments with Ansible ](https://www.digitalocean.com/community/tutorials/how-to-manage-multistage-environments-with-ansible) (The method shown is not reliable in the current version). Playbook group_vars can't be used beacuse they would override the inventory group_vars.

Resolving patch:
diff --git a/lib/ansible/plugins/vars/host_group_vars.py b/lib/ansible/plugins/vars/host_group_vars.py
--- a/lib/ansible/plugins/vars/host_group_vars.py
+++ b/lib/ansible/plugins/vars/host_group_vars.py
@@ -134,7 +134,7 @@ def _find_vars_files(self, path, name):
     def _get_dir_files(self, path):
 
         found = []
-        for spath in os.listdir(path):
+        for spath in sorted(os.listdir(path)):
             if not spath.startswith(u'.') and not spath.endswith(u'~'):  # skip hidden and backups
 
                 ext = os.path.splitext(spath)[-1]


Test patch: