Repository: ansible/ansible

Issue:
Unexpected Exception, this is probably a bug: 'NoneType' object is not iterable
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
ansible-playbook
##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.0 (devel 101377768b) last updated 2017/10/03 14:54:07 (GMT +000)
  config file = /sbux_ansible/ansible.cfg
  configured module search path = [u'/sbux_ansible/library', u'/usr/share/ansible']
  ansible python module location = /home/ubuntu/ansible/lib/ansible
  executable location = /home/ubuntu/ansible/bin/ansible
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
```
also in 2.4, and observed in 2.3
```
ansible 2.3.1.0
  config file = /home/administrator/ansible/playbooks/ansible.cfg
  configured module search path = [u'/usr/share/ansible/']
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
<!---
If using Ansible 2.4 or above, paste the results of "ansible-config dump --only-changed"

Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).

-->
```
DEFAULT_HOST_LIST(/sbux_ansible/ansible.cfg) = [u'/sbux_ansible/inventory.txt']
DEFAULT_MODULE_PATH(/sbux_ansible/ansible.cfg) = [u'/sbux_ansible/library', u'/usr/share/ansible']
RETRY_FILES_ENABLED(/sbux_ansible/ansible.cfg) = False
```
##### OS / ENVIRONMENT
<!---
Mention the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.
-->
`ubuntu/xenial64`
##### SUMMARY
<!--- Explain the problem briefly -->
Method `play.get_vars_files()` returns None rather than an empty list. In very large playbooks, it is not obvious what is causing the exception.
##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->
If `vars_files` has no entries (note the comment), the play fails with `TypeError: 'NoneType' object is not iterable`
<!--- Paste example playbooks or commands between quotes below -->
```yaml
#!/usr/bin/ansible-playbook
---
#
#
#      Copyright (c) 2017 World Wide Technology, Inc.
#      All rights reserved.
#
#      author: Joel W. King,  World Wide Technology
#
#
- hosts: localhost
  connection: local
  user: administrator

  vars_files:
   #  - "{{playbook_dir}}/passwords.yml"

  tasks:
  - name: debug the output
    debug: msg="{{f5.username}}"

```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Ignore the empty list or issue a warning to the user.
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
ubuntu@ubuntu-xenial:~/ansible/playbooks$ ansible-playbook ./debug_vars_files.yml -vvv
ansible-playbook 2.5.0 (devel 101377768b) last updated 2017/10/03 14:54:07 (GMT +000)
  config file = /sbux_ansible/ansible.cfg
  configured module search path = [u'/sbux_ansible/library', u'/usr/share/ansible']
  ansible python module location = /home/ubuntu/ansible/lib/ansible
  executable location = /home/ubuntu/ansible/bin/ansible-playbook
  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
Using /sbux_ansible/ansible.cfg as config file
Parsed /sbux_ansible/inventory.txt inventory source with ini plugin

PLAYBOOK: debug_vars_files.yml *****************************************************************************************************
1 plays in ./debug_vars_files.yml
ERROR! Unexpected Exception, this is probably a bug: 'NoneType' object is not iterable
the full traceback was:

Traceback (most recent call last):
  File "/home/ubuntu/ansible/bin/ansible-playbook", line 109, in <module>
    exit_code = cli.run()
  File "/home/ubuntu/ansible/lib/ansible/cli/playbook.py", line 130, in run
    results = pbex.run()
  File "/home/ubuntu/ansible/lib/ansible/executor/playbook_executor.py", line 126, in run
    all_vars = self._variable_manager.get_vars(play=play)
  File "/home/ubuntu/ansible/lib/ansible/vars/manager.py", line 348, in get_vars
    for vars_file_item in play.get_vars_files():
TypeError: 'NoneType' object is not iterable
```



Discussion:
The workaround here is, of course, to supply an empty list instead of a `None`. I notice that the vars file attributes are defaulted to `[]`, but won't override if the use supplies `None` which we could definitely handle more gracefully.

Resolving patch:
diff --git a/lib/ansible/playbook/play.py b/lib/ansible/playbook/play.py
--- a/lib/ansible/playbook/play.py
+++ b/lib/ansible/playbook/play.py
@@ -277,6 +277,8 @@ def get_vars(self):
         return self.vars.copy()
 
     def get_vars_files(self):
+        if self.vars_files is None:
+            return []
         return self.vars_files
 
     def get_handlers(self):
diff --git a/lib/ansible/vars/manager.py b/lib/ansible/vars/manager.py
--- a/lib/ansible/vars/manager.py
+++ b/lib/ansible/vars/manager.py
@@ -345,53 +345,58 @@ def plugins_by_groups():
         if play:
             all_vars = combine_vars(all_vars, play.get_vars())
 
-            for vars_file_item in play.get_vars_files():
-                # create a set of temporary vars here, which incorporate the extra
-                # and magic vars so we can properly template the vars_files entries
-                temp_vars = combine_vars(all_vars, self._extra_vars)
-                temp_vars = combine_vars(temp_vars, magic_variables)
-                templar = Templar(loader=self._loader, variables=temp_vars)
-
-                # we assume each item in the list is itself a list, as we
-                # support "conditional includes" for vars_files, which mimics
-                # the with_first_found mechanism.
-                vars_file_list = vars_file_item
-                if not isinstance(vars_file_list, list):
-                    vars_file_list = [vars_file_list]
-
-                # now we iterate through the (potential) files, and break out
-                # as soon as we read one from the list. If none are found, we
-                # raise an error, which is silently ignored at this point.
-                try:
-                    for vars_file in vars_file_list:
-                        vars_file = templar.template(vars_file)
-                        try:
-                            data = preprocess_vars(self._loader.load_from_file(vars_file, unsafe=True))
-                            if data is not None:
-                                for item in data:
-                                    all_vars = combine_vars(all_vars, item)
-                            break
-                        except AnsibleFileNotFound:
-                            # we continue on loader failures
+            vars_files = play.get_vars_files()
+            try:
+                for vars_file_item in vars_files:
+                    # create a set of temporary vars here, which incorporate the extra
+                    # and magic vars so we can properly template the vars_files entries
+                    temp_vars = combine_vars(all_vars, self._extra_vars)
+                    temp_vars = combine_vars(temp_vars, magic_variables)
+                    templar = Templar(loader=self._loader, variables=temp_vars)
+
+                    # we assume each item in the list is itself a list, as we
+                    # support "conditional includes" for vars_files, which mimics
+                    # the with_first_found mechanism.
+                    vars_file_list = vars_file_item
+                    if not isinstance(vars_file_list, list):
+                        vars_file_list = [vars_file_list]
+
+                    # now we iterate through the (potential) files, and break out
+                    # as soon as we read one from the list. If none are found, we
+                    # raise an error, which is silently ignored at this point.
+                    try:
+                        for vars_file in vars_file_list:
+                            vars_file = templar.template(vars_file)
+                            try:
+                                data = preprocess_vars(self._loader.load_from_file(vars_file, unsafe=True))
+                                if data is not None:
+                                    for item in data:
+                                        all_vars = combine_vars(all_vars, item)
+                                break
+                            except AnsibleFileNotFound:
+                                # we continue on loader failures
+                                continue
+                            except AnsibleParserError:
+                                raise
+                        else:
+                            # if include_delegate_to is set to False, we ignore the missing
+                            # vars file here because we're working on a delegated host
+                            if include_delegate_to:
+                                raise AnsibleFileNotFound("vars file %s was not found" % vars_file_item)
+                    except (UndefinedError, AnsibleUndefinedVariable):
+                        if host is not None and self._fact_cache.get(host.name, dict()).get('module_setup') and task is not None:
+                            raise AnsibleUndefinedVariable("an undefined variable was found when attempting to template the vars_files item '%s'"
+                                                           % vars_file_item, obj=vars_file_item)
+                        else:
+                            # we do not have a full context here, and the missing variable could be because of that
+                            # so just show a warning and continue
+                            display.vvv("skipping vars_file '%s' due to an undefined variable" % vars_file_item)
                             continue
-                        except AnsibleParserError:
-                            raise
-                    else:
-                        # if include_delegate_to is set to False, we ignore the missing
-                        # vars file here because we're working on a delegated host
-                        if include_delegate_to:
-                            raise AnsibleFileNotFound("vars file %s was not found" % vars_file_item)
-                except (UndefinedError, AnsibleUndefinedVariable):
-                    if host is not None and self._fact_cache.get(host.name, dict()).get('module_setup') and task is not None:
-                        raise AnsibleUndefinedVariable("an undefined variable was found when attempting to template the vars_files item '%s'" % vars_file_item,
-                                                       obj=vars_file_item)
-                    else:
-                        # we do not have a full context here, and the missing variable could be because of that
-                        # so just show a warning and continue
-                        display.vvv("skipping vars_file '%s' due to an undefined variable" % vars_file_item)
-                        continue
 
-                display.vvv("Read vars_file '%s'" % vars_file_item)
+                    display.vvv("Read vars_file '%s'" % vars_file_item)
+            except TypeError:
+                raise AnsibleParserError("Error while reading vars files - please supply a list of file names. "
+                                         "Got '%s' of type %s" % (vars_files, type(vars_files)))
 
             # By default, we now merge in all vars from all roles in the play,
             # unless the user has disabled this via a config option


Test patch: