Repository: ansible/ansible

Issue:
fix ext comparission in inv plugins
##### SUMMARY
fixes #43162


<!--- Describe the change, including rationale and design decisions -->

<!--- If you are fixing an existing issue, please include "Fixes #nnn" in your
commit message and your description; but you should still explain what
the change does.-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bugfix Pull Request

##### COMPONENT NAME
<!--- Name of the module, plugin, module or task -->
inventory
##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
2.x
```


Discussion:
TIL that `.endswith()` accepts a tuple of strings
cc @dagnello @emonty @juliakreger @omgjlk @rcarrillocruz @shrews @thingee
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
The test `ansible-test sanity --test pylint` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/pylint.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:0: syntax-error invalid syntax (&lt;string&gt;, line 509)
lib/ansible/plugins/inventory/virtualbox.py:212:0: syntax-error invalid syntax (&lt;string&gt;, line 212)
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001eea00 -->
<!-- job_id: /testresults/ansible-test-sanity-pylint.json -->
The test `ansible-test sanity --test ansible-doc --python 2.6` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/ansible-doc.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:0:0: has a documentation error formatting or is missing documentation.
lib/ansible/plugins/inventory/virtualbox.py:0:0: has a documentation error formatting or is missing documentation.
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-ansible-doc-python-2.6.json -->
The test `ansible-test sanity --test ansible-doc --python 2.7` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/ansible-doc.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:0:0: has a documentation error formatting or is missing documentation.
lib/ansible/plugins/inventory/virtualbox.py:0:0: has a documentation error formatting or is missing documentation.
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-ansible-doc-python-2.7.json -->
The test `ansible-test sanity --test ansible-doc --python 3.5` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/ansible-doc.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:0:0: has a documentation error formatting or is missing documentation.
lib/ansible/plugins/inventory/virtualbox.py:0:0: has a documentation error formatting or is missing documentation.
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-ansible-doc-python-3.5.json -->
The test `ansible-test sanity --test ansible-doc --python 3.6` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/ansible-doc.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:0:0: has a documentation error formatting or is missing documentation.
lib/ansible/plugins/inventory/virtualbox.py:0:0: has a documentation error formatting or is missing documentation.
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-ansible-doc-python-3.6.json -->
The test `ansible-test sanity --test ansible-doc --python 3.7` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/ansible-doc.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:0:0: has a documentation error formatting or is missing documentation.
lib/ansible/plugins/inventory/virtualbox.py:0:0: has a documentation error formatting or is missing documentation.
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-ansible-doc-python-3.7.json -->
The test `ansible-test sanity --test compile --python 2.6` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/compile.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:68: SyntaxError: if to_text(path.endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):
lib/ansible/plugins/inventory/virtualbox.py:212:67: SyntaxError: valid = super(InventoryModule, self).verify_file(path):
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-compile-python-2.6.json -->
The test `ansible-test sanity --test compile --python 2.7` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/compile.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:68: SyntaxError: if to_text(path.endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):
lib/ansible/plugins/inventory/virtualbox.py:212:67: SyntaxError: valid = super(InventoryModule, self).verify_file(path):
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-compile-python-2.7.json -->
The test `ansible-test sanity --test compile --python 3.5` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/compile.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:68: SyntaxError: if to_text(path.endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):
lib/ansible/plugins/inventory/virtualbox.py:212:67: SyntaxError: valid = super(InventoryModule, self).verify_file(path):
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-compile-python-3.5.json -->
The test `ansible-test sanity --test compile --python 3.6` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/compile.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:68: SyntaxError: if to_text(path.endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):
lib/ansible/plugins/inventory/virtualbox.py:212:67: SyntaxError: valid = super(InventoryModule, self).verify_file(path):
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-compile-python-3.6.json -->
The test `ansible-test sanity --test compile --python 3.7` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/compile.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:68: SyntaxError: if to_text(path.endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):
lib/ansible/plugins/inventory/virtualbox.py:212:67: SyntaxError: valid = super(InventoryModule, self).verify_file(path):
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-compile-python-3.7.json -->
The test `ansible-test sanity --test docs-build` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/docs-build.html)] failed with 11 errors:
```
docs/docsite/rst/dev_guide/developing_inventory.rst:13:0: undefined-label: undefined label: script_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:34:0: undefined-label: undefined label: host_list_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:36:0: undefined-label: undefined label: yaml_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:38:0: undefined-label: undefined label: constructed_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:40:0: undefined-label: undefined label: ini_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:42:0: undefined-label: undefined label: virtualbox_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:44:0: undefined-label: undefined label: script_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:109:0: undefined-label: undefined label: virtualbox_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:111:0: undefined-label: undefined label: host_list_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/dev_guide/developing_inventory.rst:209:0: undefined-label: undefined label: auto_inventory (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/inventory.rst:51:0: toc-tree-glob-pattern-no-match: toctree glob pattern 'inventory/*' didn't match any documents
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-docs-build.json -->
The test `ansible-test sanity --test pep8` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/pep8.html)] failed with 66 errors:
```
lib/ansible/plugins/inventory/auto.py:31:1: E302 expected 2 blank lines, found 1
lib/ansible/plugins/inventory/aws_ec2.py:510:13: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:510:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:510:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:511:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:512:13: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:513:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:515:5: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:516:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:523:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:523:16: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:523:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:528:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:529:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:530:13: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:530:35: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:530:37: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:532:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:532:16: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:532:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:533:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:533:16: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:533:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:534:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:534:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:534:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:535:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:535:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:535:29: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:537:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:539:5: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:540:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:547:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:548:13: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:548:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:548:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:550:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:551:13: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:551:25: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:551:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:553:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:554:13: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:556:9: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:558:5: E128 continuation line under-indented for visual indent
lib/ansible/plugins/inventory/aws_ec2.py:561:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:561:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:565:56: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:565:58: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:567:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:567:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:571:18: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:571:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:574:28: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:574:30: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:575:27: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:575:29: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:578:24: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:578:26: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:581:35: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:581:37: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:586:20: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:586:22: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:588:32: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:588:34: E251 unexpected spaces around keyword / parameter equals
lib/ansible/plugins/inventory/aws_ec2.py:594:1: E901 TokenError: EOF in multi-line statement
lib/ansible/plugins/inventory/virtualbox.py:213:9: E112 expected an indented block
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-pep8.json -->
The test `ansible-test sanity --test yamllint` [[explain](https://docs.ansible.com/ansible/devel/dev_guide/testing/sanity/yamllint.html)] failed with 2 errors:
```
lib/ansible/plugins/inventory/aws_ec2.py:509:68: python-syntax-error invalid syntax (<unknown>, line 509)
lib/ansible/plugins/inventory/virtualbox.py:212:67: python-syntax-error invalid syntax (<unknown>, line 212)
```
<!-- job_id: 5b57553974cdab07007f8915 -->
<!-- job_id: 5b575546c5d4ad09001ee9ff -->
<!-- job_id: /testresults/ansible-test-sanity-yamllint.json -->

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: shippable_test_result --->
Build succeeded (third-party-check pipeline).

- [openstacksdk-ansible-devel-functional-devstack ](http://logs.openstack.org/07/43207/012b876dfdd693d82ee7cebe738bb6696cc523b5/third-party-check/openstacksdk-ansible-devel-functional-devstack/e436418/) : SUCCESS in 40m 46s

Reported errors:
*aws_ec2.py*
```Python
# to text is not closing parenthesis
if to_text(path.endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):

if to_text(path).endswith(('.aws_ec2.yml', '.aws_ec2.yaml')):
```

*virtualBox.py*
```Python
# colon at the end of assignation
            valid = super(InventoryModule, self).verify_file(path): 

# I'd put the whole comparison within an if statement instead
    def verify_file(self, path):
        valid = False
        if (
            super(InventoryModule, self).verify_file(path) and
            to_text(path).endswith(('.vbox.yaml', '.vbox.yml'))
        ):
            valid = True
        return valid
```



Seeing to_text in so many places makes it feel like this conversion is happening at the wrong level.  It's supposed to be converted to text when it enters the system.  So figure out where path is entering ansible and convert it to text there.
Okay, I think what's happening is that you end up using options.inventory before the options are converted into text (IIRC, that was done when options are used as default values for globals.  Someone should probably check whether config manager broke that as well).
config manager is safe... I've traced this back further and found that we're converting the command line options into text type in bin/ansible.  So path should be a text type by the time we handle it in constructed.  Are you calling Ansible as an API?  If so, you need to pass text strings in not byte strings (on Python2, text strings would be unicode type.)
this is a tough one @abadger (convert it `to_text` only once) `path` is declared in several places for different contexts.
My finding is that the variable to be changed is `source`, in the `parse_souce` method from the `InventoryManager` class and well it can go backwards to the class initialization where it is passed as an argument of type `list`; see: [ansible/inventory/manager.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/inventory/manager.py)

I might be wrong, but that's as far as I have tracked this value

Resolving patch:
diff --git a/lib/ansible/cli/__init__.py b/lib/ansible/cli/__init__.py
--- a/lib/ansible/cli/__init__.py
+++ b/lib/ansible/cli/__init__.py
@@ -180,12 +180,6 @@ def run(self):
             ver = deprecated[1]['version']
             display.deprecated("%s option, %s %s" % (name, why, alt), version=ver)
 
-        # Errors with configuration entries
-        if C.config.UNABLE:
-            for unable in C.config.UNABLE:
-                display.error("Unable to set correct type for configuration entry for %s: %s" % (unable, C.config.UNABLE[unable]))
-            raise AnsibleError("Invalid configuration settings")
-
     @staticmethod
     def split_vault_id(vault_id):
         # return (before_@, after_@)
diff --git a/lib/ansible/config/manager.py b/lib/ansible/config/manager.py
--- a/lib/ansible/config/manager.py
+++ b/lib/ansible/config/manager.py
@@ -4,12 +4,12 @@
 from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
+import io
 import os
 import sys
 import stat
 import tempfile
-
-import io
+import traceback
 from collections import namedtuple
 
 from yaml import load as yaml_load
@@ -26,9 +26,11 @@
 from ansible.module_utils._text import to_text, to_bytes, to_native
 from ansible.module_utils.parsing.convert_bool import boolean
 from ansible.parsing.quoting import unquote
+from ansible.utils import py3compat
 from ansible.utils.path import unfrackpath
 from ansible.utils.path import makedirs_safe
 
+
 Plugin = namedtuple('Plugin', 'name type')
 Setting = namedtuple('Setting', 'name value origin type')
 
@@ -180,7 +182,6 @@ def find_ini_config_file(warnings=None):
 
 class ConfigManager(object):
 
-    UNABLE = {}
     DEPRECATED = []
     WARNINGS = set()
 
@@ -309,13 +310,14 @@ def get_config_value(self, config, cfile=None, plugin_type=None, plugin_name=Non
         try:
             value, _drop = self.get_config_value_and_origin(config, cfile=cfile, plugin_type=plugin_type, plugin_name=plugin_name,
                                                             keys=keys, variables=variables, direct=direct)
+        except AnsibleError:
+            raise
         except Exception as e:
-            raise AnsibleError("Invalid settings supplied for %s: %s" % (config, to_native(e)))
+            raise AnsibleError("Unhandled exception when retrieving %s:\n%s" % (config, traceback.format_exc()))
         return value
 
     def get_config_value_and_origin(self, config, cfile=None, plugin_type=None, plugin_name=None, keys=None, variables=None, direct=None):
         ''' Given a config key figure out the actual value and report on the origin of the settings '''
-
         if cfile is None:
             # use default config
             cfile = self._config_file
@@ -351,7 +353,7 @@ def get_config_value_and_origin(self, config, cfile=None, plugin_type=None, plug
 
                 # env vars are next precedence
                 if value is None and defs[config].get('env'):
-                    value, origin = self._loop_entries(os.environ, defs[config]['env'])
+                    value, origin = self._loop_entries(py3compat.environ, defs[config]['env'])
                     origin = 'env: %s' % origin
 
                 # try config file entries next, if we have one
@@ -438,10 +440,17 @@ def update_config_data(self, defs=None, configfile=None):
             try:
                 value, origin = self.get_config_value_and_origin(config, configfile)
             except Exception as e:
-                # when building constants.py we ignore invalid configs
-                # CLI takes care of warnings once 'display' is loaded
-                self.UNABLE[config] = to_text(e)
-                continue
+                # Printing the problem here because, in the current code:
+                # (1) we can't reach the error handler for AnsibleError before we
+                #     hit a different error due to lack of working config.
+                # (2) We don't have access to display yet because display depends on config
+                #     being properly loaded.
+                #
+                # If we start getting double errors printed from this section of code, then the
+                # above problem #1 has been fixed.  Revamp this to be more like the try: except
+                # in get_config_value() at that time.
+                sys.stderr.write("Unhandled error:\n %s\n\n" % traceback.format_exc())
+                raise AnsibleError("Invalid settings supplied for %s: %s\n%s" % (config, to_native(e), traceback.format_exc()))
 
             # set the constant
             self.data.update_setting(Setting(config, value, origin, defs[config].get('type', 'string')))
diff --git a/lib/ansible/utils/py3compat.py b/lib/ansible/utils/py3compat.py
new file mode 100644
--- /dev/null
+++ b/lib/ansible/utils/py3compat.py
@@ -0,0 +1,64 @@
+# -*- coding: utf-8 -*-
+#
+# (c) 2018, Toshio Kuratomi <a.badger@gmail.com>
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+#
+# Note that the original author of this, Toshio Kuratomi, is trying to submit this to six.  If
+# successful, the code in six will be available under six's more liberal license:
+# https://mail.python.org/pipermail/python-porting/2018-July/000539.html
+
+# Make coding more python3-ish
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+import os
+import sys
+from collections import MutableMapping
+
+from ansible.module_utils.six import PY3
+from ansible.module_utils._text import to_bytes, to_text
+
+__all__ = ('environ',)
+
+
+class _TextEnviron(MutableMapping):
+    """
+    Utility class to return text strings from the environment instead of byte strings
+
+    Mimics the behaviour of os.environ on Python3
+    """
+    def __init__(self, env=None):
+        if env is None:
+            env = os.environ
+        self._raw_environ = env
+        self._value_cache = {}
+        # Since we're trying to mimic Python3's os.environ, use sys.getfilesystemencoding()
+        # instead of utf-8
+        self.encoding = sys.getfilesystemencoding()
+
+    def __delitem__(self, key):
+        del self._raw_environ[key]
+
+    def __getitem__(self, key):
+        value = self._raw_environ[key]
+        if PY3:
+            return value
+        # Cache keys off of the undecoded values to handle any environment variables which change
+        # during a run
+        if value not in self._value_cache:
+            self._value_cache[value] = to_text(value, encoding=self.encoding,
+                                               nonstring='passthru', errors='surrogate_or_strict')
+        return self._value_cache[value]
+
+    def __setitem__(self, key, value):
+        self._raw_environ[key] = to_bytes(value, encoding=self.encoding, nonstring='strict',
+                                          errors='surrogate_or_strict')
+
+    def __iter__(self):
+        return self._raw_environ.__iter__()
+
+    def __len__(self):
+        return len(self._raw_environ)
+
+
+environ = _TextEnviron()


Test patch: