CVE-2017-7481: Improper Input Validation (CWE-20)
value: Ansible before versions 2.3.1.0 and 2.4.0.0 fails to properly mark lookup-plugin results as unsafe. If an attacker could control the results of lookup() calls, they could inject Unicode strings to be parsed by the jinja2 templating system, resulting in code execution. By default, the jinja2 templating language is now marked as 'unsafe' and is not evaluated.
CWE description: The product receives input or data, but it does
        not validate or incorrectly validates that the input has the
        properties that are required to process the data safely and
        correctly.
CVSS v3 base score: 9.8
Published: 2018-07-19T13:29Z
Repository: https://github.com/ansible/ansible
Language: Python
Files changed: docs/docsite/rst/intro_configuration.rst, examples/ansible.cfg, lib/ansible/constants.py, lib/ansible/template/__init__.py
Commit date: 2017-05-08 12:43:46 -0500
Commit message: Fixing security issue with lookup returns not tainting the jinja2 environment

CVE-2017-7481

Lookup returns wrap the result in unsafe, however when used through the
standard templar engine, this does not result in the jinja2 environment being
marked as unsafe as a whole. This means the lookup result looses the unsafe
protection and may become simple unicode strings, which can result in bad
things being re-templated.

This also adds a global lookup param and cfg options for lookups to allow
unsafe returns, so users can force the previous (insecure) behavior.

Diff: diff --git a/docs/docsite/rst/intro_configuration.rst b/docs/docsite/rst/intro_configuration.rst
index 3647e22b7b..259e1074de 100644
--- a/docs/docsite/rst/intro_configuration.rst
+++ b/docs/docsite/rst/intro_configuration.rst
@@ -84,10 +84,24 @@ different locations::
    action_plugins = ~/.ansible/plugins/action_plugins/:/usr/share/ansible_plugins/action_plugins
 
 Most users will not need to use this feature.  See :doc:`dev_guide/developing_plugins` for more details.
 
 
+.. _allow_unsafe_lookups:
+
+allow_unsafe_lookups
+====================
+
+.. versionadded:: 2.2.3, 2.3.1
+
+When enabled, this option allows lookup plugins (whether used in variables as `{{lookup('foo')}}` or as a loop as `with_foo`) to return data that is **not** marked "unsafe". By default, such data is marked as unsafe to prevent the templating engine from evaluating any jinja2 templating language, as this could represent a security risk.
+
+This option is provided to allow for backwards-compatibility, however users should first consider adding `allow_unsafe=True` to any lookups which may be expected to contain data which may be run through the templating engine later. For example::
+
+    {{lookup('pipe', '/path/to/some/command', allow_unsafe=True)}}
+
+
 .. _allow_world_readable_tmpfiles:
 
 allow_world_readable_tmpfiles
 =============================
 
diff --git a/examples/ansible.cfg b/examples/ansible.cfg
index e283064603..77ba5d20d4 100644
--- a/examples/ansible.cfg
+++ b/examples/ansible.cfg
@@ -280,11 +280,11 @@
 #merge_multiple_cli_flags = True
 
 # Controls showing custom stats at the end, off by default
 #show_custom_stats = True
 
-# Controlls which files to ignore when using a directory as inventory with
+# Controls which files to ignore when using a directory as inventory with
 # possibly multiple sources (both static and dynamic)
 #inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo
 
 # This family of modules use an alternative execution path optimized for network appliances
 # only update this setting if you know how this works, otherwise it can break module execution
@@ -292,10 +292,16 @@
 
 # This keeps facts from polluting the main namespace as variables.
 # Setting to True keeps them under the ansible_facts namespace, the default is False
 #restrict_facts_namespace: True
 
+# When enabled, this option allows lookups (via variables like {{lookup('foo')}} or when used as
+# a loop with `with_foo`) to return data that is not marked "unsafe". This means the data may contain
+# jinja2 templating language which will be run through the templating engine.
+# ENABLING THIS COULD BE A SECURITY RISK
+#allow_unsafe_lookups = False
+
 [privilege_escalation]
 #become=True
 #become_method=sudo
 #become_user=root
 #become_ask_pass=False
diff --git a/lib/ansible/constants.py b/lib/ansible/constants.py
index da4503753e..40d1038d16 100644
--- a/lib/ansible/constants.py
+++ b/lib/ansible/constants.py
@@ -234,10 +234,11 @@ DEFAULT_LOG_PATH          = get_config(p, DEFAULTS, 'log_path',           'ANSIB
 DEFAULT_FORCE_HANDLERS    = get_config(p, DEFAULTS, 'force_handlers', 'ANSIBLE_FORCE_HANDLERS', False, value_type='boolean')
 DEFAULT_INVENTORY_IGNORE  = get_config(p, DEFAULTS, 'inventory_ignore_extensions', 'ANSIBLE_INVENTORY_IGNORE',
                                        ["~", ".orig", ".bak", ".ini", ".cfg", ".retry", ".pyc", ".pyo"], value_type='list')
 DEFAULT_VAR_COMPRESSION_LEVEL = get_config(p, DEFAULTS, 'var_compression_level', 'ANSIBLE_VAR_COMPRESSION_LEVEL', 0, value_type='integer')
 DEFAULT_INTERNAL_POLL_INTERVAL = get_config(p, DEFAULTS, 'internal_poll_interval', None, 0.001, value_type='float')
+DEFAULT_ALLOW_UNSAFE_LOOKUPS = get_config(p, DEFAULTS, 'allow_unsafe_lookups', None, False, value_type='boolean')
 ERROR_ON_MISSING_HANDLER  = get_config(p, DEFAULTS, 'error_on_missing_handler', 'ANSIBLE_ERROR_ON_MISSING_HANDLER', True, value_type='boolean')
 SHOW_CUSTOM_STATS = get_config(p, DEFAULTS, 'show_custom_stats', 'ANSIBLE_SHOW_CUSTOM_STATS', False, value_type='boolean')
 NAMESPACE_FACTS = get_config(p, DEFAULTS, 'restrict_facts_namespace', 'ANSIBLE_RESTRICT_FACTS', False, value_type='boolean')
 
 # static includes
diff --git a/lib/ansible/template/__init__.py b/lib/ansible/template/__init__.py
index 5d551d7b2f..49de8aacc1 100644
--- a/lib/ansible/template/__init__.py
+++ b/lib/ansible/template/__init__.py
@@ -250,10 +250,13 @@ class Templar:
             extensions=self._get_extensions(),
             finalize=self._finalize,
             loader=FileSystemLoader(self._basedir),
         )
 
+        # the current rendering context under which the templar class is working
+        self.cur_context = None
+
         self.SINGLE_VAR = re.compile(r"^%s\s*(\w*)\s*%s$" % (self.environment.variable_start_string, self.environment.variable_end_string))
 
         self._clean_regex   = re.compile(r'(?:%s|%s|%s|%s)' % (
             self.environment.variable_start_string,
             self.environment.block_start_string,
@@ -572,10 +575,11 @@ class Templar:
     def _lookup(self, name, *args, **kwargs):
         instance = self._lookup_loader.get(name.lower(), loader=self._loader, templar=self)
 
         if instance is not None:
             wantlist = kwargs.pop('wantlist', False)
+            allow_unsafe = kwargs.pop('allow_unsafe', C.DEFAULT_ALLOW_UNSAFE_LOOKUPS)
 
             from ansible.utils.listify import listify_lookup_plugin_terms
             loop_terms = listify_lookup_plugin_terms(terms=args, templar=self, loader=self._loader, fail_on_undefined=True, convert_bare=False)
             # safely catch run failures per #5059
             try:
@@ -586,11 +590,12 @@ class Templar:
                 if self._fail_on_lookup_errors:
                     raise AnsibleError("An unhandled exception occurred while running the lookup plugin '%s'. Error was a %s, "
                                        "original message: %s" % (name, type(e), e))
                 ran = None
 
-            if ran:
+            if ran and not allow_unsafe:
+                from ansible.vars.unsafe_proxy import UnsafeProxy, wrap_var
                 if wantlist:
                     ran = wrap_var(ran)
                 else:
                     try:
                         ran = UnsafeProxy(",".join(ran))
@@ -598,10 +603,12 @@ class Templar:
                         if isinstance(ran, list) and len(ran) == 1:
                             ran = wrap_var(ran[0])
                         else:
                             ran = wrap_var(ran)
 
+                if self.cur_context:
+                    self.cur_context.unsafe = True
             return ran
         else:
             raise AnsibleError("lookup plugin (%s) not found" % name)
 
     def do_template(self, data, preserve_trailing_newlines=True, escape_backslashes=True, fail_on_undefined=None, overrides=None, disable_lookups=False):
@@ -654,11 +661,11 @@ class Templar:
 
             t.globals['finalize'] = self._finalize
 
             jvars = AnsibleJ2Vars(self, t.globals)
 
-            new_context = t.new_context(jvars, shared=True)
+            self.cur_context = new_context = t.new_context(jvars, shared=True)
             rf = t.root_render_func(new_context)
 
             try:
                 res = j2_concat(rf)
                 if new_context.unsafe:

Vulnerable code: # Controlls which files to ignore when using a directory as inventory with
            if ran:
            new_context = t.new_context(jvars, shared=True)
Fixed code: .. _allow_unsafe_lookups:

allow_unsafe_lookups
====================

.. versionadded:: 2.2.3, 2.3.1

When enabled, this option allows lookup plugins (whether used in variables as `{{lookup('foo')}}` or as a loop as `with_foo`) to return data that is **not** marked "unsafe". By default, such data is marked as unsafe to prevent the templating engine from evaluating any jinja2 templating language, as this could represent a security risk.

This option is provided to allow for backwards-compatibility, however users should first consider adding `allow_unsafe=True` to any lookups which may be expected to contain data which may be run through the templating engine later. For example::

    {{lookup('pipe', '/path/to/some/command', allow_unsafe=True)}}


# Controls which files to ignore when using a directory as inventory with
# When enabled, this option allows lookups (via variables like {{lookup('foo')}} or when used as
# a loop with `with_foo`) to return data that is not marked "unsafe". This means the data may contain
# jinja2 templating language which will be run through the templating engine.
# ENABLING THIS COULD BE A SECURITY RISK
#allow_unsafe_lookups = False

DEFAULT_ALLOW_UNSAFE_LOOKUPS = get_config(p, DEFAULTS, 'allow_unsafe_lookups', None, False, value_type='boolean')
        # the current rendering context under which the templar class is working
        self.cur_context = None

            allow_unsafe = kwargs.pop('allow_unsafe', C.DEFAULT_ALLOW_UNSAFE_LOOKUPS)
            if ran and not allow_unsafe:
                from ansible.vars.unsafe_proxy import UnsafeProxy, wrap_var
                if self.cur_context:
                    self.cur_context.unsafe = True
            self.cur_context = new_context = t.new_context(jvars, shared=True)