Repository: ansible/ansible
Pull request: Force collections to be static
Templating of collection names does not work at all. Force them to
be static so that a warning is generated for the user.

##### SUMMARY

Attempting to template a collection name within a playbook results in any of the collection roles/modules/etc not being found and generates an error. This will force the collection to be a static attribute, disallowing templating, and generating a warning message for the user.

##### ISSUE TYPE

- Bugfix Pull Request

##### COMPONENT NAME

collections

##### ADDITIONAL INFORMATION

Closes #68704


Problem statement:
Using a variable in FQCN of the collection called inside the playbook does not find the roles
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Explain the problem briefly below -->
While trying to include roles from collections in a dynamic manner, I used a variable as name for the collection name (eg. my_namespace.{{ collection_var_name }}.role1 ). By doing so, the role is not found. Replacing the collection_var_name with the collection name correctly i can use the role without issues.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
Collections

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```

ansible 2.9.6
  configured module search path = [u'/Users/alexandruvinogradov/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/alexandruvinogradov/Library/Python/2.7/lib/python/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.16 (default, Jan 27 2020, 04:46:15) [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)]

```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```
no modification
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
MacOS Mojave 10.14.6


##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->

<!--- Paste example playbooks or commands between quotes below -->
Use the playbook below to try to execute role1 from a pre-created collection. Call the collection by passing the var collection_name to match the collection name. 
ansible-playbook playbook.yml  -e'{"collection_name":"name"}'

```
---
- hosts: all  
  collections:
   - namespace.{{ collection_name }}
  tasks:
    - import_role:
        name: role1
```

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- Describe what you expected to happen when running the steps above -->
The expected behaviour would be that the role from the collection passed as variable gets executed. If i were to replace the variable in the playbook with the actual passed name, the role gets executed correctly.


##### ACTUAL RESULTS
<!--- Describe what actually happened. If possible run with extra verbosity (-vvvv) -->
The role is not recognized as part of the collection.
<!--- Paste verbatim command output between quotes -->
```
ERROR! the role 'role1' was not found

The error appears to be in 'playbook.yml': line 7, column 15, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

    - import_role:
        name: role1
              ^ here

```

Hints from issue discussion:
Files identified in the description:
* [`lib/ansible/module_utils/common/collections.py`](https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/common/collections.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->

Commit message: Force collections to be static

Templating of collection names does not work at all. Force them to
be static so that a warning is generated for the user.

Patch to review:
diff --git a/lib/ansible/playbook/collectionsearch.py b/lib/ansible/playbook/collectionsearch.py
index 93b80a8665761e..ef988d742d58c0 100644
--- a/lib/ansible/playbook/collectionsearch.py
+++ b/lib/ansible/playbook/collectionsearch.py
@@ -7,6 +7,10 @@
 from ansible.module_utils.six import string_types
 from ansible.playbook.attribute import FieldAttribute
 from ansible.utils.collection_loader import AnsibleCollectionLoader
+from ansible.template import is_template, Environment
+from ansible.utils.display import Display
+
+display = Display()
 
 
 def _ensure_default_collection(collection_list=None):
@@ -32,9 +36,18 @@ def _ensure_default_collection(collection_list=None):
 class CollectionSearch:
 
     # this needs to be populated before we can resolve tasks/roles/etc
-    _collections = FieldAttribute(isa='list', listof=string_types, priority=100, default=_ensure_default_collection)
+    _collections = FieldAttribute(isa='list', listof=string_types, priority=100, default=_ensure_default_collection,
+                                  always_post_validate=True, static=True)
 
     def _load_collections(self, attr, ds):
+        # This duplicates static attr checking logic from post_validate()
+        # because if the user attempts to template a collection name, it will
+        # error before it ever gets to the post_validate() warning.
+        for collection_name in ds:
+            if is_template(collection_name, Environment()):
+                display.warning('"collections" is not templatable, but we found: %s, '
+                                'it will not be templated and will be used "as is".' % (collection_name))
+
         # this will only be called if someone specified a value; call the shared value
         _ensure_default_collection(collection_list=ds)
 


Code review comments:
diff_hunk: @@ -32,9 +36,18 @@ def _ensure_default_collection(collection_list=None):
 class CollectionSearch:
 
     # this needs to be populated before we can resolve tasks/roles/etc
-    _collections = FieldAttribute(isa='list', listof=string_types, priority=100, default=_ensure_default_collection)
+    _collections = FieldAttribute(isa='list', listof=string_types, priority=100, default=_ensure_default_collection,
+                                  always_post_validate=True, static=True)
 
     def _load_collections(self, attr, ds):
+        # This duplicates static attr checking logic from post_validate()
+        # because if the user attempts to template a collection name, it will
+        # error before it ever gets to the post_validate() warning.
+        for collection_name in ds:
+            if is_template(collection_name, Environment()):
path: lib/ansible/playbook/collectionsearch.py
text: @user1:
Instantiate `Environment` 1 time and re-use it.

To address other potential questions here, we don't allow for things like `variable_start` to be overriden in the playbook "DSL", as such, we don't need to account for that here.  Although we could in the future, but I'm not concerned.

Also, `Environment.lex` doesn't care about undefined vars, or non-existent filters, etc...So we don't need that context anyway.

@user1:
I think this code may actually need to be moved after `_ensure_default_collection`.  `_ensure_default_collection` will make sure the value is formatted correctly.  In the case where someone just does `collections: whatever` it will be turned into a list as well.  So maybe it needs moved all the way to the bottom.

@author:
Doh! on the Environment usage. All requested changes applied.