Repository: ansible/ansible

Issue:
with_subelements lookup plugin (or similar)
The question comes up how to distribute SSH keys to different users.

In this case, suppose we have:

```
users:
   - name:  xyz
     ssh_keys: [ /path/to/x, y, z ]
   - name: abc
     ssh_keys: [ xx, yy ]
```

We might want to use the authorized_key module in a with_items loop, so it would be nice to say:

```
 with_subelements:
     - users:
     - 'ssh_keys'
```

And in this invocation, it would return {{ item.0 }} as each record, and {{ item.1 }} as each subelement of the given field name, as such:

   # configure all authorized keys for all users

```
- authorized_key:  name={{ item.0.name }} key={{ lookup('file', item[1]) }}
   with_subelements:
      - users
      - 'ssh_keys'
```

Alternative proposals welcome.

Having a sufficient number of lookup plugins for the 'odd' cases.



Discussion:
Alternative proposals:

Example playbook:

```
---
- hosts: all
  connection: local
  vars:
    - users:
        mark:
          name: Real Name
          ssh_key: "ssh-rsa AAAAA...."
        foo:
          name: Foo Barrington
          ssh_key: "ssh-rsa AAAA...."

  tasks:
    - debug: msg="{{users[item]['name']}}"
      with_items: users.keys()
```

This prints out: "Real Name" and "Foo Barrington" in turn.

If you relax the with_items plugin to allow a dict instead of a list (only
required on run function check, the flatten check can stay the same), then the
with_items check can be changed to just say `with_items: users`.

I just realized this is a little different than your original layout of users,
but the layut described above looks like it shouldn't need any changes. The
following works perfectly:

```
- hosts: all
  connection: local
  vars:
    - users:
        - name: foo
          ssh_key: "ssh-rsa AAAAA...."
        - name: bar
          ssh_key: "ssh-rsa AAAA...."

  tasks:
    - debug: msg="{{item.name}} - {{item.ssh_key}}"
      with_items: users
```

I realized my comment didn't address the specific issue described in the issue above, namely how to do a nested loop where the inner loop is a key of a dictionary instead of a straight list.

An alternative proposal for the issue mentioned could be something like:

```
with_nested:
  - users
  - item['ssh_keys']
```

Problems I'm seeing with doing it this way:
- The nested plugin currently doesn't identify anything other than a basic variable name, so foo['bar'] wouldn't be recognized as something that should be expanded/evaluated.
- The nested plugin would need to potentially evaluate the inner loop for every invocation of the outer loop, which doesn't seem to be done currently.

Advantages:
- No new plugin, just extending the functionality of an existing one.
- You could do something more complex on the variable name, just like with the with_items plugin (e.g. jinja2 filters, applying python functions), which may cover more cases that aren't being considered right now.

The first example you suggested might not actually do it, because what I'm suggesting is dealing with a user who has multiple SSH keys.

The second example is perhaps interesting, if we can tweak with_nested without much evils.   It's probably possible and would be worth exploring.   item.ssh_keys would also be acceptable if that were done.

Proposal is that it should be able to do iterate over

hash of 

```
x:
     foo: splat
     keys: []
y: 
     foo: splat2
     keys: []
```

And also lists of users like my 1st post too

Ouch, looks like a missed this one, was thinking of exact the same thing this week.

Question: should this support just with 1 subloop, or also going deeper? (just like nested and together can take more than two list elements and do more than one subloop)

I am thinking one fits nearly all use cases?

Can you post an example datastructure and what that might look like in playbook syntax?

I don't have a use case for that, just wanted to know what you expect.

So the proposal is: with_subelements takes a list of two items, the first being a hash/dict, or a list, the second being a string pointing to a (sub)key in that hash/dict or list.

In case it's a hash like

``` yaml
x:
     foo: splat
     keys: []
y: 
     foo: splat2
     keys: []
```

will the main keys x and y appear in the resulting list?

I'm not sure if updating with_nested would be the best way to go; with_nested can also take more than two list items. Might also confuse users.

Yes, the proposal is about returning the element x or y as item[0] and then
iterating keys as item[1]

On Sat, Sep 7, 2013 at 9:36 AM, Serge van Ginderachter <
notifications@github.com> wrote:

> I don't have a use case for that, just wanted to know what you expect.
> 
> So the proposal is: with_subelements takes a list of two items, the first
> being a hash/dict, or a list, the second being a string pointing to a
> (sub)key in that hash/dict or list.
> 
> In case it's a hash like
> 
> x:
>      foo: splat
>      keys: []y:
>      foo: splat2
>      keys: []
> 
> will the main keys x and y appear in the resulting list?
> 
> I'm not sure if updating with_nested would be the best way to go;
> with_nested can also take more than two list items. Might also confuse
> users.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/ansible/ansible/issues/3752#issuecomment-23994319
> .

OK, and in case of a list the iterating keys just as item.


Resolving patch:
diff --git a/lib/ansible/runner/lookup_plugins/subelements.py b/lib/ansible/runner/lookup_plugins/subelements.py
new file mode 100644
--- /dev/null
+++ b/lib/ansible/runner/lookup_plugins/subelements.py
@@ -0,0 +1,61 @@
+# (c) 2013, Serge van Ginderachter <serge@vanginderachter.be>
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+import ansible.utils as utils
+import ansible.errors as errors
+
+
+class LookupModule(object):
+
+    def __init__(self, basedir=None, **kwargs):
+        self.basedir = basedir
+
+
+    def run(self, terms, inject=None, **kwargs):
+        terms = utils.listify_lookup_plugin_terms(terms, self.basedir, inject)
+        terms[0] = utils.listify_lookup_plugin_terms(terms[0], self.basedir, inject)
+
+        if not isinstance(terms, list) or not len(terms) == 2:
+            raise errors.AnsibleError(
+                "subelements lookup expects a list of two items, first a dict or a list, and second a string")
+        terms[0] = utils.listify_lookup_plugin_terms(terms[0], self.basedir, inject)
+        if not isinstance(terms[0], (list, dict)) or not isinstance(terms[1], basestring):
+            raise errors.AnsibleError(
+                "subelements lookup expects a list of two items, first a dict or a list, and second a string")
+
+        if isinstance(terms[0], dict): # convert to list:
+            elementlist = []
+            for key in terms[0].iterkeys():
+                elementlist.append(terms[0][key])
+        else: 
+            elementlist = terms[0]
+        subelement = terms[1]
+
+        ret = []
+        for item0 in elementlist:
+            if not isinstance(item0, dict):
+                raise errors.AnsibleError("subelements lookup expects a dictionary, got '%s'" %item0)
+            if not subelement in item0:
+                raise errors.AnsibleError("could not find '%s' key in iterated item '%s'" % (subelement, item0))
+            if not isinstance(item0[subelement], list):
+                raise errors.AnsibleError("the key %s should point to a list, got '%s'" % (subelement, item0[subelement]))
+            sublist = item0.pop(subelement, [])
+            for item1 in sublist:
+                ret.append((item0, item1))
+
+        return ret
+


Test patch: