Repository: ansible-collections/community.general

Issue:
Proxmox inventory plugin fails to parse with auto plugin: VM-Template is not a known host nor group
### Summary

When using `community.general.proxmox` as an inventory plugin to connect to a proxmox cluster, ansible crashes with an error `[WARNING]:  * Failed to parse /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml with auto plugin: VM-Template is not a known host nor group`

I have not setup the proxmox environment nor do I have great knowledge of proxmox to begin with.

### Issue Type

Bug Report

### Component Name

proxmox

### Ansible Version

```
❯ ansible --version
ansible 2.10.6
  config file = /home/user/Documents/work/customer/proxmox_inventering/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible
  executable location = /home/user/.pyenv/versions/customer-inventory-hypervisor-migration/bin/ansible
  python version = 3.9.2 (default, Feb 20 2021, 18:40:11) [GCC 10.2.0]
```

```
❯ ansible-galaxy collection list | ag "(community.general|^#)"
# /home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible_collections
community.general             2.2.0
# /home/user/.ansible/collections/ansible_collections
community.general    2.2.0
```

### Configuration

```
$ ansible-config dump --only-changed
DEFAULT_HOST_LIST(/home/user/Documents/work/customer/proxmox_inventering/ansible.cfg) = ['/home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml']
HOST_KEY_CHECKING(/home/user/Documents/work/customer/proxmox_inventering/ansible.cfg) = False
INTERPRETER_PYTHON(/home/user/Documents/work/customer/proxmox_inventering/ansible.cfg) = /home/user/.pyenv/versions/customer-inventory-hypervisor-migration/bin/python
```

### OS / Environment

Proxmox 6.3-3

### Steps to Reproduce

```
❯ cat inventory.proxmox.yaml
plugin: community.general.proxmox
url: https://1.2.3.4:8006
user: user@pve
password: redacted
validate_certs: no
#want_facts: yes
```

```
❯ ansible-inventory --list -i inventory.proxmox.yaml -vvvv
```

I modified the source inventory plugin (`community/general/plugins/inventory/proxmox.py`) to drop to debug if that particular object is being hit;

```
340                 for member in self._get_members_per_pool(pool['poolid']):
341                     if member.get('name'):
342                         try:
343                             self.inventory.add_child(pool_group, member['name'])
344                         except:
345                             breakpoint()
```

and the object looks like:
```
-> for member in self._get_members_per_pool(pool['poolid']):
(Pdb) member
{'uptime': 0, 'netin': 0, 'name': 'VM-Template', 'mem': 0, 'node': 'a-bc12-pve3', 'maxdisk': 34359738368, 'disk': 0, 'template': 1, 'netout': 0, 'cpu': 0, 'type': 'qemu', 'diskread': 0, 'diskwrite': 0, 'id': 'qemu/9997', 'status': 'stopped', 'vmid': 9997, 'maxmem': 4294967296, 'maxcpu': 4}
```

### Expected Results

I expected ansible-inventory to iterate over the specific proxmox inventory and return a list of vms as hosts to connect to.

### Actual Results

```
❯ ansible-inventory --list -i inventory.proxmox.yaml -vvvv
ansible-inventory 2.10.6
  config file = /home/user/Documents/work/customer/proxmox_inventering/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible
  executable location = /home/user/.pyenv/versions/customer-inventory-hypervisor-migration/bin/ansible-inventory
  python version = 3.9.2 (default, Feb 20 2021, 18:40:11) [GCC 10.2.0]
Using /home/user/Documents/work/customer/proxmox_inventering/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml as it did not pass its verify_file() method
script declined parsing /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml as it did not pass its verify_file() method
Loading collection community.general from /home/user/.ansible/collections/ansible_collections/community/general
/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/urllib3/connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host '1.2.3.4'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
toml declined parsing /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml as it did not pass its verify_file() method
[WARNING]:  * Failed to parse /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml with auto plugin: VM-Template is not a known host nor group
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/inventory/manager.py", line 289, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
    plugin.parse(inventory, loader, path, cache=cache)
  File "/home/user/.ansible/collections/ansible_collections/community/general/plugins/inventory/proxmox.py", line 362, in parse
    self._populate()
  File "/home/user/.ansible/collections/ansible_collections/community/general/plugins/inventory/proxmox.py", line 342, in _populate
    self.inventory.add_child(pool_group, member['name'])
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/inventory/data.py", line 265, in add_child
    raise AnsibleError("%s is not a known host nor group" % child)
[WARNING]:  * Failed to parse /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml with yaml plugin: Plugin configuration YAML file, not YAML inventory
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/inventory/manager.py", line 289, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/plugins/inventory/yaml.py", line 112, in parse
    raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
[WARNING]:  * Failed to parse /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/inventory/manager.py", line 289, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/home/user/.pyenv/versions/customer-inventory-hypervisor-migration/lib/python3.9/site-packages/ansible/plugins/inventory/ini.py", line 138, in parse
    raise AnsibleParserError(e)
[WARNING]: Unable to parse /home/user/Documents/work/customer/proxmox_inventering/inventory.proxmox.yaml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
{
    "_meta": {
        "hostvars": {}
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    }
}
```

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/cloud/misc/proxmox_snap.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/cloud/misc/proxmox_snap.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Aversiste @Thulium-Drake @joshainglis @karmab
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
!component =plugins/inventory/proxmox.py
Files identified in the description:
* [`plugins/inventory/proxmox.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/inventory/proxmox.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
Hi @fiskhest ,

Thanks for reporting this! But I seem unable to reproduce this on my proxmox 6.3-4 server. I'm running with the following stuff:

```
ansible-galaxy collection list

# /root/projects/playground/collections/ansible_collections
Collection              Version
----------------------- -------
community.general       2.2.0  

ansible 2.10.6
  config file = /root/projects/playground/ansible.cfg
  configured module search path = ['/root/projects/playground/library/modules']
  ansible python module location = /opt/ansible/2.10.6/lib/python3.7/site-packages/ansible
  executable location = /opt/ansible/2.10.6/bin/ansible
  python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
```

When I create a qemu machine named 'VM-Template' with the same ID you used, it shows up like any other qemu machine on my laptop. Once I convert it to a template, it disappears (as I'd expect, templates are inert VM's).

Is there anything else weird going on with that template? Did you check if you have this issue on a different Proxmox system as well?
> Hi @fiskhest ,
> 
> Thanks for reporting this! But I seem unable to reproduce this on my proxmox 6.3-4 server. I'm running with the following stuff:
> 
> ```
> ansible-galaxy collection list
> 
> # /root/projects/playground/collections/ansible_collections
> Collection              Version
> ----------------------- -------
> community.general       2.2.0  
> 
> ansible 2.10.6
>   config file = /root/projects/playground/ansible.cfg
>   configured module search path = ['/root/projects/playground/library/modules']
>   ansible python module location = /opt/ansible/2.10.6/lib/python3.7/site-packages/ansible
>   executable location = /opt/ansible/2.10.6/bin/ansible
>   python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
> ```
> 
> When I create a qemu machine named 'VM-Template' with the same ID you used, it shows up like any other qemu machine on my laptop. Once I convert it to a template, it disappears (as I'd expect, templates are inert VM's).
> 
> Is there anything else weird going on with that template? Did you check if you have this issue on a different Proxmox system as well?

The issue is that the template is in a pool.

It is intentionally not added to the inventory as a host:

```
  for qemu in self._get_qemu_per_node(node['node']):
                    if qemu['template']:
                        continue

                    self.inventory.add_host(qemu['name'])
```

But that same exclusion logic is not applied after pool members are fetched so this loop attempts to add a non-existent child to the pool group.
```
 for member in self._get_members_per_pool(pool['poolid']):
                    if member.get('name'):
                        self.inventory.add_child(pool_group, member['name'])
```
@Ajpantuso Right.. Thanks for pointing that out :-)

I made a fix for it! ;-)
Confirm the fix in #1991 works perfectly, thank you @Thulium-Drake @Ajpantuso  :+1: 

Resolving patch:
diff --git a/plugins/inventory/proxmox.py b/plugins/inventory/proxmox.py
--- a/plugins/inventory/proxmox.py
+++ b/plugins/inventory/proxmox.py
@@ -325,7 +325,8 @@ def _populate(self):
 
                 for member in self._get_members_per_pool(pool['poolid']):
                     if member.get('name'):
-                        self.inventory.add_child(pool_group, member['name'])
+                        if not member.get('template'):
+                            self.inventory.add_child(pool_group, member['name'])
 
     def parse(self, inventory, loader, path, cache=True):
         if not HAS_REQUESTS:


Test patch: