Repository: ansible-collections/community.general

Issue:
Proxmox Inventory fails when a cluster node is offline
### Summary

When querying the proxmox API with the ansible inventory plugin. If one server of the cluster is offline then the inventory fails to build.

i.e. Proxmox inventory connects to server2 and if server1 is offline then the proxmox inventory is unable to generate an inventory.

### Issue Type

Bug report

### Component Name

plugins/inventory/proxmox.py

### Ansible Version

```console (paste below)
$ ansible --version
ansible 2.10.7
  config file = /home/adamus1red/ansible/ansible.cfg
  configured module search path = ['/home/adamus1red/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/adamus1red/ansible/python-debian/lib/python3.7/site-packages/ansible
  executable location = /home/adamus1red/ansible/python-debian/bin/ansible
  python version = 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed
ANSIBLE_SSH_ARGS(/home/adamus1red/ansible/ansible.cfg) = -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no
INVENTORY_ENABLED(/home/adamus1red/ansible/ansible.cfg) = ['host_list', 'script', 'auto', 'yaml', 'ini', 'toml', 'vmware_vm_inventory', 'community.general.proxmox']
```


### OS / Environment

Proxmox Debian

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
### Inventory
```yaml
plugin: community.general.proxmox
url: https://pve-02.lan:8006
user: Ansible@pve
password: ****
validate_certs: yes
want_facts: yes
```
### Playbook
```yaml
---
- hosts: all
  become: true
  remote_user: root
  roles:
    - base
```

### Expected Results

I expect an inventory to be generated from the remaining hosts in the cluster.

### Actual Results

```console (paste below)
[WARNING]:  * Failed to parse /home/adamus1red/ansible/pve.proxmox.yml with auto plugin: 595 Server Error: No route to host for url: https://pve-02.lan:8006/api2/json/nodes/pve-01/network
[WARNING]:  * Failed to parse /home/adamus1red/ansible/pve.proxmox.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory
[WARNING]:  * Failed to parse /home/adamus1red/ansible/pve.proxmox.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
[WARNING]:  * Failed to parse /home/adamus1red/ansible/pve.proxmox.yml with ansible_collections.community.general.plugins.inventory.proxmox plugin: 595 Server Error: No route to host for url:
https://pve-02.lan:8006/api2/json/nodes/pve-01/network
[WARNING]: Unable to parse /home/adamus1red/ansible/pve.proxmox.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
[WARNING]: Could not match supplied host pattern, ignoring: ven-*

```

When server pve-01 is online the inventory is generated as expected.

### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/cloud/misc/proxmox.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/cloud/misc/proxmox.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Ajpantuso @Thulium-Drake @UnderGreen @joshainglis @karmab @tleguern
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
!component =plugins/inventory/proxmox.py
Files identified in the description:
* [`plugins/inventory/proxmox.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/inventory/proxmox.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @ilijamt
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
if the pve02 host is down there is no way to generate the inventory, you need to point it to an active host. 
In the log I see that the pve02 host is down and then the behavior is expected.  (If you point it to pve01 then it will generate the inventory for all except the downed node)

The inventory has no idea of a proxmox, even when the node is up, the node that you query generates the data from all the nodes.

Currently the inventory doesn't support multiple hosts when that is implemented then this issue won't be a problem.
> if the pve02 host is down there is no way to generate the inventory, you need to point it to an active host. 

Pve02 is the online host. Pve01 is the offline host
Given the specific error message:
```
[WARNING]:  * Failed to parse /home/adamus1red/ansible/pve.proxmox.yml with ansible_collections.community.general.plugins.inventory.proxmox plugin: 595 Server Error: No route to host for url:
https://pve-02.lan:8006/api2/json/nodes/pve-01/network
```
I reckon it popped up from: 
https://github.com/ansible-collections/community.general/blob/main/plugins/inventory/proxmox.py#L225-L232

But I am not very familiar with inventory plugins nor proxmox. Not sure how to mitigate (if possible).
If you run the same curl command what does the server give you?

I will need to spin up two proxmox installations to test this.

On Sun, 4 Jul 2021, 02:16 Alexei Znamensky, ***@***.***>
wrote:

> Give the specific error message, I reckon it popped up from:
> https://github.com/ansible-collections/community.general/blob/main/plugins/inventory/proxmox.py#L225-L232
>
> But I am not very familiar with inventory plugins nor proxmox. Not sure
> how to mitigate (if possible).
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/ansible-collections/community.general/issues/2931#issuecomment-873489198>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAIWFY3FQW666FBZEVMA2H3TV6R6VANCNFSM47YD6EPA>
> .
>

The only way to mitigate this is if the plugin accepts a list of URLs, and then tries all of them until one works (or fails if none work). Right now the plugin only accepts one URL.

So this is basically a feature request.
Hmm, looks like I didn't completely reload, and missed some of the messages. Sorry :)
> 
> If you run the same curl command what does the server give you? 
>
> I will need to spin up two proxmox installations to test this.

The actual response from the server is empty.

```bash
$ curl -I -s -k -H "CSRFPreventionToken: 60E61BLAHBLAHBLAH" -b 'PVEAuthCookie=PVE:root@pam:60E6158BLAHBLAHBLAH' https://pve-02.lan:8006/api2/json/nodes/pve-01/network
HTTP/1.1 501 method 'HEAD' not available
Cache-Control: max-age=0
Connection: close
Date: Wed, 07 Jul 2021 21:03:59 GMT
Pragma: no-cache
Server: pve-api-daemon/3.0
Expires: Wed, 07 Jul 2021 21:03:59 GM
$ curl -s -k -H "CSRFPreventionToken: 60E61BLAHBLAHBLAH" -b 'PVEAuthCookie=PVE:root@pam:60E6158BLAHBLAHBLAH' https://pve-02.lan:8006/api2/json/nodes/pve-01/network
$
```

Obviously when I query the api for pve-02 the data returns as expected
```bash
$ curl -I -s -k -H "CSRFPreventionToken: 60E61BLAHBLAHBLAH" -b 'PVEAuthCookie=PVE:root@pam:60E6158BLAHBLAHBLAH' https://pve-02.lan:8006/api2/json/nodes/pve-02/network
HTTP/1.1 501 method 'HEAD' not available
Cache-Control: max-age=0
Connection: close
Date: Wed, 07 Jul 2021 21:07:45 GMT
Pragma: no-cache
Server: pve-api-daemon/3.0
Expires: Wed, 07 Jul 2021 21:07:45 GMT
$ curl -s -k -H "CSRFPreventionToken: 60E61BLAHBLAHBLAH" -b 'PVEAuthCookie=PVE:root@pam:60E6158BLAHBLAHBLAH' https://pve-02.lan:8006/api2/json/nodes/pve-02/network
{"data":[{"bridge_stp":"off","method6":"manual", ... }
$
```

From the on system proxmox `pvesh` I get the following error, which while not related to the inventory directly might give an idea behind the scenes.
```bash
root@pve-02:~# pvesh get /nodes/pve-01/network
proxy handler failed: ssh: connect to host 10.100.0.1 port 22: No route to host
```


Resolving patch:
diff --git a/plugins/inventory/proxmox.py b/plugins/inventory/proxmox.py
--- a/plugins/inventory/proxmox.py
+++ b/plugins/inventory/proxmox.py
@@ -369,6 +369,9 @@ def _populate(self):
                 if node['type'] == 'node':
                     self.inventory.add_child(nodes_group, node['node'])
 
+                if node['status'] == 'offline':
+                    continue
+
                 # get node IP address
                 if self.get_option("want_proxmox_nodes_ansible_host"):
                     ip = self._get_node_ip(node['node'])


Test patch:
diff --git a/tests/unit/plugins/inventory/test_proxmox.py b/tests/unit/plugins/inventory/test_proxmox.py
--- a/tests/unit/plugins/inventory/test_proxmox.py
+++ b/tests/unit/plugins/inventory/test_proxmox.py
@@ -9,7 +9,6 @@
 
 import pytest
 
-from ansible.errors import AnsibleError, AnsibleParserError
 from ansible.inventory.data import InventoryData
 from ansible_collections.community.general.plugins.inventory.proxmox import InventoryModule
 
@@ -52,7 +51,12 @@ def get_json(url):
                  "disk": 1000,
                  "maxmem": 1000,
                  "uptime": 10000,
-                 "level": ""}]
+                 "level": ""},
+                {"type": "node",
+                 "node": "testnode2",
+                 "id": "node/testnode2",
+                 "status": "offline",
+                 "ssl_fingerprint": "yy"}]
     elif url == "https://localhost:8006/api2/json/pools":
         # _get_pools
         return [{"poolid": "test"}]
@@ -554,7 +558,6 @@ def test_populate(inventory, mocker):
     host_qemu_multi_nic = inventory.inventory.get_host('test-qemu-multi-nic')
     host_qemu_template = inventory.inventory.get_host('test-qemu-template')
     host_lxc = inventory.inventory.get_host('test-lxc')
-    host_node = inventory.inventory.get_host('testnode')
 
     # check if qemu-test is in the proxmox_pool_test group
     assert 'proxmox_pool_test' in inventory.inventory.groups
@@ -584,3 +587,6 @@ def test_populate(inventory, mocker):
 
     # check if qemu template is not present
     assert host_qemu_template is None
+
+    # check that offline node is in inventory
+    assert inventory.inventory.get_host('testnode2')