Repository: ansible-collections/community.general

Issue:
Proxmox disk - Read time out when importing a disk
### Summary

I am using community.general.proxmox_disk to import a disk from a cloud image which always fails on the first attempt with the following error:

fatal: [lab-pve2]: FAILED! => {"attempts": 1, "changed": false, "msg": "Unable to create/update disk scsi1 in VM 201: HTTPSConnectionPool(host='127.0.0.1', port=8006): Read timed out. (read timeout=5)", "vmid": 201}

I have tried adding retries+until which has no effect, ansible treats this as unexpected error and stops/skips the task. When I rerun the same thing, it passes as unchanged therefore I assume it's just an async process which takes too long and on the second attempt the disk is already imported.

### Issue Type

Bug Report

### Component Name

community.general.proxmox_disk module

### Ansible Version

```console (paste below)
$ ansible --version

ansible [core 2.13.6]
  config file = None
  configured module search path = ['/home/lholota/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/lholota/.local/lib/python3.8/site-packages/ansible
  ansible collection location = /home/lholota/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/lholota/.local/bin/ansible
  python version = 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
  jinja version = 3.0.3
  libyaml = True

```


### Community.general Version

```console (paste below)
$ ansible-galaxy collection list community.general

# /home/..../.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 5.7.0
```


### Configuration

```console (paste below)
$ ansible-config dump --only-changed

```
(empty output)

### OS / Environment

Executed from: Debian 11 (in WSL)
Targets: Proxmox 7.2

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
- name: Import disk from cloud image
  register: pve_vm_disk
  retries: 5 # transient timeouts
  until: pve_vm_disk.vmid == vm.vmid
  community.general.proxmox_disk:
    api_user: root@pam
    api_password: "{{ users_root_password }}"
    api_host: 127.0.0.1
    vmid: "{{ vm.vmid }}"
    disk: scsi1
    import_from: "/var/lib/vz/images/{{ vm.cloud_image_filename }}"
    storage: "{{ vm.os_storage }}"
    ssd: true
    timeout: 600
    state: present
```


### Expected Results

Disk should be imported without throwing the error

### Actual Results

```console (paste below)
fatal: [lab-pve2]: FAILED! => {"attempts": 1, "changed": false, "msg": "Unable to create/update disk scsi1 in VM 201: HTTPSConnectionPool(host='127.0.0.1', port=8006): Read timed out. (read timeout=5)", "vmid": 201}
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct


Discussion:
Files identified in the description:
* [`plugins/modules/proxmox_disk.py`](https://github.com/ansible-collections/community.general/blob/main/plugins/modules/proxmox_disk.py)

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Ajpantuso @Thulium-Drake @castorsky @joshainglis @karmab @tleguern
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Confirmed: it is actual for slow disks or big images. I will try to fix this bug asap.
@castorsky thanks for the effort! Regarding the slow disk - I have encountered this issue in my dev environment where I'm running Proxmox inside of HyperV VMs and I don't care about performance too much on the other hand the image I am importing is a Debian Bullseye cloud init (qcow2) which is ~350 MB and the VM disk is placed on an NVMe SSDs so it shouldn't be that slow

Resolving patch:
diff --git a/plugins/modules/proxmox_disk.py b/plugins/modules/proxmox_disk.py
--- a/plugins/modules/proxmox_disk.py
+++ b/plugins/modules/proxmox_disk.py
@@ -104,6 +104,7 @@
       - Move the disk to this storage when I(state=moved).
       - You can move between storages only in scope of one VM.
       - Mutually exclusive with I(target_vmid).
+      - Consider increasing I(timeout) in case of large disk images or slow storage backend.
     type: str
   target_vmid:
     description:
@@ -113,8 +114,8 @@
     type: int
   timeout:
     description:
-      - Timeout in seconds to wait when moving disk.
-      - Used only when I(state=moved).
+      - Timeout in seconds to wait for slow operations such as importing disk or moving disk between storages.
+      - Used only when I(state) is C(present) or C(moved).
     type: int
     default: 600
   aio:
@@ -172,6 +173,7 @@
       - C(<STORAGE>:<VMID>/<FULL_NAME>) or C(<ABSOLUTE_PATH>/<FULL_NAME>)
       - Attention! Only root can use absolute paths.
       - This parameter is mutually exclusive with I(size).
+      - Increase I(timeout) parameter when importing large disk images or using slow storage.
     type: str
   iops:
     description:
@@ -471,6 +473,16 @@ def get_create_attributes(self):
         params.update(dict((k, int(v)) for k, v in params.items() if isinstance(v, bool)))
         return params
 
+    def wait_till_complete_or_timeout(self, node_name, task_id):
+        timeout = self.module.params['timeout']
+        while timeout:
+            if self.api_task_ok(node_name, task_id):
+                return True
+            timeout -= 1
+            if timeout <= 0:
+                return False
+            sleep(1)
+
     def create_disk(self, disk, vmid, vm, vm_config):
         create = self.module.params['create']
         if create == 'disabled' and disk not in vm_config:
@@ -484,20 +496,23 @@ def create_disk(self, disk, vmid, vm, vm_config):
 
             if import_string:
                 config_str = "%s:%s,import-from=%s" % (self.module.params["storage"], "0", import_string)
+                timeout_str = "Reached timeout while importing VM disk. Last line in task before timeout: %s"
+                ok_str = "Disk %s imported into VM %s"
             else:
                 config_str = "%s:%s" % (self.module.params["storage"], self.module.params["size"])
+                ok_str = "Disk %s created in VM %s"
+                timeout_str = "Reached timeout while creating VM disk. Last line in task before timeout: %s"
 
             for k, v in attributes.items():
                 config_str += ',%s=%s' % (k, v)
 
-            create_disk = {self.module.params["disk"]: config_str}
-            self.proxmox_api.nodes(vm['node']).qemu(vmid).config.set(**create_disk)
-            return True, "Disk %s created in VM %s" % (disk, vmid)
+            disk_config_to_apply = {self.module.params["disk"]: config_str}
 
         if create in ['disabled', 'regular'] and disk in vm_config:
             # UPDATE
             disk_config = disk_conf_str_to_dict(vm_config[disk])
             config_str = disk_config["volume"]
+            ok_str = "Disk %s updated in VM %s"
             attributes = self.get_create_attributes()
             # 'import_from' fails on disk updates
             attributes.pop('import_from', None)
@@ -513,9 +528,16 @@ def create_disk(self, disk, vmid, vm, vm_config):
             if disk_config == attributes:
                 return False, "Disk %s is up to date in VM %s" % (disk, vmid)
 
-            update_disk = {self.module.params["disk"]: config_str}
-            self.proxmox_api.nodes(vm['node']).qemu(vmid).config.set(**update_disk)
-            return True, "Disk %s updated in VM %s" % (disk, vmid)
+            disk_config_to_apply = {self.module.params["disk"]: config_str}
+
+        current_task_id = self.proxmox_api.nodes(vm['node']).qemu(vmid).config.post(**disk_config_to_apply)
+        task_success = self.wait_till_complete_or_timeout(vm['node'], current_task_id)
+        if task_success:
+            return True, ok_str % (disk, vmid)
+        else:
+            self.module.fail_json(
+                msg=timeout_str % self.proxmox_api.nodes(vm['node']).tasks(current_task_id).log.get()[:1]
+            )
 
     def move_disk(self, disk, vmid, vm, vm_config):
         params = dict()
@@ -535,20 +557,15 @@ def move_disk(self, disk, vmid, vm, vm_config):
             if params['storage'] == disk_config['storage_name']:
                 return False
 
-        taskid = self.proxmox_api.nodes(vm['node']).qemu(vmid).move_disk.post(**params)
-        timeout = self.module.params['timeout']
-        while timeout:
-            status_data = self.proxmox_api.nodes(vm['node']).tasks(taskid).status.get()
-            if status_data['status'] == 'stopped' and status_data['exitstatus'] == 'OK':
-                return True
-            if timeout <= 0:
-                self.module.fail_json(
-                    msg='Reached timeout while waiting for moving VM disk. Last line in task before timeout: %s' %
-                        self.proxmox_api.nodes(vm['node']).tasks(taskid).log.get()[:1])
-
-            sleep(1)
-            timeout -= 1
-        return True
+        task_id = self.proxmox_api.nodes(vm['node']).qemu(vmid).move_disk.post(**params)
+        task_success = self.wait_till_complete_or_timeout(vm['node'], task_id)
+        if task_success:
+            return True
+        else:
+            self.module.fail_json(
+                msg='Reached timeout while waiting for moving VM disk. Last line in task before timeout: %s' %
+                    self.proxmox_api.nodes(vm['node']).tasks(task_id).log.get()[:1]
+            )
 
 
 def main():


Test patch: