Repository: ansible-collections/community.general

Issue:
proxmox_kvm: VM created with OVMF BIOS are invalid
<!--- Verify first that your issue is not already reported on GitHub -->
<!--- Also test if the latest release and devel branch are affected too -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
EFI virtual machines created with proxmox_kvm don't work, the EFI disk is missing and the boot order is missing the disks.

##### ISSUE TYPE
- Bug Report

##### COMPONENT NAME
proxmox_kvm

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes -->
```paste below
ansible 2.10.4
  config file = /home/johann/blade/ansible/ansible.cfg
  configured module search path = ['/home/johann/blade/ansible/library/modules', '/home/johann/blade/ansible/.imported_roles/lae.proxmox/library']
  ansible python module location = /home/johann/.local/lib/python3.9/site-packages/ansible
  executable location = /home/johann/.local/bin/ansible
  python version = 3.9.1+ (default, Jan 10 2021, 15:42:50) [GCC 10.2.1 20201224]

❯ ansible-galaxy collection list

# /home/johann/blade/ansible/.imported_collections/ansible_collections
Collection           Version
-------------------- -------
ansible.netcommon    1.4.1  
ansible.posix        1.1.1  
community.general    1.3.3  
community.kubernetes 1.1.1  
google.cloud         1.0.1  
netbox.netbox        1.2.0  
```

##### CONFIGURATION
<!--- Paste verbatim output from "ansible-config dump --only-changed" between quotes -->
```paste below
CACHE_PLUGIN(/home/johann/blade/ansible/ansible.cfg) = jsonfile
CACHE_PLUGIN_CONNECTION(/home/johann/blade/ansible/ansible.cfg) = /tmp/facts_cache
CACHE_PLUGIN_TIMEOUT(/home/johann/blade/ansible/ansible.cfg) = 7200
COLLECTIONS_PATHS(/home/johann/blade/ansible/ansible.cfg) = ['/home/johann/blade/ansible/.imported_collections', '/home/johann/.ansible/coll>
COMMAND_WARNINGS(/home/johann/blade/ansible/ansible.cfg) = True
DEFAULT_BECOME(/home/johann/blade/ansible/ansible.cfg) = False
DEFAULT_FILTER_PLUGIN_PATH(/home/johann/blade/ansible/ansible.cfg) = ['/home/johann/blade/ansible/library/filters']
DEFAULT_FORKS(/home/johann/blade/ansible/ansible.cfg) = 10
DEFAULT_GATHERING(/home/johann/blade/ansible/ansible.cfg) = smart
DEFAULT_LOOKUP_PLUGIN_PATH(/home/johann/blade/ansible/ansible.cfg) = ['/home/johann/blade/ansible/library/lookup']
DEFAULT_MODULE_PATH(/home/johann/blade/ansible/ansible.cfg) = ['/home/johann/blade/ansible/library/modules', '/home/johann/blade/ansible/.im>
DEFAULT_MODULE_UTILS_PATH(/home/johann/blade/ansible/ansible.cfg) = ['/home/johann/blade/ansible/.imported_roles/lae.proxmox/module_utils']
DEFAULT_ROLES_PATH(/home/johann/blade/ansible/ansible.cfg) = ['/home/johann/blade/ansible/.imported_roles', '/home/johann/blade/ansible/role>
DEPRECATION_WARNINGS(/home/johann/blade/ansible/ansible.cfg) = True
HOST_KEY_CHECKING(/home/johann/blade/ansible/ansible.cfg) = False
INTERPRETER_PYTHON(/home/johann/blade/ansible/ansible.cfg) = auto_legacy_silent
SYSTEM_WARNINGS(/home/johann/blade/ansible/ansible.cfg) = True
TRANSFORM_INVALID_GROUP_CHARS(/home/johann/blade/ansible/ansible.cfg) = silently
```

##### OS / ENVIRONMENT
<!--- Provide all relevant information below, e.g. target OS versions, network device firmware, etc. -->
Targeted Proxmox server is running 6.3-3

##### STEPS TO REPRODUCE
<!--- Describe exactly how to reproduce the problem, using a minimal test-case -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: Create KVM instances
  proxmox_kvm:
    api_host: "{{ proxmox.api_host }}"
    api_user: "root@pam"
    api_password: "{{ proxmox.password }}"
    state: present
    name: "{{ item.name }}"
    description: "{{ item.description }}"
    node: "{{ item.node }}"
    boot: "cn"
    ostype: "{{ item.ostype }}"
    cores: "{{ item.cpu_cores }}"
    memory: "{{ item.memory }}"
    balloon: 0
    net: "{{ item.net | items2dict }}"
    scsi: "{{ item.disks }}"
    scsihw: virtio-scsi-pci
    agent: yes
    bios: ovmf
    machine: q35
    vga: qxl
    serial: '{"serial0": "socket"}'
    cpu: "{{ proxmox.cpu_type }}"
    cpuunits: 1000
    kvm: yes
    acpi: yes
    autostart: yes
  loop: "{{ proxmox_admin_kvm }}"
```

```yaml
proxmox:
  api_user: automation@pve
  api_password: YYYYYYYYYY
  api_host: XXXXXXXXXXXXX
  node: XXXX
  searchdomain: "{{ ansible_domain }}"
  password: ZZZZZZZZZZZZ
  cpu_type: EPYC

proxmox_admin_kvm:
- name: smtp1
  description: SMTP relay
  ostype: l26
  node: XXXX
  cpu_cores: 1
  memory: 1024
  net:
    - key: net0
      value: virtio,bridge=vmbr101
  disks: {"scsi0": "local:15,format=qcow2"}
```

I stripped the bootstrap part, but short story, we're using qemu args to start debian-installer images with direct boot and preseed parameters.

<!--- HINT: You can paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
VM is created with a valid specification

##### ACTUAL RESULTS
The VM specification has two issues:

- The Proxmox EFI disk used to store EFI vars for the VM is missing, should be added before bootstrap (creation should be required if ovmf is chosen as bios, the web UI won't let you proceed without one)
- The boot order is missing the disks, mentionning only the network cards. I need to manually enable scsi0 and put it at the top of the list or the VM won't start after bootstraping.


Discussion:
Files identified in the description:
* [`plugins/modules/cloud/misc/proxmox_kvm.py`](https://github.com/ansible-collections/community.general/blob/master/plugins/modules/cloud/misc/proxmox_kvm.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Aversiste @Thulium-Drake @helldorado @joshainglis @karmab
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
cc @Ajpantuso
[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/plugins/modules/cloud/misc/proxmox_kvm.py b/plugins/modules/cloud/misc/proxmox_kvm.py
--- a/plugins/modules/cloud/misc/proxmox_kvm.py
+++ b/plugins/modules/cloud/misc/proxmox_kvm.py
@@ -120,6 +120,39 @@
       - Specify if to prevent changes if current configuration file has different SHA1 digest.
       - This can be used to prevent concurrent modifications.
     type: str
+  efidisk0:
+    description:
+      - Specify a hash/dictionary of EFI disk options.
+      - Requires I(bios=ovmf) to be set to be able to use it.
+    type: dict
+    suboptions:
+      storage:
+        description:
+          - C(storage) is the storage identifier where to create the disk.
+        type: str
+      format:
+        description:
+          - C(format) is the drive's backing file's data format. Please refer to the Proxmox VE Administrator Guide,
+           section Proxmox VE Storage (see U(https://pve.proxmox.com/pve-docs/chapter-pvesm.html) for the latest
+           version, tables 3 to 14) to find out format supported by the provided storage backend.
+        type: str
+      efitype:
+        description:
+          - C(efitype) indicates the size of the EFI disk.
+          - C(2m) will allow for a 2MB EFI disk, which will be enough to persist boot order and new boot entries.
+          - C(4m) will allow for a 4MB EFI disk, which will additionally allow to store EFI keys in order to enable
+           Secure Boot
+        type: str
+        choices:
+          - 2m
+          - 4m
+      pre_enrolled_keys:
+        description:
+          - C(pre_enrolled_keys) indicates whether EFI keys for Secure Boot should be enrolled C(1) in the VM firmware
+           upon creation or not (0).
+          - If set to C(1), Secure Boot will also be enabled by default when the VM is created.
+        type: bool
+    version_added: 4.5.0
   force:
     description:
       - Allow to force stop VM.
@@ -530,6 +563,38 @@
     cores: 4
     vcpus: 2
 
+- name: Create VM with 1 10GB SATA disk and an EFI disk, with Secure Boot disabled by default
+  community.general.proxmox_kvm:
+    api_user: root@pam
+    api_password: secret
+    api_host: helldorado
+    name: spynal
+    node: sabrewulf
+    sata:
+      sata0: 'VMs_LVM:10,format=raw'
+    bios: ovmf
+    efidisk0:
+      storage: VMs_LVM_thin
+      format: raw
+      efitype: 4m
+      pre_enrolled_keys: False
+
+- name: Create VM with 1 10GB SATA disk and an EFI disk, with Secure Boot enabled by default
+  community.general.proxmox_kvm:
+    api_user: root@pam
+    api_password: secret
+    api_host: helldorado
+    name: spynal
+    node: sabrewulf
+    sata:
+      sata0: 'VMs_LVM:10,format=raw'
+    bios: ovmf
+    efidisk0:
+      storage: VMs_LVM
+      format: raw
+      efitype: 4m
+      pre_enrolled_keys: 1
+
 - name: >
     Clone VM with only source VM name.
     The VM source is spynal.
@@ -776,7 +841,7 @@ def get_vminfo(self, node, vmid, **kwargs):
 
         # Split information by type
         re_net = re.compile(r'net[0-9]')
-        re_dev = re.compile(r'(virtio|ide|scsi|sata)[0-9]')
+        re_dev = re.compile(r'(virtio|ide|scsi|sata|efidisk)[0-9]')
         for k in kwargs.keys():
             if re_net.match(k):
                 mac[k] = parse_mac(vm[k])
@@ -847,7 +912,7 @@ def create_vm(self, vmid, newid, node, name, memory, cpu, cores, sockets, update
             urlencoded_ssh_keys = quote(kwargs['sshkeys'], safe='')
             kwargs['sshkeys'] = str(urlencoded_ssh_keys)
 
-        # If update, don't update disk (virtio, ide, sata, scsi) and network interface
+        # If update, don't update disk (virtio, efidisk0, ide, sata, scsi) and network interface
         # pool parameter not supported by qemu/<vmid>/config endpoint on "update" (PVE 6.2) - only with "create"
         if update:
             if 'virtio' in kwargs:
@@ -858,6 +923,8 @@ def create_vm(self, vmid, newid, node, name, memory, cpu, cores, sockets, update
                 del kwargs['scsi']
             if 'ide' in kwargs:
                 del kwargs['ide']
+            if 'efidisk0' in kwargs:
+                del kwargs['efidisk0']
             if 'net' in kwargs:
                 del kwargs['net']
             if 'force' in kwargs:
@@ -865,6 +932,26 @@ def create_vm(self, vmid, newid, node, name, memory, cpu, cores, sockets, update
             if 'pool' in kwargs:
                 del kwargs['pool']
 
+        # Check that the bios option is set to ovmf if the efidisk0 option is present
+        if 'efidisk0' in kwargs:
+            if ('bios' not in kwargs) or ('ovmf' != kwargs['bios']):
+                self.module.fail_json(msg='efidisk0 cannot be used if bios is not set to ovmf. ')
+
+        # Flatten efidisk0 option to a string so that it's a string which is what Proxmoxer and the API expect
+        if 'efidisk0' in kwargs:
+            efidisk0_str = ''
+            # Regexp to catch underscores in keys name, to replace them after by hypens
+            hyphen_re = re.compile(r'_')
+            # If present, the storage definition should be the first argument
+            if 'storage' in kwargs['efidisk0']:
+                efidisk0_str += kwargs['efidisk0'].get('storage') + ':1,'
+                kwargs['efidisk0'].pop('storage')
+            # Join other elements from the dict as key=value using commas as separator, replacing any underscore in key
+            # by hyphens (needed for pre_enrolled_keys to pre-enrolled-keys)
+            efidisk0_str += ','.join([hyphen_re.sub('-', k) + "=" + str(v) for k, v in kwargs['efidisk0'].items()
+                                      if 'storage' != k])
+            kwargs['efidisk0'] = efidisk0_str
+
         # Convert all dict in kwargs to elements.
         # For hostpci[n], ide[n], net[n], numa[n], parallel[n], sata[n], scsi[n], serial[n], virtio[n], ipconfig[n]
         for k in list(kwargs.keys()):
@@ -971,6 +1058,13 @@ def main():
         delete=dict(type='str'),
         description=dict(type='str'),
         digest=dict(type='str'),
+        efidisk0=dict(type='dict',
+                      options=dict(
+                          storage=dict(type='str'),
+                          format=dict(type='str'),
+                          efitype=dict(type='str', choices=['2m', '4m']),
+                          pre_enrolled_keys=dict(type='bool'),
+                      )),
         force=dict(type='bool'),
         format=dict(type='str', choices=['cloop', 'cow', 'qcow', 'qcow2', 'qed', 'raw', 'vmdk', 'unspecified']),
         freeze=dict(type='bool'),
@@ -1160,6 +1254,7 @@ def main():
                               cpuunits=module.params['cpuunits'],
                               description=module.params['description'],
                               digest=module.params['digest'],
+                              efidisk0=module.params['efidisk0'],
                               force=module.params['force'],
                               freeze=module.params['freeze'],
                               hostpci=module.params['hostpci'],


Test patch: