Repository: ansible-collections/community.vmware

Issue:
Add option to disable vCLS for a given cluster
<!--- Verify first that your feature was not already discussed on GitHub -->
<!--- Complete *all* sections as described, this form is processed automatically -->

##### SUMMARY
<!--- Describe the new feature/improvement briefly below -->
With vCenter 7, VMware introduced the VMware cluster services concept that uses "vCLS" vm's. Unfortunately these spin up automatically, but it is not possible to - easily - remove them (they spin back up automatically as well), while they count as actual VM's when trying to remove hosts/clusters. This makes it fairly impossible to remove a cluster automatically purely via ansible at the moment. 

This [page](https://communities.vmware.com/t5/VMware-vCenter-Discussions/disable-vCLS-need-to-delete-vCenter-advanced-option-name/td-p/2305547) states that setting some entry in a config file and restarting vpxd would clean the VM's up, but it would be way nicer to have an Ansible module that implements management for this for a given cluster. 

##### ISSUE TYPE
- Feature Idea

##### COMPONENT NAME
<!--- Write the short name of the module, plugin, task or feature below, use your best guess if unsure -->
Could be part of community.vmware.vmware_cluster, but a new community.vmware.vmware_cluster_vcls is also an option.

##### ADDITIONAL INFORMATION
<!--- Describe how the feature would be used, why it is needed and what it would solve -->
The feature should probably be on by default, but a switch to disable it could be usable as follows:

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: Disable cluster vCLS
  community.vmware.vmware_cluster:
    hostname: '{{ vcenter_hostname }}'
    username: '{{ vcenter_username }}'
    password: '{{ vcenter_password }}'
    datacenter_name: datacenter
    cluster_name: cluster
    enable_vcls: false
  delegate_to: localhost
```

<!--- HINT: You can also paste gist.github.com links for larger files -->



Discussion:
Files identified in the description:
None

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
@lvlie Thanks for reporting this feature.
@lvlie For now, you can use this playbook to enable retreat mode - 

```yaml
---
- hosts: localhost
  vars_files:
    - vcenter_vars.yml
  vars:
    c1: "Asia-Cluster1"
  tasks:
  - name: Get Cluster Managed Object ID
    vmware_cluster_info:
      cluster_name: "{{ c1 }}"
      schema: vsphere
      properties:
      - name
      - _moId
    register: r

  - set_fact:
      key: "config.vcls.clusters.{{ r.clusters[c1]['moid'] }}.enabled"
      value: "false"
  
  - name: Enable retreat mode on cluster {{ c1 }}
    vmware_guest:
      datacenter: "{{ dc1 }}"
      cluster: "{{ c1 }}"
      name: "{{ vcsa }}"
      folder: /Asia-Datacenter1/vm/
      advanced_settings:
        - key: "{{ key }}"
          value: "{{ value }}"
      state: present
```

Target your VCSA virtual machine. Let me know if it works for you.

Thanks.

needs_info
Oh, cool, didn't know you could do that. Will try it later today or tomorrow
Applied the above on the VM that is the vcsa instance where the cluster is configured in. This results in the following extra config parameters on the vcsa vm:
![image](https://user-images.githubusercontent.com/8437831/116553514-7d281000-a8fa-11eb-9a20-7f9f266d7201.png)

Which seems to be correct. However, the vCLS VM's do not disappear in the vcsa instance. As the page says to restart vpxd I restarted the vcsa VM (that should do it right? :) ) but to no avail. 
As per https://kb.vmware.com/s/article/80472, point 7 in 'Using the vSphere Client', 

```
vCLS monitoring service will initiate the clean-up of vCLS VMs and user will start noticing the tasks with the VM deletion. 
```

Do you get the same behavior when you do it manually?

btw, here is the crude patch (untested) - 

```diff
diff --git a/plugins/modules/vmware_vcenter_settings.py b/plugins/modules/vmware_vcenter_settings.py
index e31c43f..af084ce 100644
--- a/plugins/modules/vmware_vcenter_settings.py
+++ b/plugins/modules/vmware_vcenter_settings.py
@@ -783,6 +783,34 @@ class VmwareVcenterSettings(PyVmomi):
                 )
                 diff_config['before']['logging_options'] = setting.value
 
+        for option in self.module.params.get('options'):
+            option_name = option.get('name', '')
+            option_value = option.get('value')
+            if option_name in [setting.key for setting in self.option_manager.setting]:
+                try:
+                    option_obj = self.option_manager.QueryView(name=option_name)
+                except vim.fault.InvalidName as invalid_name:
+                    self.module.fail_json(
+                        msg="Failed to query option(s) as one or more OptionValue objects refers to a "
+                        "non-existent option : %s" % to_native(invalid_name.msg)
+                    )
+                if option_obj[0].value != option_value:
+                    changed = True
+                    changed_list.append(option_name)
+                    result[option_name] = option_value
+                    change_option_list.append(
+                        vim.option.OptionValue(key=option_name, value=option_value)
+                    )
+                    diff_config['before'][option_name] = option_value
+            else:
+                changed = True
+                changed_list.append(option_name)
+                result[option_name] = option_value
+                change_option_list.append(
+                    vim.option.OptionValue(key=option_name, value=option_value)
+                )
+                diff_config['before'][option_name] = option_value
+
         if changed:
             if self.module.check_mode:
                 changed_suffix = ' would be changed'
@@ -927,6 +955,7 @@ def main():
             ),
         ),
         logging_options=dict(default='info', choices=['none', 'error', 'warning', 'info', 'verbose', 'trivia']),
+        options=dict(type='list', elements='dict'),
     )
 
     module = AnsibleModule(

```
Which will work with the following playbook - 

```yaml
---
- hosts: localhost
  vars_files:
    - vcenter_vars.yml
  tasks:
  - name: Set values
    community.vmware.vmware_vcenter_settings:
      options:
        - name: 'config.vcls.clusters.domain-c8.enabled'
          value: 'false'

  - name: Get values
    community.vmware.vmware_vcenter_settings_info:
```


[vSphere Cluster Services (vCLS)](http://www.yellow-bricks.com/2020/10/09/vmware-vsphere-clustering-services-vcls-considerations-questions-and-answers/) VMs are managed by vCenter and an important part of how DRS works. Over time, they will be the backbone for all clustering services. I should say that it's generally a good idea to **not** mess around with them manually. Having an option to do this in `vmware_cluster` sounds to me like an accident waiting to happen. Before we implement this, we should really understand the possible implications. And, quite frankly, at the moment _I_ don't. vCLS VMs are just to new.

> This makes it fairly impossible to remove a cluster automatically purely via ansible at the moment.

I don't think so. Duncan Epping:

> If I need to power off my cluster, what do I do with these VMs?
> 
> These VMs are migrated by DRS to the next host until the last host needs to go into maintenance mode and then they are automatically powered off by EAM.

So it looks like you just have to place all the hosts in the cluster in maintenance mode (there is a module for this, `vmware_maintenancemode`) and the vCLS VMs will be powered off automatically.
Thanks for your answer. I have disabled drs and ha on the specific cluster, which is a nested lab where I would like to frequently redeploy most of the resources, but not always vcsa itself. 
When putting the nodes into maintenance mode one by one, the vCLS vm's hang often, making the maintenance mode task hang and eventually fail. The task continues when I poweroff the vCLS vm's manually when this happens, which is why I ventured into this avenue. I can test further, but it seems like the vCLS vm's get relocated and this starts to fail after 2 or 3 nodes. Could be some other issue, but since I'm tearing everything down I don't want this to fail...

If there is some other way to *force* the maintenance mode of all hosts, or forcefully remove all content of a vdc without properly tearing down services one by one, I'd be happy to hear if you know of other options. 

Thanks
@mariolenz I totally agree with your opinion. But like @lvlie is facing issues, I think we should have the facility to modify configurations.

My proposed solutions allow changing existing configurations (not just vCLS enabling retreat mode). Let me know if you feel otherwise.
Ah, did not think about why it started to fail because of the ongoing relocations (but focussed on preventing them), but that is probably because the vCLS vm's are running on a vsan datastore and I'm using
```yaml
- name: Enter maintenance mode
  community.vmware.vmware_maintenancemode:
    vsan: noAction
    evacuate: false
```

This would make the datastore inaccessible after 2 nodes are down in my setup. However, I cannot remove the datastore or shutdown vsan before starting maintenance mode because of the vCLS vm's... Which I think brings me back to having to go into "retreat mode".
> This would make the datastore inaccessible after 2 nodes are down in my setup. However, I cannot remove the datastore or shutdown vsan before starting maintenance mode because of the vCLS vm's... Which I think brings me back to having to go into "retreat mode".

Yes. I can see the problem there. We also had issues with those vCLS VMs in our CI pipeline. I think they were fixed with PR #568 but we don't have VSAN which makes it easier for us.

> My proposed solutions allow changing existing configurations (not just vCLS enabling retreat mode). Let me know if you feel otherwise.

If I understand your solution correct, you propose a new parameter `options` to `vmware_vcenter_settings` to configure advanced settings on the vCenter itself. Well, personally I would prefer to call the parameter `advanced_settings` but apart from this: I should say it's probably a very good idea to have a way to manage them on the vCenter. I wonder why this hasn't been implemented yet.

However, I'm afk from now on for a couple of days and can't work on it. But feel free to open a PR implementing this yourself.
@lvlie 

> However, I'm afk from now on for a couple of days and can't work on it. But feel free to open a PR implementing this yourself.

I'm back :-) Did you work on a PR to implement your solution? If not, I'll do if that's OK for you.
@mariolenz Not yet. Please go ahead. Thanks for asking.
Ah if that question was aimed at me: sorry, not that good at python unfortunately and didn't catch it was aimed at me. I'd be happy to test anything in my env tho. Have a week off but can scrape some time together :-)
@Akasurde @lvlie I've assigned this issue to myself and will work on it. Please give me a couple of days.

Resolving patch:
diff --git a/plugins/modules/vmware_vcenter_settings.py b/plugins/modules/vmware_vcenter_settings.py
--- a/plugins/modules/vmware_vcenter_settings.py
+++ b/plugins/modules/vmware_vcenter_settings.py
@@ -227,6 +227,12 @@
         type: str
         choices: ['none', 'error', 'warning', 'info', 'verbose', 'trivia']
         default: 'info'
+    advanced_settings:
+      description:
+      - A dictionary of advanced settings.
+      default: {}
+      type: dict
+      version_added: '1.11.0'
 extends_documentation_fragment:
 - community.vmware.vmware.documentation
 
@@ -267,6 +273,15 @@
       long_operations: 120
     logging_options: info
   delegate_to: localhost
+
+- name: Enable Retreat Mode for cluster with MOID domain-c8 (https://kb.vmware.com/kb/80472)
+  community.vmware.vmware_vcenter_settings:
+    hostname: '{{ vcenter_hostname }}'
+    username: '{{ vcenter_username }}'
+    password: '{{ vcenter_password }}'
+    advanced_settings:
+      'config.vcls.clusters.domain-c8.enabled': 'false'
+  delegate_to: localhost
 '''
 
 RETURN = r'''
@@ -388,7 +403,7 @@
         pass
 
 from ansible.module_utils.basic import AnsibleModule
-from ansible_collections.community.vmware.plugins.module_utils.vmware import PyVmomi, vmware_argument_spec
+from ansible_collections.community.vmware.plugins.module_utils.vmware import PyVmomi, option_diff, vmware_argument_spec
 from ansible.module_utils._text import to_native
 
 
@@ -455,6 +470,7 @@ def ensure(self):
         timeout_normal_operations = self.params['timeout_settings'].get('normal_operations')
         timeout_long_operations = self.params['timeout_settings'].get('long_operations')
         logging_options = self.params.get('logging_options')
+
         changed = False
         changed_list = []
 
@@ -499,6 +515,18 @@ def ensure(self):
             exec("diff_config['after']['snmp_receiver_%s_community'] = snmp_receiver_%s_community" % (n, n))
         result['diff'] = {}
 
+        advanced_settings = self.params['advanced_settings']
+        changed_advanced_settings = option_diff(advanced_settings, self.option_manager.setting, False)
+
+        if changed_advanced_settings:
+            changed = True
+            change_option_list += changed_advanced_settings
+
+        for advanced_setting in advanced_settings:
+            result[advanced_setting] = advanced_settings[advanced_setting]
+            diff_config['before'][advanced_setting] = result[advanced_setting]
+            diff_config['after'][advanced_setting] = result[advanced_setting]
+
         for setting in self.option_manager.setting:
             # Database
             if setting.key == 'VirtualCenter.MaxDBConnection' and setting.value != db_max_connections:
@@ -783,6 +811,19 @@ def ensure(self):
                 )
                 diff_config['before']['logging_options'] = setting.value
 
+            # Advanced settings
+            for advanced_setting in changed_advanced_settings:
+                if setting.key == advanced_setting.key and setting.value != advanced_setting.value:
+                    changed_list.append(advanced_setting.key)
+                    result[advanced_setting.key + '_previous'] = advanced_setting.value
+                    diff_config['before'][advanced_setting.key] = advanced_setting.value
+
+        for advanced_setting in changed_advanced_settings:
+            if advanced_setting.key not in changed_list:
+                changed_list.append(advanced_setting.key)
+                result[advanced_setting.key + '_previous'] = "N/A"
+                diff_config['before'][advanced_setting.key] = "N/A"
+
         if changed:
             if self.module.check_mode:
                 changed_suffix = ' would be changed'
@@ -927,6 +968,7 @@ def main():
             ),
         ),
         logging_options=dict(default='info', choices=['none', 'error', 'warning', 'info', 'verbose', 'trivia']),
+        advanced_settings=dict(type='dict', default=dict(), required=False),
     )
 
     module = AnsibleModule(


Test patch:
diff --git a/tests/integration/targets/vmware_vcenter_settings/tasks/main.yml b/tests/integration/targets/vmware_vcenter_settings/tasks/main.yml
--- a/tests/integration/targets/vmware_vcenter_settings/tasks/main.yml
+++ b/tests/integration/targets/vmware_vcenter_settings/tasks/main.yml
@@ -6,7 +6,7 @@
     name: prepare_vmware_tests
 
 - name: Configure general settings in check mode
-  vmware_vcenter_settings:
+  community.vmware.vmware_vcenter_settings:
     hostname: "{{ vcenter_hostname }}"
     username: "{{ vcenter_username }}"
     password: "{{ vcenter_password }}"
@@ -45,7 +45,7 @@
 - debug: var=all_settings_results_check_mode
 
 - name: Configure general settings
-  vmware_vcenter_settings:
+  community.vmware.vmware_vcenter_settings:
     hostname: "{{ vcenter_hostname }}"
     username: "{{ vcenter_username }}"
     password: "{{ vcenter_password }}"
@@ -83,7 +83,7 @@
 - debug: var=all_settings_results
 
 - name: Configure settings with out runtime_settings parameter
-  vmware_vcenter_settings:
+  community.vmware.vmware_vcenter_settings:
     hostname: "{{ vcenter_hostname }}"
     username: "{{ vcenter_username }}"
     password: "{{ vcenter_password }}"
@@ -96,7 +96,7 @@
 - debug: var=without_runtime_settings_results
 
 - name: Configure settings with check mode and diff mode
-  vmware_vcenter_settings:
+  community.vmware.vmware_vcenter_settings:
     hostname: "{{ vcenter_hostname }}"
     username: "{{ vcenter_username }}"
     password: "{{ vcenter_password }}"
@@ -121,7 +121,7 @@
       - configure_settings_check_diff_mode_result.diff is defined
 
 - name: Configure settings without check mode and diff mode
-  vmware_vcenter_settings:
+  community.vmware.vmware_vcenter_settings:
     hostname: "{{ vcenter_hostname }}"
     username: "{{ vcenter_username }}"
     password: "{{ vcenter_password }}"
@@ -143,7 +143,7 @@
       - configure_settings_result.changed is sameas true
 
 - name: Configure settings without check mode and diff mode(idempotency check)
-  vmware_vcenter_settings:
+  community.vmware.vmware_vcenter_settings:
     hostname: "{{ vcenter_hostname }}"
     username: "{{ vcenter_username }}"
     password: "{{ vcenter_password }}"
@@ -163,3 +163,53 @@
   assert:
     that:
       - configure_settings_idempotency_check_result.changed is sameas false
+
+- when: vcsim is not defined
+  block:
+  - name: Configure advanced settings
+    community.vmware.vmware_vcenter_settings:
+      hostname: "{{ vcenter_hostname }}"
+      username: "{{ vcenter_username }}"
+      password: "{{ vcenter_password }}"
+      validate_certs: false
+      database:
+        max_connections: 50
+        task_cleanup: true
+        task_retention: 180
+        event_cleanup: true
+        event_retention: 180
+      mail:
+        server: mail.example.local
+        sender: vcenter@{{ inventory_hostname }}
+      advanced_settings:
+        'config.vcls.clusters.domain-c8.enabled': 'false'
+    register: configure_advanced_settings
+
+  - name: Make sure that advanced settings are configured
+    assert:
+      that:
+        - configure_advanced_settings.changed
+
+  - name: Configure advanced settings again (idempotency)
+    community.vmware.vmware_vcenter_settings:
+      hostname: "{{ vcenter_hostname }}"
+      username: "{{ vcenter_username }}"
+      password: "{{ vcenter_password }}"
+      validate_certs: false
+      database:
+        max_connections: 50
+        task_cleanup: true
+        task_retention: 180
+        event_cleanup: true
+        event_retention: 180
+      mail:
+        server: mail.example.local
+        sender: vcenter@{{ inventory_hostname }}
+      advanced_settings:
+        'config.vcls.clusters.domain-c8.enabled': 'false'
+    register: configure_advanced_settings_again
+
+  - name: Make sure that advanced settings are not configured (idempotency)
+    assert:
+      that:
+        - not configure_advanced_settings_again.changed