Repository: ansible/ansible

Issue:
openvswitch_db: Missing support for column types other than a map
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
<!--- Explain the problem briefly -->

Open vSwitch schema provides columns with various types like int, bool, string that don't require a "key" which is required only by a map type. Current openvswitch_db ansible module has a 'key' parameter as required. This makes it not possible to set columns that are types other than a map.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
openvswitch_db

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.7.0.dev0
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
DEFAULT_LOG_PATH(/etc/ansible/ansible.cfg) = /opt/stack/new/ansible.log
DEFAULT_ROLES_PATH(/etc/ansible/ansible.cfg) = [u'/etc/ansible/roles', u'/opt/stack/new/networking-ansible/etc/ansible/roles']

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
CentOS Linux release 7.5.1804 (Core)

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: "openvswitch: run update_port"
  hosts: localhost
  tasks:
    - openvswitch_db:
        table: Port
        record: "port0"
        col: tag
        value: "1"
      become: yes
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
The 'tag' column for port0 at Port table should be set to 1

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
ansible-playbook 2.5.5
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.5 (default, Apr 11 2018, 07:36:10) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Parsed /etc/ansible/hosts inventory source with ini plugin
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Loading callback plugin default of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/default.pyc

PLAYBOOK: test-playbook.yml *************************************************************************************************************************************************************
1 plays in test-playbook.yml

PLAY [openvswitch: run update_port] *****************************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************************
task path: /home/vagrant/networking-ansible/test-playbook.yml:2
Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<127.0.0.1> EXEC /bin/sh -c 'echo ~vagrant && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315 `" && echo ansible-tmp-1530280901.98-145822743403315="` echo /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315 `" ) && sleep 0'
<127.0.0.1> PUT /home/vagrant/.ansible/tmp/ansible-local-29171pVygKZ/tmpFnjIYb TO /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315/setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315/ /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2 /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/vagrant/.ansible/tmp/ansible-tmp-1530280901.98-145822743403315/ > /dev/null 2>&1 && sleep 0'
ok: [localhost]
META: ran handlers

TASK [openvswitch_db] *******************************************************************************************************************************************************************
task path: /home/vagrant/networking-ansible/test-playbook.yml:5
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/ovs/openvswitch_db.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<127.0.0.1> EXEC /bin/sh -c 'echo ~vagrant && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014 `" && echo ansible-tmp-1530280902.48-128404859669014="` echo /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014 `" ) && sleep 0'
<127.0.0.1> PUT /home/vagrant/.ansible/tmp/ansible-local-29171pVygKZ/tmpQBJLuq TO /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014/openvswitch_db.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014/ /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014/openvswitch_db.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-rnyzkgtjykoxtnypsistbvemltyxwkpb; /usr/bin/python2 /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014/openvswitch_db.py'"'"' && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/vagrant/.ansible/tmp/ansible-tmp-1530280902.48-128404859669014/ > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "col": "tag",
            "record": "port0",
            "state": "present",
            "table": "Port",
            "timeout": 5,
            "value": "1"
        }
    },
    "msg": "missing required arguments: key"
}
	to retry, use: --limit @/home/vagrant/networking-ansible/test-playbook.retry

PLAY RECAP ******************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1
```



Discussion:

Resolving patch:
diff --git a/lib/ansible/modules/network/ovs/openvswitch_db.py b/lib/ansible/modules/network/ovs/openvswitch_db.py
--- a/lib/ansible/modules/network/ovs/openvswitch_db.py
+++ b/lib/ansible/modules/network/ovs/openvswitch_db.py
@@ -48,9 +48,10 @@
         description:
             - Identifies the column in the record.
     key:
-        required: true
+        required: false
         description:
-            - Identifies the key in the record column
+            - Identifies the key in the record column, when the column is a map
+              type.
     value:
         required: true
         description:
@@ -87,11 +88,23 @@
     record: br-int
     col: other_config
     key: disable-in-band
+
+# Mark port with tag 10
+- openvswitch_db:
+    table: Port
+    record: port0
+    col: tag
+    value: 10
 '''
 import re
 
 from ansible.module_utils.basic import AnsibleModule
 
+# Regular expression for map type, must not be empty
+NON_EMPTY_MAP_RE = re.compile(r'{.+}')
+# Regular expression for a map column type
+MAP_RE = re.compile(r'{.*}')
+
 
 def map_obj_to_commands(want, have, module):
     """ Define ovs-vsctl command to meet desired state """
@@ -102,8 +115,16 @@ def map_obj_to_commands(want, have, module):
             templatized_command = "%(ovs-vsctl)s -t %(timeout)s remove %(table)s %(record)s " \
                                   "%(col)s %(key)s=%(value)s"
             commands.append(templatized_command % module.params)
+        elif module.params['key'] is None:
+            templatized_command = "%(ovs-vsctl)s -t %(timeout)s remove %(table)s %(record)s " \
+                                  "%(col)s"
+            commands.append(templatized_command % module.params)
     else:
-        if 'key' not in have.keys():
+        if module.params['key'] is None:
+            templatized_command = "%(ovs-vsctl)s -t %(timeout)s set %(table)s %(record)s " \
+                                  "%(col)s=%(value)s"
+            commands.append(templatized_command % module.params)
+        elif 'key' not in have.keys():
             templatized_command = "%(ovs-vsctl)s -t %(timeout)s add %(table)s %(record)s " \
                                   "%(col)s %(key)s=%(value)s"
             commands.append(templatized_command % module.params)
@@ -125,8 +146,16 @@ def map_config_to_obj(module):
     match = re.search(r'^' + module.params['col'] + r'(\s+):(\s+)(.*)$', out, re.M)
 
     col_value = match.group(3)
+
+    # Map types require key argument
+    has_key = module.params['key'] is not None
+    is_map = MAP_RE.match(col_value)
+    if is_map and not has_key:
+        module.fail_json(
+            msg="missing required arguments: key for map type of column")
+
     col_value_to_dict = {}
-    if col_value and col_value != '{}':
+    if NON_EMPTY_MAP_RE.match(col_value):
         for kv in col_value[1:-1].split(', '):
             k, v = kv.split('=')
             col_value_to_dict[k.strip()] = v.strip()
@@ -137,9 +166,12 @@ def map_config_to_obj(module):
         'col': module.params['col'],
     }
 
-    if module.params['key'] in col_value_to_dict:
-        obj['key'] = module.params['key']
-        obj['value'] = col_value_to_dict[module.params['key']]
+    if has_key and is_map:
+        if module.params['key'] in col_value_to_dict:
+            obj['key'] = module.params['key']
+            obj['value'] = col_value_to_dict[module.params['key']]
+    else:
+            obj['value'] = col_value.strip()
 
     return obj
 
@@ -149,10 +181,13 @@ def map_params_to_obj(module):
         'table': module.params['table'],
         'record': module.params['record'],
         'col': module.params['col'],
-        'key': module.params['key'],
         'value': module.params['value']
     }
 
+    key = module.params['key']
+    if key is not None:
+        obj['key'] = key
+
     return obj
 
 
@@ -163,7 +198,7 @@ def main():
         'table': {'required': True},
         'record': {'required': True},
         'col': {'required': True},
-        'key': {'required': True},
+        'key': {'required': False},
         'value': {'required': True},
         'timeout': {'default': 5, 'type': 'int'},
     }


Test patch:
diff --git a/test/integration/targets/openvswitch_db/tests/basic.yaml b/test/integration/targets/openvswitch_db/tests/basic.yaml
--- a/test/integration/targets/openvswitch_db/tests/basic.yaml
+++ b/test/integration/targets/openvswitch_db/tests/basic.yaml
@@ -33,7 +33,7 @@
     that:
       - "result.changed == false"
 
-- name: Change column value
+- name: Change column value in a map
   openvswitch_db:
     table: Bridge
     record: br-test
@@ -46,7 +46,7 @@
     that:
       - "result.changed == true"
 
-- name: Change column value again (idempotent)
+- name: Change column value in a map again (idempotent)
   openvswitch_db:
     table: Bridge
     record: br-test
@@ -59,6 +59,43 @@
     that:
       - "result.changed == false"
 
+- name: Change column value
+  openvswitch_db:
+    table: Bridge
+    record: br-test
+    col: stp_enable
+    value: true
+  register: result
+
+- assert:
+    that:
+      - "result.changed == true"
+
+- name: Change column value again (idempotent)
+  openvswitch_db:
+    table: Bridge
+    record: br-test
+    col: stp_enable
+    value: true
+  register: result
+
+- assert:
+    that:
+      - "result.changed == false"
+
+- name: Try to set value on a map type without a key (negative)
+  ignore_errors: true
+  openvswitch_db:
+    table: Bridge
+    record: br-test
+    col: other_config
+    value: true
+  register: result
+
+- assert:
+    that:
+      - "result.failed == true"
+
 - name: Remove bridge
   openvswitch_db:
     table: Bridge
diff --git a/test/units/modules/network/ovs/test_openvswitch_db.py b/test/units/modules/network/ovs/test_openvswitch_db.py
--- a/test/units/modules/network/ovs/test_openvswitch_db.py
+++ b/test/units/modules/network/ovs/test_openvswitch_db.py
@@ -41,6 +41,12 @@
     'test_openvswitch_db_present_updates_key': [
         (0, 'openvswitch_db_disable_in_band_true.cfg', None),
         (0, None, None)],
+    'test_openvswitch_db_present_missing_key_on_map': [
+        (0, 'openvswitch_db_disable_in_band_true.cfg', None),
+        (0, None, None)],
+    'test_openvswitch_db_present_stp_enable': [
+        (0, 'openvswitch_db_disable_in_band_true.cfg', None),
+        (0, None, None)],
 }
 
 
@@ -122,3 +128,20 @@ def test_openvswitch_db_present_updates_key(self):
             commands=['/usr/bin/ovs-vsctl -t 5 set Bridge test-br other_config'
                       ':disable-in-band=False'],
             test_name='test_openvswitch_db_present_updates_key')
+
+    def test_openvswitch_db_present_missing_key_on_map(self):
+        set_module_args(dict(state='present',
+                             table='Bridge', record='test-br',
+                             col='other_config',
+                             value='False'))
+        self.execute_module(
+            failed=True,
+            test_name='test_openvswitch_db_present_idempotent')
+
+    def test_openvswitch_db_present_stp_enable(self):
+        set_module_args(dict(state='present',
+                             table='Bridge', record='test-br',
+                             col='stp_enable',
+                             value='False'))
+        self.execute_module(changed=True,
+                            test_name='test_openvswitch_db_present_stp_enable')