Repository: ansible/ansible

Issue:
nxos_evpn_vni and nxos_interface module failures
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and master branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest: -->
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->
nxos_evpn_vni
nxos_interface

##### ANSIBLE VERSION
<!--- Paste verbatim output from “ansible --version” between quotes below -->
```

ansible 2.4.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Nov  6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]

```

##### CONFIGURATION
<!---
Mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).
-->
no changes

##### OS / ENVIRONMENT
<!---

-->
RHEL 7.2

##### SUMMARY
<!--- Explain the problem briefly -->
I recently upgraded ansible from 2.2 to 2.4 to fix an NXOS module issue where nxos_vlan_vtep_vni was not deleting the interface when putting it on an absent state. However now I get errors when I try to create objects using nxos_evpn_vni and nxos_interface. 

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->
```
    - name: Configure EVPN
      nxos_evpn_vni:
        vni: "{{ vnid }}"
        route_distinguisher: auto
        route_target_import: auto
        route_target_export: auto
        username: ""
        password: ""
        host: "{{ item }}"
      with_items:
        - "{{ leaf1ip }}"
      register: evpnconfig
      ignore_errors: yes

    - name: Configure fabric forwarding anycast gateway
      nxos_interface:
        interface: vlan {{vlan_id}}
        description: "{{ vlan_name }}"
        transport: cli
        state: present
        username: ""
        password: ""
        host: "{{ item }}"
        fabric_forwarding_anycast_gateway: true
      with_items:
        - "{{ leaf1ip }}"
      ignore_errors: yes
      register: fabforconfig



```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
Creation of the objects, worked in version 2.2

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

<!--- Paste verbatim command output between quotes below -->
```
TASK [Configure EVPN] ***********************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: 'NoneType' object is not iterable
failed: [localhost] (item=x.x.x.x) => {"failed": true, "item": "x.x.x.x", "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_DplDrg/ansible_module_nxos_evpn_vni.py\", line 363, in <module>\n    main()\n  File \"/tmp/ansible_DplDrg/ansible_module_nxos_evpn_vni.py\", line 346, in main\n    result = execute_config(module, candidate)\n  File \"/tmp/ansible_DplDrg/ansible_module_nxos_evpn_vni.py\", line 272, in execute_config\n    result.update(response)\nTypeError: 'NoneType' object is not iterable\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}


TASK [Configure fabric forwarding anycast gateway] ******************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'dict' object has no attribute 'split'
failed: [localhost] (itemx.x.x.x) => {"failed": true, "item": "x.x.x.x", "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_E1__fU/ansible_module_nxos_interface.py\", line 711, in <module>\n    main()\n  File \"/tmp/ansible_E1__fU/ansible_module_nxos_interface.py\", line 637, in main\n    existing, is_default = smart_existing(module, intf_type, normalized_interface)\n  File \"/tmp/ansible_E1__fU/ansible_module_nxos_interface.py\", line 536, in smart_existing\n    existing = get_interface(normalized_interface, module)\n  File \"/tmp/ansible_E1__fU/ansible_module_nxos_interface.py\", line 303, in get_interface\n    attributes = get_manual_interface_attributes(intf, module)\n  File \"/tmp/ansible_E1__fU/ansible_module_nxos_interface.py\", line 228, in get_manual_interface_attributes\n    command_list = body.split('\\n')\nAttributeError: 'dict' object has no attribute 'split'\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}

```



Discussion:
cc @jedelman8 @GGabriele @rahushen @privateip @rcarrillocruz @mikewiebe
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/network/nxos/nxos_evpn_vni.py b/lib/ansible/modules/network/nxos/nxos_evpn_vni.py
--- a/lib/ansible/modules/network/nxos/nxos_evpn_vni.py
+++ b/lib/ansible/modules/network/nxos/nxos_evpn_vni.py
@@ -16,9 +16,11 @@
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-ANSIBLE_METADATA = {'metadata_version': '1.0',
-                    'status': ['preview'],
-                    'supported_by': 'community'}
+ANSIBLE_METADATA = {
+    'metadata_version': '1.0',
+    'status': ['preview'],
+    'supported_by': 'community'
+}
 
 
 DOCUMENTATION = '''
@@ -28,146 +30,116 @@
 version_added: "2.2"
 short_description: Manages Cisco EVPN VXLAN Network Identifier (VNI).
 description:
-    - Manages Cisco Ethernet Virtual Private Network (EVPN) VXLAN Network
-      Identifier (VNI) configurations of a Nexus device.
+  - Manages Cisco Ethernet Virtual Private Network (EVPN) VXLAN Network
+    Identifier (VNI) configurations of a Nexus device.
 author: Gabriele Gerbino (@GGabriele)
 notes:
-    - default, where supported, restores params default value.
-    - RD override is not permitted. You should set it to the default values
-      first and then reconfigure it.
-    - C(route_target_both), C(route_target_import) and
-      C(route_target_export valid) values are a list of extended communities,
-      (i.e. ['1.2.3.4:5', '33:55']) or the keywords 'auto' or 'default'.
-    - The C(route_target_both) property is discouraged due to the inconsistent
-      behavior of the property across Nexus platforms and image versions.
-      For this reason it is recommended to use explicit C(route_target_export)
-      and C(route_target_import) properties instead of C(route_target_both).
-    - RD valid values are a string in one of the route-distinguisher formats,
-      the keyword 'auto', or the keyword 'default'.
+  - default, where supported, restores params default value.
+  - RD override is not permitted. You should set it to the default values
+    first and then reconfigure it.
+  - C(route_target_both), C(route_target_import) and
+    C(route_target_export valid) values are a list of extended communities,
+    (i.e. ['1.2.3.4:5', '33:55']) or the keywords 'auto' or 'default'.
+  - The C(route_target_both) property is discouraged due to the inconsistent
+    behavior of the property across Nexus platforms and image versions.
+    For this reason it is recommended to use explicit C(route_target_export)
+    and C(route_target_import) properties instead of C(route_target_both).
+  - RD valid values are a string in one of the route-distinguisher formats,
+    the keyword 'auto', or the keyword 'default'.
 options:
-    vni:
-        description:
-            - The EVPN VXLAN Network Identifier.
-        required: true
-        default: null
-    route_distinguisher:
-        description:
-            - The VPN Route Distinguisher (RD). The RD is combined with
-              the IPv4 or IPv6 prefix learned by the PE router to create a
-              globally unique address.
-        required: true
-        default: null
-    route_target_both:
-        description:
-            - Enables/Disables route-target settings for both import and
-              export target communities using a single property.
-        required: false
-        default: null
-    route_target_import:
-        description:
-            - Sets the route-target 'import' extended communities.
-        required: false
-        default: null
-    route_target_export:
-        description:
-            - Sets the route-target 'import' extended communities.
-        required: false
-        default: null
-    state:
-        description:
-            - Determines whether the config should be present or not
-              on the device.
-        required: false
-        default: present
-        choices: ['present','absent']
+  vni:
+    description:
+      - The EVPN VXLAN Network Identifier.
+    required: true
+    default: null
+  route_distinguisher:
+    description:
+      - The VPN Route Distinguisher (RD). The RD is combined with
+        the IPv4 or IPv6 prefix learned by the PE router to create a
+        globally unique address.
+    required: true
+    default: null
+  route_target_both:
+    description:
+      - Enables/Disables route-target settings for both import and
+        export target communities using a single property.
+    required: false
+    default: null
+  route_target_import:
+    description:
+      - Sets the route-target 'import' extended communities.
+    required: false
+    default: null
+  route_target_export:
+    description:
+      - Sets the route-target 'import' extended communities.
+    required: false
+    default: null
+  state:
+    description:
+      - Determines whether the config should be present or not
+        on the device.
+    required: false
+    default: present
+    choices: ['present','absent']
 '''
+
 EXAMPLES = '''
-- nxos_evpn_vni:
+- name: vni configuration
+  nxos_evpn_vni:
     vni: 6000
     route_distinguisher: "60:10"
     route_target_import:
-        - "5000:10"
-        - "4100:100"
+      - "5000:10"
+      - "4100:100"
     route_target_export: auto
     route_target_both: default
-    username: "{{ un }}"
-    password: "{{ pwd }}"
-    host: "{{ inventory_hostname }}"
 '''
 
 RETURN = '''
-proposed:
-    description: k/v pairs of parameters passed into module
-    returned: verbose mode
-    type: dict
-    sample: {"route_target_import": ["5000:10", "4100:100",
-             "5001:10"],"vni": "6000"}
-existing:
-    description: k/v pairs of existing EVPN VNI configuration
-    returned: verbose mode
-    type: dict
-    sample: {"route_distinguisher": "70:10", "route_target_both": [],
-            "route_target_export": [], "route_target_import": [
-            "4100:100", "5000:10"], "vni": "6000"}
-end_state:
-    description: k/v pairs of EVPN VNI configuration after module execution
-    returned: verbose mode
-    type: dict
-    sample: {"route_distinguisher": "70:10", "route_target_both": [],
-             "route_target_export": [], "route_target_import": [
-             "4100:100", "5000:10", "5001:10"], "vni": "6000"}
-updates:
+commands:
     description: commands sent to the device
     returned: always
     type: list
     sample: ["evpn", "vni 6000 l2", "route-target import 5001:10"]
-changed:
-    description: check to see if a change was made on the device
-    returned: always
-    type: boolean
-    sample: true
 '''
 
 import re
+import time
 from ansible.module_utils.nxos import get_config, load_config, run_commands
 from ansible.module_utils.nxos import nxos_argument_spec, check_args
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils.netcfg import CustomNetworkConfig
 
+
 PARAM_TO_COMMAND_KEYMAP = {
     'vni': 'vni',
+    'route_distinguisher': 'rd',
     'route_target_both': 'route-target both',
     'route_target_import': 'route-target import',
-    'route_target_export': 'route-target export',
-    'route_distinguisher': 'rd'
+    'route_target_export': 'route-target export'
 }
-WARNINGS = []
-
-import time
-
-def invoke(name, *args, **kwargs):
-    func = globals().get(name)
-    if func:
-        return func(*args, **kwargs)
 
 
 def get_value(arg, config, module):
-    REGEX = re.compile(r'(?:{0}\s)(?P<value>.*)$'.format(PARAM_TO_COMMAND_KEYMAP[arg]), re.M)
+    command = PARAM_TO_COMMAND_KEYMAP.get(arg)
+    command_re = re.compile(r'(?:{0}\s)(?P<value>.*)$'.format(command), re.M)
     value = ''
-    if PARAM_TO_COMMAND_KEYMAP[arg] in config:
-        value = REGEX.search(config).group('value')
+    if command in config:
+        value = command_re.search(config).group('value')
     return value
 
 
 def get_route_target_value(arg, config, module):
     splitted_config = config.splitlines()
     value_list = []
-    REGEX = re.compile(r'(?:{0}\s)(?P<value>.*)$'.format(PARAM_TO_COMMAND_KEYMAP[arg]), re.M)
+    command = PARAM_TO_COMMAND_KEYMAP.get(arg)
+    command_re = re.compile(r'(?:{0}\s)(?P<value>.*)$'.format(command), re.M)
 
     for line in splitted_config:
         value = ''
-        if PARAM_TO_COMMAND_KEYMAP[arg] in line.strip():
-            value = REGEX.search(line).group('value')
+        if command in line.strip():
+            value = command_re.search(line).group('value')
             value_list.append(value)
     return value_list
 
@@ -197,14 +169,10 @@ def get_existing(module, args):
 
 def apply_key_map(key_map, table):
     new_dict = {}
-    for key, value in table.items():
+    for key in table:
         new_key = key_map.get(key)
         if new_key:
-            value = table.get(key)
-            if value:
-                new_dict[new_key] = value
-            else:
-                new_dict[new_key] = value
+            new_dict[new_key] = table.get(key)
     return new_dict
 
 
@@ -236,26 +204,25 @@ def state_present(module, existing, proposed):
                 if existing_value:
                     for target in existing_value:
                         commands.append('no {0} {1}'.format(key, target))
-            else:
-                if not isinstance(value, list):
-                    value = [value]
-                for target in value:
-                    if existing:
-                        if target not in existing.get(key.replace('-', '_').replace(' ', '_')):
-                            commands.append('{0} {1}'.format(key, target))
-                    else:
+            elif not isinstance(value, list):
+                value = [value]
+            for target in value:
+                if existing:
+                    if target not in existing.get(key.replace('-', '_').replace(' ', '_')):
                         commands.append('{0} {1}'.format(key, target))
+                else:
+                    commands.append('{0} {1}'.format(key, target))
+        elif value == 'default':
+            existing_value = existing_commands.get(key)
+            if existing_value:
+                commands.append('no {0} {1}'.format(key, existing_value))
         else:
-            if value == 'default':
-                existing_value = existing_commands.get(key)
-                if existing_value:
-                    commands.append('no {0} {1}'.format(key, existing_value))
-            else:
-                command = '{0} {1}'.format(key, value)
-                commands.append(command)
+            command = '{0} {1}'.format(key, value)
+            commands.append(command)
 
-    if commands:
-        parents = ['evpn', 'vni {0} l2'.format(module.params['vni'])]
+    else:
+        commands = ['vni {0} l2'.format(module.params['vni'])]
+        parents = ['evpn']
 
     return commands, parents
 
@@ -266,13 +233,6 @@ def state_absent(module, existing, proposed):
     return commands, parents
 
 
-def execute_config(module, candidate):
-    result = {}
-    response = load_config(module, candidate)
-    result.update(response)
-    return result
-
-
 def main():
     argument_spec = dict(
         vni=dict(required=True, type='str'),
@@ -280,8 +240,7 @@ def main():
         route_target_both=dict(required=False, type='list'),
         route_target_import=dict(required=False, type='list'),
         route_target_export=dict(required=False, type='list'),
-        state=dict(choices=['present', 'absent'], default='present',
-                       required=False),
+        state=dict(choices=['present', 'absent'], default='present', required=False),
         include_defaults=dict(default=True),
         config=dict(),
         save=dict(type='bool', default=False)
@@ -289,26 +248,19 @@ def main():
 
     argument_spec.update(nxos_argument_spec)
 
-    module = AnsibleModule(argument_spec=argument_spec,
-                        supports_check_mode=True)
+    module = AnsibleModule(argument_spec=argument_spec, supports_check_mode=True)
 
     warnings = list()
     check_args(module, warnings)
-
+    results = dict(changed=False, warnings=warnings)
 
     state = module.params['state']
-    args =  [
-        'vni',
-        'route_distinguisher',
-        'route_target_both',
-        'route_target_import',
-        'route_target_export'
-    ]
-
-    existing = invoke('get_existing', module, args)
-    end_state = existing
+    args = PARAM_TO_COMMAND_KEYMAP.keys()
+    existing = get_existing(module, args)
     proposed_args = dict((k, v) for k, v in module.params.items()
-                    if v is not None and k in args)
+                         if v is not None and k in args)
+    commands = []
+    parents = []
 
     proposed = {}
     for key, value in proposed_args.items():
@@ -317,48 +269,42 @@ def main():
                 value = True
             elif value == 'false':
                 value = False
-            if existing.get(key) or (not existing.get(key) and value):
+            if existing.get(key) != value:
                 proposed[key] = value
-    result = {}
-    if state == 'present' or (state == 'absent' and existing):
-        candidate = CustomNetworkConfig(indent=3)
-        commands, parents = invoke('state_%s' % state, module, existing,
-                                                proposed)
-        if commands:
-            if (existing.get('route_distinguisher') and
-                    proposed.get('route_distinguisher')):
-                if (existing['route_distinguisher'] != proposed['route_distinguisher'] and
-                        proposed['route_distinguisher'] != 'default'):
-                    WARNINGS.append('EVPN RD {0} was automatically removed. '
-                                         'It is highly recommended to use a task '
-                                         '(with default as value) to explicitly '
-                                         'unconfigure it.'.format(
-                                             existing['route_distinguisher']))
-                    remove_commands = ['no rd {0}'.format(
-                        existing['route_distinguisher'])]
-
-                    candidate.add(remove_commands, parents=parents)
-                    result = execute_config(module, candidate)
-                    time.sleep(30)
 
+    if state == 'present':
+        commands, parents = state_present(module, existing, proposed)
+    elif state == 'absent' and existing:
+        commands, parents = state_absent(module, existing, proposed)
+
+    if commands:
+        if (existing.get('route_distinguisher') and
+                proposed.get('route_distinguisher')):
+            if (existing['route_distinguisher'] != proposed['route_distinguisher'] and
+                    proposed['route_distinguisher'] != 'default'):
+                warnings.append('EVPN RD {0} was automatically removed. '
+                                'It is highly recommended to use a task '
+                                '(with default as value) to explicitly '
+                                'unconfigure it.'.format(existing['route_distinguisher']))
+                remove_commands = ['no rd {0}'.format(existing['route_distinguisher'])]
+
+                candidate = CustomNetworkConfig(indent=3)
+                candidate.add(remove_commands, parents=parents)
+                load_config(module, candidate)
+                results['changed'] = True
+                results['commands'] = candidate.items_text()
+                time.sleep(30)
+
+        else:
             candidate = CustomNetworkConfig(indent=3)
             candidate.add(commands, parents=parents)
-            result = execute_config(module, candidate)
+            load_config(module, candidate)
+            results['changed'] = True
+            results['commands'] = candidate.items_text()
     else:
-        result['updates'] = []
-
-    if module._verbosity > 0:
-        end_state = invoke('get_existing', module, args)
-        result['end_state'] = end_state
-        result['existing'] = existing
-        result['proposed'] = proposed_args
-
-    if WARNINGS:
-        result['warnings'] = WARNINGS
-
-    module.exit_json(**result)
+        results['commands'] = []
+    module.exit_json(**results)
 
 
 if __name__ == '__main__':
     main()
-


Test patch:
diff --git a/test/sanity/pep8/legacy-files.txt b/test/sanity/pep8/legacy-files.txt
--- a/test/sanity/pep8/legacy-files.txt
+++ b/test/sanity/pep8/legacy-files.txt
@@ -495,7 +495,6 @@ lib/ansible/modules/network/nxos/nxos_bgp_neighbor_af.py
 lib/ansible/modules/network/nxos/nxos_command.py
 lib/ansible/modules/network/nxos/nxos_config.py
 lib/ansible/modules/network/nxos/nxos_evpn_global.py
-lib/ansible/modules/network/nxos/nxos_evpn_vni.py
 lib/ansible/modules/network/nxos/nxos_facts.py
 lib/ansible/modules/network/nxos/nxos_feature.py
 lib/ansible/modules/network/nxos/nxos_file_copy.py
diff --git a/test/units/modules/network/nxos/test_nxos_evpn_vni.py b/test/units/modules/network/nxos/test_nxos_evpn_vni.py
new file mode 100644
--- /dev/null
+++ b/test/units/modules/network/nxos/test_nxos_evpn_vni.py
@@ -0,0 +1,59 @@
+# (c) 2016 Red Hat Inc.
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+# Make coding more python3-ish
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+import json
+
+from ansible.compat.tests.mock import patch
+from ansible.modules.network.nxos import nxos_evpn_vni
+from .nxos_module import TestNxosModule, load_fixture, set_module_args
+
+
+class TestNxosEvpnVniModule(TestNxosModule):
+
+    module = nxos_evpn_vni
+
+    def setUp(self):
+        self.mock_run_commands = patch('ansible.modules.network.nxos.nxos_evpn_vni.run_commands')
+        self.run_commands = self.mock_run_commands.start()
+
+        self.mock_load_config = patch('ansible.modules.network.nxos.nxos_evpn_vni.load_config')
+        self.load_config = self.mock_load_config.start()
+
+        self.mock_get_config = patch('ansible.modules.network.nxos.nxos_evpn_vni.get_config')
+        self.get_config = self.mock_get_config.start()
+
+    def tearDown(self):
+        self.mock_run_commands.stop()
+        self.mock_load_config.stop()
+        self.mock_get_config.stop()
+
+    def load_fixtures(self, commands=None):
+        self.load_config.return_value = None
+
+    def test_nxos_evpn_vni_absent(self):
+        set_module_args(dict(vni='6000', state='absent'))
+        result = self.execute_module(changed=False)
+        self.assertEqual(result['commands'], [])
+
+    def test_nxos_evpn_vni_present(self):
+        set_module_args(dict(vni='6000', state='present'))
+        result = self.execute_module(changed=True)
+        self.assertEqual(result['commands'], ['evpn', 'vni 6000 l2'])