Repository: ansible/ansible

Issue:
nxos_pim errors out for both transports (nxapi and cli)
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
 - nxos_pim

##### NXOS VERSION
7.0(3)I3(1)

##### ANSIBLE VERSION
```
ansible-playbook 2.4.0 (devel 781bb44b02) last updated 2017/06/08 15:20:16 (GMT -400)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/usr/share/ansible']
  ansible python module location = /root/agents-ci/ansible/lib/ansible
  executable location = /root/agents-ci/ansible/bin/ansible-playbook
  python version = 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4]
```

##### OS / ENVIRONMENT
running on Ubuntu 14.04

##### SUMMARY
The module errors out in latest `devel` when using transport `nxapi` or `cli` but it errors out differently for each.

##### STEPS TO REPRODUCE
```yaml
---
- name: configure
  nxos_pim:
    ssm_range: "232.0.0.0/8"
    provider: "{{ nxapi }}"
```

##### EXPECTED RESULTS
Expect nxapi_pim module to run properly.

##### ACTUAL RESULTS FOR TRANSPORT NXAPI

```shell
TASK [nxos_pim : debug] ***********************************************************************************************************************************************************************************************************************
task path: /root/agents-ci/ansible/test/integration/targets/nxos_pim/tests/nxapi/sanity.yaml:2
ok: [n9k] => {
    "failed": false, 
    "msg": "START TRANSPORT:NXAPI nxos_pim sanity test"
}

TASK [nxos_pim : configure] *******************************************************************************************************************************************************************************************************************
task path: /root/agents-ci/ansible/test/integration/targets/nxos_pim/tests/nxapi/sanity.yaml:4
<n9k> connection transport is nxapi
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/basic.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/netcfg.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/nxos.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/_text.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/pycompat24.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/six/__init__.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/six/_six.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/network_common.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/connection.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/urls.py
Using module file /root/agents-ci/ansible/lib/ansible/modules/network/nxos/nxos_pim.py
<n9k> ESTABLISH LOCAL CONNECTION FOR USER: root
<n9k> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398 `" && echo ansible-tmp-1496951930.35-277675936470398="` echo $HOME/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398 `" ) && sleep 0'
<n9k> PUT /tmp/tmpsqIPxa TO /root/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398/nxos_pim.py
<n9k> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398/ /root/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398/nxos_pim.py && sleep 0'
<n9k> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398/nxos_pim.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1496951930.35-277675936470398/" > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_t5Q4mg/ansible_module_nxos_pim.py", line 192, in <module>
    main()
  File "/tmp/ansible_t5Q4mg/ansible_module_nxos_pim.py", line 176, in main
    response = load_config(module, candidate)
  File "/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py", line 399, in load_config
  File "/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py", line 346, in load_config
  File "/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py", line 262, in send_request
  File "/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py", line 227, in _request_builder
TypeError: sequence item 0: expected string, CustomNetworkConfig found

fatal: [n9k]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_t5Q4mg/ansible_module_nxos_pim.py\", line 192, in <module>\n    main()\n  File \"/tmp/ansible_t5Q4mg/ansible_module_nxos_pim.py\", line 176, in main\n    response = load_config(module, candidate)\n  File \"/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py\", line 399, in load_config\n  File \"/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py\", line 346, in load_config\n  File \"/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py\", line 262, in send_request\n  File \"/tmp/ansible_t5Q4mg/ansible_modlib.zip/ansible/module_utils/nxos.py\", line 227, in _request_builder\nTypeError: sequence item 0: expected string, CustomNetworkConfig found\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}
	to retry, use: --limit @/root/agents-ci/ansible/test/integration/nxos.retry

PLAY RECAP ************************************************************************************************************************************************************************************************************************************
n9k          : ok=6    changed=0    unreachable=0    failed=1   
```

##### ACTUAL RESULTS FOR TRANSPORT CLI

```shell
TASK [nxos_pim : debug] ***********************************************************************************************************************************************************************************************************************
task path: /root/agents-ci/ansible/test/integration/targets/nxos_pim/tests/cli/sanity.yaml:2
ok: [n9k] => {
    "failed": false, 
    "msg": "START TRANSPORT:CLI nxos_pim sanity test"
}

TASK [nxos_pim : configure] *******************************************************************************************************************************************************************************************************************
task path: /root/agents-ci/ansible/test/integration/targets/nxos_pim/tests/cli/sanity.yaml:4
<n9k> connection transport is cli
<n9k> using connection plugin network_cli
<n9k> socket_path: /root/.ansible/pc/4cc10f215f
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/basic.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/netcfg.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/nxos.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/_text.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/pycompat24.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/six/__init__.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/six/_six.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/network_common.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/connection.py
Using module_utils file /root/agents-ci/ansible/lib/ansible/module_utils/urls.py
Using module file /root/agents-ci/ansible/lib/ansible/modules/network/nxos/nxos_pim.py
<n9k> ESTABLISH LOCAL CONNECTION FOR USER: root
<n9k> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745 `" && echo ansible-tmp-1496952076.16-122450992659745="` echo $HOME/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745 `" ) && sleep 0'
<n9k> PUT /tmp/tmpbWAeyp TO /root/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745/nxos_pim.py
<n9k> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745/ /root/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745/nxos_pim.py && sleep 0'
<n9k> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745/nxos_pim.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1496952076.16-122450992659745/" > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_bjyvu4/ansible_module_nxos_pim.py", line 192, in <module>
    main()
  File "/tmp/ansible_bjyvu4/ansible_module_nxos_pim.py", line 177, in main
    result.update(response)
TypeError: 'NoneType' object is not iterable

fatal: [n9k]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_bjyvu4/ansible_module_nxos_pim.py\", line 192, in <module>\n    main()\n  File \"/tmp/ansible_bjyvu4/ansible_module_nxos_pim.py\", line 177, in main\n    result.update(response)\nTypeError: 'NoneType' object is not iterable\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}
	to retry, use: --limit @/root/agents-ci/ansible/test/integration/nxos.retry

PLAY RECAP ************************************************************************************************************************************************************************************************************************************
n9k          : ok=5    changed=0    unreachable=0    failed=1   

```


Discussion:
cc @jedelman8 @GGabriele @rahushen @privateip @rcarrillocruz @trishnaguha
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/network/nxos/nxos_pim.py b/lib/ansible/modules/network/nxos/nxos_pim.py
--- a/lib/ansible/modules/network/nxos/nxos_pim.py
+++ b/lib/ansible/modules/network/nxos/nxos_pim.py
@@ -16,9 +16,11 @@
 # along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
 #
 
-ANSIBLE_METADATA = {'metadata_version': '1.0',
-                    'status': ['preview'],
-                    'supported_by': 'community'}
+ANSIBLE_METADATA = {
+    'metadata_version': '1.0',
+    'status': ['preview'],
+    'supported_by': 'community',
+}
 
 
 DOCUMENTATION = '''
@@ -31,84 +33,51 @@
     - Manages configuration of a Protocol Independent Multicast (PIM) instance.
 author: Gabriele Gerbino (@GGabriele)
 options:
-    ssm_range:
-        description:
-            - Configure group ranges for Source Specific Multicast (SSM).
-              Valid values are multicast addresses or the keyword 'none'.
-        required: true
+  ssm_range:
+    description:
+      - Configure group ranges for Source Specific Multicast (SSM).
+        Valid values are multicast addresses or the keyword 'none'.
+    required: true
 '''
 EXAMPLES = '''
 - nxos_pim:
     ssm_range: "232.0.0.0/8"
-    username: "{{ un }}"
-    password: "{{ pwd }}"
-    host: "{{ inventory_hostname }}"
 '''
 
 RETURN = '''
-proposed:
-    description: k/v pairs of parameters passed into module
-    returned: verbose mode
-    type: dict
-    sample: {"ssm_range": "232.0.0.0/8"}
-existing:
-    description: k/v pairs of existing PIM configuration
-    returned: verbose mode
-    type: dict
-    sample: {"ssm_range": none}
-end_state:
-    description: k/v pairs of BGP configuration after module execution
-    returned: verbose mode
-    type: dict
-    sample: {"ssm_range": "232.0.0.0/8"}
-updates:
+commands:
     description: commands sent to the device
     returned: always
     type: list
     sample: ["ip pim ssm range 232.0.0.0/8"]
-changed:
-    description: check to see if a change was made on the device
-    returned: always
-    type: boolean
-    sample: true
 '''
 
 
 import re
-from ansible.module_utils.nxos import get_config, load_config, run_commands
+
+from ansible.module_utils.nxos import get_config, load_config
 from ansible.module_utils.nxos import nxos_argument_spec, check_args
 from ansible.module_utils.basic import AnsibleModule
 from ansible.module_utils.netcfg import CustomNetworkConfig
 
-import re
-
 
 PARAM_TO_COMMAND_KEYMAP = {
     'ssm_range': 'ip pim ssm range'
 }
-PARAM_TO_DEFAULT_KEYMAP = {}
-WARNINGS = []
-
-
-def invoke(name, *args, **kwargs):
-    func = globals().get(name)
-    if func:
-        return func(*args, **kwargs)
-
-
-def get_value(arg, config, module):
-    REGEX = re.compile(r'(?:{0}\s)(?P<value>.*)$'.format(PARAM_TO_COMMAND_KEYMAP[arg]), re.M)
-    value = ''
-    if PARAM_TO_COMMAND_KEYMAP[arg] in config:
-        value = REGEX.search(config).group('value')
-    return value
 
 
 def get_existing(module, args):
     existing = {}
     config = str(get_config(module))
+
     for arg in args:
-        existing[arg] = get_value(arg, config, module)
+        command = PARAM_TO_COMMAND_KEYMAP[arg]
+        has_command = re.match(r'(?:{0}\s)(?P<value>.*)$'.format(command), config, re.M)
+
+        value = ''
+        if has_command:
+            value = has_command.group('value')
+        existing[arg] = value
     return existing
 
 
@@ -116,19 +85,14 @@ def apply_key_map(key_map, table):
     new_dict = {}
     for key, value in table.items():
         new_key = key_map.get(key)
-        if new_key:
-            value = table.get(key)
-            if value:
-                new_dict[new_key] = value
-            else:
-                new_dict[new_key] = value
+        if value:
+            new_dict[new_key] = value
     return new_dict
 
 
 def get_commands(module, existing, proposed, candidate):
     commands = list()
     proposed_commands = apply_key_map(PARAM_TO_COMMAND_KEYMAP, proposed)
-    existing_commands = apply_key_map(PARAM_TO_COMMAND_KEYMAP, existing)
 
     for key, value in proposed_commands.items():
         command = '{0} {1}'.format(key, value)
@@ -141,53 +105,39 @@ def get_commands(module, existing, proposed, candidate):
 def main():
     argument_spec = dict(
         ssm_range=dict(required=True, type='str'),
-        m_facts=dict(required=False, default=False, type='bool'),
-        include_defaults=dict(default=False),
-        config=dict(),
-        save=dict(type='bool', default=False)
     )
 
     argument_spec.update(nxos_argument_spec)
 
     module = AnsibleModule(argument_spec=argument_spec,
-                                supports_check_mode=True)
+                           supports_check_mode=True)
 
     warnings = list()
     check_args(module, warnings)
-
+    result = {'changed': False, 'commands': [], 'warnings': warnings}
 
     splitted_ssm_range = module.params['ssm_range'].split('.')
     if len(splitted_ssm_range) != 4 and module.params['ssm_range'] != 'none':
         module.fail_json(msg="Valid ssm_range values are multicast addresses "
                              "or the keyword 'none'.")
 
-    args =  [
-        'ssm_range'
-    ]
+    args = PARAM_TO_COMMAND_KEYMAP.keys()
 
-    existing = invoke('get_existing', module, args)
-    end_state = existing
+    existing = get_existing(module, args)
     proposed = dict((k, v) for k, v in module.params.items()
-                    if v is not None and k in args)
+                    if k in args and v != existing[k])
 
-    result = {}
     candidate = CustomNetworkConfig(indent=3)
-    invoke('get_commands', module, existing, proposed, candidate)
-    response = load_config(module, candidate)
-    result.update(response)
-
-    if module._verbosity > 0:
-        end_state = invoke('get_existing', module, args)
-        result['end_state'] = end_state
-        result['existing'] = existing
-        result['proposed'] = proposed
+    get_commands(module, existing, proposed, candidate)
 
-    if WARNINGS:
-        result['warnings'] = WARNINGS
+    if candidate:
+        candidate = candidate.items_text()
+        result['commands'] = candidate
+        result['changed'] = True
+        load_config(module, candidate)
 
     module.exit_json(**result)
 
 
 if __name__ == '__main__':
     main()
-


Test patch:
diff --git a/test/sanity/pep8/legacy-files.txt b/test/sanity/pep8/legacy-files.txt
--- a/test/sanity/pep8/legacy-files.txt
+++ b/test/sanity/pep8/legacy-files.txt
@@ -371,7 +371,6 @@ lib/ansible/modules/network/nxos/nxos_ntp_auth.py
 lib/ansible/modules/network/nxos/nxos_ntp_options.py
 lib/ansible/modules/network/nxos/nxos_nxapi.py
 lib/ansible/modules/network/nxos/nxos_overlay_global.py
-lib/ansible/modules/network/nxos/nxos_pim.py
 lib/ansible/modules/network/nxos/nxos_pim_interface.py
 lib/ansible/modules/network/nxos/nxos_pim_rp_address.py
 lib/ansible/modules/network/nxos/nxos_ping.py
diff --git a/test/units/modules/network/nxos/fixtures/nxos_pim/config.cfg b/test/units/modules/network/nxos/fixtures/nxos_pim/config.cfg
new file mode 100644
--- /dev/null
+++ b/test/units/modules/network/nxos/fixtures/nxos_pim/config.cfg
@@ -0,0 +1 @@
+ip pim ssm range 127.0.0.0/31
diff --git a/test/units/modules/network/nxos/test_nxos_pim.py b/test/units/modules/network/nxos/test_nxos_pim.py
new file mode 100644
--- /dev/null
+++ b/test/units/modules/network/nxos/test_nxos_pim.py
@@ -0,0 +1,58 @@
+# (c) 2016 Red Hat Inc.
+#
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+
+# Make coding more python3-ish
+from __future__ import (absolute_import, division, print_function)
+__metaclass__ = type
+
+import os
+
+from ansible.compat.tests.mock import patch
+from ansible.modules.network.nxos import nxos_pim
+from .nxos_module import TestNxosModule, load_fixture, set_module_args
+
+
+class TestNxosPimModule(TestNxosModule):
+
+    module = nxos_pim
+
+    def setUp(self):
+        self.mock_get_config = patch('ansible.modules.network.nxos.nxos_pim.get_config')
+        self.get_config = self.mock_get_config.start()
+
+        self.mock_load_config = patch('ansible.modules.network.nxos.nxos_pim.load_config')
+        self.load_config = self.mock_load_config.start()
+
+    def tearDown(self):
+        self.mock_get_config.stop()
+        self.mock_load_config.stop()
+
+    def load_fixtures(self, commands=None, device=''):
+        self.get_config.return_value = load_fixture('nxos_pim', 'config.cfg')
+        self.load_config.return_value = None
+
+    def test_nxos_pim(self):
+        set_module_args(dict(ssm_range='232.0.0.0/8'))
+        self.execute_module(changed=True, commands=['ip pim ssm range 232.0.0.0/8'])
+
+    def test_nxos_pim_none(self):
+        set_module_args(dict(ssm_range='none'))
+        self.execute_module(changed=True, commands=['ip pim ssm range none'])
+
+    def test_nxos_pim_no_change(self):
+        set_module_args(dict(ssm_range='127.0.0.0/31'))
+        self.execute_module(changed=False, commands=[])