Repository: ansible/ansible

Issue:
Ansible 2.5 - ios_facts fails when running in IOS-XE 3850 switch
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ios-facts

##### ANSIBLE VERSION
ansible 2.5.0
ansible 2.5.0
  config file = /home/mfisher/.ansible.cfg
  configured module search path = [u'/home/mfisher/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]

##### CONFIGURATION
DEFAULT_EXECUTABLE(/home/mfisher/.ansible.cfg) = /bin/sh
DEFAULT_HOST_LIST(/home/mfisher/.ansible.cfg) = [u'/home/mfisher/SIPS_AUTOMATION/sips_development/hosts', u'/etc/ansible/hosts']
DEFAULT_JINJA2_EXTENSIONS(/home/mfisher/.ansible.cfg) = jinja2.ext.do,jinja2.ext.i18n
DEFAULT_LOCAL_TMP(/home/mfisher/.ansible.cfg) = /home/mfisher/.ansible/tmp/ansible-local-24706SGstyP
DEFAULT_LOG_PATH(/home/mfisher/.ansible.cfg) = /home/mfisher/.ansible/log
DEFAULT_REMOTE_PORT(/home/mfisher/.ansible.cfg) = 22
DEFAULT_ROLES_PATH(/home/mfisher/.ansible.cfg) = [u'/etc/ansible/roles', u'/usr/share/ansible/roles']
DEFAULT_STDOUT_CALLBACK(/home/mfisher/.ansible.cfg) = skippy
DEFAULT_TRANSPORT(/home/mfisher/.ansible.cfg) = smart
HOST_KEY_CHECKING(/home/mfisher/.ansible.cfg) = False
PERSISTENT_CONNECT_RETRY_TIMEOUT(/home/mfisher/.ansible.cfg) = 30

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->

RHEL 7.4 - Server
IOS-XE 03.07,05E - Cisco 3850 Switch

##### SUMMARY
ios_facts module fails when run against IOS-XE switch. While IOS and IOS-XE share similar command sets, they are not identical. 

##### STEPS TO REPRODUCE
Happens on every run. This is the role that is failing. The role is in a playbook being called. Paramters at the top are as follows:
- name: playbook name
  hosts: Cisco_3850_Switches
  connection: local
  gather_facts: no
```
- name: Check if VLAN 100 is in VLAN database
  ios_command:
    commands:
      - show vlan id 100
    wait_for:
      - result[0] contains "100"

- name: Validate Gather Facts
  ios_facts:
    gather_subset: all
  register: output

- debug: var=output
```

##### EXPECTED RESULTS
Would expect the module to run and return all the pertinent information gathered from the ios_facts module.

##### ACTUAL RESULTS

```
[<user name>@niddserver01 sips_playbooks]$ ansible-playbook -i ~//hosts --limit 3850-01 playbook.yml -u <user name>ar -k --tags PRE-CHECK -vvvv
ansible-playbook 2.5.1
  config file = /home/<user name>/.ansible.cfg
  configured module search path = [u'/home/<user name>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
Using /home/<user name>/.ansible.cfg as config file
SSH password: 
setting up inventory plugins
Parsed /home/<user name>//hosts inventory source with ini plugin
Loading callback plugin skippy of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/skippy.pyc

PLAYBOOK: playbook.yml ***************************************************************************************
1 plays in playbook.yml

PLAY [Playbook] ******************************************************************************
META: ran handlers

TASK [Retail-Pico-Guest-Vlan_Pre-Check : Check if VLAN 100 is in VLAN database] *********************************************
task path: /home/<user name>/SIPS_AUTOMATION/sips_playbooks/roles/Retail-Pico-Guest-Vlan_Pre-Check/tasks/main.yml:11
<<ip address>> using connection plugin network_cli (was local)
<<ip address>> starting connection from persistent connection plugin
<<ip address>> local domain socket does not exist, starting it
<<ip address>> control socket path is /home/<user name>/.ansible/pc/2707a0f7ff
<<ip address>> <<ip address>> ESTABLISH CONNECTION FOR USER: <user name>ar on PORT 22 TO <ip address>
<<ip address>> <<ip address>> ssh connection done, setting terminal
<<ip address>> <<ip address>> loaded terminal plugin for network_os ios
<<ip address>> <<ip address>> loaded cliconf plugin for network_os ios
<<ip address>> <<ip address>> firing event: on_open_shell()
<<ip address>> <<ip address>> ssh connection has completed successfully
<<ip address>> connection to remote device started successfully
<<ip address>> local domain socket listeners started successfully
<<ip address>> 
<<ip address>> local domain socket path is /home/<user name>/.ansible/pc/2707a0f7ff
<<ip address>> socket_path: /home/<user name>/.ansible/pc/2707a0f7ff
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/ios/ios_command.py
<<ip address>> ESTABLISH LOCAL CONNECTION FOR USER: <user name>
<<ip address>> EXEC /bin/bash -c '/usr/bin/python && sleep 0'
ok: [3850-01] => {
    "changed": false, 
    "invocation": {
        "module_args": {
            "auth_pass": null, 
            "authorize": null, 
            "commands": [
                "show vlan id 100"
            ], 
            "host": null, 
            "interval": 1, 
            "match": "all", 
            "password": null, 
            "port": null, 
            "provider": {
                "auth_pass": null, 
                "authorize": null, 
                "host": null, 
                "password": null, 
                "port": null, 
                "ssh_keyfile": null, 
                "timeout": null, 
                "username": null
            }, 
            "retries": 10, 
            "ssh_keyfile": null, 
            "timeout": null, 
            "username": null, 
            "wait_for": [
                "result[0] contains \"100\""
            ]
        }
    }, 
    "stdout": [
        "VLAN Name                             Status    Ports\n---- -------------------------------- --------- -------------------------------\n100  GUEST_WIFI                       active    Gi1/0/4, Gi1/0/40\n\nVLAN Type  SAID       MTU   Parent RingNo BridgeNo Stp  BrdgMode Trans1 Trans2\n---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------\n100  enet  100100     1500  -      -      -        -    -        0      0   \n\nRemote SPAN VLAN\n----------------\nDisabled\n\nPrimary Secondary Type              Ports\n------- --------- ----------------- ------------------------------------------"
    ], 
    "stdout_lines": [
        [
            "VLAN Name                             Status    Ports", 
            "---- -------------------------------- --------- -------------------------------", 
            "100  GUEST_WIFI                       active    Gi1/0/4, Gi1/0/40", 
            "", 
            "VLAN Type  SAID       MTU   Parent RingNo BridgeNo Stp  BrdgMode Trans1 Trans2", 
            "---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------", 
            "100  enet  100100     1500  -      -      -        -    -        0      0   ", 
            "", 
            "Remote SPAN VLAN", 
            "----------------", 
            "Disabled", 
            "", 
            "Primary Secondary Type              Ports", 
            "------- --------- ----------------- ------------------------------------------"
        ]
    ]
}

TASK [Vlan_Pre-Check : Validate Gather Facts] *************************************************************
task path: /home/<user name>/<file path>/tasks/main.yml:18
<<ip address>> using connection plugin network_cli (was local)
<<ip address>> starting connection from persistent connection plugin
<<ip address>> found existing local domain socket, using it!
<<ip address>> updating play_context for connection
<<ip address>> 
<<ip address>> local domain socket path is /home/<user name>/.ansible/pc/2707a0f7ff
<<ip address>> socket_path: /home/<user name>/.ansible/pc/2707a0f7ff
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/ios/ios_facts.py
<<ip address>> ESTABLISH LOCAL CONNECTION FOR USER: <user name>
<<ip address>> EXEC /bin/bash -c '/usr/bin/python && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_tTTP3y/ansible_module_ios_facts.py", line 516, in <module>
    main()
  File "/tmp/ansible_tTTP3y/ansible_module_ios_facts.py", line 501, in main
    inst.populate()
  File "/tmp/ansible_tTTP3y/ansible_module_ios_facts.py", line 224, in populate
    super(Hardware, self).populate()
  File "/tmp/ansible_tTTP3y/ansible_module_ios_facts.py", line 160, in populate
    self.responses = run_commands(self.module, commands=self.COMMANDS, check_rc=False)

  File "/tmp/ansible_tTTP3y/ansible_modlib.zip/ansible/module_utils/network/ios/ios.py", line 147, in run_commands

  File "/tmp/ansible_tTTP3y/ansible_modlib.zip/ansible/module_utils/connection.py", line 149, in __rpc__

ansible.module_utils.connection.ConnectionError: show memory statistics
                           ^
% Invalid input detected at '^' marker.


<ip address>#
fatal: [3850-01]: FAILED! => {
    "changed": false, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_tTTP3y/ansible_module_ios_facts.py\", line 516, in <module>\n    main()\n  File \"/tmp/ansible_tTTP3y/ansible_module_ios_facts.py\", line 501, in main\n    inst.populate()\n  File \"/tmp/ansible_tTTP3y/ansible_module_ios_facts.py\", line 224, in populate\n    super(Hardware, self).populate()\n  File \"/tmp/ansible_tTTP3y/ansible_module_ios_facts.py\", line 160, in populate\n    self.responses = run_commands(self.module, commands=self.COMMANDS, check_rc=False)\n  File \"/tmp/ansible_tTTP3y/ansible_modlib.zip/ansible/module_utils/network/ios/ios.py\", line 147, in run_commands\n  File \"/tmp/ansible_tTTP3y/ansible_modlib.zip/ansible/module_utils/connection.py\", line 149, in __rpc__\nansible.module_utils.connection.ConnectionError: show memory statistics\r\n                           ^\r\n% Invalid input detected at '^' marker.\r\n\r\n<ip address>#\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 1
}

PLAY RECAP ******************************************************************************************************************
3850-01                    : ok=1    changed=0    unreachable=0    failed=1   

[<user name>@niddserver01 sips_playbooks]$ ansible --version
ansible 2.5.1
  config file = /home/<user name>/.ansible.cfg
  configured module search path = [u'/home/<user name>/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, May  3 2017, 07:55:04) [GCC 4.8.5 20150623 (Red Hat 4.8.5-14)]
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/network/ios/ios_facts.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/ios/ios_facts.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @privateip @rcarrillocruz
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
I fixed the fundamental issue here in #40109, the rest of the issue should follow like the changes in ios_facts in that PR

Resolving patch:
diff --git a/lib/ansible/modules/network/ios/ios_facts.py b/lib/ansible/modules/network/ios/ios_facts.py
--- a/lib/ansible/modules/network/ios/ios_facts.py
+++ b/lib/ansible/modules/network/ios/ios_facts.py
@@ -228,12 +228,15 @@ def populate(self):
 
         data = self.responses[1]
         if data:
-            processor_line = [l for l in data.splitlines()
-                              if 'Processor' in l].pop()
-            match = re.findall(r'\s(\d+)\s', processor_line)
-            if match:
-                self.facts['memtotal_mb'] = int(match[0]) / 1024
-                self.facts['memfree_mb'] = int(match[3]) / 1024
+            if 'Invalid input detected' in data:
+                warnings.append('Unable to gather memory statistics')
+            else:
+                processor_line = [l for l in data.splitlines()
+                                  if 'Processor' in l].pop()
+                match = re.findall(r'\s(\d+)\s', processor_line)
+                if match:
+                    self.facts['memtotal_mb'] = int(match[0]) / 1024
+                    self.facts['memfree_mb'] = int(match[3]) / 1024
 
     def parse_filesystems(self, data):
         return re.findall(r'^Directory of (\S+)/', data, re.M)
@@ -446,6 +449,9 @@ def parse_lldp_port(self, data):
 
 VALID_SUBSETS = frozenset(FACT_SUBSETS.keys())
 
+global warnings
+warnings = list()
+
 
 def main():
     """main entry point for module execution
@@ -508,7 +514,6 @@ def main():
         key = 'ansible_net_%s' % key
         ansible_facts[key] = value
 
-    warnings = list()
     check_args(module, warnings)
 
     module.exit_json(ansible_facts=ansible_facts, warnings=warnings)


Test patch: