Repository: ansible/ansible

Issue:
ios_vrf slow with big config
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
<!--- Explain the problem briefly -->
When using ios_vrf, the function parse_interfaces increases the time for the task from ~20 seconds to ~120 seconds which causes connection timeouts with default values and a config of ~12k lines, 200 VRFs and 500 interfaces
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
ios_vrf
##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.6.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/e600790/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.13 (default, Nov 24 2017, 17:33:09) [GCC 6.3.0 20170516]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False
##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
Debian 9.5 managing Cisco 15.5(3)S6b/03.16.06b.S
##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
ios_vrf:
  name: "{{ vrf_name }}"
  rd: "{{ vrf_route_target }}"
  description: "{{ vrf_description }}"
  route_export: '{{ vrf_route_target }}'
  route_import: '{{ vrf_route_target }}'
  state: present
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->
real    0m16,741s
user    0m8,664s
sys     0m0,856s
##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
real    1m52,279s
user    1m41,468s
sys     0m1,788s
<!--- Paste verbatim command output between quotes below -->



Discussion:
Files identified in the description:
* [lib/ansible/modules/network/ios/ios_vrf.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/ios/ios_vrf.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @privateip @rcarrillocruz
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Hi @denngie,

Thank you for the issue, just so you are aware we have a dedicated Working Group for network.
You can find other people interested in this in #ansible-network on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: community_workgroups --->
I know this is related to the ansible host's performance, my machine at the moment only has 1 vCPU (Virtualbox) and 1GB RAM, but I still think the easiest solution would be only parsing the interface configuration if needed i.e. when either interfaces or associated_interfaces is entered as module parameter input? I can't see the benefit of getting and working with information that is not needed
I tried modifying the get_config to only get the relevant config with flags = ['| section ^vrf definition |^interface .+'] added, shaved off roughly 20%. Just wanted to see the difference when config lines was reducted to ~800 on my device.

Resolving patch:
diff --git a/lib/ansible/modules/network/ios/ios_vrf.py b/lib/ansible/modules/network/ios/ios_vrf.py
--- a/lib/ansible/modules/network/ios/ios_vrf.py
+++ b/lib/ansible/modules/network/ios/ios_vrf.py
@@ -405,12 +405,16 @@ def parse_rd(configobj, name):
         return match.group(1)
 
 
-def parse_interfaces(configobj, name):
-    vrf_cfg = 'vrf forwarding %s' % name
-    interfaces = list()
-    for intf in re.findall('^interface .+', str(configobj), re.M):
-        if vrf_cfg in '\n'.join(configobj[intf].children):
-            interfaces.append(intf.split(' ')[1])
+def parse_interfaces(configobj):
+    vrf_cfg = 'vrf forwarding'
+    interfaces = dict()
+    for intf in set(re.findall('^interface .+', str(configobj), re.M)):
+        for line in configobj[intf].children:
+            if vrf_cfg in line:
+                try:
+                    interfaces[line.split()[-1]].append(intf.split(' ')[1])
+                except KeyError:
+                    interfaces[line.split()[-1]] = [intf.split(' ')[1]]
     return interfaces
 
 
@@ -507,13 +511,16 @@ def map_config_to_obj(module):
         return list()
 
     instances = list()
+
+    interfaces = parse_interfaces(configobj)
+
     for item in set(match):
         obj = {
             'name': item,
             'state': 'present',
             'description': parse_description(configobj, item),
             'rd': parse_rd(configobj, item),
-            'interfaces': parse_interfaces(configobj, item),
+            'interfaces': interfaces.get(item),
             'route_import': parse_import(configobj, item),
             'route_export': parse_export(configobj, item),
             'route_both': parse_both(configobj, item),


Test patch: