Repository: ansible/ansible

Issue:
ios_config fails when creating macro "timeout trying to send command:"
##### ISSUE TYPE
 - Bug Report


##### COMPONENT NAME
ios_config
##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Aug  4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]

```

##### CONFIGURATION
HOST_KEY_CHECKING(/etc/ansible/ansible.cfg) = False
PERSISTENT_CONNECT_TIMEOUT(/etc/ansible/ansible.cfg) = 30

##### OS / ENVIRONMENT
CentOS Linux release 7.4.1708 (Core)
Cisco IOS XE Software, Version 16.06.03

##### SUMMARY
ios_config timeout when sending "macro name MYMACRO" command

##### STEPS TO REPRODUCE
Rerun playbook

<!--- Paste example playbooks or commands between quotes below -->
```yaml
---

- hosts: access_switches

  tasks:

  - name: MACRO PHONE
    ios_config:
      lines:
      - 'switchport voice vlan 60'
      - 'switchport port-security maximum 2'
      - 'spanning-tree portfast'
      - 'no cdp enable'
      - 'no macro desc'
      - '#macro keywords $DESCRIPTION'
      - '@'
      parents: macro name PHONE
      before: no macro name PHONE
      after: exit
      match: line
      replace: block

```



##### EXPECTED RESULTS
Creates the macro.

##### ACTUAL RESULTS
```
ansible-playbook 2.4.2.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible-playbook
  python version = 2.7.5 (default, Aug  4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Using /etc/ansible/ansible.cfg as config file
setting up inventory plugins
Parsed /etc/ansible/production inventory source with ini plugin
Loading callback plugin default of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/__init__.pyc

PLAYBOOK: MACRO_PHONE.yml ********************************************************************************************************
1 plays in ./roles/network/switch/access/MACRO_PHONE.yml

PLAY [access_switches] ***********************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************
Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py
<10.X.X.X> ESTABLISH LOCAL CONNECTION FOR USER: root
<10.X.X.X> EXEC /bin/sh -c 'echo ~ && sleep 0'
<10.X.X.X> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695 `" && echo ansible-tmp-1523044289.76-194673473453695="` echo /root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695 `" ) && sleep 0'
<10.X.X.X> PUT /tmp/tmpZi78WP TO /root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695/setup.py
<10.X.X.X> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695/ /root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695/setup.py && sleep 0'
<10.X.X.X> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695/setup.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1523044289.76-194673473453695/" > /dev/null 2>&1 && sleep 0'
ok: [10.X.X.X]
META: ran handlers

TASK [MACRO PHONE] ***************************************************************************************************************
task path: /etc/ansible/roles/network/switch/access/MACRO_PHONE.yml:7
<10.X.X.X> using connection plugin network_cli
<10.X.X.X> socket_path: /root/.ansible/pc/70f3934ea5
Using module file /usr/lib/python2.7/site-packages/ansible/modules/network/ios/ios_config.py
<10.X.X.X> ESTABLISH LOCAL CONNECTION FOR USER: root
<10.X.X.X> EXEC /bin/sh -c 'echo ~ && sleep 0'
<10.X.X.X> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034 `" && echo ansible-tmp-1523044295.51-69820156660034="` echo /root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034 `" ) && sleep 0'
<10.X.X.X> PUT /tmp/tmpVVnmKX TO /root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034/ios_config.py
<10.X.X.X> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034/ /root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034/ios_config.py && sleep 0'
<10.X.X.X> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034/ios_config.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1523044295.51-69820156660034/" > /dev/null 2>&1 && sleep 0'
fatal: [10.X.X.X]: FAILED! => {
    "changed": false,
    "command": "macro name PHONE",
    "invocation": {
        "module_args": {
            "after": [
                "exit"
            ],
            "auth_pass": null,
            "authorize": null,
            "backup": false,
            "before": [
                "no macro name PHONE"
            ],
            "defaults": false,
            "diff_against": null,
            "diff_ignore_lines": null,
            "force": false,
            "host": null,
            "intended_config": null,
            "lines": [
                "switchport voice vlan 60",
                "switchport port-security maximum 5 ! was 2",
                "spanning-tree portfast",
                "no cdp enable",
                "no macro desc",
                "#macro keywords $DESCRIPTION",
                "@"
            ],
            "match": "line",
            "multiline_delimiter": "@",
            "parents": [
                "macro name PHONE"
            ],
            "password": null,
            "port": null,
            "provider": null,
            "replace": "block",
            "running_config": null,
            "save": false,
            "save_when": "never",
            "src": null,
            "ssh_keyfile": null,
            "timeout": null,
            "username": null
        }
    },
    "msg": "timeout trying to send command: macro name PHONE",
    "rc": 1
}
        to retry, use: --limit @/etc/ansible/roles/network/switch/access/MACRO_PHONE.retry

PLAY RECAP ***********************************************************************************************************************
10.X.X.X               : ok=1    changed=0    unreachable=0    failed=1
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/network/ios/ios_config.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/ios/ios_config.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @privateip @rcarrillocruz
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Just a guess but a macro is a multi-line command but does not allow you to choose the delimiter like "banner".  So perhaps ansible its assuming the command should return a prompt or something instead of treating it as a multi-line command?

The output of "macro name MYMACRO" is:
```
Enter macro commands one per line. End with the character '@'.
```
Config on switch is like:
```
macro name PHONE
  switchport voice vlan 60
  switchport port-security maximum 2
  spanning-tree portfast
  no cdp enable
  no macro desc
#macro keywords $DESCRIPTION
@
```
Adding this to our feature idea list. There is a workaround by using the ios_config module and enter the command as one long line.
I've raised https://github.com/ansible/ansible/issues/38591 to track giving an error message
@calfonso 

Perhaps I misunderstand is there an example of this somewhere?  I have tried some variations to enter the whole thing as a single command but nothing has worked so far.

```
---

- hosts: access_switches

  tasks:

  - name: MACRO PHONE
    ios_config:
      lines: |
       - 'macro name PHONE'
       - 'switchport voice vlan 60'
       - 'switchport port-security maximum 5 ! was 2'
       - 'spanning-tree portfast'
       - 'no cdp enable'
       - 'no macro desc'
       - '#macro keywords $DESCRIPTION'
       - '@'
      before: no macro name PHONE
      after: exit
      match: line
      replace: block
      timeout: 60
```

Had not known about this command, but this seems like another example of multi-line inputs such as with the banner commands.  Instead of having another specially focused feature or module on macros, I'd propose to just have a detection for `^Enter.*End with the character '\(.\)'` that puts the rest of the task into a  `sendonly` mode for the duration of the commands in that task?

I did a quick test of this and was able to get this play to work (after fixing the structure issues in above), 

Willing to create the PR if this sounds like an acceptable approach.

e.g.

```
  - name: MACRO PHONE
    ios_config:
      lines:
       - macro name PHONE
       - switchport voice vlan 60
       - switchport port-security maximum 5 ! was 2
       - spanning-tree portfast
       - no cdp enable
       - no macro desc
       - #macro keywords $DESCRIPTION
       - "@"
      before: no macro name PHONE
      match: line
      replace: block
      timeout: 60
```
@bdowling 

I think a general method to handle multi-line inputs instead of only handling the "banner" command would be great.

Does your solution work if your using "parents:macro name PHONE"?  If you end up making a PR I would certainly be willing to help test it.
@yaplej  fwiw, on my 9500 it doesn't appear that a macro is indented, so parents would not apply.  

Which, IMHO is really broken on Cisco's part not to indent that.  It doesn't for banner either.  But that's the way it is.
@bdowling @calfonso 
I never found a functional workaround for this.  Did you ever create a PR?

Thanks.
@bdowling

I tried the structural changes you mentioned but I was unable to get it to work.  Still getting "timeout trying to send command: macro name PHONE"

```
  - name: MACRO PHONE
    ios_config:
      lines:
      - macro name PHONE
      - switchport voice vlan 60
      - switchport port-security maximum 5
      - spanning-tree portfast
      - no cdp enable
      - no macro desc
      - #macro keywords $DESCRIPTION
      - "@"
      before: no macro name PHONE
      match: line
      replace: block
      timeout: 60

```

I also updated to ansible 2.6.1 but no luck.
```
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_hc4T4F/ansible_module_ios_config.py", line 561, in <module>
    main()
  File "/tmp/ansible_hc4T4F/ansible_module_ios_config.py", line 490, in main
    load_config(module, commands)
  File "/tmp/ansible_hc4T4F/ansible_modlib.zip/ansible/module_utils/network/ios/ios.py", line 168, in load_config
  File "/tmp/ansible_hc4T4F/ansible_modlib.zip/ansible/module_utils/connection.py", line 149, in __rpc__
ansible.module_utils.connection.ConnectionError: timeout trying to send command: macro name PHONE

fatal: [10.1.100.11]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_hc4T4F/ansible_module_ios_config.py\", line 561, in <module>\n    main()\n  File \"/tmp/ansible_hc4T4F/ansible_module_ios_config.py\", line 490, in main\n    load_config(module, commands)\n  File \"/tmp/ansible_hc4T4F/ansible_modlib.zip/ansible/module_utils/network/ios/ios.py\", line 168, in load_config\n  File \"/tmp/ansible_hc4T4F/ansible_modlib.zip/ansible/module_utils/connection.py\", line 149, in __rpc__\nansible.module_utils.connection.ConnectionError: timeout trying to send command: macro name PHONE\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
```
I'm still trying to get it to work in 2.6.2.  Its a total hack job but it seems like its close to running.  There is probably a much better solution like detecting the line "Enter macro commands one per line. End with the character '@'." and going into sendonly mode as previously mentioned but I was not sure where that could be done.

In /modules/network/ios/ios_config.py created a modified function based on load_banners().
```
def load_config2(module, commands):
    delimiter = module.params['multiline_delimiter']
    commands.insert(0,'config terminal')
    commands.append(delimiter)
    commands.append('end')
    thecommands = []
    for line in thecommands:
        thecommands.appent(line + '\n')
    for cmd in [thecommands]:
        obj = {'command': cmd, 'sendonly': True}
        run_commands(module, [cmd])
    time.sleep(0.1)
    run_commands(module, ['\n'])
```

Temporarily send all commands through load_config2() function.  Because I don't know what I am doing.
```
            if not module.check_mode:
                if commands:
                    load_config2(module, commands)
                if banners:
                    load_banners(module, banners)
```

Using a very simple playbook.  I had to take the '@' out of the play and add it as the delimiter in  load_config2() because it kept getting stripped out of the updates.
```
---

- hosts: access_switches

  tasks:

  - name: MACRO PHONE
    ios_config:
      provider:
        timeout: 60
      lines:
      - macro name PHONE
      - switchport voice vlan 60
      - switchport port-security maximum 5
      - spanning-tree portfast
      - no cdp enable
      - no macro desc
      - "#macro keywords $DESCRIPTION"
      before: no macro name PHONE
      match: line
      replace: block
```

It appears to run now but does not actually put the config on the device for some reason.
```
changed: [1.1.1.1] => {
    "banners": {},
    "changed": true,
    "commands": [
        "config terminal",
        "macro name PHONE",
        "switchport voice vlan 60",
        "switchport port-security maximum 5",
        "spanning-tree portfast",
        "no cdp enable",
        "no macro desc",
        "#macro keywords $DESCRIPTION",
        "@",
        "end"
    ],
    "invocation": {
        "module_args": {
            "after": null,
            "auth_pass": null,
            "authorize": null,
            "backup": false,
            "before": null,
            "defaults": false,
            "diff_against": null,
            "diff_ignore_lines": null,
            "force": false,
            "host": null,
            "intended_config": null,
            "lines": [
                "macro name PHONE",
                "switchport voice vlan 60",
                "switchport port-security maximum 5",
                "spanning-tree portfast",
                "no cdp enable",
                "no macro desc",
                "#macro keywords $DESCRIPTION"
            ],
            "match": "line",
            "multiline_delimiter": "@",
            "parents": null,
            "password": null,
            "port": null,
            "provider": {
                "auth_pass": null,
                "authorize": null,
                "host": null,
                "password": null,
                "port": null,
                "ssh_keyfile": null,
                "timeout": 60,
                "username": null
            },
            "replace": "block",
            "running_config": null,
            "save": false,
            "save_when": "never",
            "src": null,
            "ssh_keyfile": null,
            "timeout": null,
            "username": null
        }
    },
    "updates": [
        "config terminal",
        "macro name PHONE",
        "switchport voice vlan 60",
        "switchport port-security maximum 5",
        "spanning-tree portfast",
        "no cdp enable",
        "no macro desc",
        "#macro keywords $DESCRIPTION",
        "@",
        "end"
    ]
}
```

I am stumped why the updates are not actually getting executed on the device.  I enable command logging on the device and can see nothing is getting executed.

Another thing I noticed with macro's in general is that they cannot be modified without first removing them.  So the "before" statement of 'no macro name WORD' must be executed before the macro is re-created.  If the macro partially exists or is modified the "updates" section is missing the commands that were formerly on the device.
@bdowling Your correct that the IOS does not indent anything under the macro command itself but you can manually indent them by prefixing the lines with a space or two when you enter them.
```
macro name PHONE
 switchport voice vlan 60
 switchport port-security maximum 5
 spanning-tree portfast
 no cdp enable
 no macro desc
 #macro keywords $DESCRIPTION
@
```

If you do that your able to treat each macro as a section in the config.  With the exception of the terminating '@'.
```
show run | sec macro name PHONE
macro name PHONE
 switchport voice vlan 60
 switchport port-security maximum 5
 spanning-tree portfast
 no cdp enable
 no macro desc
 #macro keywords $DESCRIPTION
```
I actually got it to work.  No its not pretty.

Created a new function to separate normal commands from macros.
```
def load_config_or_macro(module, commands):
    if "macro name" in commands[0]:
        load_macro(module, commands)
    else:
        load_config(module, commands)
```
Edit: Should be looking for 'macro name' not just 'macro'.

Send all commands through load_config_or_macro() to filter out macros to load_macro().
```
            if not module.check_mode:
                if commands:
                    load_config_or_macro(module, commands)
                if banners:
                    load_banners(module, banners)
```

Took all the lines from the macro and converted it into one big blob of commands similar to how the banner command is handled.
```
def load_macro(module, commands):
    thecommands = ''
    thecommands += 'config terminal\n'
    if "#macro keywords" in commands[2]:
        return
    for line in commands:
        if line != 'None':
             thecommands += (' ' + line + '\n')
    thecommands += 'end\n'
    for cmd in [thecommands]:
        obj = {'command': cmd, 'sendonly': True}
        run_commands(module, [cmd])
    time.sleep(0.1)
    run_commands(module, ['\n'])
```
Edit: Adding the delimiter is not needed after fixing the replace block functionality by adding the 'parent:' to the play.

The macro play had to get two lines quoted or else they were handled as 'None' or blank lines.  
```
---

- hosts: access_switches

  tasks:

  - name: MACRO PHONE
    ios_config:
      provider:
        timeout: 60
      lines:
      - switchport voice vlan 60
      - switchport port-security maximum 5
      - spanning-tree portfast
      - no cdp enable
      - no macro desc
      - "#macro keywords $DESCRIPTION"
      - "@"
      before: 'no macro name PHONE'
      parents: macro name PHONE
      match: line
      replace: block
```
Edit: Replace block only works when used with 'parents:'. https://github.com/ansible/ansible/issues/29716

Edit: Tried using 'diff_ignore_lines:' for '@' and '#macro.*' does this work at all? https://github.com/ansible/ansible/issues/30562

Last thing is it needs improvement to be idempotent.  I really do not want it to be running every time the play is executed but right now it seems to detect that the lines '#macro keyword.*' and '@' are different before each run.  So I think the diff detection needs improved to handle macros better too.
I am not sure how to make it idempotent.  

The difference is generated by candidate.difference() from the NetworkConfig class.  In that it has DEFAULT_COMMENT_TOKENS and includes '#'.  That is why the line '#macro keywords $DESCRIPTION' line is always detected as a difference/change.

I was able to resolve the issue with '@' being detected as a change by moving that to 'after:' in the play.  Putting each line in single quotes keeps '#macro keywords' line from being treated as 'None' so I put them all in quotes for consistency.
````
---

- hosts: access_switches

  tasks:

  - name: MACRO PHONE
    ios_config:
      provider:
        timeout: 60
      lines:
      - 'switchport voice vlan 60'
      - 'switchport port-security maximum 5'
      - 'spanning-tree portfast'
      - 'no cdp enable'
      - 'no macro desc'
      - '#macro keywords $DESCRIPTION'
      before: no macro name PHONE
      after: '@'
      parents: macro name PHONE
      match: line
      replace: block
````

Edit: I confirmed by removing '#' from 'DEFAULT_COMMENT_TOKENS' in   '/module_utils/network/common/config.py' and running my playbook again.  This seems like a separate issue so I opened https://github.com/ansible/ansible/issues/43861.
```
ok: [1.1.1.1] => {
    "changed": false,
    "invocation": {
        "module_args": {
            "after": [
                "@"
            ],
            "auth_pass": null,
            "authorize": null,
            "backup": false,
            "before": [
                "no macro name PHONE"
            ],
            "defaults": false,
            "diff_against": null,
            "diff_ignore_lines": null,
            "force": false,
            "host": null,
            "intended_config": null,
            "lines": [
                "switchport voice vlan 60",
                "switchport port-security maximum 5",
                "spanning-tree portfast",
                "no cdp enable",
                "no macro desc",
                "#macro keywords $DESCRIPTION"
            ],
            "match": "line",
            "multiline_delimiter": "@",
            "parents": [
                "macro name PHONE"
            ],
            "password": null,
            "port": null,
            "provider": {
                "auth_pass": null,
                "authorize": null,
                "host": null,
                "password": null,
                "port": null,
                "ssh_keyfile": null,
                "timeout": 60,
                "username": null
            },
            "replace": "block",
            "running_config": null,
            "save": false,
            "save_when": "never",
            "src": null,
            "ssh_keyfile": null,
            "timeout": null,
            "username": null
        }
    }
}
```

Resolving patch:
diff --git a/lib/ansible/modules/network/ios/ios_config.py b/lib/ansible/modules/network/ios/ios_config.py
--- a/lib/ansible/modules/network/ios/ios_config.py
+++ b/lib/ansible/modules/network/ios/ios_config.py
@@ -311,6 +311,13 @@ def check_args(module, warnings):
                                  'single character')
 
 
+def edit_config_or_macro(connection, commands):
+    if "macro name" in commands[0]:
+        connection.edit_macro(candidate=commands)
+    else:
+        connection.edit_config(candidate=commands)
+
+
 def get_candidate_config(module):
     candidate = ''
     if module.params['src']:
@@ -447,7 +454,7 @@ def main():
             # them with the current running config
             if not module.check_mode:
                 if commands:
-                    connection.edit_config(candidate=commands)
+                    edit_config_or_macro(connection, commands)
                 if banner_diff:
                     connection.edit_banner(candidate=json.dumps(banner_diff), multiline_delimiter=module.params['multiline_delimiter'])
 
diff --git a/lib/ansible/plugins/cliconf/ios.py b/lib/ansible/plugins/cliconf/ios.py
--- a/lib/ansible/plugins/cliconf/ios.py
+++ b/lib/ansible/plugins/cliconf/ios.py
@@ -151,6 +151,32 @@ def edit_config(self, candidate=None, commit=True, replace=None, comment=None):
         resp['response'] = results
         return resp
 
+    def edit_macro(self, candidate=None, commit=True, replace=None, comment=None):
+        resp = {}
+        operations = self.get_device_operations()
+        self.check_edit_config_capabiltiy(operations, candidate, commit, replace, comment)
+
+        results = []
+        requests = []
+        if commit:
+            commands = ''
+            for line in candidate:
+                if line != 'None':
+                    commands += (' ' + line + '\n')
+                self.send_command('config terminal', sendonly=True)
+                obj = {'command': commands, 'sendonly': True}
+                results.append(self.send_command(**obj))
+                requests.append(commands)
+
+            self.send_command('end', sendonly=True)
+            time.sleep(0.1)
+            results.append(self.send_command('\n'))
+            requests.append('\n')
+
+        resp['request'] = requests
+        resp['response'] = results
+        return resp
+
     def get(self, command=None, prompt=None, answer=None, sendonly=False, output=None):
         if not command:
             raise ValueError('must provide value of command to execute')


Test patch: