Repository: ansible/ansible

Issue:
junos_config: override is not overriding the current configuration
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
When using `junos_config` with `update: override`, we are not seeing the module completely overwrite the running configuration in the device.  It _appears_ to be using `merge` functionality instead.

I'm not sure if this applicable, but we were using the netconf transport plugin, **not** the network_cli plugin.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
junos_config

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.5.5
  config file = /Users/fxfitz/dev/cyclops-ansible/ansible.cfg
  configured module search path = [u'/Users/fxfitz/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/fxfitz/.pyenv/versions/2.7.15/envs/cyclops-ansible/lib/python2.7/site-packages/ansible
  executable location = /Users/fxfitz/.pyenv/versions/cyclops-ansible/bin/ansible
  python version = 2.7.15 (default, May 29 2018, 20:16:38) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)]
```

##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
```
DEFAULT_CALLBACK_WHITELIST(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = ['slack_backported']
DEFAULT_FORCE_HANDLERS(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = True
DEFAULT_FORKS(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = 15
DEFAULT_ROLES_PATH(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = [u'/Users/fxfitz/dev/cyclops-ansible/roles']
HOST_KEY_CHECKING(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = False
PERSISTENT_COMMAND_TIMEOUT(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = 90
RETRY_FILES_ENABLED(/Users/fxfitz/dev/cyclops-ansible/ansible.cfg) = False
```

##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->
Step 1) ensure a Juniper device has a pre-existing security zone with an interface defined:
```
set security zones security-zone EXTERNAL-VPN host-inbound-traffic system-services all
set security zones security-zone EXTERNAL-VPN host-inbound-traffic protocols all
set security zones security-zone EXTERNAL-VPN interfaces ge-0/0/8.0
```

Step 2) Have a configuration that DOES NOT include `EXTERNAL-VPN` security zone, but have a new zone that has the same interface defined.
```
set security zones security-zone trust host-inbound-traffic system-services all
set security zones security-zone trust host-inbound-traffic protocols all
set security zones security-zone trust interfaces ge-0/0/8.0
```

Step 3) Run junos_config with `update: override`
```yaml
- name: Ensure the JunOS Configuration is up-to-date
  junos_config:
    src: your_configuration.txt
    update: override
```

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
I would expect the EXTERNAL-VPN security zone to be removed, and the `trust` security zone to be added/committed successfully.

##### ACTUAL RESULTS
The task did not complete successfully because EXTERNAL-VPN still existed and the same interface (ge-0/0/8.0) was already assigned to it, so the trust zone could not be added.

This is the detailed netconf error from the logs that we retrieved from the Juniper device:
<!--- Paste verbatim command output between quotes below -->
```
*** netconf ***
Jul 30 17:08:19 [11217] Outgoing: <rpc-error>
Jul 30 17:08:19 [11217] Outgoing: <error-type>protocol</error-type>
Jul 30 17:08:19 [11217] Outgoing: <error-tag>operation-failed</error-tag>
Jul 30 17:08:19 [11217] Outgoing: <error-severity>error</error-severity>
Jul 30 17:08:19 [11217] Outgoing: <source-daemon>
Jul 30 17:08:19 [11217] Outgoing: nsd
Jul 30 17:08:19 [11217] Outgoing: </source-daemon>
Jul 30 17:08:19 [11217] Outgoing: <error-path>
Jul 30 17:08:19 [11217] Outgoing: [edit security zones security-zone EXTERNAL-VPN]
Jul 30 17:08:19 [11217] Outgoing: </error-path>
Jul 30 17:08:19 [11217] Outgoing: <error-info>
Jul 30 17:08:19 [11217] Outgoing: <bad-element>
Jul 30 17:08:19 [11217] Outgoing: interfaces ge-0/0/8.0
Jul 30 17:08:19 [11217] Outgoing: </bad-element>
Jul 30 17:08:19 [11217] Outgoing: </error-info>
Jul 30 17:08:19 [11217] Outgoing: <error-message>
Jul 30 17:08:19 [11217] Outgoing: Interface ge-0/0/8.0 already assigned to another zone
Jul 30 17:08:19 [11217] Outgoing: </error-message>
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/network/junos/junos_config.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/junos/junos_config.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Qalthos @ganeshrn @privateip
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Hi @fxfitz,

Thank you for the issue, just so you are aware we have a dedicated Working Group for network.
You can find other people interested in this in #ansible-network on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: community_workgroups --->
@fxfitz The junos_config module relies on load-configuration netconf module that sends config in set/xml/json/text format to remote device. As [load-configuration](https://www.juniper.net/documentation/en_US/junos/topics/reference/tag-summary/junos-xml-protocol-load-configuration.html) rpc doesn't support `override` with `set` format `update:overide` is not working as expected (this needs to be documented in module note section)

To get around this issue 
1) Use xml or text format confgiuration
2) If option-1 is not preferred you can query the set config using `junos_command` module and if `set security zones security-zone` is present in config only then execute `delete security zones security-zone` and after that load the required source file.

Resolving patch:
diff --git a/lib/ansible/modules/network/junos/junos_config.py b/lib/ansible/modules/network/junos/junos_config.py
--- a/lib/ansible/modules/network/junos/junos_config.py
+++ b/lib/ansible/modules/network/junos/junos_config.py
@@ -131,6 +131,7 @@
     L(Network FAQ,../network/user_guide/faq.html#why-do-the-config-modules-always-return-changed-true-with-abbreviated-commands).
   - Loading JSON-formatted configuration I(json) is supported
     starting in Junos OS Release 16.1 onwards.
+  - Update C(override) not currently compatible with C(set) notation.
   - Tested against vSRX JUNOS version 15.1X49-D15.4, vqfx-10000 JUNOS Version 15.1X53-D60.4.
   - Recommended connection is C(netconf). See L(the Junos OS Platform Options,../network/user_guide/platform_junos.html).
   - This module also works with C(local) connections for legacy playbooks.


Test patch: