Repository: ansible/ansible

Issue:
junos_netconf should not configure a port unless explicitly defined
<!---
Verify first that your issue/request is not already reported on GitHub.
THIS FORM WILL BE READ BY A MACHINE, COMPLETE ALL SECTIONS AS DESCRIBED.
Also test if the latest release, and devel branch are affected too.
ALWAYS add information AFTER (OUTSIDE) these html comments.
Otherwise it may end up being automatically closed by our bot. -->

##### SUMMARY
junos_netconf is used to enable netconf on juniper devices. The `netconf_port:` argument allows the user to specify a specific port to enable netconf on. The problem is that ansible will explicitly configure netconf on port 830 even when no port is configured (as per ansible documentation).

Junos however does not require an explicit port to be configured to enable netconf. If no port is specified, netconf service will be started on both port 22 and 830.  

junos_netconf should not configure a port unless the tasks specifically defines one, and the documentation should be updated to reflect this as well.

When defined in a play, `netconf_port:` should continue to work as currently implemented.

##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Insert, BELOW THIS COMMENT, the name of the module, plugin, task or feature.
Do not include extra details here, e.g. "vyos_command" not "the network module vyos_command" or the full path-->
junos_netconf

##### ANSIBLE VERSION
<!--- Paste, BELOW THIS COMMENT, verbatim output from "ansible --version" between quotes below -->
```
ansible 2.6.1
  config file = /home/4032956/network-automation/ansible.cfg
  configured module search path = ['/home/4032956/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/4032956/venv/ansiblePSN/lib/python3.6/site-packages/ansible
  executable location = /home/4032956/venv/ansiblePSN/bin/ansible
  python version = 3.6.4 (default, Dec 19 2017, 14:48:27) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
```
##### CONFIGURATION
<!--- If using Ansible 2.4 or above, paste, BELOW THIS COMMENT, the results of "ansible-config dump --only-changed"
Otherwise, mention any settings you have changed/added/removed in ansible.cfg
(or using the ANSIBLE_* environment variables).-->
```
(ansiblePSN) ~/network-automation (master) $ ansible-config dump --only-changed
DEFAULT_GATHERING(/home/4032956/network-automation/ansible.cfg) = explicit
DEFAULT_HOST_LIST(/home/4032956/network-automation/ansible.cfg) = ['/home/4032956/network-automation/inventory']
DEFAULT_STDOUT_CALLBACK(/home/4032956/network-automation/ansible.cfg) = debug
HOST_KEY_CHECKING(/home/4032956/network-automation/ansible.cfg) = False
PERSISTENT_COMMAND_TIMEOUT(/home/4032956/network-automation/ansible.cfg) = 20
PERSISTENT_CONNECT_TIMEOUT(/home/4032956/network-automation/ansible.cfg) = 40
```
##### OS / ENVIRONMENT
<!--- Mention, BELOW THIS COMMENT, the OS you are running Ansible from, and the OS you are
managing, or say "N/A" for anything that is not platform-specific.
Also mention the specific version of what you are trying to control,
e.g. if this is a network bug the version of firmware on the network device.-->
From RHEL 7 to junos 15.1 and others.

##### STEPS TO REPRODUCE
<!--- For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used. -->

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- name: Enable NETCONF on junos devices
  junos_netconf:
  when: ansible_network_os == 'junos'
  vars:
    ansible_connection: network_cli
```
<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
```junos
> show configuration system services netconf | display set
set system services netconf ssh
```

<!--- What did you expect to happen when running the steps above? -->

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->
Result:
<!--- Paste verbatim command output between quotes below -->
```junos
> show configuration system services netconf | display set
set system services netconf ssh port 830
```



Discussion:
Files identified in the description:
* [lib/ansible/modules/network/junos/junos_netconf.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/junos/junos_netconf.py)

If these files are inaccurate, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
cc @Qalthos @ganeshrn @privateip
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->
Also worth noting that `junos_config` does not permit `ansible_connection: network_cli` to configure netconf more generally.
Finally, `ansible_connection: local` achieves the desired result:

```yaml

- name: Enable NETCONF on junos devices using junos_config
  junos_config:
    lines:
      - delete system services netconf ssh port 830
      - delete system services netconf ssh port 22
      - set system services netconf ssh
  when: ansible_network_os == 'junos'
  vars:
    ansible_connection: local
```

Result:
```junos
> show configuration system services netconf | display set
set system services netconf ssh
```

@LukeRussell junos_config works with `netconf` and `local` connection type only. Please refer junos platform guide for more details https://docs.ansible.com/ansible/latest/network/user_guide/platform_junos.html#connections-available

As per Netconf rfc the netconf port defaults to port 830 that's the reason for `junos_netconf` module to default port 830 even if not mentioned. 

 As per junos doc [here](https://www.juniper.net/documentation/en_US/junos/topics/topic-map/netconf-ssh-connection.html) `` `set system services netconf ssh`  enables netconf on port 830. 
To enable netconf on port 22 need to explicitly add
```
[edit system services]
user@host# set ssh
```

Also the doc recommends against enabling Netconf on default port used by other services.
```
Note: Although NETCONF-over-SSH can be configured on any port from 1 through 65535, you should avoid configuring access on a port that is normally assigned for another service. This practice avoids potential resource conflicts. 
```

Hence the current behaviour of `junos_netconf` is right imo.
Hi Ganesh.

No arguments that network_cli is not supported. It should be supported in my opinion, but that's a different issue.

To the netconf port issue however, I don't think I've done the best job of explaining this Issue.

Using your example, `set system services netconf ssh` will enable netconf on both 830 *and* 22. Regardless of RFC, this is standard Junos behaviour and it's actually desirable, as it sidesteps the issue of firewalls typically blocking 830.

The problem in this Issue is that Ansible doesn't send the command `set system services netconf ssh`, it actually sends the command `set system services netconf ssh port 830`. This command in Junos will actually _disable_ netconf on port 22. This mandates firewall rule changes.

We've also ran into a bug with Junos where if netconf is configured for port 22, when a router or switch reboots sometimes the netconf service starts before the interactive ssh service. Netconf then takes exclusive access of port 22, preventing standard CLI access.
(edit: seems you've mentioned that already)


In short, `junos_netconf` should only be sending `set system services netconf ssh` unless a playbook designer explicitly defines a port. In that case, the command should be `set system services netconf ssh port {{netconf_port}}`




Thank you sharing more information on this.

I still feel the current behaviour of junos_netconf module of enabling netconf on default port 830 **only** is correct even more so after reading the problem you described in your comment :-)

To get around the firewall issue you can explicitly set `netconf_port: 22`. By doing so the operator at least is aware of what is happening on the device.

I have not come across a use case that requires netconf to be enabled on both ports (22 and 830) simultaneously. If you have any valid use case please let me know.

If it helps we can work on improving the module documentation to describe the exact behaviour.

AKAIK this behaviour on junos of enabling netconf port simultaneously on port 22 and 830 is for legacy reasons. `junosscript` the predecessor of Netconf worked with default port 22
Regardless of the reason, I have half dozen senior network engineers (at JNCIE level) saying they want `set system services netconf ssh` and they specifically *do not* want `set system services netconf ssh port 830`. Now I can and have used junos_config to get the desired result. But that doesn't mean junos_netconf can't be improved.

The way I see it, Ansible should not be so opinionated about desired outcomes. It's not really Ansible's role to determine what is and what isn't a _legitimate_ use case for network devices. 

Further, junos_netconf loses absolutely no functionality by not sending port 830 by default. Plus, I don't even have a mechanism to turn that behaviour off. If junos_netconf doesn't send port 830 we still end up with a working netconf service. Those that want port 830 can still specify it. But by sending port 830 even when I don't ask for it, junos_netconf is no longer fit for purpose.


@lukerussell 

I think you make some valid points and it has sparked some internal discussion but: 

>The way I see it, Ansible should not be so opinionated about desired outcomes. It's not really Ansible's role to determine what is and what isn't a legitimate use case for network devices.

The developer here is trying to do the opposite. Ansible playbooks are supposed to be super human readable and not bogged into vendor implementations when possible. Having port 22 configured automatically (even though that’s what Junos docs (https://www.juniper.net/documentation/en_US/junos/topics/task/configuration/netconf-over-ssh-connections-port-configuring.html) seems like something the operator wouldn’t want.  From reading a playbook it would not be apparent that you enabled netconf on two ports.  That is something specific to junos that we probably need to consider.

Keep in mind that just because someone has a JNCIE doesn’t make them right every time. The ansible network team has multiple certifications from multiple vendors but we don’t use them to chest bump and force solutions we think are right.  A stronger argument would be around a workflow for current juniper operators that have expectations that don’t match the defaults that junos has set.
> From reading a playbook it would not be apparent that you enabled netconf on two ports. That is something specific to junos that we probably need to consider.

This is a pretty valid point - you'd have to be into the weeds with JUNOS to know it was on both ports.

Based on that, how about an additional argument? This would not change existing behaviour, and allow a playbook designer to enable netconf on both ports without having to use junos_config. I do admit though that someone not intimately familiar with junos might look at this argument and scratch their head trying to think of a use case.
```
set_netconf_port:
    description:
      - Specifies whether a port should be configured for the netconf service.
    required: false
    default: true
    version_added: "2.7?"
```

> Keep in mind that just because someone has a JNCIE doesn’t make them right every time. The ansible network team has multiple certifications from multiple vendors but we don’t use them to chest bump and force solutions we think are right. A stronger argument would be around a workflow for current juniper operators that have expectations that don’t match the defaults that junos has set.

Chest bumping was not my intent at all, but I can see from the 3 thumbs up in 9 hours on the weekend you and others have seen it that way. I do feel fairly sheepish right now, consider me appropriately humbled on a Sunday morning.

If I can try again to explain my comment, I was just trying to convey that this isn't an obscure corner case that I've come up with independently. We have a fairly large fleet of Juniper devices and a fairly large team of network engineers and they are in agreement. If we've had this use case, then I'm sure others will as well. I also admit to a general feeling that some of the newer role style modules and net_ modules may not have had a lot of production use and validation as yet, and I want to contribute our experiences to Ansible.


P.S. I have gone back and checked that juniper_netconf was added back in 2.1, so it's hardly a new module any more.
Adding a new option `set_netconf_port` might be confusing to novice junos users. 

How about improving the module doc string to make things much more clear for both experienced and novice junos operators?
```
options:
  netconf_port:
    description:
      - This argument specifies the port the netconf service should
        listen on for SSH connections.  The default port as defined
        in RFC 6242 is 830. **If the C(netconf_port) is not mentioned in the task by default
        netconf will be enabled on port 830 only.** 

notes:
  - Tested against vSRX JUNOS version 15.1X49-D15.4, vqfx-10000 JUNOS Version 15.1X53-D60.4.
  - Recommended connection is C(network_cli). See L(the Junos OS Platform Options,../network/user_guide/platform_junos.html).
  - This module also works with C(local) connections for legacy playbooks.
  **-  If C(netconf_port) value is not mentioned in task by default it will be enabled on port 830 only.
     Although C(netconf_port) value can be from 1 through 65535, avoid configuring access on a port 
     that is normally assigned for another service. This practice avoids potential resource conflicts.**
```


>  I also admit to a general feeling that some of the newer role style modules and net_ modules may not have had a lot of production use and validation as yet, 

Not quite clear when you say newer role style module. Are you referring to network role development [here](https://github.com/ansible-network). BTW we have stopped active development of new net_ modules.



> and I want to contribute our experiences to Ansible.


Happy to have help in form of PR's, feature request and Github issues.
FYI we have weekly irc meeting in `ansible-network` channel on freenode, more detailts [here](https://github.com/ansible/community/tree/master/group-network#meetings). Feel free to drop in and share your experience with Ansible
By "role-style" modules I was referring to things like `ios_vrf` or `ios_logging`. The functions that can be pieced together using the more generic `*_config` and `*_command` modules. My intent when creating this Issue was to get more power into `junos_netconf` to cover more use cases (including ours).

I'm getting the impression though that I'm not getting any other traction, so content to close with your proposed documentation update.

(I do semi-regularly get up at 4am for Ansible Networking webinars, but 2am AEST for the weekly meeting is a bit of drag for me.)
>  By "role-style" modules I was referring to things like ios_vrf or ios_logging. The functions that can be pieced together using the more generic *_config and *_command modules.

That's is one of the reason we are moving to roles which is more faster and scalable approach. I have shared the link to Github repo is above comment. The work is under active development.

 > My intent when creating this Issue was to get more power into junos_netconf to cover more use cases (including ours).

I completely understand the intent, but I don't get a compelling 'technical' reason to enable netconf on two ports simultaneously. As i understand the current behaviour of junos_netconf is not blocking any functional use case.

>  (I do semi-regularly get up at 4am for Ansible Networking webinars, but 2am AEST for the weekly meeting is a bit of drag for me.)

That's rough, other option is you can remain idle in `ansible-network` irc channel, someone who is working in your timezone can reply

Resolving patch:
diff --git a/lib/ansible/modules/network/junos/junos_netconf.py b/lib/ansible/modules/network/junos/junos_netconf.py
--- a/lib/ansible/modules/network/junos/junos_netconf.py
+++ b/lib/ansible/modules/network/junos/junos_netconf.py
@@ -24,7 +24,9 @@
     the netconf system service running on Junos devices.  This module
     can be used to easily enable the Netconf API. Netconf provides
     a programmatic interface for working with configuration and state
-    resources as defined in RFC 6242.
+    resources as defined in RFC 6242. If the C(netconf_port) is not
+    mentioned in the task by default netconf will be enabled on port 830
+    only.
 extends_documentation_fragment: junos
 options:
   netconf_port:
@@ -50,6 +52,9 @@
   - Tested against vSRX JUNOS version 15.1X49-D15.4, vqfx-10000 JUNOS Version 15.1X53-D60.4.
   - Recommended connection is C(network_cli). See L(the Junos OS Platform Options,../network/user_guide/platform_junos.html).
   - This module also works with C(local) connections for legacy playbooks.
+  - If C(netconf_port) value is not mentioned in task by default it will be enabled on port 830 only.
+    Although C(netconf_port) value can be from 1 through 65535, avoid configuring access on a port
+    that is normally assigned for another service. This practice avoids potential resource conflicts.
 """
 
 EXAMPLES = """


Test patch: