Repository: ktbyers/netmiko

Issue:
No EOF checks after calling `self.remote_conn.recv()`
Both `_read_channel()` and `_read_channel_expect()` do not check for EOF after calling `self.remote_conn.recv()`.  This causes a connection which has closed to raise

`NetMikoTimeoutException("Timed-out reading channel, pattern not found in output: {}".format(pattern))`

This isn't exactly desired behavior as it makes it hard to discern whether the command just took too long or the connection is dead.  Most of the time, a subsequent call to `self.send_command()` will cause paramiko to raise `socket.error("Socket is closed")`, but I have ran into a case where the socket only half closed, so the `socket.error` never got raised.

I can fix this, but I'd like guidance on the preferred way to signal this error to the user.  `EOFError`?  A netmiko exception?  Something else?


Discussion:
@kbirkeland `_read_channel()` checks if data is available (in the case of SSH) i.e. it uses `recv_ready()` call or else it breaks out of method (so above should be very unlikely in case of `read_channel()`.

Yes, differentiating between the SSH socket closing and not getting the expected data back is probably hard with `_read_channel_expect()`.


Can you just check this?

```
>>> net_connect.remote_conn.get_transport().is_alive()
True
# Terminated the SSH session on the router
>>> net_connect.remote_conn.get_transport().is_alive()
False
```


This is probably not something I will incorporate in via a PR (as I don't see enough importance in making a distinction here). In practice, it is usually the expected pattern didn't come back (and pretty rarely that the SSH connection closed mid-usage).

I am open to arguments that it is important, however.
While it may be unlikely, testing error cases with sockets is a good thing as unexpected things are bound to happen at some point.  

The assumption that connections don't usually close mid-usage is probably true for one-shot scripts, but when using a long running connection, such as napalm with salt, this happens quite often if an exec-timeout is set.  When the exec-timeout hits, the connection dies and the salt-proxy minion must be restarted.  I'll be sending a PR to napalm-ios to reconnect on `socket.error`, but I'd also like to know if the socket received EOF.
@kbirkeland Fair enough on Salt + NAPALM.

If you are going to submit it on napalm-ios, we should probably discuss the most logical place to include it (netmiko or napalm-ios). It goes to me either way.

Adding @mirceaulinic into this...as this mostly pertains to Salt.
I don't have preference here (mainly want to avoid adding much complexity)...so if you have proposals, let me know.

Resolving patch:
diff --git a/netmiko/base_connection.py b/netmiko/base_connection.py
--- a/netmiko/base_connection.py
+++ b/netmiko/base_connection.py
@@ -226,7 +226,10 @@ def _read_channel(self):
             output = ""
             while True:
                 if self.remote_conn.recv_ready():
-                    output += self.remote_conn.recv(MAX_BUFFER).decode('utf-8', 'ignore')
+                    outbuf = self.remote_conn.recv(MAX_BUFFER)
+                    if len(outbuf) == 0:
+                        raise EOFError
+                    output += outbuf.decode('utf-8', 'ignore')
                 else:
                     break
         elif self.protocol == 'telnet':
@@ -276,7 +279,10 @@ def _read_channel_expect(self, pattern='', re_flags=0, max_loops=None):
                 try:
                     # If no data available will wait timeout seconds trying to read
                     self._lock_netmiko_session()
-                    new_data = self.remote_conn.recv(MAX_BUFFER).decode('utf-8', 'ignore')
+                    new_data = self.remote_conn.recv(MAX_BUFFER)
+                    if len(new_data) == 0:
+                        raise EOFError
+                    new_data = new_data.decode('utf-8', 'ignore')
                     log.debug("_read_channel_expect read_data: {}".format(new_data))
                     output += new_data
                 except socket.timeout:


Test patch: