Repository: Flexget/Flexget

Issue:
Client ended connection while still streaming output
### Expected behaviour:
Execution goes from start to end

### Actual behaviour:
Execution stops throwing an error.

### Steps to reproduce:
Run command:
`flexget execute`

#### Config:
[https://github.com/effemmeffe/flexget/blob/master/config.yml](https://github.com/effemmeffe/flexget/blob/master/config.yml)
  
#### Log:
<details>
 <summary>(click to expand)</summary>
flexget.log:

```
2019-02-01 15:28 INFO     scheduler                     Starting scheduler
2019-02-01 15:28 INFO     web_server_daemon                 Running web server at IP 0.0.0.0:5050
2019-02-01 15:28 INFO     web_server_daemon                 Initiating API
2019-02-01 15:28 INFO     web_server_daemon                 Registering WebUI v1
2019-02-01 15:28 INFO     web_server                    Web interface available at http://127.0.1.1:5050
2019-02-01 15:28 DEBUG    manager                       Figuring out config load paths
2019-02-01 15:28 DEBUG    manager                       Found config: /media/video-nas/flexget/config.yml
2019-02-01 15:28 DEBUG    manager                       Config file /media/video-nas/flexget/config.yml selected
2019-02-01 15:28 DEBUG    manager                       sys.defaultencoding: ascii
2019-02-01 15:28 DEBUG    manager                       sys.getfilesystemencoding: UTF-8
2019-02-01 15:28 DEBUG    manager                       flexget detected io encoding: utf-8
2019-02-01 15:28 DEBUG    manager                       os.path.supports_unicode_filenames: False
2019-02-01 15:28 DEBUG    manager                       Sending command to running FlexGet process: [u'-L', u'debug', u'execute', u'--task', u'download-series-discover']
2019-02-01 15:33 ERROR    ipc           download-series-discover Client ended connection while still streaming output.

```

Output on screen:

```
Traceback (most recent call last):
  File "/usr/local/bin/flexget", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/flexget/__init__.py", line 42, in main
    manager.start()
  File "/usr/local/lib/python2.7/dist-packages/flexget/manager.py", line 344, in start
    client.handle_cli(self.args)
  File "/usr/local/lib/python2.7/dist-packages/rpyc/core/netref.py", line 199, in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/rpyc/core/netref.py", line 75, in syncreq
    return conn.sync_request(handler, proxy, *args)
  File "/usr/local/lib/python2.7/dist-packages/rpyc/core/protocol.py", line 471, in sync_request
    return self.async_request(handler, *args, timeout=timeout).value
  File "/usr/local/lib/python2.7/dist-packages/rpyc/core/async_.py", line 95, in value
    self.wait()
  File "/usr/local/lib/python2.7/dist-packages/rpyc/core/async_.py", line 47, in wait
    raise AsyncResultTimeout("result expired")
rpyc.lib.compat.TimeoutError: result expired
```
</details>

### Additional information:

- FlexGet version: 2.19.0
- Python version: Python 2.7.15rc1
- Installation method: pip
- Using daemon (yes/no): yes
- OS and version: ubuntu 18.04
- Link to crash log: no crash log present

The log is taken with the download-series-discover task, but it happens randomly with every task in my config whan I run flexget execute.

The error happens only when the daemon is running and I run the command flexget execute.
If I stop the daemon and run flexget execute I don't see any error.
The daemon is started via systemctl, the service is this:

```
fmf@kodi:~/.flexget$ cat /etc/systemd/system/flexget.service
[Unit]
Description=Flexget Daemon
After=network.target

[Service]
Type=simple
User=fmf
Group=fmf
UMask=000
WorkingDirectory=/home/fmf/.flexget
ExecStart=/usr/local/bin/flexget daemon start
ExecStop=/usr/local/bin/flexget daemon stop
ExecReload=/usr/local/bin/flexget daemon reload

[Install]
WantedBy=multi-user.target
```



Discussion:
Please see https://github.com/Flexget/Flexget/pull/2268
Thanks.
Seen that, but the question there is same as mine: what about the daemon configuration? Should I add the flag in my service configuration?
Hmm. Do we need to be sending keepalives or something now to keep the connection open?
Looks like there are some reports on the forum of issues as well. https://discuss.flexget.com/t/error-message-raise-asyncresulttimeout-result-expired-rpyc-lib-compat-timeouterror-result-expired/4430

I don't think the cli flag was a very good solution to this problem, just deciding 5 minutes as the longest we allow a client connection to be alive isn't good, even if the user can change it with that flag. We should figure out a better way for it to only time out if the connection actually dies.
I did some experiments, keepalive doesn't help. The timeout is how long it will wait for a remote function to finish. In our case, that remote function may be executing many tasks from an users config, which can take quite a long time. I think we should either set the timeout very high, or disable it altogether.

Resolving patch:
diff --git a/flexget/ipc.py b/flexget/ipc.py
--- a/flexget/ipc.py
+++ b/flexget/ipc.py
@@ -160,7 +160,7 @@ def shutdown(self):
 
 
 class IPCClient(object):
-    def __init__(self, port, password, sync_request_timeout=300):
+    def __init__(self, port, password):
         channel = rpyc.Channel(rpyc.SocketStream.connect('127.0.0.1', port))
         channel.send(password.encode('utf-8'))
         response = channel.recv()
@@ -168,7 +168,7 @@ def __init__(self, port, password, sync_request_timeout=300):
             # TODO: What to raise here. I guess we create a custom error
             raise ValueError('Invalid password for daemon')
         self.conn = rpyc.utils.factory.connect_channel(
-            channel, service=ClientService, config={'sync_request_timeout': sync_request_timeout})
+            channel, service=ClientService, config={'sync_request_timeout': None})
 
     def close(self):
         self.conn.close()
diff --git a/flexget/manager.py b/flexget/manager.py
--- a/flexget/manager.py
+++ b/flexget/manager.py
@@ -339,7 +339,7 @@ def start(self):
             console('There is a FlexGet process already running for this config, sending execution there.')
             log.debug('Sending command to running FlexGet process: %s' % self.args)
             try:
-                client = IPCClient(ipc_info['port'], ipc_info['password'], sync_request_timeout=self.options.timeout)
+                client = IPCClient(ipc_info['port'], ipc_info['password'])
             except ValueError as e:
                 log.error(e)
             else:
diff --git a/flexget/options.py b/flexget/options.py
--- a/flexget/options.py
+++ b/flexget/options.py
@@ -406,7 +406,6 @@ def _debug_tb_callback(self, *dummy):
 manager_parser.add_argument('--cron', action=CronAction, default=False, nargs=0,
                             help='use when executing FlexGet non-interactively: allows background '
                                  'maintenance to run, disables stdout and stderr output, reduces logging level')
-manager_parser.add_argument('--timeout', default=300, help='number of seconds to run before timing out', type=int)
 
 
 class CoreArgumentParser(ArgumentParser):


Test patch: