Repository: dask/distributed

Issue:
Using progress leads to broken pipe error in distributed.core
Running this:

~~~python
from dask import delayed
from dask.distributed import Client, progress


def run():
    client = Client('127.0.0.1:8786')
    result = delayed(lambda x: x+2)(3)
    progress(result)


if __name__ == '__main__':
    run()
~~~

I see the following warning in the scheduler's log:

~~~
2017-08-03 08:12:37,325 INFO     Receive client connection: Client-82e3804f-7823-11e7-9b21-0800278e0874 (distributed.scheduler)
2017-08-03 08:12:37,330 INFO     Remove client Client-82e3804f-7823-11e7-9b21-0800278e0874 (distributed.scheduler)
2017-08-03 08:12:37,331 INFO     Close client connection: Client-82e3804f-7823-11e7-9b21-0800278e0874 (distributed.scheduler)
2017-08-03 08:12:37,533 WARNING  Lost connection to 'tcp://127.0.0.1:58912' while sending result for op 'feed': in <closed TCP>: error: [Errno 32] Broken pipe (distributed.core)


Discussion:
`progress` works on futures, but you have provided a `delayed` object. You probably wanted a `future = client.compute(result)` first and then pass the future, or to use `client.submit` instead of delayed.

The message you see is saying that the client finished, as the function `run` had finished, and there was nothing for the client to do. I assume you mean to actually do something with the result; but note with in this very simple function, the future may well be done before the first time that progress checks on its status.
OK, so after correcting the mistake, I have the following code:

~~~python
from dask import delayed
from dask.distributed import Client, progress


def run():
    client = Client('127.0.0.1:8786')
    result = delayed(lambda x: x+2)(3)
    future = client.compute(result)
    progress(future)
    client.close()


if __name__ == '__main__':
    run()
~~~

This will still result in a warning showing up on the schedulers log:

~~~
2017-08-10 10:56:48,718 INFO     Receive client connection: Client-9baad202-7dba-11e7-a28a-0800278e0874 (distributed.scheduler)
2017-08-10 10:56:48,841 INFO     Remove client Client-9baad202-7dba-11e7-a28a-0800278e0874 (distributed.scheduler)
2017-08-10 10:56:48,843 INFO     Close client connection: Client-9baad202-7dba-11e7-a28a-0800278e0874 (distributed.scheduler)
2017-08-10 10:56:49,186 WARNING  Lost connection to 'tcp://127.0.0.1:35565' while sending result for op 'feed': in <closed TCP>: error: [Errno 32] Broken pipe (distributed.core)
~~~

Do I understand you correctly, that this is expected, since the future is done too quickly?

I created this issue because I get quite a lot of these warnings on my schedulers log for a bigger application. 

Resolving patch:
diff --git a/distributed/core.py b/distributed/core.py
--- a/distributed/core.py
+++ b/distributed/core.py
@@ -285,8 +285,8 @@ def handle_comm(self, comm, shutting_down=shutting_down):
                     try:
                         yield comm.write(result)
                     except EnvironmentError as e:
-                        logger.warning("Lost connection to %r while sending result for op %r: %s",
-                                       address, op, e)
+                        logger.debug("Lost connection to %r while sending result for op %r: %s",
+                                     address, op, e)
                         break
                 msg = result = None
                 if close_desired:


Test patch: