Repository: dask/distributed

Issue:
Web UI missing components?
When trying out the UI for the first time today I get a figure without two of the components - for example:

![image](https://cloud.githubusercontent.com/assets/377869/14552239/a093457c-031f-11e6-9583-2fe0450f0c18.png)

Using chrome version 49.0.2623.87 (64-bit)



Discussion:
Are there any errors coming from the scheduler logs?

Or errors coming from the browser I suppose.  In chrome the development console is available by hitting F12

No errors in logs. I am using the imperative interface to form a graph if that helps..

The scheduler doesn't know/care how the graph was formed (it's all the same soup at that stage).  

One similarity between the missing graphs is that both connect to the scheduler over TCP at the address `localhost:8786`.  This is incorrect if the scheduler runs on a different port.  We should make these programmable.  Is this possibly an issue for you?

I will check it out. I am using docker to setup a scheduler and pin down ports. It's possible I am goofing. Generally I think it's good to be able to pin all ports. One thing I ran into was a white list problem for bokeh... it made my docker-compose script a bit of a drag when my scheduler was on a remote host I had to explicitly list that hostname in the scheduler command.

Do you think it might be a good idea to have a configuration file for the scheduler?

I agree about the bokeh whitelist.  This will be fixed in the next release of Bokeh. See https://github.com/bokeh/bokeh/pull/4052

I think that a configuration file for the scheduler sounds like a very fine idea.  It would be good to get your thoughts on what you would find valuable.

I think it might be good to initially pull all the command line args into config.

OK - I think the use of localhost might be an issue for schedulers running in containers. Seems to be a world of pain associated with mapping things appropriately. 

If you specify a "host" argument then perhaps you should use that instead of localhost?

Yes, that seems quite sensible.  So does making it accessible from the command line.  It might be a bit tricky to pass things down to the bokeh server.  I'll have time to look into things more deeply tomorrow.

Thanks Matthew -- cannot emphasize enough how useful this work is. Please keep the momentum up!

OK, so it looks like Bokeh does allow you to pass keyword arguments down to Bokeh server applications (see https://github.com/bokeh/bokeh/pull/4010/files#diff-2295c07d712da9ccef87a86e3e955672R53) but this isn't yet in a release.  Looking around for alternatives.

@nfaggian are you able to test a couple things on your setup?  I'm curious if using `127.0.0.1` in place of `localhost` is more robust for docker setups.

Actually, I'm reproducing this locally.  Bisecting down to commit that causes the failure doesn't clarify why it's happening.  However @nfaggian you might want to try installing from `487ee78cabad4cd4eede4174ffc3214a64e01b5f` which seems to run well, at least on my machine.

OK, this appears to be a spawn vs fork issue.  I can resolve it easily in Python 3, hunting around for a Python 2 solution.


Resolving patch:
diff --git a/distributed/bokeh/status/server_lifecycle.py b/distributed/bokeh/status/server_lifecycle.py
--- a/distributed/bokeh/status/server_lifecycle.py
+++ b/distributed/bokeh/status/server_lifecycle.py
@@ -25,7 +25,10 @@
 def http_get(route):
     """ Get data from JSON route, store in messages deques """
     with log_errors():
-        response = yield client.fetch('http://localhost:9786/%s.json' % route)
+        try:
+            response = yield client.fetch('http://localhost:9786/%s.json' % route)
+        except ConnectionRefusedError:
+            import sys; sys.exit(0)
         msg = json.loads(response.body.decode())
         messages[route]['deque'].append(msg)
         messages[route]['times'].append(time())
diff --git a/distributed/cli/dscheduler.py b/distributed/cli/dscheduler.py
--- a/distributed/cli/dscheduler.py
+++ b/distributed/cli/dscheduler.py
@@ -4,7 +4,8 @@
 import multiprocessing
 import os
 import socket
-from sys import argv, exit
+import subprocess
+import sys
 from time import sleep
 
 import click
@@ -22,7 +23,6 @@
 
 import signal
 
-
 def handle_signal(sig, frame):
     IOLoop.instance().add_callback(IOLoop.instance().stop)
 
@@ -58,7 +58,8 @@ def main(center, host, port, http_port, bokeh_port, show, _bokeh):
             dirname = os.path.dirname(distributed.__file__)
             paths = [os.path.join(dirname, 'bokeh', name)
                      for name in ['status', 'tasks']]
-            args = (['bokeh', 'serve'] + paths +
+            binname = sys.argv[0][:-len('dscheduler')] + 'bokeh'
+            args = ([binname, 'serve'] + paths +
                     ['--log-level', 'warning',
                      '--check-unused-sessions=50',
                      '--unused-session-lifetime=1',
@@ -66,10 +67,15 @@ def main(center, host, port, http_port, bokeh_port, show, _bokeh):
                      sum([['--host', host] for host in hosts], []))
             if show:
                 args.append('--show')
-            from bokeh.command.bootstrap import main
-            proc = multiprocessing.Process(target=main, args=(args,))
-            proc.daemon = True
-            proc.start()
+
+            if sys.version_info[0] >= 3:
+                from bokeh.command.bootstrap import main
+                ctx = multiprocessing.get_context('spawn')
+                bokeh_proc = ctx.Process(target=main, args=(args,))
+                bokeh_proc.daemon = True
+                bokeh_proc.start()
+            else:
+                bokeh_proc = subprocess.Popen(args)
 
             logger.info(" Start Bokeh UI at:        http://%s:%d/status/"
                         % (ip, bokeh_port))
@@ -81,7 +87,7 @@ def main(center, host, port, http_port, bokeh_port, show, _bokeh):
     loop.start()
     loop.close()
     scheduler.stop()
-    proc.terminate()
+    bokeh_proc.terminate()
 
     logger.info("End scheduler at %s:%d", ip, port)
 


Test patch: