Repository: dask/distributed

Issue:
Occasional "KeyError: 'cpu'" error message in dscheduler logs
I am not sure how to reproduce this error but it seems that when I start a new cluster I sometimes get the following error in the logs of the scheduler:

```
[...]
distributed.scheduler - INFO - Register 10.112.2.4:53995
distributed.utils - ERROR - 'cpu'
Traceback (most recent call last):
  File "/work/miniconda/lib/python3.5/site-packages/distributed/utils.py", line 205, in log_errors
    yield
  File "/work/miniconda/lib/python3.5/site-packages/distributed/bokeh/status/server_lifecycle.py", line 59, in workers
    resource_append(messages['workers']['plot-data'], msg)
  File "/work/miniconda/lib/python3.5/site-packages/distributed/bokeh/worker_monitor.py", line 72, in resource_append
    lists[k].append(mean(pluck(k, L)) / 100)
  File "/work/miniconda/lib/python3.5/site-packages/distributed/bokeh/worker_monitor.py", line 87, in mean
    seq = list(seq)
KeyError: 'cpu'
distributed.scheduler - INFO - Starting worker compute stream, 10.112.18.3:58457
distributed.scheduler - INFO - Starting worker compute stream, 10.112.19.3:52639
distributed.scheduler - INFO - Starting worker compute stream, 10.112.5.4:35957
distributed.scheduler - INFO - Starting worker compute stream, 10.112.1.3:47780
distributed.scheduler - INFO - Starting worker compute stream, 10.112.8.4:36129
distributed.scheduler - INFO - Starting worker compute stream, 10.112.0.3:37636
distributed.scheduler - INFO - Starting worker compute stream, 10.112.16.4:48440
distributed.scheduler - INFO - Starting worker compute stream, 10.112.14.4:48649
distributed.scheduler - INFO - Starting worker compute stream, 10.112.4.3:43642
distributed.scheduler - INFO - Starting worker compute stream, 10.112.7.4:44042
distributed.scheduler - INFO - Starting worker compute stream, 10.112.3.3:49770
distributed.scheduler - INFO - Starting worker compute stream, 10.112.10.4:47595
distributed.scheduler - INFO - Starting worker compute stream, 10.112.2.3:37964
distributed.scheduler - INFO - Starting worker compute stream, 10.112.11.3:41369
distributed.scheduler - INFO - Starting worker compute stream, 10.112.9.3:54121
distributed.scheduler - INFO - Starting worker compute stream, 10.112.19.4:45283
distributed.scheduler - INFO - Starting worker compute stream, 10.112.14.3:41628
distributed.scheduler - INFO - Starting worker compute stream, 10.112.17.3:53510
distributed.scheduler - INFO - Starting worker compute stream, 10.112.2.4:55194
distributed.scheduler - INFO - Starting worker compute stream, 10.112.6.3:50701
bokeh.util.tornado - ERROR - Error thrown from periodic callback: KeyError('cpu',)
distributed.core - INFO - Connection from 10.112.16.3:39481 to Scheduler
```

This does seem to cause any problem using the cluster though.



Discussion:

Resolving patch:
diff --git a/distributed/bokeh/worker_monitor.py b/distributed/bokeh/worker_monitor.py
--- a/distributed/bokeh/worker_monitor.py
+++ b/distributed/bokeh/worker_monitor.py
@@ -68,8 +68,11 @@ def resource_append(lists, msg):
     L = list(msg.values())
     if not L:
         return
-    for k in ['cpu', 'memory-percent']:
-        lists[k].append(mean(pluck(k, L)) / 100)
+    try:
+        for k in ['cpu', 'memory-percent']:
+            lists[k].append(mean(pluck(k, L)) / 100)
+    except KeyError:  # initial messages sometimes lack resource data
+        return        # this is safe to skip
 
     lists['time'].append(mean(pluck('time', L)) * 1000)
     if len(lists['time']) >= 2:


Test patch: