Repository: dask/distributed

Issue:
Worker memory chart is very jittery
Currently the worker memory chart moves around a lot as data is allocated and deallocated rapidly.  This makes it distracting and hard to read.  I suggest that we consider slowing down the rate at which we scale down the x-axis after deallocation.  Maybe at every cycle we only allow it to decrease by 5% or something.  Inside the code for updating this plot we would look at the maximum of the x-range before updating things, then we would update, and then we would increase the maximum of the x-range if it was below 95% of the previous value.

This is just one suggested approach.  There may be others.


Discussion:
Actually, looking at it now while it's still at rest it appears to be jumping around on its own faster than the cycle time.  I suspect that there is some fanciness going on (maybe even fanciness like what I propose above)
xref-ing https://github.com/dask/distributed/issues/4675 as these issues might be related
cc also @crusaderky 
> I suggest that we consider slowing down the rate at which we scale down the x-axis after deallocation.

If you're talking about x_range.end, it only changes when 
1. you add or remove a worker
2. the amount of spilled memory causes process + spilled memory for a worker to go beyond its memory limit

In both cases rate limiting won't solve the problem. On chrome the graph keeps bouncing around well after x_range.end has stopped moving, and only stabilizes after a browser page refresh.

Resolving patch:
diff --git a/distributed/dashboard/components/scheduler.py b/distributed/dashboard/components/scheduler.py
--- a/distributed/dashboard/components/scheduler.py
+++ b/distributed/dashboard/components/scheduler.py
@@ -281,7 +281,6 @@ def __init__(self, scheduler, width=600, **kwargs):
             self.root.axis[0].ticker = BasicTicker(**TICKS_1024)
             self.root.xaxis[0].formatter = NumeralTickFormatter(format="0.0 b")
             self.root.xaxis.major_label_orientation = XLABEL_ORIENTATION
-            self.root.x_range.start = 0
             self.root.xaxis.minor_tick_line_alpha = 0
             self.root.yaxis.visible = False
             self.root.ygrid.visible = False
@@ -341,12 +340,9 @@ def update(self):
                 "unmanaged_recent": [meminfo.unmanaged_recent] * 4,
                 "spilled": [meminfo.managed_spilled] * 4,
             }
-            # FIXME https://github.com/dask/distributed/issues/4675
-            #       This causes flickering after adding workers and when enough memory
-            #       is spilled out
-            self.root.x_range.end = max(
-                limit, meminfo.process + meminfo.managed_spilled
-            )
+
+            x_end = max(limit, meminfo.process + meminfo.managed_spilled)
+            self.root.x_range = DataRange1d(start=0, end=x_end, range_padding=0)
 
             title = f"Bytes stored: {format_bytes(meminfo.process)}"
             if meminfo.managed_spilled:
@@ -402,7 +398,6 @@ def __init__(self, scheduler, width=600, **kwargs):
             self.root.axis[0].ticker = BasicTicker(**TICKS_1024)
             self.root.xaxis[0].formatter = NumeralTickFormatter(format="0.0 b")
             self.root.xaxis.major_label_orientation = XLABEL_ORIENTATION
-            self.root.x_range.start = 0
             self.root.xaxis.minor_tick_line_alpha = 0
             self.root.yaxis.visible = False
             self.root.ygrid.visible = False
@@ -510,10 +505,8 @@ def quadlist(i) -> list:
             result = {
                 k: [vi for vi, w in zip(v, width) if w] for k, v in result.items()
             }
-            # FIXME https://github.com/dask/distributed/issues/4675
-            #       This causes flickering after adding workers and when enough memory
-            #       is spilled to disk
-            self.root.x_range.end = max_limit
+
+            self.root.x_range = DataRange1d(start=0, end=max_limit, range_padding=0)
             update(self.source, result)
 
 


Test patch: