Repository: dask/distributed

Issue:
The dashboard is not cleaning Bokeh Sessions after the browser disconnects.
<!-- Please include a self-contained copy-pastable example that generates the issue if possible.

Please be concise with code posted. See guidelines below on how to provide a good bug report:

- Craft Minimal Bug Reports http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports
- Minimal Complete Verifiable Examples https://stackoverflow.com/help/mcve

Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly.
-->

**What happened**:
The Dask Scheduler Dashboard CPU usage increases when the dashboard is used but it keeps being high after the user closes the dashboard.  I think that this could be related with other issues like dask/dask#8493.

Steps to reproduce: 
1. Start dask-scheduler
2. Open the browser at localhost:8787
3. Reload several times. 

You can expect the CPU usage to increase the more you reload.

**What you expected to happen**:
The CPU usage should be stable or at least decrease when the user closes the tab.

**More information:**
I've added logs to see the active Bokeh sessions
```python
#dashboard/scheduler.py
...
def connect(application, http_server, scheduler, prefix=""):
    bokeh_app = BokehApplication(
        applications, scheduler, prefix=prefix, template_variables=template_variables
    )
    application.add_application(bokeh_app)
    bokeh_app.initialize(IOLoop.current())
    bokeh_app.add_handlers(
        r".*",
        [
            (
                r"/",
                web.RedirectHandler,
                {"url": urljoin((prefix or "").strip("/") + "/", r"status")},
            )
        ],
    )

    import threading
    threading.Thread(
        target=run_stuff,
        args=(bokeh_app,),
    ).start()

from time import sleep

def run_stuff(bokeh_app):
    while True:
        sleep(1)
        sessions = bokeh_app.get_sessions("/status")
        print('Total sessions', len(sessions))
        for (c, session) in enumerate(sessions):
            print('conn:', c, session.connection_count)
```

Each time the browser reloads the number of sessions increases even though the connection count for all sessions but one is 0.

**How to fix this? (I think)**

The problem is that the cleaning callback loop is not started. To start one simply has to call `bokeh_app.start()` after the initialization.

```python
#dashboard/scheduler.py
...
def connect(application, http_server, scheduler, prefix=""):
    bokeh_app = BokehApplication(
        applications, scheduler, prefix=prefix, template_variables=template_variables
    )
    application.add_application(bokeh_app)
    bokeh_app.initialize(IOLoop.current())
    bokeh_app.add_handlers(
        r".*",
        [
            (
                r"/",
                web.RedirectHandler,
                {"url": urljoin((prefix or "").strip("/") + "/", r"status")},
            )
        ],
    )
    bokeh_app.start()
```

Also setting the option `unused_session_lifetime_milliseconds` to 500 helps cleaning the sessions faster.

**Environment**:

- Dask version: 2022.6.1
- Python version: 3.10
- Operating System: Ubuntu 22.04
- Install method (conda, pip, source): pip




Discussion:
@z4m0 Thank you for a well-explained report, and for including a possible solution!
@pavithraes Any news on this? Do you need a PR?
@z4m0 a PR would be welcome if you feel like opening one, thanks!

Resolving patch:
diff --git a/distributed/dashboard/scheduler.py b/distributed/dashboard/scheduler.py
--- a/distributed/dashboard/scheduler.py
+++ b/distributed/dashboard/scheduler.py
@@ -168,3 +168,5 @@ def connect(application, http_server, scheduler, prefix=""):
             )
         ],
     )
+
+    bokeh_app.start()
diff --git a/distributed/dashboard/worker.py b/distributed/dashboard/worker.py
--- a/distributed/dashboard/worker.py
+++ b/distributed/dashboard/worker.py
@@ -20,6 +20,7 @@ def connect(application, http_server, worker, prefix=""):
     )
     application.add_application(bokeh_app)
     bokeh_app.initialize(IOLoop.current())
+    bokeh_app.start()
 
 
 applications = {
diff --git a/distributed/node.py b/distributed/node.py
--- a/distributed/node.py
+++ b/distributed/node.py
@@ -73,6 +73,10 @@ def start_services(self, default_listen_ip):
                 )
 
     def stop_services(self):
+        if hasattr(self, "http_application"):
+            for application in self.http_application.applications:
+                if hasattr(application, "stop") and callable(application.stop):
+                    application.stop()
         for service in self.services.values():
             service.stop()
 


Test patch: