Repository: dask/distributed

Issue:
[FEA] Include total number of tasks in the performance report summary
Dask performance reports can be invaluable when profiling complex workloads. When evaluating multiple workloads at once, it can be difficult to remember the size of the task graphs in each workload. Including the total number of tasks in the Summary tab of the performance report would be a nice addition, perhaps under the Duration section.




Discussion:
Thanks for raising this issue @beckernick, adding the total number of tasks seems like a reasonable addition. Is this something you're interested in working on? (no obligation though)
Thanks @jrbourbeau . I don't have the bandwidth at the moment to explore this segment of the codebase (I'm not already familiar with it), so if someone wants to jump on this to implement this feature in the short term that's fine with me. Perhaps it might be a "Good First Issue" for someone looking to contribute.
Not a problem, we'll just leave this issue open to track progress. As this does require some knowledge of how the performance report is produced, I've marked it as a good second issue

Resolving patch:
diff --git a/distributed/scheduler.py b/distributed/scheduler.py
--- a/distributed/scheduler.py
+++ b/distributed/scheduler.py
@@ -1067,7 +1067,7 @@ def __init__(
         preload=None,
         preload_argv=(),
         plugins=(),
-        **kwargs
+        **kwargs,
     ):
         self._setup_logging(logger)
 
@@ -1350,7 +1350,7 @@ def __init__(
             connection_limit=connection_limit,
             deserialize=False,
             connection_args=self.connection_args,
-            **kwargs
+            **kwargs,
         )
 
         if self.worker_ttl:
@@ -2084,7 +2084,7 @@ def stimulus_task_erred(
                     exception=exception,
                     traceback=traceback,
                     worker=worker,
-                    **kwargs
+                    **kwargs,
                 )
         else:
             recommendations = {}
@@ -3384,7 +3384,7 @@ async def retire_workers(
         close_workers=False,
         names=None,
         lock=True,
-        **kwargs
+        **kwargs,
     ):
         """ Gracefully retire workers from cluster
 
@@ -4085,7 +4085,7 @@ def transition_processing_memory(
         typename=None,
         worker=None,
         startstops=None,
-        **kwargs
+        **kwargs,
     ):
         try:
             ts = self.tasks[key]
@@ -5050,6 +5050,15 @@ def profile_to_figure(state):
 
         # Task stream
         task_stream = self.get_task_stream(start=start)
+        total_tasks = len(task_stream)
+        timespent = defaultdict(int)
+        for d in task_stream:
+            for x in d["startstops"]:
+                timespent[x["action"]] += x["stop"] - x["start"]
+        tasks_timings = ""
+        for k in sorted(timespent.keys()):
+            tasks_timings += f"\n<li> {k} time: {format_time(timespent[k])} </li>"
+
         from .diagnostics.task_stream import rectangles
         from .dashboard.components.scheduler import task_stream_figure
 
@@ -5076,6 +5085,11 @@ def profile_to_figure(state):
         <i> Select different tabs on the top for additional information </i>
 
         <h2> Duration: {time} </h2>
+        <h2> Tasks Information </h2>
+        <ul>
+         <li> number of tasks: {ntasks} </li>
+         {tasks_timings}
+        </ul>
 
         <h2> Scheduler Information </h2>
         <ul>
@@ -5091,6 +5105,8 @@ def profile_to_figure(state):
         </pre>
         """.format(
             time=format_time(stop - start),
+            ntasks=total_tasks,
+            tasks_timings=tasks_timings,
             address=self.address,
             nworkers=len(self.workers),
             threads=sum(w.nthreads for w in self.workers.values()),


Test patch: