Repository: fitbenchmarking/fitbenchmarking

Issue:
When using load_checkpoint on old results (previous to last run), Dash pp plots are not shown
When one uses load_checkpoint to visualise results that are older than the one from the last run, the Dash app fails to display the performance profiles. When one clicks the online button, a blank space is shown in place of the pp plot, and no warning is given. 

From the terminal it's possible to see that a Dash app is running, but we get an error in display_page (see traceback attached).
<img width="622" alt="traceback" src="https://github.com/fitbenchmarking/fitbenchmarking/assets/28632706/d4ca4952-a7f2-4a28-befa-21e8ee84c4a2">

Checking the url for the broken pp plot, I saw that the Dash app available at that url was broken (it just appeared as a blank page), while the only Dash app still running was the one associated with the latest run, so at a different URL.


Discussion:
github-actions[bot]: ## Unit & system test results
  3 files   -   1    3 suites   - 1   10m 34s :stopwatch: - 1m 30s
561 tests  -   8  547 :white_check_mark:  -  12  10 :zzz: ± 0  4 :x: +4 
566 runs   - 500  547 :white_check_mark:  - 443  15 :zzz:  - 61  4 :x: +4 

For more details on these failures, see [this check](https://github.com/fitbenchmarking/fitbenchmarking/runs/31018226072).

Results for commit cdb3ba52. ± Comparison against base commit 05bd1e70.

<details>
  <summary>This pull request <b>removes</b> 8 tests.</summary>

```
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_factory[nist-/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/basic.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_factory[nist-/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/start_end_x.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_function_evaluation[/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/function_evaluations.json-nist]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_hessian_evaluation[/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/hessian_evaluations.json-nist]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_jacobian_evaluation[/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/jacobian_evaluations.json-nist]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected0-nist-/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/start_end_x.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_parsers[expected1-nist-/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/basic.dat]
fitbenchmarking.parsing.tests.test_parsers.TestParsers ‑ test_sparsej_evaluation[/home/runner/work/fitbenchmarking/fitbenchmarking/fitbenchmarking/parsing/tests/nist/sparse_jacobian_evaluations.json-nist]
```
</details>

[test-results]:data:application/gzip;base64,H4sIAPBl/mYC/1WMyw6DIBBFf8Ww7gIUpqY/Y2CAhFSl4bEy/feO9VG7POfenIX5MLrMHk13a1iuoZxga9IlxJkQOkmCprKOCsRBQ66Iq5L3n3qGFynBT+N1GMmcjcGlFBOZ9ZLqvEVhh//mZvakOsSl+OVrEOM0hULA0JrOaNWKFrXuPYA06JBbq7gFib1DLQGFYe8Pkrne9AcBAAA=

coveralls: 
[![Coverage Status](https://coveralls.io/builds/70360441/badge)](https://coveralls.io/builds/70360441)

Changes unknown
when pulling **5de2dcd1ab5238f2ba4f412a45cd01b1c9739f72 on 1253-dash_uuid**
into ** on master**.


Patch:
diff --git a/fitbenchmarking/core/results_output.py b/fitbenchmarking/core/results_output.py
index b9d0cd73..6d660eb8 100644
--- a/fitbenchmarking/core/results_output.py
+++ b/fitbenchmarking/core/results_output.py
@@ -592,7 +592,8 @@ def display_page(
     pathname: str,
     profile_instances_all_groups: "dict[str, dict[str, DashPerfProfile]]",
     layout: "list",
-    max_solvers: int
+    max_solvers: int,
+    run_id: str
 ):
     """
     Update the layout of the dash app.
@@ -605,18 +606,25 @@ def display_page(
     :type layout: list of dcc or html components
     :param max_solvers: Maximum number of solvers that can be plotted
     :type max_solvers: int
+    :param run_id: The id for the run that is plotted in dash
+    :type run_id: str
 
     :return: The updated layout
     :rtype: html.Div
     """
 
     try:
-        _, group, plot, metric_str = pathname.split('/')
+        _, plot_id, group, plot, metric_str = pathname.split('/')
     except ValueError:
-        return ("404 Page Error! Path does not have the expected shape. "
+        return ("404 Page Error! Path does not have the expected format. "
                 "Please provide it in the following form:  \n"
-                "ip-address:port/problem_set/plot/performance_profile.")
+                "ip-address:port/run_id/problem_set/plot/performance_profile.")
 
+    if run_id != plot_id:
+        return (
+            "404 Page Error! Dash plots are not available for these results."
+            "You can use `fitbenchmarking --load-checkpoint` to fix this."
+        )
     if plot != "pp":
         return f"404 Page Error! Plot type '{plot}' not available."
 
@@ -744,7 +752,9 @@ def run_dash_app(options, pp_dfs_all_prob_sets) -> None:
                 x,
                 profile_instances_all_groups=profile_instances_all_groups,
                 layout=layout,
-                max_solvers=max_solvers)
+                max_solvers=max_solvers,
+                run_id=options.run_id,
+            )
     )
 
     app.run(port=options.port)
diff --git a/fitbenchmarking/results_processing/tables.py b/fitbenchmarking/results_processing/tables.py
index b8321e72..0e159376 100644
--- a/fitbenchmarking/results_processing/tables.py
+++ b/fitbenchmarking/results_processing/tables.py
@@ -140,6 +140,7 @@ def create_results_tables(options, results, best_results, group_dir, fig_dir,
                             os.path.relpath(table.pp_locations[p], group_dir)
                             for p in table.pps],
                         pp_dash_url=f'http://127.0.0.1:{options.port}/'
+                                    f"{options.run_id}/"
                                     f'{os.path.basename(group_dir)}/'
                                     f'pp/{"+".join(table.pps)}',
                         cbar=cbar,
diff --git a/fitbenchmarking/utils/options.py b/fitbenchmarking/utils/options.py
index ebc600a1..d2f257d4 100644
--- a/fitbenchmarking/utils/options.py
+++ b/fitbenchmarking/utils/options.py
@@ -6,6 +6,7 @@ This file will handle all interaction with the options configuration file.
 import configparser
 import glob
 import os
+import uuid
 
 import matplotlib.pyplot as plt
 
@@ -346,6 +347,8 @@ class Options:
         :type additional_options: dict
 
         """
+        self.run_id = uuid.uuid4().hex
+
         # additional_options is initialied to an empty dict if
         # no value is given
         if additional_options is None:


Test patch:
diff --git a/fitbenchmarking/core/tests/test_results_output.py b/fitbenchmarking/core/tests/test_results_output.py
index 3c61e58b..26c8d09f 100644
--- a/fitbenchmarking/core/tests/test_results_output.py
+++ b/fitbenchmarking/core/tests/test_results_output.py
@@ -13,6 +13,7 @@ from unittest.mock import patch
 
 import pandas as pd
 from dash import dcc, html
+from parameterized import parameterized
 
 from fitbenchmarking import test_files
 from fitbenchmarking.core.results_output import (_extract_tags,
@@ -646,11 +647,13 @@ class DisplayPageTests(unittest.TestCase):
         Test that the expected layout components are returned when the
         pathname refers to one plot only.
         """
-        pathname = "127.0.0.1:5009/NIST_low_difficulty/pp/acc"
-        output_div = display_page(pathname,
-                                  self.profile_instances_all_groups,
-                                  self.layout,
-                                  self.max_solvers)
+        pathname = "127.0.0.1:5009/abc/NIST_low_difficulty/pp/acc"
+        output_div = display_page(
+            pathname=pathname,
+            profile_instances_all_groups=self.profile_instances_all_groups,
+            layout=self.layout,
+            max_solvers=self.max_solvers,
+            run_id="abc")
         output_ids = list(output_div)
         expected_ids = ['Log axis toggle', 'dropdown', 'warning',
                         'visual NIST_low_difficulty-Accuracy']
@@ -661,23 +664,46 @@ class DisplayPageTests(unittest.TestCase):
         Test that the expected layout components are returned when the
         pathname refers to two plots.
         """
-        pathname = "127.0.0.1:5009/NIST_low_difficulty/pp/acc+runtime"
-        output_div = display_page(pathname,
-                                  self.profile_instances_all_groups,
-                                  self.layout,
-                                  self.max_solvers)
+        pathname = "127.0.0.1:5009/123/NIST_low_difficulty/pp/acc+runtime"
+        output_div = display_page(
+            pathname=pathname,
+            profile_instances_all_groups=self.profile_instances_all_groups,
+            layout=self.layout,
+            max_solvers=self.max_solvers,
+            run_id="123",
+        )
         output_ids = list(output_div)
         expected_ids = ['Log axis toggle', 'dropdown', 'warning',
                         'visual NIST_low_difficulty-Accuracy',
                         'visual NIST_low_difficulty-Runtime']
         self.assertEqual(output_ids, expected_ids)
 
+    @parameterized.expand([
+        ("127.0.0.1:5009/old_id/NIST_low_difficulty/pp/acc+runtime", "new_id"),
+        ("127.0.0.1:5009/abc/NIST_low_difficulty/??/acc+runtime", "abc"),
+        ("127.0.0.1:5009/abc/pp/acc+runtime", "abc"),
+    ])
+    def test_dash_url_404(self, pathname, run_id):
+        """
+        Tests that errors are reported when the dash url is not correct.
+        """
+        output_div = display_page(
+            pathname=pathname,
+            profile_instances_all_groups=self.profile_instances_all_groups,
+            layout=self.layout,
+            max_solvers=self.max_solvers,
+            run_id=run_id,
+        )
+        assert (
+            isinstance(output_div, str) and output_div.startswith("404")
+        )
+
     def test_styles_consistent_when_two_plts(self):
         """
         Test that the styles of lines on the graphs are consistent when
         pathname refers to two plots.
         """
-        pathname = "127.0.0.1:5009/NIST_low_difficulty/pp/acc+runtime"
+        pathname = "127.0.0.1:5009/abc/NIST_low_difficulty/pp/acc+runtime"
         pps = self.profile_instances_all_groups["NIST_low_difficulty"]
         pp = pps["acc"]
         pp.current_styles["solver1"] = pp.avail_styles.pop()
@@ -686,7 +712,8 @@ class DisplayPageTests(unittest.TestCase):
             pathname,
             self.profile_instances_all_groups,
             self.layout,
-            self.max_solvers
+            self.max_solvers,
+            run_id="abc"
         )
 
         self.assertDictEqual(
diff --git a/fitbenchmarking/utils/tests/test_options_generic.py b/fitbenchmarking/utils/tests/test_options_generic.py
index eb4e4dbc..6c877b39 100644
--- a/fitbenchmarking/utils/tests/test_options_generic.py
+++ b/fitbenchmarking/utils/tests/test_options_generic.py
@@ -80,6 +80,10 @@ class OptionsWriteTests(unittest.TestCase):
         options.stored_file_name = ""
         new_options.stored_file_name = ""
 
+        # Overwrite run_id
+        options.run_id = ""
+        new_options.run_id = ""
+
         self.assertDictEqual(options.__dict__, new_options.__dict__)
 
     def test_write_to_stream(self):
@@ -101,6 +105,10 @@ class OptionsWriteTests(unittest.TestCase):
         options.stored_file_name = ""
         new_options.stored_file_name = ""
 
+        # Overwrite run_id
+        options.run_id = ""
+        new_options.run_id = ""
+
         os.remove(new_file_name)
         self.assertDictEqual(options.__dict__, new_options.__dict__)