Repository: fitbenchmarking/fitbenchmarking

Issue:
parameter_ranges with incorrect parameter names are accepted
**Description of the error**
Currently if `parameter_ranges` are set in a problem definition file and incorrect parameter names are used e.g. `parameter_ranges = {'r': (1,50), 'l': (1,500)}` where r and l are used instead of radius and length, fitbenchmarking still runs and produces output as if bounds are set.

**Describe the expected result**
As the parameter names in `parameter_ranges` don't correspond to any actual parameter names, I would expect a warning/error message.

**Describe the actual result**
Instead, in `fitbenchmark_parser`, the `parameter_ranges` line is parsed and this triggers the `set_value_ranges` function. This function assigns all parameters not listed in `parameter_ranges` lower and upper bounds of `-inf` and `inf` respectively. Because of this, even though every parameter is unbounded, each minimizer is run as if bounds are set. This could be a problem if a user has set bounds and made a typo in a parameter name.

**Suggested fix**
In the function `set_value_ranges` in `fitting_problem.py`, a check should be added to make sure that the parameter names listed in `parameter_ranges` are a sublist of `fitting_problem._param_names`. If this is not the case, then a warning message should be displayed to the user and the problem skipped.

Discussion:

Patch:
diff --git a/fitbenchmarking/parsing/fitting_problem.py b/fitbenchmarking/parsing/fitting_problem.py
index 13e51537..dd6d721b 100644
--- a/fitbenchmarking/parsing/fitting_problem.py
+++ b/fitbenchmarking/parsing/fitting_problem.py
@@ -12,7 +12,8 @@ except ImportError:
     from itertools import zip_longest as izip_longest
 import numpy as np
 
-from fitbenchmarking.utils.exceptions import FittingProblemError
+from fitbenchmarking.utils.exceptions import FittingProblemError, \
+    IncorrectBoundsError
 
 
 # Using property getters and setters means that the setter does not always use
@@ -223,13 +224,20 @@ class FittingProblem:
         :type params: dict
 
         """
+        lower_param_names = [name.lower()
+                             for name in self.starting_values[0].keys()]
+        if not all(name in lower_param_names for name in value_ranges):
+            raise IncorrectBoundsError('One or more of the parameter names in '
+                                       'the `parameter_ranges` dictionary is '
+                                       'incorrect, please check the problem '
+                                       'definiton file for this problem.')
+
         self.value_ranges = []
-        for name in self.starting_values[0].keys():
-            param_name = name.lower()
-            if param_name in value_ranges:
+        for name in lower_param_names:
+            if name in value_ranges:
                 self.value_ranges.append(
-                    (value_ranges[param_name][0],
-                        value_ranges[param_name][1]))
+                    (value_ranges[name][0],
+                        value_ranges[name][1]))
             else:
                 self.value_ranges.append((-np.inf, np.inf))
 
diff --git a/fitbenchmarking/utils/exceptions.py b/fitbenchmarking/utils/exceptions.py
index 5bd42bb7..a51ff508 100644
--- a/fitbenchmarking/utils/exceptions.py
+++ b/fitbenchmarking/utils/exceptions.py
@@ -237,3 +237,17 @@ class IncompatibleTableError(FitBenchmarkException):
                               'compatible with the selected ' \
                               'cost function'
         self.error_code = 18
+        
+
+class IncorrectBoundsError(FitBenchmarkException):
+    """
+    Indicates that `parameter_ranges` have been set incorrectly
+    """
+
+    def __init__(self, message=''):
+        super(IncorrectBoundsError, self).__init__(message)
+
+        self._class_message = 'Bounds for this problem are ' \
+                              'unable to be set, so this ' \
+                              'problem will be skipped.'
+        self.error_code = 19


Test patch:
diff --git a/fitbenchmarking/parsing/tests/test_fitting_problem.py b/fitbenchmarking/parsing/tests/test_fitting_problem.py
index 733fbf5c..783b4ffb 100644
--- a/fitbenchmarking/parsing/tests/test_fitting_problem.py
+++ b/fitbenchmarking/parsing/tests/test_fitting_problem.py
@@ -108,6 +108,19 @@ class TestFittingProblem(TestCase):
         fitting_problem.set_value_ranges(value_ranges_prob_def)
         self.assertEqual(fitting_problem.value_ranges, expected_value_ranges)
 
+    def test_set_value_ranges_incorrect_names(self):
+        """
+        Tests that an exception is raised if parameter names
+        in `parameter_ranges` are incorrect
+        """
+        fitting_problem = FittingProblem(self.options)
+        fitting_problem.starting_values = [OrderedDict(
+            [('param1', 0), ('param2', 0), ('param3', 0)])]
+        value_ranges_prob_def = {'incorrect_name': (0, 5), 'param2': (5, 10)}
+        self.assertRaises(exceptions.IncorrectBoundsError,
+                          fitting_problem.set_value_ranges,
+                          value_ranges_prob_def)
+
     def test_correct_data_single_fit(self):
         """
         Tests that correct data gives the expected result