Repository: novonordisk-research/ProcessOptimizer

Issue:
LHS sampling is different if limits are continuous or discrete
The points returned by the latin hypercube sampling differs, depending on whether the min/max limits are defined as integers or reals. An example that illustrates it is the following code:

```
from ProcessOptimizer import Optimizer
n_init = 5
space1 = [(0,100),(0,100),(0,100)]
space2 = [(0.0,100.0),(0.0,100.0),(0.0,100.0)]
opt1 = Optimizer(space1,n_initial_points=n_init)
opt2 = Optimizer(space2,n_initial_points=n_init)
opt1.ask(5)
opt2.ask(5)
```

op1 returns:
```[[25, 75, 0], [100, 50, 50], [50, 25, 25], [0, 0, 75], [75, 100, 100]]```
while opt2 returns:
```[[30.0, 70.0, 10.0], [90.0, 50.0, 50.0], [50.0, 30.0, 30.0], [10.0, 10.0, 70.0], [70.0, 90.0, 90.0]]```

In my opinion, these lists should be the same within rounding. The difference is caused by different definitions of `lhs_arange` for the class Real and the class Integer in space.py. In the case of Real the points are spaced equally, with a half-step buffer to the limit values like this:  `a = (np.arange(n)+0.5)/n`, while Integers do not include the buffer at the limits: `rounded_numbers = np.round(np.linspace(self.low, self.high, n))`.

I'll work on a fix for the integer class.



Comments:

Patch:
diff --git a/ProcessOptimizer/bokeh_plot.py b/ProcessOptimizer/bokeh_plot.py
index e71b40a..24d8d69 100644
--- a/ProcessOptimizer/bokeh_plot.py
+++ b/ProcessOptimizer/bokeh_plot.py
@@ -280,7 +280,7 @@ def get_plot_list(layout, result, active_list, n_points, x_eval, confidence):
             elif i == j:  # Diagonal
                 # Passing j = None to dependence
                 # makes it calculate a diagonal plot
-                xi, yi = dependence(
+                xi, yi, _ = dependence(
                     space,
                     model,
                     i,
diff --git a/ProcessOptimizer/space/space.py b/ProcessOptimizer/space/space.py
index a31b04f..e66e504 100644
--- a/ProcessOptimizer/space/space.py
+++ b/ProcessOptimizer/space/space.py
@@ -305,10 +305,13 @@ class Real(Dimension):
         * `n` [int]
             Number of samples.
         """
-        a = (np.arange(n)+0.5)/n  # Evenly distributed betweeen 0 and 1
+        # Generate sample points by splitting the space 0 to 1 into n pieces
+        # and picking the middle of each piece. Samples are spaced 1/n apart
+        # inside the interval with a buffer of half a step size to the extremes
+        samples = (np.arange(n)+0.5)/n
 
-        # Transform to the bounds of this dimension
-        return a*(self.high-self.low)+self.low
+        # Transform the samples to the range used for this dimension
+        return samples*(self.high - self.low) + self.low
 
 
 class Integer(Dimension):
@@ -413,9 +416,17 @@ class Integer(Dimension):
         * `n` [int]
             Number of samples.
         """
-        rounded_numbers = np.round(np.linspace(self.low, self.high, n))
-        # convert to a list of integers
-        return [int(a) for a in rounded_numbers]
+        # Generate sample points by splitting the space 0 to 1 into n pieces
+        # and picking the middle of each piece. Samples are spaced 1/n apart
+        # inside the interval with a buffer of half a step size to the extremes
+        samples = (np.arange(n)+0.5)/n
+        # Transform the samples to the range used for this dimension and then 
+        # round them back to integers. If your space is less than n wide, some 
+        # of your samples will be rounded to the same number
+        samples = np.round(samples*(self.high - self.low) + self.low)
+        
+        # Convert samples to a list of integers
+        return samples.astype(int)
 
 
 class Categorical(Dimension):


Test patch:
diff --git a/ProcessOptimizer/tests/test_space.py b/ProcessOptimizer/tests/test_space.py
index 82e0c43..7f458fe 100644
--- a/ProcessOptimizer/tests/test_space.py
+++ b/ProcessOptimizer/tests/test_space.py
@@ -671,12 +671,21 @@ def test_purely_categorical_space():
 
 @pytest.mark.fast_test
 def test_lhs_arange():
-    dim = Categorical(["a", "b", "c"])
-    dim.lhs_arange(10)
-    dim = Integer(1, 20)
-    dim.lhs_arange(10)
-    dim = Real(-10, 20)
-    dim.lhs_arange(10)
+    # Test that the latin hypercube sampling functions run normally
+    dim_cat = Categorical(["a", "b", "c"])
+    dim_cat.lhs_arange(10)
+    dim_int = Integer(-10, 20)
+    lhs_int = dim_int.lhs_arange(10)
+    dim_real = Real(-10, 20)
+    lhs_real = dim_real.lhs_arange(10)
+    # Test that the Integer and Real classes return identical points after
+    # rounding, using Numpy's allclose function
+    assert np.allclose(lhs_int, lhs_real, atol=0.5)
+    # Test that the Integer class returns values that are ints for all intents 
+    # and purposes
+    assert all([np.mod(x,1) == 0 for x in lhs_int])
+    # Test that the Real class returns flots in all locations
+    assert all([isinstance(x,np.float64) for x in lhs_real])
 
 
 @pytest.mark.fast_test
@@ -687,3 +696,4 @@ def test_lhs():
     samples = SPACE.lhs(10)
     assert len(samples) == 10
     assert len(samples[0]) == 3
+