Repository: onnx/sklearn-onnx

Issue:
CalibratedClassifierCV returns wrong predictions when used in pipeline
When using the CalibratedClassifierCV in a Pipeline that is exported to ONNX, then running the ONNX session returns completely different results than running the pipeline directly.

To reproduce the issue, take the tutorial example "Convert a pipeline with ColumnTransformer" (http://onnx.ai/sklearn-onnx/auto_examples/plot_complex_pipeline.html) and replace

clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('classifier', LogisticRegression(solver='lbfgs'))])

with 

clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('calibrator', CalibratedClassifierCV(base_estimator=LogisticRegression(solver='lbfgs', random_state=42), method='sigmoid', cv='warn'))])  

Then predict the same row once using the classifier and once using the ONNX model. The predicted probabilities are completely different.

(Attached is also the complete code).
[titanic.zip](https://github.com/onnx/sklearn-onnx/files/3784470/titanic.zip)


CalibratedClassifierCV returns wrong predictions when used in pipeline
When using the CalibratedClassifierCV in a Pipeline that is exported to ONNX, then running the ONNX session returns completely different results than running the pipeline directly.

To reproduce the issue, take the tutorial example "Convert a pipeline with ColumnTransformer" (http://onnx.ai/sklearn-onnx/auto_examples/plot_complex_pipeline.html) and replace

clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('classifier', LogisticRegression(solver='lbfgs'))])

with 

clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('calibrator', CalibratedClassifierCV(base_estimator=LogisticRegression(solver='lbfgs', random_state=42), method='sigmoid', cv='warn'))])  

Then predict the same row once using the classifier and once using the ONNX model. The predicted probabilities are completely different.

(Attached is also the complete code).
[titanic.zip](https://github.com/onnx/sklearn-onnx/files/3784470/titanic.zip)




Discussion:
After further investigation, it turned out that the CalibratedClassifierCV returns also wrong results if it is used directly (not in a pipeline).

Here's the code that shows the difference between the predictions when calling the trained classifier directly and when calling the loaded ONNX model:

`
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.calibration import CalibratedClassifierCV
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
import onnxruntime as rt
import numpy

#Load data
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

#Fit model
#clr = LogisticRegression()
clr = CalibratedClassifierCV(base_estimator=LogisticRegression(), method='sigmoid')

clr.fit(X_train, y_train)

#Predict with the fitted model
pred_skl = clr.predict(X_test)
print("Predictions from skl:", pred_skl)

#Export to ONNX
initial_type = [('float_input', FloatTensorType([-1, X_train.shape[1]]))]
onx = convert_sklearn(clr, initial_types=initial_type)
with open("exported_model.onnx", "wb") as f:
    f.write(onx.SerializeToString())

#Predict with exported ONNX model
sess = rt.InferenceSession("exported_model.onnx")
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred_onx = sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]
print("Predictions from ONNX:", pred_onx)
`

**Returned results:**
`
Predictions from skl: [1 0 2 1 1 0 1 2 1 1 2 0 0 0 0 1 2 1 1 2 0 2 0 2 2 2 2 2 0 0 0 0 1 0 0 2 1 0]

Predictions from ONNX: [1 0 1 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 0 1 0 1 1 1 1 2 0 0 0 0 1 0 0 1 1 0]
`

(Besides, changing the CalibratedClassifierCV in the code above to LogisticRegression, the two outputs match).
Which version of skl2onnx are you using? This should raise an error in the current implementation. Currently, skl2onnx doesn't support classifiers which have decision_function() as don't yet implement decision_function(). If the base classifier supports decision_function(), it's return value is used to calculate probability scores, else predict_proba()'s return value is used(https://github.com/scikit-learn/scikit-learn/blob/1495f69242646d239d89a5713982946b8ffcf9d9/sklearn/calibration.py#L297). We only support the one's that use predict_proba(). Please update your skl2onnx package.
I was using skl2onn 1.5.1.
After updating to 1.5.2, I'm now getting the expected exception:
`<class 'sklearn.linear_model.logistic.LogisticRegression'>' is not supported with CalibratedClassifierCV yet.`

I've tried to adapt the calibration.py file that you've linked by switching the order of the if statements to the following:

```
        if hasattr(self.base_estimator, "predict_proba"):
            df = self.base_estimator.predict_proba(X)
            if n_classes == 2:
                df = df[:, 1:]

        elif hasattr(self.base_estimator, "decision_function"):
            df = self.base_estimator.decision_function(X)
            if df.ndim == 1:
                df = df[:, np.newaxis]
```

Unfortunately, this did not resolve the issue and still returns the same exception.

Is there any other way that I can use a logistic regression classifier within a CalibratedClassifierCV?

Oh, I didn't mention that for you to tinker with the file itself. I just linked it to point out the difference. Since we only implement predict_proba() and not decision_function(), we don't support CalibratedClassifierCV with predictors with decision_function(). That's something we could look at implementing, but it's going to be a major change and would require some time. Is there any particular reason you need to go with LogisticRegression? You can pick a predictor which doesn't have decision_function().
Actually, I've just used LogisticRegression, as it was in the tutorial. Our preference is the AdaBoostClassifier, which has the same issue.
Therefore, we're currently looking into the possibility to replace the AdaBoostClassifier with one of the following (supported) classifiers:
- ExtraTreesClassifier
- RandomForestClassifer
- XGBClassifer

Do you have any estimates, whether and when you would add support for predictors with decision_function()?

Yes, you may use these for the time being. We have started planning on adding decision_function() in the converters, and I'll prioritise this work. I'll be able to give an estimated timeline soon.
Perfect. Looking forward to hear from you.
After further investigation, it turned out that the CalibratedClassifierCV returns also wrong results if it is used directly (not in a pipeline).

Here's the code that shows the difference between the predictions when calling the trained classifier directly and when calling the loaded ONNX model:

`
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.calibration import CalibratedClassifierCV
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
import onnxruntime as rt
import numpy

#Load data
iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

#Fit model
#clr = LogisticRegression()
clr = CalibratedClassifierCV(base_estimator=LogisticRegression(), method='sigmoid')

clr.fit(X_train, y_train)

#Predict with the fitted model
pred_skl = clr.predict(X_test)
print("Predictions from skl:", pred_skl)

#Export to ONNX
initial_type = [('float_input', FloatTensorType([-1, X_train.shape[1]]))]
onx = convert_sklearn(clr, initial_types=initial_type)
with open("exported_model.onnx", "wb") as f:
    f.write(onx.SerializeToString())

#Predict with exported ONNX model
sess = rt.InferenceSession("exported_model.onnx")
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred_onx = sess.run([label_name], {input_name: X_test.astype(numpy.float32)})[0]
print("Predictions from ONNX:", pred_onx)
`

**Returned results:**
`
Predictions from skl: [1 0 2 1 1 0 1 2 1 1 2 0 0 0 0 1 2 1 1 2 0 2 0 2 2 2 2 2 0 0 0 0 1 0 0 2 1 0]

Predictions from ONNX: [1 0 1 1 1 0 1 1 1 1 1 0 0 0 0 1 1 1 1 1 0 1 0 1 1 1 1 2 0 0 0 0 1 0 0 1 1 0]
`

(Besides, changing the CalibratedClassifierCV in the code above to LogisticRegression, the two outputs match).
Which version of skl2onnx are you using? This should raise an error in the current implementation. Currently, skl2onnx doesn't support classifiers which have decision_function() as don't yet implement decision_function(). If the base classifier supports decision_function(), it's return value is used to calculate probability scores, else predict_proba()'s return value is used(https://github.com/scikit-learn/scikit-learn/blob/1495f69242646d239d89a5713982946b8ffcf9d9/sklearn/calibration.py#L297). We only support the one's that use predict_proba(). Please update your skl2onnx package.
I was using skl2onn 1.5.1.
After updating to 1.5.2, I'm now getting the expected exception:
`<class 'sklearn.linear_model.logistic.LogisticRegression'>' is not supported with CalibratedClassifierCV yet.`

I've tried to adapt the calibration.py file that you've linked by switching the order of the if statements to the following:

```
        if hasattr(self.base_estimator, "predict_proba"):
            df = self.base_estimator.predict_proba(X)
            if n_classes == 2:
                df = df[:, 1:]

        elif hasattr(self.base_estimator, "decision_function"):
            df = self.base_estimator.decision_function(X)
            if df.ndim == 1:
                df = df[:, np.newaxis]
```

Unfortunately, this did not resolve the issue and still returns the same exception.

Is there any other way that I can use a logistic regression classifier within a CalibratedClassifierCV?

Oh, I didn't mention that for you to tinker with the file itself. I just linked it to point out the difference. Since we only implement predict_proba() and not decision_function(), we don't support CalibratedClassifierCV with predictors with decision_function(). That's something we could look at implementing, but it's going to be a major change and would require some time. Is there any particular reason you need to go with LogisticRegression? You can pick a predictor which doesn't have decision_function().
Actually, I've just used LogisticRegression, as it was in the tutorial. Our preference is the AdaBoostClassifier, which has the same issue.
Therefore, we're currently looking into the possibility to replace the AdaBoostClassifier with one of the following (supported) classifiers:
- ExtraTreesClassifier
- RandomForestClassifer
- XGBClassifer

Do you have any estimates, whether and when you would add support for predictors with decision_function()?

Yes, you may use these for the time being. We have started planning on adding decision_function() in the converters, and I'll prioritise this work. I'll be able to give an estimated timeline soon.
Perfect. Looking forward to hear from you.

Resolving patch:
diff --git a/skl2onnx/_supported_operators.py b/skl2onnx/_supported_operators.py
--- a/skl2onnx/_supported_operators.py
+++ b/skl2onnx/_supported_operators.py
@@ -202,11 +202,6 @@
 # each cluster in most cases.
 cluster_list = [KMeans, MiniBatchKMeans]
 
-# Classifiers with converters supporting decision_function().
-decision_function_classifiers = (
-    SGDClassifier,
-)
-
 # Outlier detection algorithms:
 # produces two outputs, label and scores
 outlier_list = [OneClassSVM]
diff --git a/skl2onnx/operator_converters/calibrated_classifier_cv.py b/skl2onnx/operator_converters/calibrated_classifier_cv.py
--- a/skl2onnx/operator_converters/calibrated_classifier_cv.py
+++ b/skl2onnx/operator_converters/calibrated_classifier_cv.py
@@ -12,7 +12,6 @@
     apply_div, apply_exp, apply_mul, apply_reshape, apply_sub)
 from ..common._topology import FloatTensorType
 from ..common._registration import register_converter
-from .._supported_operators import decision_function_classifiers
 from .._supported_operators import sklearn_operator_name_map
 
 
@@ -396,14 +395,6 @@ def convert_sklearn_calibrated_classifier_cv(scope, operator, container):
                               [], [clf_length])
 
     for clf in op.calibrated_classifiers_:
-        if (hasattr(clf.base_estimator, 'decision_function') and
-                not isinstance(clf.base_estimator,
-                               decision_function_classifiers)):
-            raise NotImplementedError(
-                "'{0}' is not supported with CalibratedClassifierCV yet. "
-                "You may raise an issue at "
-                "https://github.com/onnx/sklearn-onnx/issues"
-                "".format(type(clf.base_estimator)))
         prob_scores_name.append(convert_calibrated_classifier_base_estimator(
             scope, operator, container, clf))
 


Test patch:
diff --git a/tests/test_sklearn_bagging_converter.py b/tests/test_sklearn_bagging_converter.py
--- a/tests/test_sklearn_bagging_converter.py
+++ b/tests/test_sklearn_bagging_converter.py
@@ -166,7 +166,8 @@ def test_bagging_classifier_sgd_multiclass(self):
 
     def test_bagging_classifier_sgd_multiclass_decision_function(self):
         model, X = fit_classification_model(
-            BaggingClassifier(GradientBoostingClassifier(random_state=42),
+            BaggingClassifier(GradientBoostingClassifier(
+                                  random_state=42, n_estimators=10),
                               random_state=42), 4)
         options = {id(model): {'raw_scores': True}}
         model_onnx = convert_sklearn(
@@ -189,7 +190,8 @@ def test_bagging_classifier_sgd_multiclass_decision_function(self):
 
     def test_bagging_classifier_gradient_boosting_binary(self):
         model, X = fit_classification_model(
-            BaggingClassifier(GradientBoostingClassifier()), 2)
+            BaggingClassifier(
+                GradientBoostingClassifier(n_estimators=10)), 2)
         model_onnx = convert_sklearn(
             model,
             "bagging classifier",
@@ -208,7 +210,8 @@ def test_bagging_classifier_gradient_boosting_binary(self):
 
     def test_bagging_classifier_gradient_boosting_multiclass(self):
         model, X = fit_classification_model(
-            BaggingClassifier(GradientBoostingClassifier()), 3)
+            BaggingClassifier(
+                GradientBoostingClassifier(n_estimators=10)), 3)
         model_onnx = convert_sklearn(
             model,
             "bagging classifier",
@@ -265,7 +268,8 @@ def test_bagging_regressor_sgd(self):
 
     def test_bagging_regressor_gradient_boosting(self):
         model, X = fit_regression_model(
-            BaggingRegressor(GradientBoostingRegressor()))
+            BaggingRegressor(
+                GradientBoostingRegressor(n_estimators=10)))
         model_onnx = convert_sklearn(
             model,
             "bagging regressor",
diff --git a/tests/test_sklearn_calibrated_classifier_cv_converter.py b/tests/test_sklearn_calibrated_classifier_cv_converter.py
--- a/tests/test_sklearn_calibrated_classifier_cv_converter.py
+++ b/tests/test_sklearn_calibrated_classifier_cv_converter.py
@@ -7,8 +7,11 @@
 import numpy as np
 from sklearn.calibration import CalibratedClassifierCV
 from sklearn.datasets import load_digits, load_iris
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.linear_model import LogisticRegression
 from sklearn.naive_bayes import MultinomialNB
 from sklearn.neighbors import KNeighborsClassifier
+from sklearn.svm import SVC
 import onnxruntime
 from skl2onnx import convert_sklearn
 from skl2onnx.common.data_types import FloatTensorType, Int64TensorType
@@ -26,7 +29,7 @@ def test_model_calibrated_classifier_cv_float(self):
         model = CalibratedClassifierCV(clf, cv=2, method="sigmoid").fit(X, y)
         model_onnx = convert_sklearn(
             model,
-            "scikit-learn CalibratedClassifierCV",
+            "scikit-learn CalibratedClassifierCVMNB",
             [("input", FloatTensorType([None, X.shape[1]]))],
         )
         self.assertTrue(model_onnx is not None)
@@ -48,7 +51,7 @@ def test_model_calibrated_classifier_cv_int(self):
         model = CalibratedClassifierCV(clf, cv=2, method="sigmoid").fit(X, y)
         model_onnx = convert_sklearn(
             model,
-            "scikit-learn CalibratedClassifierCV",
+            "scikit-learn CalibratedClassifierCVMNB",
             [("input", Int64TensorType([None, X.shape[1]]))],
         )
         self.assertTrue(model_onnx is not None)
@@ -73,7 +76,7 @@ def test_model_calibrated_classifier_cv_isotonic_float(self):
         model = CalibratedClassifierCV(clf, cv=2, method="isotonic").fit(X, y)
         model_onnx = convert_sklearn(
             model,
-            "scikit-learn CalibratedClassifierCV",
+            "scikit-learn CalibratedClassifierCVKNN",
             [("input", FloatTensorType([None, X.shape[1]]))],
         )
         self.assertTrue(model_onnx is not None)
@@ -101,7 +104,7 @@ def test_model_calibrated_classifier_cv_binary(self):
             X.astype(np.float32),
             model,
             model_onnx,
-            basename="SklearnCalibratedClassifierCVBinary",
+            basename="SklearnCalibratedClassifierCVBinaryMNB",
             allow_failure="StrictVersion(onnxruntime.__version__)"
             "<= StrictVersion('0.2.1')",
         )
@@ -122,16 +125,78 @@ def test_model_calibrated_classifier_cv_isotonic_binary(self):
             "scikit-learn CalibratedClassifierCV",
             [("input", FloatTensorType([None, X.shape[1]]))],
         )
+        self.assertTrue(model_onnx is not None)
+        dump_data_and_model(
+            X.astype(np.float32),
+            model,
+            model_onnx,
+            basename="SklearnCalibratedClassifierCVIsotonicBinaryKNN")
+
+    @unittest.skipIf(not onnx_built_with_ml(),
+                     reason="Requires ONNX-ML extension.")
+    @unittest.skipIf(
+        StrictVersion(onnxruntime.__version__) < StrictVersion("0.5.0"),
+        reason="not available")
+    def test_model_calibrated_classifier_cv_logistic_regression(self):
+        data = load_iris()
+        X, y = data.data, data.target
+        y[y > 1] = 1
+        model = CalibratedClassifierCV(
+            base_estimator=LogisticRegression(), method='sigmoid').fit(X, y)
+        model_onnx = convert_sklearn(
+            model, "unused",
+            [("input", FloatTensorType([None, X.shape[1]]))],
+        )
+        self.assertTrue(model_onnx is not None)
+        dump_data_and_model(
+            X.astype(np.float32),
+            model,
+            model_onnx,
+            basename="SklearnCalibratedClassifierCVBinaryLogReg",
+            allow_failure="StrictVersion(onnxruntime.__version__)"
+            "<= StrictVersion('0.2.1')",
+        )
+
+    @unittest.skipIf(not onnx_built_with_ml(),
+                     reason="Requires ONNX-ML extension.")
+    @unittest.skipIf(
+        StrictVersion(onnxruntime.__version__) < StrictVersion("0.5.0"),
+        reason="not available")
+    def test_model_calibrated_classifier_cv_rf(self):
+        data = load_iris()
+        X, y = data.data, data.target
+        y[y > 1] = 1
+        model = CalibratedClassifierCV(
+            base_estimator=RandomForestClassifier(n_estimators=2),
+            method='sigmoid').fit(X, y)
+        try:
+            convert_sklearn(
+                model, "unused",
+                [("input", FloatTensorType([None, X.shape[1]]))])
+            raise AssertionError(
+                "RandomForestClassifier has no decision_function")
+        except RuntimeError as e:
+            assert "cannot implement decision_function" in str(e)
+
+    @unittest.skipIf(not onnx_built_with_ml(),
+                     reason="Requires ONNX-ML extension.")
+    @unittest.skipIf(
+        StrictVersion(onnxruntime.__version__) < StrictVersion("0.5.0"),
+        reason="not available")
+    def test_model_calibrated_classifier_cv_svc(self):
+        data = load_iris()
+        X, y = data.data, data.target
+        model = CalibratedClassifierCV(
+            base_estimator=SVC(),
+            method='sigmoid').fit(X, y)
         try:
-            self.assertTrue(model_onnx is not None)
-            dump_data_and_model(
-                X.astype(np.float32),
-                model,
-                model_onnx,
-                basename="SklearnCalibratedClassifierCVIsotonicBinary")
-        except Exception as e:
-            raise AssertionError("Issue with model\n{}".format(
-                str(model_onnx))) from e
+            convert_sklearn(
+                model, "unused",
+                [("input", FloatTensorType([None, X.shape[1]]))])
+            raise AssertionError(
+                "SVC has no decision_function for ovr")
+        except RuntimeError as e:
+            assert "'ovr' is not supported" in str(e)
 
 
 if __name__ == "__main__":
diff --git a/tests/test_sklearn_grid_search_cv_converter.py b/tests/test_sklearn_grid_search_cv_converter.py
--- a/tests/test_sklearn_grid_search_cv_converter.py
+++ b/tests/test_sklearn_grid_search_cv_converter.py
@@ -27,7 +27,7 @@ class TestSklearnGridSearchCVModels(unittest.TestCase):
     @unittest.skipIf(not onnx_built_with_ml(),
                      reason="Requires ONNX-ML extension.")
     def test_grid_search_binary_float(self):
-        tuned_parameters = [{'C': np.logspace(-1, 0, 30)}]
+        tuned_parameters = [{'C': np.logspace(-1, 0, 4)}]
         clf = GridSearchCV(
             LogisticRegression(random_state=42, max_iter=100, solver='lbfgs',
                                multi_class='ovr'),
@@ -52,7 +52,7 @@ def test_grid_search_binary_float(self):
     @unittest.skipIf(not onnx_built_with_ml(),
                      reason="Requires ONNX-ML extension.")
     def test_grid_search_multiclass_float(self):
-        tuned_parameters = [{'C': np.logspace(-1, 0, 30)}]
+        tuned_parameters = [{'C': np.logspace(-1, 0, 4)}]
         clf = GridSearchCV(
             SVC(random_state=42, probability=True, gamma='auto'),
             tuned_parameters, cv=5, iid=False)
@@ -76,7 +76,7 @@ def test_grid_search_multiclass_float(self):
     @unittest.skipIf(not onnx_built_with_ml(),
                      reason="Requires ONNX-ML extension.")
     def test_grid_search_binary_int(self):
-        tuned_parameters = [{'C': np.logspace(-1, 0, 30)}]
+        tuned_parameters = [{'C': np.logspace(-1, 0, 4)}]
         clf = GridSearchCV(
             LogisticRegression(random_state=42, max_iter=100, solver='lbfgs',
                                multi_class='ovr'),
@@ -101,7 +101,7 @@ def test_grid_search_binary_int(self):
     @unittest.skipIf(not onnx_built_with_ml(),
                      reason="Requires ONNX-ML extension.")
     def test_grid_search_multiclass_int(self):
-        tuned_parameters = [{'C': np.logspace(-1, 0, 30)}]
+        tuned_parameters = [{'C': np.logspace(-1, 0, 4)}]
         clf = GridSearchCV(
             LogisticRegression(random_state=42, max_iter=100, solver='lbfgs',
                                multi_class='multinomial'),
@@ -124,7 +124,7 @@ def test_grid_search_multiclass_int(self):
         )
 
     def test_grid_search_regression_int(self):
-        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 30)}]
+        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 4)}]
         clf = GridSearchCV(Lasso(max_iter=100),
                            tuned_parameters, cv=5, iid=False)
         model, X = fit_regression_model(clf, is_int=True)
@@ -145,7 +145,7 @@ def test_grid_search_regression_int(self):
         )
 
     def test_grid_search_regressor_float(self):
-        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 30)}]
+        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 4)}]
         clf = GridSearchCV(LassoLars(max_iter=100),
                            tuned_parameters, cv=5, iid=False)
         model, X = fit_regression_model(clf)
@@ -169,7 +169,7 @@ def test_grid_search_regressor_float(self):
         StrictVersion(ort_version) <= StrictVersion('0.4.0'),
         reason="onnxruntime %s" % '0.4.0')
     def test_grid_search_gaussian_regressor_float(self):
-        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 30)}]
+        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 4)}]
         clf = GridSearchCV(GaussianProcessRegressor(),
                            tuned_parameters, cv=5, iid=False)
         model, X = fit_regression_model(clf)
@@ -194,7 +194,7 @@ def test_grid_search_gaussian_regressor_float(self):
         StrictVersion(ort_version) <= StrictVersion('0.4.0'),
         reason="onnxruntime %s" % '0.4.0')
     def test_grid_search_gaussian_regressor_double(self):
-        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 30)}]
+        tuned_parameters = [{'alpha': np.logspace(-4, -0.5, 4)}]
         clf = GridSearchCV(GaussianProcessRegressor(),
                            tuned_parameters, cv=3, iid=False)
         model, X = fit_regression_model(clf)