Repository: fairlearn/fairlearn

Issue:
Creating group metric sets for regression fails
#### Describe the bug

Unable to create group metric set for `regression` prediction types.

#### Steps/Code to Reproduce

Try to create a `regression` group metric set.

```python
dash_dict = _create_group_metric_set(y_true=y,
                                    predictions=y_pred,
                                    sensitive_features=sf,
                                    prediction_type='regression')
```

#### Expected Results
A dictionary containing the group metrics is created.

#### Actual Results
A dictionary containing the group metrics is, however, not created. Instead, we get the error below.

```python
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-24-96aab337f351> in <module>
      9                                     predictions=y_pred,
     10                                     sensitive_features=sf,
---> 11                                     prediction_type='regression')
     12 
     13 

/anaconda/envs/azureml_py36/lib/python3.6/site-packages/fairlearn/metrics/_group_metric_set.py in _create_group_metric_set(y_true, predictions, sensitive_features, prediction_type)
    145         function_dict = BINARY_CLASSIFICATION_METRICS
    146     elif prediction_type == REGRESSION:
--> 147         result[_PREDICTION_TYPE] == _PREDICTION_REGRESSION
    148         function_dict = REGRESSION_METRICS
    149     else:

KeyError: 'predictionType'

```

#### Versions

- OS: Linux
- Browser (if you're reporting a dashboard bug in jupyter): Chrome
- Python version: 3.6.9
- Fairlearn version: 0.4.6
- version of Python packages: 
```

System:
    python: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)  [GCC 7.3.0]
executable: /anaconda/envs/azureml_py36/bin/python
   machine: Linux-4.15.0-1098-azure-x86_64-with-debian-stretch-sid

Python dependencies:
       pip: 20.1.1
setuptools: 50.3.2
   sklearn: 0.22.2.post1
     numpy: 1.18.5
     scipy: 1.5.2
    Cython: 0.29.21
    pandas: 0.25.3
matplotlib: 3.2.1
    tempeh: None
```

Discussion:
romanlutz: Thank you for reporting, fixing the issue, AND adding a test case @vladiliescu ! 

@riedgar-ms @MiroDudik this is pretty bad. Should we consider a patch release to v0.5.0 with just this change? 

Patch:
diff --git a/AUTHORS.md b/AUTHORS.md
index 3927aca..9cdf0bc 100644
--- a/AUTHORS.md
+++ b/AUTHORS.md
@@ -28,3 +28,4 @@ All names are sorted alphabetically by last name. Contributors, please add your
 - [Hilde Weerts](https://github.com/hildeweerts)
 - [Vincent Xu](https://github.com/vingu)
 - [Beth Zeranski](https://github.com/bethz)
+- [Vlad Iliescu](https://vladiliescu.net)
diff --git a/fairlearn/metrics/_group_metric_set.py b/fairlearn/metrics/_group_metric_set.py
index 6f079fb..83eafde 100644
--- a/fairlearn/metrics/_group_metric_set.py
+++ b/fairlearn/metrics/_group_metric_set.py
@@ -138,7 +138,7 @@ def _create_group_metric_set(y_true,
         result[_PREDICTION_TYPE] = _PREDICTION_BINARY_CLASSIFICATION
         function_dict = BINARY_CLASSIFICATION_METRICS
     elif prediction_type == REGRESSION:
-        result[_PREDICTION_TYPE] == _PREDICTION_REGRESSION
+        result[_PREDICTION_TYPE] = _PREDICTION_REGRESSION
         function_dict = REGRESSION_METRICS
     else:
         raise NotImplementedError(


Test patch:
diff --git a/test/unit/metrics/test_create_group_metric_set.py b/test/unit/metrics/test_create_group_metric_set.py
index 15afb07..97c7e5b 100644
--- a/test/unit/metrics/test_create_group_metric_set.py
+++ b/test/unit/metrics/test_create_group_metric_set.py
@@ -166,7 +166,7 @@ class TestCreateGroupMetricSet:
             y_pred[name] = y_p_ts[i](expected['predictedY'][i])
 
         sensitive_features = {}
-        t_sfs = [lambda x: x, t_sf, lambda x:x]  # Only transform one sf
+        t_sfs = [lambda x: x, t_sf, lambda x: x]  # Only transform one sf
         for i, sf_file in enumerate(expected['precomputedFeatureBins']):
             sf = [sf_file['binLabels'][x] for x in sf_file['binVector']]
             sensitive_features[sf_file['featureBinName']] = t_sfs[i](sf)
@@ -212,3 +212,17 @@ class TestCreateGroupMetricSet:
         actual_roc = actual['precomputedMetrics'][0][0]['balanced_accuracy_score']
         assert actual_roc['global'] == expected.overall['roc_auc_score']
         assert actual_roc['bins'] == list(expected.by_group['roc_auc_score'])
+
+    def test_regression_prediction_type(self):
+        y_t = [0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1]
+        y_p = [1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0]
+        s_f = [0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1]
+
+        predictions = {"some model": y_p}
+        sensitive_feature = {"my sf": s_f}
+
+        # Using the `regression` prediction type should not crash
+        _create_group_metric_set(y_t,
+                                 predictions,
+                                 sensitive_feature,
+                                 'regression')