Repository: DistrictDataLabs/yellowbrick

Issue:
ROCAUC treats binary classification as multiclass for estimators with predict_proba available
**Describe the bug**
``ROCAUC`` plots 4 different curves for a binary classification problem when using an estimator with ``predict_proba``, which returns predictions of shape ``nx2``. Only one curve is plotted when estimator only has ``decision_function``, which returns predictions of shape ``nx1``. I think only one ROC curve should be drawn for a binary classification problem, as in the binary case the two ROC curves before averaging are just a reflection of each other.

For comparison, ``PrecisionRecallCurve`` plots 1 curve for binary classification irrespective, whether provided estimator uses ``predict_proba`` or ``decision_function``.

**To Reproduce**

To see this behavior it is enough to look at 
``yellowbrick/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision.png`` and ``yellowbrick/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability.png``

**Expected behavior**

I expected to see just one ROC for binary classification instead of separate curves for class 0 and 1 and their micro and macro averaging.



Discussion:
To draw just 1 ROC curve minor changes are needed in ``ROCAUC``, I can work on this.
@VladSkripniuk thank you so much for opening this issue and for the two PRs - I've been away on travel and we've been working to get the v1.1 release out; but I'll get to your PRs soon! 
@VladSkripniuk which sklearn estimators do you believe should produce single curves?
@lwgray basically all estimators which solve binary classification problem, i.e. estimators which have `predict_proba` or `decision_function` which returns array of shape (n_samples,2) or (n_samples,)
Relates to #1041 

Resolving patch:
diff --git a/yellowbrick/classifier/rocauc.py b/yellowbrick/classifier/rocauc.py
--- a/yellowbrick/classifier/rocauc.py
+++ b/yellowbrick/classifier/rocauc.py
@@ -24,6 +24,7 @@
 from scipy import interp
 from sklearn.metrics import auc, roc_curve
 from sklearn.preprocessing import label_binarize
+from sklearn.utils.multiclass import type_of_target
 
 from yellowbrick.exceptions import ModelError
 from yellowbrick.style.palettes import LINE_COLOR
@@ -35,6 +36,10 @@
 MACRO = "macro"
 MICRO = "micro"
 
+# Target Type Constants
+BINARY = "binary"
+MULTICLASS = "multiclass"
+
 
 ##########################################################################
 ## ROCAUC Visualizer
@@ -85,9 +90,17 @@ class ROCAUC(ClassificationScoreVisualizer):
 
     per_class : bool, default: True
         Plot the ROC curves for each individual class. This should be set
-        to false if only the macro or micro average curves are required. Per-
-        class classification is not defined for binary classification problems
-        with estimators with only a decision_function method.
+        to false if only the macro or micro average curves are required. For true
+        binary classifiers, setting per_class=False will plot the positive class
+        ROC curve, and per_class=True will use ``1-P(1)`` to compute the curve of
+        the negative class if only a decision_function method exists on the estimator.
+
+    binary : bool, default: False
+        This argument quickly resets the visualizer for true binary classification
+        by updating the micro, macro, and per_class arguments to False (do not use
+        in conjunction with those other arguments). Note that this is not a true
+        hyperparameter to the visualizer, it just collects other parameters into
+        a single, simpler argument.
 
     classes : list of str, defult: None
         The class labels to use for the legend ordered by the index of the sorted
@@ -131,6 +144,9 @@ class classification is not defined for binary classification problems
         generally better. For classifiers, this score is usually accuracy, but
         if micro or macro is specified this returns an F1 score.
 
+    target_type_ : string
+        Specifies if the detected classification target was binary or multiclass.
+
     Notes
     -----
     ROC curves are typically used in binary classification, and in fact the
@@ -173,6 +189,7 @@ def __init__(
         micro=True,
         macro=True,
         per_class=True,
+        binary=False,
         classes=None,
         encoder=None,
         is_fitted="auto",
@@ -190,7 +207,33 @@ def __init__(
         )
 
         # Set the visual parameters for ROCAUC
-        self.set_params(micro=micro, macro=macro, per_class=per_class)
+        # NOTE: the binary flag breaks our API since it's really just a meta parameter
+        # for micro, macro, and per_class. We knew this going into it, but did it anyway.
+        if binary:
+            self.set_params(micro=False, macro=False, per_class=False)
+        else:
+            self.set_params(micro=micro, macro=macro, per_class=per_class)
+
+    def fit(self, X, y=None):
+        """
+        Fit the classification model.
+        """
+        # The target determines what kind of estimator is fit
+        ttype = type_of_target(y)
+        if ttype.startswith(MULTICLASS):
+            self.target_type_ = MULTICLASS
+        elif ttype.startswith(BINARY):
+            self.target_type_ = BINARY
+        else:
+            raise YellowbrickValueError(
+                (
+                    "{} does not support target type '{}', "
+                    "please provide a binary or multiclass single-output target"
+                ).format(self.__class__.__name__, ttype)
+            )
+
+        # Fit the model and return self
+        return super(ROCAUC, self).fit(X, y)
 
     def score(self, X, y=None):
         """
@@ -217,27 +260,14 @@ def score(self, X, y=None):
         # Compute the predictions for the test data
         y_pred = self._get_y_scores(X)
 
-        # Note: In the above, _get_y_scores calls either a decision_function or
-        # predict_proba, which should return a 2D array. But in a binary
-        # classification using an estimator with only a decision_function, y_pred
-        # will instead be 1D, meaning only one curve can be plotted. In this case,
-        # we set the _binary_decision attribute to True to ensure only one curve is
-        # computed and plotted later on.
-        if y_pred.ndim == 1:
-            self._binary_decision = True
-
-            # Raise an error if it's a binary decision and user has set micro,
-            # macro, or per_class to True
-            if self.micro or self.macro or self.per_class:
+        if self.target_type_ == BINARY:
+            # If it's binary classification, to draw micro or macro curves per_class must be True
+            if (self.micro or self.macro) and not self.per_class:
                 raise ModelError(
-                    "Micro, macro, and per-class scores are not defined for "
-                    "binary classification for estimators with only "
-                    "decision_function methods; set micro, macro, and "
-                    "per-class params to False."
+                    "no curves will be drawn; set per_class=True or micro=False and macro=False."
                 )
-        else:
-            self._binary_decision = False
-            # If it's not a binary decision, at least one of micro, macro, or
+        if self.target_type_ == MULTICLASS:
+            # If it's multiclass classification, at least one of micro, macro, or
             # per_class must be True
             if not self.micro and not self.macro and not self.per_class:
                 raise YellowbrickValueError(
@@ -254,16 +284,48 @@ def score(self, X, y=None):
         self.tpr = dict()
         self.roc_auc = dict()
 
-        # If the decision is binary, compute the ROC curve and ROC area
-        if self._binary_decision is True:
-            self.fpr[0], self.tpr[0], _ = roc_curve(y, y_pred)
+        # If the decision is binary draw only ROC curve for the postitive class
+        if self.target_type_ is BINARY and not self.per_class:
+            # In this case predict_proba returns an array of shape (n, 2) which
+            # specifies the probabilities of both the negative and positive classes.
+            if len(y_pred.shape) == 2 and y_pred.shape[1] == 2:
+                self.fpr[BINARY], self.tpr[BINARY], _ = roc_curve(y, y_pred[:,1])
+            else:
+                # decision_function returns array of shape (n,), so plot it directly
+                self.fpr[BINARY], self.tpr[BINARY], _ = roc_curve(y, y_pred)
+            self.roc_auc[BINARY] = auc(self.fpr[BINARY], self.tpr[BINARY])
+
+        # Per-class binary decisions may have to have the negative class curve computed
+        elif self.target_type_ is BINARY and self.per_class:
+            # draw a curve for class 1 (the positive class)
+            if len(y_pred.shape) == 2 and y_pred.shape[1] == 2:
+                # predict_proba returns array of shape (n, 2), so use
+                # probability of class 1 to compute ROC
+                self.fpr[1], self.tpr[1], _ = roc_curve(y, y_pred[:,1])
+            else:
+                # decision_function returns array of shape (n,)
+                self.fpr[1], self.tpr[1], _ = roc_curve(y, y_pred)
+            self.roc_auc[1] = auc(self.fpr[1], self.tpr[1])
+
+            # draw a curve for class 0 (the negative class)
+            if len(y_pred.shape) == 2 and y_pred.shape[1] == 2:
+                # predict_proba returns array of shape (n, 2), so use
+                # probability of class 0 to compute ROC
+                self.fpr[0], self.tpr[0], _ = roc_curve(1-y, y_pred[:,0])
+            else:
+                # decision_function returns array of shape (n,).
+                # To draw a ROC curve for class 0 we swap the classes 0 and 1 in y
+                # and reverse classifiers predictions y_pred.
+                self.fpr[0], self.tpr[0], _ = roc_curve(1-y, -y_pred)
             self.roc_auc[0] = auc(self.fpr[0], self.tpr[0])
+
         else:
             # Otherwise compute the ROC curve and ROC area for each class
             for i, c in enumerate(classes):
                 self.fpr[i], self.tpr[i], _ = roc_curve(y, y_pred[:, i], pos_label=c)
                 self.roc_auc[i] = auc(self.fpr[i], self.tpr[i])
 
+
         # Compute micro average
         if self.micro:
             self._score_micro_average(y, y_pred, classes, n_classes)
@@ -298,11 +360,11 @@ def draw(self):
         n_classes = len(colors)
 
         # If it's a binary decision, plot the single ROC curve
-        if self._binary_decision is True:
+        if self.target_type_ == BINARY and not self.per_class:
             self.ax.plot(
-                self.fpr[0],
-                self.tpr[0],
-                label="ROC for binary decision, AUC = {:0.2f}".format(self.roc_auc[0]),
+                self.fpr[BINARY],
+                self.tpr[BINARY],
+                label="ROC for binary decision, AUC = {:0.2f}".format(self.roc_auc[BINARY]),
             )
 
         # If per-class plotting is requested, plot ROC curves for each class
@@ -459,6 +521,7 @@ def roc_auc(
     micro=True,
     macro=True,
     per_class=True,
+    binary=False,
     classes=None,
     encoder=None,
     is_fitted="auto",
@@ -494,7 +557,7 @@ def roc_auc(
 
     X_train : array-like, 2D
         The table of instance data or independent variables that describe the outcome of
-        the dependent variable, y. Used to fit the visualizer and also to score the 
+        the dependent variable, y. Used to fit the visualizer and also to score the
         visualizer if test splits are not specified.
 
     y_train : array-like, 2D
@@ -504,9 +567,9 @@ def roc_auc(
     X_test: array-like, 2D, default: None
         The table of instance data or independent variables that describe the outcome of
         the dependent variable, y. Used to score the visualizer if specified.
-    
+
     y_test: array-like, 1D, default: None
-        The vector of target data or the dependent variable predicted by X. 
+        The vector of target data or the dependent variable predicted by X.
         Used to score the visualizer if specified.
 
     ax : matplotlib Axes, default: None
@@ -533,9 +596,17 @@ def roc_auc(
 
     per_class : bool, default: True
         Plot the ROC curves for each individual class. This should be set
-        to false if only the macro or micro average curves are required. Per-
-        class classification is not defined for binary classification problems
-        with estimators with only a decision_function method.
+        to false if only the macro or micro average curves are required. For true
+        binary classifiers, setting per_class=False will plot the positive class
+        ROC curve, and per_class=True will use ``1-P(1)`` to compute the curve of
+        the negative class if only a decision_function method exists on the estimator.
+
+    binary : bool, default: False
+        This argument quickly resets the visualizer for true binary classification
+        by updating the micro, macro, and per_class arguments to False (do not use
+        in conjunction with those other arguments). Note that this is not a true
+        hyperparameter to the visualizer, it just collects other parameters into
+        a single, simpler argument.
 
     classes : list of str, defult: None
         The class labels to use for the legend ordered by the index of the sorted
@@ -611,6 +682,7 @@ class classification is not defined for binary classification problems
         micro=micro,
         macro=macro,
         per_class=per_class,
+        binary=binary,
         classes=classes,
         encoder=encoder,
         is_fitted=is_fitted,
@@ -626,7 +698,7 @@ class classification is not defined for binary classification problems
         visualizer.score(X_test, y_test)
     else:
         visualizer.score(X_train,  y_train)
-    
+
     if show:
         visualizer.show()
     else:


Test patch:
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision.png b/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision.png and b/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision_per_class.png b/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision_per_class.png
new file mode 100644
Binary files /dev/null and b/tests/baseline_images/test_classifier/test_rocauc/test_binary_decision_per_class.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability.png b/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability.png and b/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision.png b/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision.png and b/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision_single_curve.png b/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision_single_curve.png
new file mode 100644
Binary files /dev/null and b/tests/baseline_images/test_classifier/test_rocauc/test_binary_probability_decision_single_curve.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_multiclass_rocauc.png b/tests/baseline_images/test_classifier/test_rocauc/test_multiclass_rocauc.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_multiclass_rocauc.png and b/tests/baseline_images/test_classifier/test_rocauc/test_multiclass_rocauc.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_pandas_integration.png b/tests/baseline_images/test_classifier/test_rocauc/test_pandas_integration.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_pandas_integration.png and b/tests/baseline_images/test_classifier/test_rocauc/test_pandas_integration.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_classes.png b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_classes.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_classes.png and b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_classes.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro.png b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro.png and b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro_no_micro.png b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro_no_micro.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro_no_micro.png and b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_macro_no_micro.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_micro.png b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_micro.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_micro.png and b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_no_micro.png differ
diff --git a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_quickmethod.png b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_quickmethod.png
Binary files a/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_quickmethod.png and b/tests/baseline_images/test_classifier/test_rocauc/test_rocauc_quickmethod.png differ
diff --git a/tests/test_classifier/test_rocauc.py b/tests/test_classifier/test_rocauc.py
--- a/tests/test_classifier/test_rocauc.py
+++ b/tests/test_classifier/test_rocauc.py
@@ -26,8 +26,8 @@
 from tests.base import VisualTestCase
 
 from yellowbrick.classifier.rocauc import *
+from yellowbrick.exceptions import ModelError
 from yellowbrick.datasets import load_occupancy
-from yellowbrick.exceptions import ModelError, YellowbrickValueError
 
 from sklearn.svm import LinearSVC
 from sklearn.naive_bayes import GaussianNB
@@ -41,11 +41,11 @@
 except ImportError:
     pd = None
 
+
 ##########################################################################
 ## Fixtures
 ##########################################################################
 
-
 class FakeClassifier(BaseEstimator, ClassifierMixin):
     """
     A fake classifier for testing noops on the visualizer.
@@ -124,6 +124,29 @@ def test_binary_probability_decision(self):
         visualizer.finalize()
         self.assert_images_similar(visualizer, tol=0.1, windows_tol=10)
 
+    def test_binary_probability_decision_single_curve(self):
+        """
+        Test ROCAUC binary classifier with both decision & predict_proba with per_class=False
+        """
+        # Create and fit the visualizer
+        visualizer = ROCAUC(AdaBoostClassifier(), micro=False, macro=False, per_class=False)
+        visualizer.fit(self.binary.X.train, self.binary.y.train)
+
+        # Score the visualizer
+        s = visualizer.score(self.binary.X.test, self.binary.y.test)
+
+        # Test that score method successfully returns a value between 0 and 1
+        assert 0 <= s <= 1
+
+        # Check the scores
+        assert len(visualizer.fpr.keys()) == 1
+        assert len(visualizer.tpr.keys()) == 1
+        assert len(visualizer.roc_auc.keys()) == 1
+
+        # Compare the images
+        visualizer.finalize()
+        self.assert_images_similar(visualizer, tol=0.1, windows_tol=10)
+
     def test_binary_decision(self):
         """
         Test ROCAUC with a binary classifier with a decision_function
@@ -150,12 +173,38 @@ def test_binary_decision(self):
         visualizer.finalize()
         self.assert_images_similar(visualizer, tol=10)
 
+    def test_binary_decision_per_class(self):
+        """
+        Test ROCAUC with a binary classifier with a decision_function
+        """
+        # Create and fit the visualizer
+        visualizer = ROCAUC(
+            LinearSVC(random_state=42), micro=False, macro=False, per_class=True
+        )
+        visualizer.fit(self.binary.X.train, self.binary.y.train)
+
+        # Score the visualizer
+        s = visualizer.score(self.binary.X.test, self.binary.y.test)
+
+        # Test that score method successfully returns a value between 0 and 1
+        assert 0 <= s <= 1
+
+        # Check the scores
+        assert len(visualizer.fpr.keys()) == 2
+        assert len(visualizer.tpr.keys()) == 2
+        assert len(visualizer.roc_auc.keys()) == 2
+
+        # Compare the images
+        # NOTE: increased tolerance for both AppVeyor and Travis CI tests
+        visualizer.finalize()
+        self.assert_images_similar(visualizer, tol=10)
+
     def test_binary_micro_error(self):
         """
         Test ROCAUC to see if _binary_decision with micro = True raises an error
         """
         # Create visualizer with a linear model to force a binary decision
-        visualizer = ROCAUC(LinearSVC(random_state=42), micro=True)
+        visualizer = ROCAUC(LinearSVC(random_state=42), micro=True, per_class=False)
         visualizer.fit(self.binary.X.train, self.binary.y.train)
 
         # Ensure score raises error (micro curves aren't defined for binary decisions)
@@ -167,25 +216,13 @@ def test_binary_macro_error(self):
         Test ROCAUC to see if _binary_decision with macro = True raises an error
         """
         # Create visualizer with a linear model to force a binary decision
-        visualizer = ROCAUC(LinearSVC(random_state=42), macro=True)
+        visualizer = ROCAUC(LinearSVC(random_state=42), macro=True, per_class=False)
         visualizer.fit(self.binary.X.train, self.binary.y.train)
 
         # Ensure score raises error (macro curves aren't defined for binary decisions)
         with pytest.raises(ModelError):
             visualizer.score(self.binary.X.test, self.binary.y.test)
 
-    def test_binary_per_class_error(self):
-        """
-        Test ROCAUC to see if _binary_decision with per_class = True raises an error
-        """
-        # Create visualizer with a linear model to force a binary decision
-        visualizer = ROCAUC(LinearSVC(random_state=42), per_class=True)
-        visualizer.fit(self.binary.X.train, self.binary.y.train)
-
-        # Ensure score raises error (per_class curves not defined for binary decisions)
-        with pytest.raises(ModelError):
-            visualizer.score(self.binary.X.test, self.binary.y.test)
-
     def test_multiclass_rocauc(self):
         """
         Test ROCAUC with a multiclass classifier
@@ -207,6 +244,42 @@ def test_multiclass_rocauc(self):
         visualizer.finalize()
         self.assert_images_similar(visualizer, tol=0.1, windows_tol=10)
 
+    def test_rocauc_no_classes(self):
+        """
+        Test ROCAUC without per-class curves
+        """
+        # Create and fit the visualizer
+        visualizer = ROCAUC(GaussianNB(), per_class=False)
+        visualizer.fit(self.multiclass.X.train, self.multiclass.y.train)
+
+        # Score the visualizer (should be the micro average)
+        s = visualizer.score(self.multiclass.X.test, self.multiclass.y.test)
+        assert s == pytest.approx(0.77303, abs=1e-4)
+
+        # Assert that there still are per-class scores
+        for c in (0, 1):
+            assert c in visualizer.fpr
+            assert c in visualizer.tpr
+            assert c in visualizer.roc_auc
+
+        # Compare the images
+        visualizer.finalize()
+        self.assert_images_similar(visualizer, tol=0.1, windows_tol=10)
+
+    def test_rocauc_no_curves(self):
+        """
+        Test ROCAUC with no curves specified at all
+        """
+        # Create and fit the visualizer
+        visualizer = ROCAUC(
+            GaussianNB(), per_class=False, macro=False, micro=False
+        )
+        visualizer.fit(self.multiclass.X.train, self.multiclass.y.train)
+
+        # Attempt to score the visualizer
+        with pytest.raises(YellowbrickValueError, match="no curves will be drawn"):
+            visualizer.score(self.multiclass.X.test, self.multiclass.y.test)
+
     def test_rocauc_quickmethod(self):
         """
         Test the ROCAUC quick method
@@ -305,42 +378,6 @@ def test_rocauc_no_macro_no_micro(self):
         visualizer.finalize()
         self.assert_images_similar(visualizer, tol=0.1, windows_tol=10)
 
-    def test_rocauc_no_classes(self):
-        """
-        Test ROCAUC without per-class curves
-        """
-        # Create and fit the visualizer
-        visualizer = ROCAUC(LogisticRegression(), per_class=False)
-        visualizer.fit(self.binary.X.train, self.binary.y.train)
-
-        # Score the visualizer (should be the micro average)
-        s = visualizer.score(self.binary.X.test, self.binary.y.test)
-        assert s == pytest.approx(0.8661, abs=1e-4)
-
-        # Assert that there still are per-class scores
-        for c in (0, 1):
-            assert c in visualizer.fpr
-            assert c in visualizer.tpr
-            assert c in visualizer.roc_auc
-
-        # Compare the images
-        visualizer.finalize()
-        self.assert_images_similar(visualizer, tol=0.1, windows_tol=10)
-
-    def test_rocauc_no_curves(self):
-        """
-        Test ROCAUC with no curves specified at all
-        """
-        # Create and fit the visualizer
-        visualizer = ROCAUC(
-            LogisticRegression(), per_class=False, macro=False, micro=False
-        )
-        visualizer.fit(self.binary.X.train, self.binary.y.train)
-
-        # Attempt to score the visualizer
-        with pytest.raises(YellowbrickValueError, match="no curves will be drawn"):
-            visualizer.score(self.binary.X.test, self.binary.y.test)
-
     def test_rocauc_label_encoded(self):
         """
         Test ROCAUC with a target specifying a list of classes as strings
@@ -489,3 +526,17 @@ def test_with_fitted(self):
             oz = ROCAUC(model, classes=classes, is_fitted=False)
             oz.fit(X, y)
             mockfit.assert_called_once_with(X, y)
+
+    def test_binary_meta_param(self):
+        """
+        Test the binary meta param with ROCAUC
+        """
+        oz = ROCAUC(GaussianNB(), binary=False)
+        assert oz.micro is True
+        assert oz.macro is True
+        assert oz.per_class is True
+
+        oz = ROCAUC(GaussianNB(), binary=True)
+        assert oz.micro is False
+        assert oz.macro is False
+        assert oz.per_class is False