ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].

### Describe the bug

I will be succinct. I am training a binary classification dataset on "rain" or "not rain". This is a binary target. Yet scikit-learn throws an error stating that it's not binary. Is this expected behavior / what am I missing?

![samp](https://github.com/user-attachments/assets/f34d58e3-30e7-4c94-aea9-51325fbf76dc)


### Steps/Code to Reproduce

any dataset with a binary target variable

### Expected Results

No error is thrown

### Actual Results

```

0%|                                                    | 0/6 [00:28<?, ?it/s]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[75], line 1
----> 1 final_dct = model_selection_kfold(model_lst, rain_ml_fin, "RainTomorrow_Yes")

Cell In[74], line 32, in model_selection_kfold(models, df, dep_var)
     29 scores_dct[str(model)]["model"] = model        
     30 scores_dct[str(model)]["preds"] = preds        
     31 scores_dct[model] = {
---> 32     'precision':metrics.precision_score(preds, y_test), 
     33     'recall':metrics.recall_score(preds, y_test), 
     34     'accuracy':metrics.accuracy_score(preds, y_test), 
     35     'f1':metrics.f1_score(preds, y_test),
     36     'train':clf.score(x_train, y_train),
     37     'test':clf.score(x_test, y_test),
     38     'cv':cv_score
     39 }
     41 print('\n')
     42 print('The model ', model, 'had the following Classification Report')

File [~\Languages\Lib\site-packages\sklearn\utils\_param_validation.py:213](http://localhost:8888/lab/tree/rainfall_aus/~/Languages/Lib/site-packages/sklearn/utils/_param_validation.py#line=212), in validate_params.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    207 try:
    208     with config_context(
    209         skip_parameter_validation=(
    210             prefer_skip_nested_validation or global_skip_validation
    211         )
    212     ):
--> 213         return func(*args, **kwargs)
    214 except InvalidParameterError as e:
    215     # When the function is just a wrapper around an estimator, we allow
    216     # the function to delegate validation to the estimator, but we replace
    217     # the name of the estimator by the name of the function in the error
    218     # message to avoid confusion.
    219     msg = re.sub(
    220         r"parameter of \w+ must be",
    221         f"parameter of {func.__qualname__} must be",
    222         str(e),
    223     )

File [~\Languages\Lib\site-packages\sklearn\metrics\_classification.py:2204](http://localhost:8888/lab/tree/rainfall_aus/~/Languages/Lib/site-packages/sklearn/metrics/_classification.py#line=2203), in precision_score(y_true, y_pred, labels, pos_label, average, sample_weight, zero_division)
   2037 @validate_params(
   2038     {
   2039         "y_true": ["array-like", "sparse matrix"],
   (...)
   2064     zero_division="warn",
   2065 ):
   2066     """Compute the precision.
   2067 
   2068     The precision is the ratio ``tp / (tp + fp)`` where ``tp`` is the number of
   (...)
   2202     array([0.5, 1. , 1. ])
   2203     """
-> 2204     p, _, _, _ = precision_recall_fscore_support(
   2205         y_true,
   2206         y_pred,
   2207         labels=labels,
   2208         pos_label=pos_label,
   2209         average=average,
   2210         warn_for=("precision",),
   2211         sample_weight=sample_weight,
   2212         zero_division=zero_division,
   2213     )
   2214     return p

File [~\Languages\Lib\site-packages\sklearn\utils\_param_validation.py:186](http://localhost:8888/lab/tree/rainfall_aus/~/Languages/Lib/site-packages/sklearn/utils/_param_validation.py#line=185), in validate_params.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    184 global_skip_validation = get_config()["skip_parameter_validation"]
    185 if global_skip_validation:
--> 186     return func(*args, **kwargs)
    188 func_sig = signature(func)
    190 # Map *args/**kwargs to the function signature

File [~\Languages\Lib\site-packages\sklearn\metrics\_classification.py:1789](http://localhost:8888/lab/tree/rainfall_aus/~/Languages/Lib/site-packages/sklearn/metrics/_classification.py#line=1788), in precision_recall_fscore_support(y_true, y_pred, beta, labels, pos_label, average, warn_for, sample_weight, zero_division)
   1626 """Compute precision, recall, F-measure and support for each class.
   1627 
   1628 The precision is the ratio ``tp / (tp + fp)`` where ``tp`` is the number of
   (...)
   1786  array([2, 2, 2]))
   1787 """
   1788 _check_zero_division(zero_division)
-> 1789 labels = _check_set_wise_labels(y_true, y_pred, average, labels, pos_label)
   1791 # Calculate tp_sum, pred_sum, true_sum ###
   1792 samplewise = average == "samples"

File [~\Languages\Lib\site-packages\sklearn\metrics\_classification.py:1578](http://localhost:8888/lab/tree/rainfall_aus/~/Languages/Lib/site-packages/sklearn/metrics/_classification.py#line=1577), in _check_set_wise_labels(y_true, y_pred, average, labels, pos_label)
   1576         if y_type == "multiclass":
   1577             average_options.remove("samples")
-> 1578         raise ValueError(
   1579             "Target is %s but average='binary'. Please "
   1580             "choose another average setting, one of %r." % (y_type, average_options)
   1581         )
   1582 elif pos_label not in (None, 1):
   1583     warnings.warn(
   1584         "Note that pos_label (set to %r) is ignored when "
   1585         "average != 'binary' (got %r). You may use "
   (...)
   1588         UserWarning,
   1589     )

ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].
```

### Versions

```shell
System:
    python: 3.12.6 (tags/v3.12.6:a4a2d2b, Sep  6 2024, 20:11:23) [MSC v.1940 64 bit (AMD64)]
executable: C:\Users\benno\Languages\python.exe
   machine: Windows-10-10.0.19045-SP0

Python dependencies:
      sklearn: 1.5.2
          pip: 24.2
   setuptools: 75.1.0
        numpy: 2.0.2
        scipy: 1.14.1
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.9.2
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 4
         prefix: libscipy_openblas
       filepath: C:\Users\benno\Languages\Lib\site-packages\numpy.libs\libscipy_openblas64_-caad452230ae4ddb57899b8b3a33c55c.dll
        version: 0.3.27
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 4
         prefix: vcomp
       filepath: C:\Users\benno\Languages\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 4
         prefix: libscipy_openblas
       filepath: C:\Users\benno\Languages\Lib\site-packages\scipy.libs\libscipy_openblas-5b1ec8b915dfb81d11cebc0788069d2d.dll
        version: 0.3.27.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
Yes, I'm quite certain you can use accuracy for binary classification. Consider this code:
  ```
'precision':metrics.precision_score(preds, y_test, average="weighted"), 
  'recall':metrics.recall_score(preds, y_test, average="weighted"), 
  'accuracy':metrics.accuracy_score(preds, y_test), 
  'f1':metrics.f1_score(preds, y_test, average="weighted"),
  'train':clf.score(x_train, y_train),
  'test':clf.score(x_test, y_test),
  'cv':cv_score
```
the metrics with "average" : "weighted" were the metrics that would cause an error, if averaging type was not specified.

Moreover, "accuracy" runs fine in this implementation. It is simply correct divided by total (arguably more relevant than scores like f1, which are highly uninterpretable.)

Could you clarify what you mean?

Labels: Bug, Needs Triage