Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
Will you open a PR...coz I'm very eager to contribute.....

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
@PragyanTiwari this first needs extensive discussion via a SLEP, which @amueller will work on, and then we'll get to the implementation. This is also not a good first contribution candidate.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
> Will you open a PR...coz I'm very eager to contribute.....

@PragyanTiwari This is probably not the best issue to start with because we are going to need a scikit-learn enhancement proporal (SLEP) that will require much more thinking before to actually doing a PR that is going to be merged.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
> > I would say yes but it means that we need to properly define what is the scorer API. For instance, there already this difference between metric scorer and curve scorer (that does not exist yet but it probably should).

> Can you elaborate on that? I think I'm not familiar with the proposal.

When speaking about the curve scorer, I'm referring to `roc_curve` and `precision_recall_curve`. Basically, those metrics are returning 3 arrays. In some way, the PR linked to the `TunedThresholdClassifierCV` shown that you could abuse the `make_scorer` with `make_scorer(roc_curve, response_method="predict_proba")` that will return the scores and threshold. I assume that it make sense to have a "scorer" API in the sense of being able to call `curve_scorer(estimator, X, y)` that returns the info but it should not be allowed with `make_scorer` that should only accept metric that return a single value.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
Ok....thanks for the info

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
> @betatim I suggested `scoring` because that's the argument in `cross_validate` and `GridSearchCV` and they are probably that just because we had `scorer` before with a different API and we had to do a deprecation cycle?

Ah ok, well, then lets stick to it :D

> The problem on both accounts is that people don't do the second step of actually thinking about it.

Agreed. For me the question is if forcing people to think about it is the solution. My guess as to why people keep doing "crazy" things like accuracy for imbalanced problems is that somehow they have no idea and don't realise it and have no idea where to even start thinking about this. Which isn't a novel insight, so probably has been discussed to death. The philosophical question (for me) is if we actually do these kinds of people a service by forcing them to make a decision about something that they don't even know where to start reading/informing themselves about. I am sure people use tools that are easy to use and get started with. No matter how methodologically flawed they are. Which makes me think we need to maintain an easy on ramp and then find a way of educating people after the fact, somehow (the hard part). Otherwise they will leave and use `simple-scikit-learn` - the fork with all the wrong defaults :D



> I proposed two changes, adding they keyword arg, and removing the default. Just adding the keyword arg would at least smooth over the path from using accuracy to using something more relevant.

The keyword arg change is an easy yes for me. Removing the default is a more difficult/philosophical question for me on which I flipflop regarding "the thing we should do" :-/

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
The question on my mind is this: will `scoring` be able to take callable as an input (after `make_scorer` transformation)?

Because, IMO, this is crucial functionality, especially if we are talking about choosing the right metric for the task. A lot of real world problems require specific mathematical or business metrics, that users will need to code themselves.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
@glevv scoring will accept "scorer" as provided by `make_scorer`.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
@betatim 

> The philosophical question (for me) is if we actually do these kinds of people a service by forcing them to make a decision about something that they don't even know where to start reading/informing themselves about.

That's why to me it's crucial to have a page where people can easily understand what they should use. We should write that first. And then when they don't provide a scorer, we raise with an error which has a link to the page where they can read about it.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
Related discussion in #29065. I'm using the `scoring` parameter for `GridSearchCV`, but currently it only applies to `fit`, not to `score`. I would like to be able to compute the same metrics for both. Example code looks like:
```python
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV, train_test_split

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y)

estimator = LogisticRegression(n_jobs=-1)
param_grid = {"C": [10, 1.0, 0.1]}
score = ["accuracy", "balanced_accuracy", "precision_macro", "recall_macro", "f1_macro"]
grid_search = GridSearchCV(estimator, param_grid, scoring=score, refit="accuracy")
grid_search.fit(X_train, y_train)
grid_search.score(X_test, y_test)
```
At the moment, `score` only computes accuracy. If I want to compute the same metrics as fit, I have to replace the last line with:
```python
metrics = {}
for name, scorer in grid_search.scorer_.items():
    score = scorer(grid_search, X_test, y_test)
    metrics[name] = score
```
IMO, sklearn is all about syntactic sugar. The fact that I can perform model training, cross validation, hyperparameter tuning, and refitting on a single line is awesome. The fact that evaluating on test takes 4 lines is a bit of a surprise.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
In the case of https://github.com/scikit-learn/scikit-learn/issues/28995#issuecomment-2122718356, if we introduce `score(X, y, scoring=None)` and scoring is already set in `GridSearchCV`, then there is a valid default for grid_search.score`.

I'm liking the proposal more because it can also support multiple metrics, i.e.

```python
est.score(
    X, y,
    scoring=["accuracy", "balanced_accuracy", "precision_macro", "recall_macro", "f1_macro"],
)
```

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
Can I open a pr for this??

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
@pranav-bot no this is still under discussion, and a hard one to tackle. Please try some of more easier issues, or easy stalled PRs first.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
Meanwhile with #11430, we have at least some guidance in the docs on choosing a score(ing function).

I would love to move on with the proposal of this issue as the default accuracy for classifiers is the worst and is used by, e.g., LogisticRegressionCV.

Labels: New Feature, API, Needs Decision, RFC, module:metrics
Add "scoring" argument to estimator's ``score`` method

### Describe the workflow you want to enable

I want to enable non-accuracy metrics to ``estimator.score``, and ultimately deprecate the default values of ``accuracy`` and ``r2``. I would call it ``scoring`` though it's a bit redundant but consistent.
That would allow us to get rid of the default scoring methods, which are objectively bad and misleading, and it would require the minimum amount of code changes for anyone. 

### Describe your proposed solution

Replace

```python
est.score(X, y)
```
with
```python
est.score(X, y, scoring="accuracy")
```
or rather
```python
est.score(X, y, scoring="recall_macro")
```
(or ``r2`` for regression).

### Describe alternatives you've considered, if relevant

- Keep current status, which is bad (both ``accuracy`` and ``r2`` are bad)
- Remove ``scoring`` method.

I can't think of any other alternatives tbh.

### Additional context

I think in theory this requires a slep, as it's changing shared API, right?

COMMENT:
Maybe, let us list the possibilities we have for the goal of getting rid of the default accuracy with `classifier.score`):
1. Add a new parameter, e.g. `scoring`, to the `estimator.score()` method, as @amueller suggested [here](https://github.com/scikit-learn/scikit-learn/issues/28995#issue-2290350665).
  Start deprecating the default `scoring="accuracy"`.
2. Deprecate `etimator.score` and remove it as @adrinjalali suggested [here](https://github.com/scikit-learn/scikit-learn/issues/28995#issuecomment-2107091257).

Any other basic alternative?

Labels: New Feature, API, Needs Decision, RFC, module:metrics