Repository: scikit-learn/scikit-learn
Pull request: ENH improve error message when y_true does not provide all known classes
closes #21568 

Improve the error message raised in `top_k_accuracy_score` by stipulating to assign the known classes to labels when `y_true` does not contain all potential labels.

Problem statement:
top_k_accuracy_score fails when y_true contains labels not in labels
### Describe the bug

If, during train/test split, one of the classes in a multi-label problem is omitted from the training set but *is* included in the testing set, "top_k_accuracy_score" fails instead of just returning 0 for that particularly score. 

### Steps/Code to Reproduce

```
X_train = [[0, 0, 0, 1], 
            [1, 0, 0, 0], 
          [1, 0, 0, 0], 
          [0, 1, 0, 0], 
          [1, 0, 0, 0], 
          [1, 0, 0, 0], 
          [0, 1, 0, 0]]

y_train = [4, 1, 1, 2, 1, 1, 2]


X_test = [[0, 0, 1, 0], 
         [1, 0, 0, 0], 
         [0, 1, 0, 0], 
         [0, 0, 0, 1], 
         [1, 0, 0, 0]]

y_test = [3, 1, 2, 4, 1]

log = LogisticRegression(fit_intercept=True)
fitted_log = log.fit(X_train, y_train)
print(top_k_accuracy_score(y_test, fitted_log.predict_proba(X_test), k=2)) 
```


### Expected Results

"top_k_accuracy_score" effectively functions by looking at the top k probabilities by rank and checks to see if a given y is contained within those top k; when a given class isn't in those top K items (as would be expected if the y_test contains a class that y_train did not contain), one would usually just expect a "0" for the return on the failing check. 

### Actual Results

Instead of just returning a 0, it seems to raise an error that prevents the code from running at all. 

### Versions

System:
    python: 3.7.5 (default, Feb 23 2021, 13:22:40)  [GCC 8.4.0]
executable: /usr/bin/python
   machine: Linux-4.19.0-17-cloud-amd64-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.3.1
   setuptools: 57.4.0
      sklearn: 1.0.1
        numpy: 1.21.2
        scipy: 1.7.1
       Cython: None
       pandas: 1.2.5
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.0.0

Built with OpenMP: True

Hints from issue discussion:
Thanks for the report @meesatsquare ! I think it's an instance of https://github.com/scikit-learn/scikit-learn/issues/21250 and there is a proposed documentation improvement in https://github.com/scikit-learn/scikit-learn/pull/21255

> "top_k_accuracy_score" effectively functions by looking at the top k probabilities by rank and checks to see if a given y is contained within those top k; when a given class isn't in those top K items (as would be expected if the y_test contains a class that y_train did not contain), one would usually just expect a "0" for the return on the failing check.

The proposed solutions sounds reasonable to me, but I have not looked in the code how managable it would be to implement. Would you be interested in investigating? 
To be honest, this is something that would happen in other metrics and we rely on our users to provide the labels in `labels`. I would prefer to not change this behaviour until the error message is informative enough and provide the solution to solve it.

Commit message: DOC improve error message when y_true does not provide all known classes

Patch to review:
diff --git a/sklearn/metrics/_ranking.py b/sklearn/metrics/_ranking.py
index 0707c8d2a951d..6c8593a2da5ce 100644
--- a/sklearn/metrics/_ranking.py
+++ b/sklearn/metrics/_ranking.py
@@ -1719,6 +1719,8 @@ def top_k_accuracy_score(
             raise ValueError(
                 f"Number of classes in 'y_true' ({n_classes}) not equal "
                 f"to the number of classes in 'y_score' ({y_score_n_classes})."
+                "You can provide a list of all known classes by assigning it "
+                "to the parameter `labels`."
             )
     else:
         labels = column_or_1d(labels)


Code review comments:
diff_hunk: @@ -1719,6 +1719,8 @@ def top_k_accuracy_score(
             raise ValueError(
                 f"Number of classes in 'y_true' ({n_classes}) not equal "
                 f"to the number of classes in 'y_score' ({y_score_n_classes})."
+                "You can provide a list of all known classes by assigning it "
+                "to the parameter `labels`."
path: sklearn/metrics/_ranking.py
text: @user1:
```suggestion
                "You can provide a list of all known classes by assigning it "
                "to the `labels` parameter."
```