Repository: probabl-ai/skore

Issue:
enh: transpose the output of `report_metrics`
As discussed in https://github.com/probabl-ai/skore/issues/1260, it would make sense to transpose rows and columns in the `.report_metrics`

Discussion:

Patch:
diff --git a/.github/workflows/backend.yml b/.github/workflows/backend.yml
index 67751a7..9ca1957 100644
--- a/.github/workflows/backend.yml
+++ b/.github/workflows/backend.yml
@@ -86,8 +86,10 @@ jobs:
 
       - name: Setup Python
         uses: actions/setup-python@v5
+        id: setup-python
         with:
           python-version: ${{ matrix.python }}
+          check-latest: True
           cache: pip
 
       - name: Restore python-venv
@@ -98,7 +100,7 @@ jobs:
           key: >-
             python-venv
             -${{ matrix.os }}
-            -${{ matrix.python }}
+            -${{ steps.setup-python.outputs.python-version }}
             -${{ hashFiles(format('skore/ci/requirements/python-{0}/scikit-learn-{1}/test-requirements.txt', matrix.python, matrix.scikit-learn)) }}
 
       - name: Setup python-venv
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 86ef2a2..b10b5c7 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -9,6 +9,26 @@ everyone and appreciate you taking the time to get involved.
 
 This project is hosted on https://github.com/probabl-ai/skore.
 
+.. topic:: **Our community, our values**
+
+    We are a community based on openness and friendly, didactic
+    discussions.
+
+    We aspire to treat everybody equally, and value their contributions.  We
+    are particularly seeking people from underrepresented backgrounds in Open
+    Source Software to participate and contribute their expertise and experience.
+
+    Decisions are made based on technical merit and consensus.
+
+    Code is not the only way to help the project. Reviewing pull
+    requests, answering questions to help others on mailing lists or
+    issues, organizing and teaching tutorials, working on the website,
+    improving the documentation, are all priceless contributions.
+
+    We abide by the principles of openness, respect, and consideration of
+    others of the Python Software Foundation:
+    https://www.python.org/psf/codeofconduct/
+
 Below are some guidelines to help you get started.
 
 Questions, bugs and feature requests
@@ -85,6 +105,8 @@ Then, to use the skore-ui
     make build-skore-ui
     make serve-skore-ui
 
+Do not forget to do a hard refresh when changing the front-end (meta+shift+R) if the expected changes do not appear.
+
 PR format
 ---------
 
@@ -95,6 +117,38 @@ Generally the description of a commit should start with a verb in the imperative
 
 Examples of correct PR titles: ``docs: Update the docstrings`` or ``feat: Remove CrossValidationAggregationItem.``
 
+Tests
+-----
+
+To run the tests locally, you may run
+
+.. code-block:: bash
+
+    make test
+
+To run only back-end tests, use:
+
+.. code-block:: bash
+
+    make test-backend
+
+To run only front-end tests, use:
+
+.. code-block:: bash
+
+    make test-frontend
+
+
+Linting
+-------
+
+We use the linter ruff to make sure that the code is formatted correctly.
+
+.. code-block:: bash
+
+    make lint
+
+
 Documentation
 =============
 
@@ -103,10 +157,6 @@ Setup
 
 Our documentation uses `PyData Sphinx Theme <https://pydata-sphinx-theme.readthedocs.io/>`_.
 
-.. warning::
-
-    Modifications are to be done in the ``sphinx`` folder. The ``docs`` folder must *not* be touched!
-
 To build the docs:
 
 .. code-block:: bash
@@ -120,6 +170,8 @@ Then, you can access the local build via:
 
     open build/html/index.html
 
+The PR will also build the documentation and a bot will automatically add a comment with a link to the documentation preview to easily check the results.
+
 Contributing to the docstrings
 ------------------------------
 
@@ -132,3 +184,20 @@ When writing documentation, whether it be online, docstrings or help messages in
 #. Text is written in US english ("visualize" rather than "visualise")
 #. In the CLI, positional arguments are written in snake case (``snake_case``), keyword arguments in kebab case (``kebab-case``)
 #. When there is a default argument, it should be shown in the help message, typically with ``(default: <default value>)`` at the end of the message
+
+
+Contributing to the examples
+----------------------------
+
+The examples are stored in the folder called `examples`. They are classified in subcategories.
+They should be written in a python file, with cells marked by `# %%`. They will be automatically converted to rst files in the subfolder `sphinx/auto_examples`. This subfolder is listed in the gitignore, and cannot be pushed.
+
+The python file should start by a docstring indicating the example name and the title.
+
+No example should require to have large files in this repository. For example, no dataset should be stored, it should be downloaded in the script.
+
+Contributing to the ReadMe
+--------------------------
+
+The README.md file can be modified and is part of the documentation.
+This file is used to be presented on [PyPI](https://pypi.org/project/skore/#description).
diff --git a/examples/model_evaluation/plot_estimator_report.py b/examples/model_evaluation/plot_estimator_report.py
index 22c4f34..94e5bac 100644
--- a/examples/model_evaluation/plot_estimator_report.py
+++ b/examples/model_evaluation/plot_estimator_report.py
@@ -137,7 +137,7 @@ print(f"Time taken to compute the metrics: {end - start:.2f} seconds")
 # pandas.
 import matplotlib.pyplot as plt
 
-ax = metric_report.T.plot.barh()
+ax = metric_report.plot.barh()
 ax.set_title("Metrics report")
 plt.tight_layout()
 
diff --git a/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py b/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py
index 0257e31..d52c569 100644
--- a/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py
+++ b/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py
@@ -97,9 +97,11 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> report.metrics.report_metrics(
         ...     scoring=["precision", "recall"], pos_label=1, aggregate=["mean", "std"]
         ... )
-        Metric                   Precision (↗︎)  Recall (↗︎)
-        LogisticRegression mean        0.94...     0.96...
-                           std         0.02...     0.02...
+                    LogisticRegression
+                                    mean       std
+        Metric
+        Precision (↗︎)            0.94...  0.024...
+        Recall (↗︎)               0.96...  0.027...
         """
         return self._compute_metric_scores(
             report_metric_name="report_metrics",
@@ -161,15 +163,17 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
 
             results = pd.concat(
                 results,
-                axis=0,
+                axis=1,
                 keys=[f"Split #{i}" for i in range(len(results))],
             )
-            results = results.swaplevel(0, 1)
+            results = results.swaplevel(0, 1, axis=1)
             if aggregate:
                 if isinstance(aggregate, str):
                     aggregate = [aggregate]
-                results = results.aggregate(func=aggregate, axis=0)
-                results = pd.concat([results], keys=[self._parent.estimator_name_])
+                results = results.aggregate(func=aggregate, axis=1)
+                results = pd.concat(
+                    [results], keys=[self._parent.estimator_name_], axis=1
+                )
 
             self._parent._cache[cache_key] = results
         return results
@@ -207,9 +211,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> classifier = LogisticRegression(max_iter=10_000)
         >>> report = CrossValidationReport(classifier, X=X, y=y, cv_splitter=2)
         >>> report.metrics.accuracy()
-        Metric                       Accuracy (↗︎)
-        LogisticRegression Split #0       0.94...
-                           Split #1       0.94...
+                    LogisticRegression
+                                Split #0  Split #1
+        Metric
+        Accuracy (↗︎)            0.94...   0.94...
         """
         return self.report_metrics(
             scoring=["accuracy"],
@@ -285,10 +290,11 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> classifier = LogisticRegression(max_iter=10_000)
         >>> report = CrossValidationReport(classifier, X=X, y=y, cv_splitter=2)
         >>> report.metrics.precision()
-        Metric                      Precision (↗︎)
-        Label / Average                          0         1
-        LogisticRegression Split #0       0.96...   0.93...
-                           Split #1       0.90...   0.96...
+                                    LogisticRegression
+                                                Split #0  Split #1
+        Metric         Label / Average
+        Precision (↗︎) 0                         0.96...   0.90...
+                      1                         0.93...   0.96...
         """
         return self.report_metrics(
             scoring=["precision"],
@@ -367,10 +373,11 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> classifier = LogisticRegression(max_iter=10_000)
         >>> report = CrossValidationReport(classifier, X=X, y=y, cv_splitter=2)
         >>> report.metrics.recall()
-        Metric                      Recall (↗︎)
-        Label / Average                          0         1
-        LogisticRegression Split #0    0.87...   0.98...
-                           Split #1    0.94...   0.94...
+                                    LogisticRegression
+                                            Split #0  Split #1
+        Metric      Label / Average
+        Recall (↗︎) 0                         0.87...  0.94...
+                   1                         0.98...  0.94...
         """
         return self.report_metrics(
             scoring=["recall"],
@@ -411,9 +418,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> classifier = LogisticRegression(max_iter=10_000)
         >>> report = CrossValidationReport(classifier, X=X, y=y, cv_splitter=2)
         >>> report.metrics.brier_score()
-        Metric                       Brier score (↘︎)
-        LogisticRegression Split #0          0.04...
-                           Split #1          0.04...
+                        LogisticRegression
+                                Split #0  Split #1
+        Metric
+        Brier score (↘︎)         0.04...   0.04...
         """
         return self.report_metrics(
             scoring=["brier_score"],
@@ -494,9 +502,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> classifier = LogisticRegression(max_iter=10_000)
         >>> report = CrossValidationReport(classifier, X=X, y=y, cv_splitter=2)
         >>> report.metrics.roc_auc()
-        Metric                       ROC AUC (↗︎)
-        LogisticRegression Split #0      0.99...
-                           Split #1      0.98...
+                    LogisticRegression
+                            Split #0  Split #1
+        Metric
+        ROC AUC (↗︎)         0.99...   0.98...
         """
         return self.report_metrics(
             scoring=["roc_auc"],
@@ -538,9 +547,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> classifier = LogisticRegression(max_iter=10_000)
         >>> report = CrossValidationReport(classifier, X=X, y=y, cv_splitter=2)
         >>> report.metrics.log_loss()
-        Metric                       Log loss (↘︎)
-        LogisticRegression Split #0       0.1...
-                           Split #1       0.1...
+                    LogisticRegression
+                                Split #0  Split #1
+        Metric
+        Log loss (↘︎)           0.1...     0.1...
         """
         return self.report_metrics(
             scoring=["log_loss"],
@@ -593,9 +603,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> regressor = Ridge()
         >>> report = CrossValidationReport(regressor, X=X, y=y, cv_splitter=2)
         >>> report.metrics.r2()
-        Metric           R² (↗︎)
-        Ridge Split #0  0.36...
-              Split #1  0.39...
+                    Ridge
+                Split #0  Split #1
+        Metric
+        R² (↗︎)  0.36...   0.39...
         """
         return self.report_metrics(
             scoring=["r2"],
@@ -649,9 +660,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         >>> regressor = Ridge()
         >>> report = CrossValidationReport(regressor, X=X, y=y, cv_splitter=2)
         >>> report.metrics.rmse()
-        Metric          RMSE (↘︎)
-        Ridge Split #0  59.9...
-              Split #1  61.4...
+                    Ridge
+                    Split #0   Split #1
+        Metric
+        RMSE (↘︎)    59.9...    61.4...
         """
         return self.report_metrics(
             scoring=["rmse"],
@@ -727,9 +739,10 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         ...     response_method="predict",
         ...     metric_name="MAE (↗︎)",
         ... )
-        Metric           MAE (↗︎)
-        Ridge Split #0  50.1...
-              Split #1  52.6...
+                    Ridge
+                Split #0   Split #1
+        Metric
+        MAE (↗︎) 50.1...   52.6...
         """
         # create a scorer with `greater_is_better=True` to not alter the output of
         # `metric_function`
diff --git a/skore/src/skore/sklearn/_estimator/metrics_accessor.py b/skore/src/skore/sklearn/_estimator/metrics_accessor.py
index d460385..95795c8 100644
--- a/skore/src/skore/sklearn/_estimator/metrics_accessor.py
+++ b/skore/src/skore/sklearn/_estimator/metrics_accessor.py
@@ -112,8 +112,12 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         ...     y_test=y_test,
         ... )
         >>> report.metrics.report_metrics(pos_label=1)
-        Metric              Precision (↗︎)  Recall (↗︎)  ROC AUC (↗︎)  Brier score (↘︎)
-        LogisticRegression        0.98...     0.93...      0.99...          0.03...
+                        LogisticRegression
+        Metric
+        Precision (↗︎)              0.98...
+        Recall (↗︎)                 0.93...
+        ROC AUC (↗︎)                0.99...
+        Brier score (↘︎)            0.03...
         """
         if data_source == "X_y":
             # optimization of the hash computation to avoid recomputing it
@@ -242,89 +246,87 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
             if self._parent._ml_task == "binary-classification":
                 if isinstance(score, dict):
                     classes = list(score.keys())
-                    columns = pd.MultiIndex.from_arrays(
+                    index = pd.MultiIndex.from_arrays(
                         [[metric_name] * len(classes), classes],
                         names=["Metric", "Label / Average"],
                     )
-                    score = np.hstack([score[c] for c in classes]).reshape(1, -1)
+                    score = np.hstack([score[c] for c in classes]).reshape(-1, 1)
                 elif "average" in metrics_kwargs:
                     if metrics_kwargs["average"] == "binary":
-                        columns = pd.MultiIndex.from_arrays(
+                        index = pd.MultiIndex.from_arrays(
                             [[metric_name], [pos_label]],
                             names=["Metric", "Label / Average"],
                         )
                     elif metrics_kwargs["average"] is not None:
-                        columns = pd.MultiIndex.from_arrays(
+                        index = pd.MultiIndex.from_arrays(
                             [[metric_name], [metrics_kwargs["average"]]],
                             names=["Metric", "Label / Average"],
                         )
                     else:
-                        columns = pd.Index([metric_name], name="Metric")
-                    score = np.array(score).reshape(1, -1)
+                        index = pd.Index([metric_name], name="Metric")
+                    score = np.array(score).reshape(-1, 1)
                 else:
-                    columns = pd.Index([metric_name], name="Metric")
-                    score = np.array(score).reshape(1, -1)
+                    index = pd.Index([metric_name], name="Metric")
+                    score = np.array(score).reshape(-1, 1)
             elif self._parent._ml_task == "multiclass-classification":
                 if isinstance(score, dict):
                     classes = list(score.keys())
-                    columns = pd.MultiIndex.from_arrays(
+                    index = pd.MultiIndex.from_arrays(
                         [[metric_name] * len(classes), classes],
                         names=["Metric", "Label / Average"],
                     )
-                    score = np.hstack([score[c] for c in classes]).reshape(1, -1)
+                    score = np.hstack([score[c] for c in classes]).reshape(-1, 1)
                 elif (
                     "average" in metrics_kwargs
                     and metrics_kwargs["average"] is not None
                 ):
-                    columns = pd.MultiIndex.from_arrays(
+                    index = pd.MultiIndex.from_arrays(
                         [[metric_name], [metrics_kwargs["average"]]],
                         names=["Metric", "Label / Average"],
                     )
-                    score = np.array(score).reshape(1, -1)
+                    score = np.array(score).reshape(-1, 1)
                 else:
-                    columns = pd.Index([metric_name], name="Metric")
-                    score = np.array(score).reshape(1, -1)
+                    index = pd.Index([metric_name], name="Metric")
+                    score = np.array(score).reshape(-1, 1)
             elif self._parent._ml_task == "regression":
                 if isinstance(score, np.ndarray):
-                    columns = pd.MultiIndex.from_arrays(
+                    index = pd.MultiIndex.from_arrays(
                         [[metric_name] * len(score), list(range(len(score)))],
                         names=["Metric", "Output"],
                     )
-                    score = score.reshape(1, -1)
+                    score = score.reshape(-1, 1)
                 else:
-                    columns = pd.Index([metric_name], name="Metric")
-                    score = np.array(score).reshape(1, -1)
+                    index = pd.Index([metric_name], name="Metric")
+                    score = np.array(score).reshape(-1, 1)
             else:  # unknown task - try our best
-                columns = [metric_name] if len(score) == 1 else None
+                index = [metric_name] if len(score) == 1 else None
 
             scores.append(
-                pd.DataFrame(
-                    score, columns=columns, index=[self._parent.estimator_name_]
-                )
+                pd.DataFrame(score, index=index, columns=[self._parent.estimator_name_])
             )
 
         has_multilevel = any(
-            isinstance(score, pd.DataFrame) and isinstance(score.columns, pd.MultiIndex)
+            isinstance(score, pd.DataFrame) and isinstance(score.index, pd.MultiIndex)
             for score in scores
         )
 
         if has_multilevel:
             # Convert single-level dataframes to multi-level
             for i, score in enumerate(scores):
-                if hasattr(score, "columns") and not isinstance(
-                    score.columns, pd.MultiIndex
+                if hasattr(score, "index") and not isinstance(
+                    score.index, pd.MultiIndex
                 ):
                     if self._parent._ml_task == "regression":
                         name_index = ["Metric", "Output"]
                     else:
                         name_index = ["Metric", "Label / Average"]
 
-                    scores[i].columns = pd.MultiIndex.from_tuples(
-                        [(col, "") for col in score.columns],
+                    scores[i].index = pd.MultiIndex.from_tuples(
+                        [(idx, "") for idx in score.index],
                         names=name_index,
                     )
 
-        return pd.concat(scores, axis=1)
+        return pd.concat(scores, axis=0)
 
     def _compute_metric_scores(
         self,
diff --git a/skore/src/skore/sklearn/find_ml_task.py b/skore/src/skore/sklearn/find_ml_task.py
index d311219..bc07b5b 100644
--- a/skore/src/skore/sklearn/find_ml_task.py
+++ b/skore/src/skore/sklearn/find_ml_task.py
@@ -15,6 +15,17 @@ def _is_sequential(y) -> bool:
     return np.array_equal(y_values, sequential)
 
 
+def _is_classification(y) -> bool:
+    """Determine if `y` is a target for a classification task.
+
+    If `y` contains integers, sklearn's `type_of_target` considers
+    the task to be multiclass classification.
+    This function makes the analysis finer.
+    """
+    y = y.flatten()
+    return _is_sequential(y) and 0 in y
+
+
 def _find_ml_task(y, estimator=None) -> MLTask:
     """Guess the ML task being addressed based on a target array and an estimator.
 
@@ -53,6 +64,26 @@ def _find_ml_task(y, estimator=None) -> MLTask:
     # Discrete sequential values, not containing 0
     >>> _find_ml_task(numpy.array([1, 3, 2]))
     'regression'
+
+    # 2 values, containing 0, in a 2d array
+    >>> _find_ml_task(numpy.array([[0, 1], [1, 1]]))
+    'multioutput-binary-classification'
+
+    # Discrete sequential values, containing 0, in a 2d array
+    >>> _find_ml_task(numpy.array([[0, 1, 2], [2, 1, 1]]))
+    'multioutput-multiclass-classification'
+
+    # Discrete values, not sequential, in a 2d array
+    >>> _find_ml_task(numpy.array([[1, 5], [5, 9]]))
+    'multioutput-regression'
+
+    # Discrete values, not sequential, containing 0, in a 2d array
+    >>> _find_ml_task(numpy.array([[0, 1, 5, 9], [1, 0, 1, 1]]))
+    'multioutput-regression'
+
+    # Discrete sequential values, not containing 0, in a 2d array
+    >>> _find_ml_task(numpy.array([[1, 3, 2], [2, 1, 1]]))
+    'multioutput-regression'
     """
     if estimator is not None:
         # checking the estimator is more robust and faster than checking the type of
@@ -71,38 +102,34 @@ def _find_ml_task(y, estimator=None) -> MLTask:
                         return "binary-classification"
                     if estimator.classes_.size > 2:
                         return "multiclass-classification"
-            else:  # fallback on the target
+            else:
+                # fallback on the target
                 if y is None:
                     return "unknown"
 
-                target_type = type_of_target(y)
-                if target_type == "binary":
-                    return "binary-classification"
-                if target_type == "multiclass":
-                    # If y is a vector of integers, type_of_target considers
-                    # the task to be multiclass-classification.
-                    # We refine this analysis a bit here.
-                    if _is_sequential(y) and 0 in y:
-                        return "multiclass-classification"
-                    return "regression"
-            return "unknown"
-        return "unknown"
-    else:
-        if y is None:
-            # NOTE: The task might not be clustering
-            return "clustering"
-
-        target_type = type_of_target(y)
-
-        if target_type == "continuous":
-            return "regression"
-        if target_type == "binary":
-            return "binary-classification"
-        if target_type == "multiclass":
-            # If y is a vector of integers, type_of_target considers
-            # the task to be multiclass-classification.
-            # We refine this analysis a bit here.
-            if _is_sequential(y) and 0 in y:
-                return "multiclass-classification"
-            return "regression"
-        return "unknown"
+    # fallback on the target
+    if y is None:
+        # NOTE: The task might not be clustering
+        return "clustering"
+
+    target_type = type_of_target(y)
+
+    if target_type == "continuous":
+        return "regression"
+    if target_type == "continuous-multioutput":
+        return "multioutput-regression"
+    if target_type == "binary":
+        return "binary-classification"
+    if target_type == "multiclass":
+        if _is_classification(y):
+            return "multiclass-classification"
+        return "regression"
+    if target_type == "multiclass-multioutput":
+        if _is_classification(y):
+            return "multioutput-multiclass-classification"
+        return "multioutput-regression"
+    if target_type == "multilabel-indicator":
+        if _is_classification(y):
+            return "multioutput-binary-classification"
+        return "multioutput-regression"
+    return "unknown"
diff --git a/skore/src/skore/sklearn/types.py b/skore/src/skore/sklearn/types.py
index 3ac85af..7c58a50 100644
--- a/skore/src/skore/sklearn/types.py
+++ b/skore/src/skore/sklearn/types.py
@@ -4,8 +4,11 @@ from typing import Literal
 
 MLTask = Literal[
     "binary-classification",
+    "clustering",
     "multiclass-classification",
+    "multioutput-binary-classification",
+    "multioutput-multiclass-classification",
+    "multioutput-regression",
     "regression",
-    "clustering",
     "unknown",
 ]
diff --git a/sphinx/contributing_link.rst b/sphinx/contributing.rst
similarity index 100%
rename from sphinx/contributing_link.rst
rename to sphinx/contributing.rst
diff --git a/sphinx/index.rst b/sphinx/index.rst
index 61a5c8d..6fac154 100644
--- a/sphinx/index.rst
+++ b/sphinx/index.rst
@@ -59,4 +59,4 @@ Skore is just at the beginning of its journey, but we’re shipping fast! Freque
    install
    auto_examples/index
    api
-   contributing_link
\ No newline at end of file
+   contributing
\ No newline at end of file


Test patch:
diff --git a/skore/tests/unit/sklearn/test_cross_validation.py b/skore/tests/unit/sklearn/test_cross_validation.py
index 255b36c..b1c91ce 100644
--- a/skore/tests/unit/sklearn/test_cross_validation.py
+++ b/skore/tests/unit/sklearn/test_cross_validation.py
@@ -277,11 +277,11 @@ def test_cross_validation_report_metrics_repr(binary_classification_data):
     assert "report.metrics.help()" in repr_str
 
 
-def _normalize_metric_name(column):
-    """Helper to normalize the metric name present in a pandas column that could be
+def _normalize_metric_name(index):
+    """Helper to normalize the metric name present in a pandas index that could be
     a multi-index or single-index."""
     # if we have a multi-index, then the metric name is on level 0
-    s = column[0] if isinstance(column, tuple) else column
+    s = index[0] if isinstance(index, tuple) else index
     # Remove spaces and underscores
     return re.sub(r"[^a-zA-Z]", "", s.lower())
 
@@ -290,13 +290,13 @@ def _check_results_single_metric(report, metric, expected_n_splits, expected_nb_
     assert hasattr(report.metrics, metric)
     result = getattr(report.metrics, metric)()
     assert isinstance(result, pd.DataFrame)
-    assert result.shape[0] == expected_n_splits
+    assert result.shape[1] == expected_n_splits
     # check that we hit the cache
     result_with_cache = getattr(report.metrics, metric)()
     pd.testing.assert_frame_equal(result, result_with_cache)
 
-    # check that the index contains the expected split names
-    split_names = result.index.get_level_values(1).unique()
+    # check that the columns contains the expected split names
+    split_names = result.columns.get_level_values(1).unique()
     expected_split_names = [f"Split #{i}" for i in range(expected_n_splits)]
     assert list(split_names) == expected_split_names
 
@@ -309,14 +309,14 @@ def _check_results_single_metric(report, metric, expected_n_splits, expected_nb_
     # check the aggregate parameter
     stats = ["mean", "std"]
     result = getattr(report.metrics, metric)(aggregate=stats)
-    # check that the index contains the expected split names
-    split_names = result.index.get_level_values(1).unique()
+    # check that the columns contains the expected split names
+    split_names = result.columns.get_level_values(1).unique()
     assert list(split_names) == stats
 
     stats = "mean"
     result = getattr(report.metrics, metric)(aggregate=stats)
-    # check that the index contains the expected split names
-    split_names = result.index.get_level_values(1).unique()
+    # check that the columns contains the expected split names
+    split_names = result.columns.get_level_values(1).unique()
     assert list(split_names) == [stats]
 
 
@@ -325,13 +325,13 @@ def _check_results_report_metric(
 ):
     result = report.metrics.report_metrics(**params)
     assert isinstance(result, pd.DataFrame)
-    assert result.shape[0] == expected_n_splits
+    assert result.shape[1] == expected_n_splits
     # check that we hit the cache
     result_with_cache = report.metrics.report_metrics(**params)
     pd.testing.assert_frame_equal(result, result_with_cache)
 
-    # check that the index contains the expected split names
-    split_names = result.index.get_level_values(1).unique()
+    # check that the columns contains the expected split names
+    split_names = result.columns.get_level_values(1).unique()
     expected_split_names = [f"Split #{i}" for i in range(expected_n_splits)]
     assert list(split_names) == expected_split_names
 
@@ -340,31 +340,29 @@ def _check_results_report_metric(
     # check the aggregate parameter
     stats = ["mean", "std"]
     result = report.metrics.report_metrics(aggregate=stats, **params)
-    # check that the index contains the expected split names
-    split_names = result.index.get_level_values(1).unique()
+    # check that the columns contains the expected split names
+    split_names = result.columns.get_level_values(1).unique()
     assert list(split_names) == stats
 
     stats = "mean"
     result = report.metrics.report_metrics(aggregate=stats, **params)
-    # check that the index contains the expected split names
-    split_names = result.index.get_level_values(1).unique()
+    # check that the columns contains the expected split names
+    split_names = result.columns.get_level_values(1).unique()
     assert list(split_names) == [stats]
 
 
 def _check_metrics_names(result, expected_metrics, expected_nb_stats):
     assert isinstance(result, pd.DataFrame)
-    assert len(result.columns) == expected_nb_stats
+    assert len(result.index) == expected_nb_stats
 
     normalized_expected = {
         _normalize_metric_name(metric) for metric in expected_metrics
     }
-    for column in result.columns:
-        normalized_column = _normalize_metric_name(column)
-        matches = [
-            metric for metric in normalized_expected if metric == normalized_column
-        ]
+    for idx in result.index:
+        normalized_idx = _normalize_metric_name(idx)
+        matches = [metric for metric in normalized_expected if metric == normalized_idx]
         assert len(matches) == 1, (
-            f"No match found for column '{column}' in expected metrics: "
+            f"No match found for index '{idx}' in expected metrics: "
             f" {expected_metrics}"
         )
 
@@ -545,21 +543,21 @@ def test_cross_validation_report_report_metrics_scoring_kwargs(
     report = CrossValidationReport(estimator, X, y, cv_splitter=2)
     assert hasattr(report.metrics, "report_metrics")
     result = report.metrics.report_metrics(scoring_kwargs={"multioutput": "raw_values"})
-    assert result.shape == (2, 4)
-    assert isinstance(result.columns, pd.MultiIndex)
-    assert result.columns.names == ["Metric", "Output"]
+    assert result.shape == (4, 2)
+    assert isinstance(result.index, pd.MultiIndex)
+    assert result.index.names == ["Metric", "Output"]
 
     estimator, X, y = multiclass_classification_data
     report = CrossValidationReport(estimator, X, y, cv_splitter=2)
     assert hasattr(report.metrics, "report_metrics")
     result = report.metrics.report_metrics(scoring_kwargs={"average": None})
-    assert result.shape == (2, 10)
-    assert isinstance(result.columns, pd.MultiIndex)
-    assert result.columns.names == ["Metric", "Label / Average"]
+    assert result.shape == (10, 2)
+    assert isinstance(result.index, pd.MultiIndex)
+    assert result.index.names == ["Metric", "Label / Average"]
 
 
 @pytest.mark.parametrize(
-    "fixture_name, scoring_names, expected_columns",
+    "fixture_name, scoring_names, expected_index",
     [
         ("regression_data", ["R2", "RMSE"], ["R2", "RMSE"]),
         (
@@ -581,21 +579,21 @@ def test_cross_validation_report_report_metrics_scoring_kwargs(
     ],
 )
 def test_cross_validation_report_report_metrics_overwrite_scoring_names(
-    request, fixture_name, scoring_names, expected_columns
+    request, fixture_name, scoring_names, expected_index
 ):
     """Test that we can overwrite the scoring names in report_metrics."""
     estimator, X, y = request.getfixturevalue(fixture_name)
     report = CrossValidationReport(estimator, X, y, cv_splitter=2)
     result = report.metrics.report_metrics(scoring_names=scoring_names)
-    assert result.shape == (2, len(expected_columns))
+    assert result.shape == (len(expected_index), 2)
 
     # Get level 0 names if MultiIndex, otherwise get column names
-    result_columns = (
-        result.columns.get_level_values(0).tolist()
-        if isinstance(result.columns, pd.MultiIndex)
-        else result.columns.tolist()
+    result_index = (
+        result.index.get_level_values(0).tolist()
+        if isinstance(result.index, pd.MultiIndex)
+        else result.index.tolist()
     )
-    assert result_columns == expected_columns
+    assert result_index == expected_index
 
 
 @pytest.mark.parametrize("scoring", ["public_metric", "_private_metric"])
@@ -637,7 +635,7 @@ def test_cross_validation_report_report_metrics_with_scorer(regression_data):
     ]
     np.testing.assert_allclose(
         result.to_numpy(),
-        expected_result,
+        np.transpose(expected_result),
     )
 
 
@@ -674,7 +672,7 @@ def test_cross_validation_report_report_metrics_with_scorer_binary_classificatio
     result = report.metrics.report_metrics(
         scoring=["accuracy", accuracy_score, scorer],
     )
-    assert result.shape == (2, 3)
+    assert result.shape == (3, 2)
 
 
 def test_cross_validation_report_report_metrics_with_scorer_pos_label_error(
@@ -714,8 +712,8 @@ def test_cross_validation_report_custom_metric(binary_classification_data):
         metric_function=accuracy_score,
         response_method="predict",
     )
-    assert result.shape == (2, 1)
-    assert result.columns == ["accuracy_score"]
+    assert result.shape == (1, 2)
+    assert result.index == ["accuracy_score"]
 
 
 @pytest.mark.parametrize(
@@ -766,4 +764,4 @@ def test_cross_validation_report_interrupted(
         response_method="predict",
     )
     assert result.shape == (1, 1)
-    assert result.columns == ["accuracy_score"]
+    assert result.index == ["accuracy_score"]
diff --git a/skore/tests/unit/sklearn/test_estimator.py b/skore/tests/unit/sklearn/test_estimator.py
index f0ddbf9..237a920 100644
--- a/skore/tests/unit/sklearn/test_estimator.py
+++ b/skore/tests/unit/sklearn/test_estimator.py
@@ -584,12 +584,12 @@ def _normalize_metric_name(column):
 
 def _check_results_report_metrics(result, expected_metrics, expected_nb_stats):
     assert isinstance(result, pd.DataFrame)
-    assert len(result.columns) == expected_nb_stats
+    assert len(result.index) == expected_nb_stats
 
     normalized_expected = {
         _normalize_metric_name(metric) for metric in expected_metrics
     }
-    for column in result.columns:
+    for column in result.index:
         normalized_column = _normalize_metric_name(column)
         matches = [
             metric for metric in normalized_expected if metric == normalized_column
@@ -702,17 +702,17 @@ def test_estimator_report_report_metrics_scoring_kwargs(
     report = EstimatorReport(estimator, X_test=X_test, y_test=y_test)
     assert hasattr(report.metrics, "report_metrics")
     result = report.metrics.report_metrics(scoring_kwargs={"multioutput": "raw_values"})
-    assert result.shape == (1, 4)
-    assert isinstance(result.columns, pd.MultiIndex)
-    assert result.columns.names == ["Metric", "Output"]
+    assert result.shape == (4, 1)
+    assert isinstance(result.index, pd.MultiIndex)
+    assert result.index.names == ["Metric", "Output"]
 
     estimator, X_test, y_test = multiclass_classification_data
     report = EstimatorReport(estimator, X_test=X_test, y_test=y_test)
     assert hasattr(report.metrics, "report_metrics")
     result = report.metrics.report_metrics(scoring_kwargs={"average": None})
-    assert result.shape == (1, 10)
-    assert isinstance(result.columns, pd.MultiIndex)
-    assert result.columns.names == ["Metric", "Label / Average"]
+    assert result.shape == (10, 1)
+    assert isinstance(result.index, pd.MultiIndex)
+    assert result.index.names == ["Metric", "Label / Average"]
 
 
 @pytest.mark.parametrize(
@@ -744,15 +744,15 @@ def test_estimator_report_report_metrics_overwrite_scoring_names(
     estimator, X_test, y_test = request.getfixturevalue(fixture_name)
     report = EstimatorReport(estimator, X_test=X_test, y_test=y_test)
     result = report.metrics.report_metrics(scoring_names=scoring_names)
-    assert result.shape == (1, len(expected_columns))
+    assert result.shape == (len(expected_columns), 1)
 
     # Get level 0 names if MultiIndex, otherwise get column names
-    result_columns = (
-        result.columns.get_level_values(0).tolist()
-        if isinstance(result.columns, pd.MultiIndex)
-        else result.columns.tolist()
+    result_index = (
+        result.index.get_level_values(0).tolist()
+        if isinstance(result.index, pd.MultiIndex)
+        else result.index.tolist()
     )
-    assert result_columns == expected_columns
+    assert result_index == expected_columns
 
 
 def test_estimator_report_interaction_cache_metrics(regression_multioutput_data):
@@ -896,14 +896,12 @@ def test_estimator_report_report_metrics_with_custom_metric(regression_data):
         scoring=["r2", custom_metric],
         scoring_kwargs={"some_weights": weights, "response_method": "predict"},
     )
-    assert result.shape == (1, 2)
+    assert result.shape == (2, 1)
     np.testing.assert_allclose(
         result.to_numpy(),
         [
-            [
-                r2_score(y_test, estimator.predict(X_test)),
-                custom_metric(y_test, estimator.predict(X_test), weights),
-            ]
+            [r2_score(y_test, estimator.predict(X_test))],
+            [custom_metric(y_test, estimator.predict(X_test), weights)],
         ],
     )
 
@@ -928,15 +926,13 @@ def test_estimator_report_report_metrics_with_scorer(regression_data):
         scoring=[r2_score, median_absolute_error_scorer, custom_metric_scorer],
         scoring_kwargs={"response_method": "predict"},  # only dispatched to r2_score
     )
-    assert result.shape == (1, 3)
+    assert result.shape == (3, 1)
     np.testing.assert_allclose(
         result.to_numpy(),
         [
-            [
-                r2_score(y_test, estimator.predict(X_test)),
-                median_absolute_error(y_test, estimator.predict(X_test)),
-                custom_metric(y_test, estimator.predict(X_test), weights),
-            ]
+            [r2_score(y_test, estimator.predict(X_test))],
+            [median_absolute_error(y_test, estimator.predict(X_test))],
+            [custom_metric(y_test, estimator.predict(X_test), weights)],
         ],
     )
 
@@ -1004,17 +1000,13 @@ def test_estimator_report_report_metrics_with_scorer_binary_classification(
     result = report.metrics.report_metrics(
         scoring=["accuracy", accuracy_score, scorer],
     )
-    assert result.shape == (1, 3)
+    assert result.shape == (3, 1)
     np.testing.assert_allclose(
         result.to_numpy(),
         [
-            [
-                accuracy_score(y_test, estimator.predict(X_test)),
-                accuracy_score(y_test, estimator.predict(X_test)),
-                f1_score(
-                    y_test, estimator.predict(X_test), average="macro", pos_label=1
-                ),
-            ]
+            [accuracy_score(y_test, estimator.predict(X_test))],
+            [accuracy_score(y_test, estimator.predict(X_test))],
+            [f1_score(y_test, estimator.predict(X_test), average="macro", pos_label=1)],
         ],
     )
 
diff --git a/skore/tests/unit/sklearn/test_utils.py b/skore/tests/unit/sklearn/test_utils.py
index 08675c0..205a1d8 100644
--- a/skore/tests/unit/sklearn/test_utils.py
+++ b/skore/tests/unit/sklearn/test_utils.py
@@ -28,7 +28,7 @@ from skore.sklearn.find_ml_task import _find_ml_task
         (
             *make_multilabel_classification(random_state=42),
             MultiOutputClassifier(LogisticRegression()),
-            "unknown",
+            "multioutput-binary-classification",
         ),
     ],
 )
@@ -46,18 +46,29 @@ def test_find_ml_task_with_estimator(X, y, estimator, expected_task, should_fit)
     [
         (make_classification(random_state=42)[1], "binary-classification"),
         (
-            make_classification(n_classes=3, n_clusters_per_class=1, random_state=42)[
-                1
-            ],
+            make_classification(
+                n_classes=3,
+                n_clusters_per_class=1,
+                random_state=42,
+            )[1],
             "multiclass-classification",
         ),
         (make_regression(n_samples=100, random_state=42)[1], "regression"),
         (None, "clustering"),
-        (make_multilabel_classification(random_state=42)[1], "unknown"),
+        (
+            make_multilabel_classification(random_state=42)[1],
+            "multioutput-binary-classification",
+        ),
         (numpy.array([1, 5, 9]), "regression"),
         (numpy.array([0, 1, 2]), "multiclass-classification"),
         (numpy.array([1, 2, 3]), "regression"),
         (numpy.array([0, 1, 5, 9]), "regression"),
+        # Non-integer target
+        (numpy.array([[0.5, 2]]), "multioutput-regression"),
+        # No 0 class
+        (numpy.array([[1, 2], [2, 1]]), "multioutput-regression"),
+        # No 2 class
+        (numpy.array([[0, 3], [1, 3]]), "multioutput-regression"),
     ],
 )
 def test_find_ml_task_without_estimator(target, expected_task):