Repository: probabl-ai/skore

Issue:
Enh(project): Allow to delete a project from a command line
### Is your feature request related to a problem? Please describe.

today, the user has to do either a rm -rf, which is cumbersome, or do it by hand, which is not nice either, if they want to delete a project.  

### Describe the solution you'd like

add a parameter to the project clear function, `delete_project`, a boolean, False by default, which can be turned to True. This way, the project would be deleted.  

### Describe alternatives you've considered, if relevant

See discussion here: https://github.com/probabl-ai/skore/discussions/1258.  

### Additional context

_No response_

Discussion:
github-actions[bot]: [Documentation preview](https://skoredoc.k.probabl.dev/1322) @ db85d5fe96ad4767212ba26376b35b7ebbdfe39e
<!-- Sticky Pull Request Commentdocumentation-preview -->
auguste-probabl: The solution I gave doesn't work on Windows, because Windows is stricter about deleting files while the `project` variable still exists. I don't really know how to deal with this.
github-actions[bot]: <!-- Pytest Coverage Comment: display-backend-coverage -->
<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/README.md"><img alt="Coverage" src="https://img.shields.io/badge/Coverage-86%25-green.svg" /></a><details><summary>Coverage Report for backend </summary><table><tr><th>File</th><th>Stmts</th><th>Miss</th><th>Cover</th><th>Missing</th></tr><tbody><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/__init__.py">\_\_init\_\_.py</a></td><td>15</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/__main__.py">\_\_main\_\_.py</a></td><td>8</td><td>8</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/__main__.py#L3-L19">3&ndash;19</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/_config.py">_config.py</a></td><td>26</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/exceptions.py">exceptions.py</a></td><td>4</td><td>4</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/exceptions.py#L4-L23">4&ndash;23</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/cli</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/cli/__init__.py">\_\_init\_\_.py</a></td><td>5</td><td>5</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/cli/__init__.py#L3-L8">3&ndash;8</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/cli/cli.py">cli.py</a></td><td>22</td><td>22</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/cli/cli.py#L3-L70">3&ndash;70</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/cli/color_format.py">color_format.py</a></td><td>49</td><td>49</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/cli/color_format.py#L3-L116">3&ndash;116</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/persistence</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/__init__.py">\_\_init\_\_.py</a></td><td>0</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/persistence/item</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/__init__.py">\_\_init\_\_.py</a></td><td>56</td><td>3</td><td>93%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/__init__.py#L96-L99">96&ndash;99</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/altair_chart_item.py">altair_chart_item.py</a></td><td>19</td><td>1</td><td>91%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/altair_chart_item.py#L14">14</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/item.py">item.py</a></td><td>22</td><td>1</td><td>95%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/item.py#L86">86</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/matplotlib_figure_item.py">matplotlib_figure_item.py</a></td><td>36</td><td>1</td><td>95%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/matplotlib_figure_item.py#L19">19</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/media_item.py">media_item.py</a></td><td>22</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/numpy_array_item.py">numpy_array_item.py</a></td><td>27</td><td>1</td><td>94%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/numpy_array_item.py#L16">16</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pandas_dataframe_item.py">pandas_dataframe_item.py</a></td><td>29</td><td>1</td><td>94%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pandas_dataframe_item.py#L14">14</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pandas_series_item.py">pandas_series_item.py</a></td><td>29</td><td>1</td><td>94%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pandas_series_item.py#L14">14</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pickle_item.py">pickle_item.py</a></td><td>22</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pillow_image_item.py">pillow_image_item.py</a></td><td>25</td><td>1</td><td>93%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/pillow_image_item.py#L15">15</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/plotly_figure_item.py">plotly_figure_item.py</a></td><td>20</td><td>1</td><td>92%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/plotly_figure_item.py#L14">14</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/polars_dataframe_item.py">polars_dataframe_item.py</a></td><td>27</td><td>1</td><td>94%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/polars_dataframe_item.py#L14">14</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/polars_series_item.py">polars_series_item.py</a></td><td>22</td><td>1</td><td>92%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/polars_series_item.py#L14">14</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/primitive_item.py">primitive_item.py</a></td><td>23</td><td>2</td><td>91%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/primitive_item.py#L13-L15">13&ndash;15</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/sklearn_base_estimator_item.py">sklearn_base_estimator_item.py</a></td><td>29</td><td>1</td><td>94%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/sklearn_base_estimator_item.py#L15">15</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/skrub_table_report_item.py">skrub_table_report_item.py</a></td><td>10</td><td>1</td><td>86%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/item/skrub_table_report_item.py#L11">11</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/persistence/repository</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/repository/__init__.py">\_\_init\_\_.py</a></td><td>2</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/repository/item_repository.py">item_repository.py</a></td><td>59</td><td>5</td><td>91%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/repository/item_repository.py#L15-L16">15&ndash;16</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/repository/item_repository.py#L202-L203">202&ndash;203</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/repository/item_repository.py#L226">226</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/persistence/storage</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/storage/__init__.py">\_\_init\_\_.py</a></td><td>4</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/storage/abstract_storage.py">abstract_storage.py</a></td><td>22</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/storage/disk_cache_storage.py">disk_cache_storage.py</a></td><td>33</td><td>1</td><td>95%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/storage/disk_cache_storage.py#L44">44</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/storage/in_memory_storage.py">in_memory_storage.py</a></td><td>20</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/persistence/view</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/view/__init__.py">\_\_init\_\_.py</a></td><td>2</td><td>2</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/view/__init__.py#L3-L5">3&ndash;5</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/view/view.py">view.py</a></td><td>5</td><td>5</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/persistence/view/view.py#L3-L20">3&ndash;20</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/project</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/project/__init__.py">\_\_init\_\_.py</a></td><td>3</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/project/_launch.py">_launch.py</a></td><td>150</td><td>1</td><td>99%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/project/_launch.py#L278">278</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/project/_open.py">_open.py</a></td><td>9</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/project/project.py">project.py</a></td><td>86</td><td>1</td><td>99%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/project/project.py#L277">277</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/sklearn</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/__init__.py">\_\_init\_\_.py</a></td><td>5</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py">_base.py</a></td><td>140</td><td>14</td><td>90%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L36">36</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L48">48</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L58">58</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L91">91</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L94">94</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L147-L156">147&ndash;156</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L168-L>173">168&ndash;>173</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_base.py#L183-L184">183&ndash;184</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/find_ml_task.py">find_ml_task.py</a></td><td>49</td><td>1</td><td>97%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/find_ml_task.py#L103-L>111">103&ndash;>111</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/find_ml_task.py#L135">135</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/types.py">types.py</a></td><td>2</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation/__init__.py">\_\_init\_\_.py</a></td><td>5</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation/metrics_accessor.py">metrics_accessor.py</a></td><td>141</td><td>0</td><td>99%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation/metrics_accessor.py#L129-L>140">129&ndash;>140</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation/report.py">report.py</a></td><td>95</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/sklearn/_estimator</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/__init__.py">\_\_init\_\_.py</a></td><td>5</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py">metrics_accessor.py</a></td><td>255</td><td>9</td><td>95%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L148-L157">148&ndash;157</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L182-L>191">182&ndash;>191</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L190">190</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L230-L>232">230&ndash;>232</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L256">256</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L283-L287">283&ndash;287</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L302">302</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/metrics_accessor.py#L320">320</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/report.py">report.py</a></td><td>120</td><td>0</td><td>99%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/report.py#L213-L>219">213&ndash;>219</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/report.py#L221-L>223">221&ndash;>223</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/utils.py">utils.py</a></td><td>11</td><td>11</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_estimator/utils.py#L1-L19">1&ndash;19</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/sklearn/_plot</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/__init__.py">\_\_init\_\_.py</a></td><td>4</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/precision_recall_curve.py">precision_recall_curve.py</a></td><td>119</td><td>1</td><td>98%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/precision_recall_curve.py#L229-L>246">229&ndash;>246</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/precision_recall_curve.py#L317">317</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/prediction_error.py">prediction_error.py</a></td><td>95</td><td>1</td><td>98%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/prediction_error.py#L159">159</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/prediction_error.py#L173-L>176">173&ndash;>176</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/roc_curve.py">roc_curve.py</a></td><td>126</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/utils.py">utils.py</a></td><td>89</td><td>5</td><td>93%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/utils.py#L23">23</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/utils.py#L47-L49">47&ndash;49</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/_plot/utils.py#L53">53</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/sklearn/train_test_split</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/__init__.py">\_\_init\_\_.py</a></td><td>0</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/train_test_split.py">train_test_split.py</a></td><td>36</td><td>2</td><td>94%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/train_test_split.py#L16-L17">16&ndash;17</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/__init__.py">\_\_init\_\_.py</a></td><td>8</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/high_class_imbalance_too_few_examples_warning.py">high_class_imbalance_too_few_examples_warning.py</a></td><td>17</td><td>3</td><td>78%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/high_class_imbalance_too_few_examples_warning.py#L16-L18">16&ndash;18</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/high_class_imbalance_too_few_examples_warning.py#L80">80</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/high_class_imbalance_warning.py">high_class_imbalance_warning.py</a></td><td>18</td><td>2</td><td>88%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/high_class_imbalance_warning.py#L16-L18">16&ndash;18</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/random_state_unset_warning.py">random_state_unset_warning.py</a></td><td>11</td><td>1</td><td>87%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/random_state_unset_warning.py#L15">15</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/shuffle_true_warning.py">shuffle_true_warning.py</a></td><td>9</td><td>0</td><td>91%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/shuffle_true_warning.py#L44-L>exit">44&ndash;>exit</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/stratify_is_set_warning.py">stratify_is_set_warning.py</a></td><td>11</td><td>1</td><td>87%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/stratify_is_set_warning.py#L15">15</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/time_based_column_warning.py">time_based_column_warning.py</a></td><td>22</td><td>1</td><td>89%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/time_based_column_warning.py#L17">17</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/time_based_column_warning.py#L69-L>exit">69&ndash;>exit</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/train_test_split_warning.py">train_test_split_warning.py</a></td><td>5</td><td>1</td><td>80%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning/train_test_split_warning.py#L21">21</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/ui</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/__init__.py">\_\_init\_\_.py</a></td><td>0</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/app.py">app.py</a></td><td>32</td><td>32</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/app.py#L3-L83">3&ndash;83</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/dependencies.py">dependencies.py</a></td><td>4</td><td>4</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/dependencies.py#L3-L10">3&ndash;10</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/project_routes.py">project_routes.py</a></td><td>41</td><td>41</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/project_routes.py#L3-L92">3&ndash;92</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/serializers.py">serializers.py</a></td><td>77</td><td>77</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/serializers.py#L3-L194">3&ndash;194</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/server.py">server.py</a></td><td>17</td><td>17</td><td>0%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/ui/server.py#L3-L40">3&ndash;40</a></td></tr><tr><td colspan="5"><b>venv/lib/python3.12/site-packages/skore/utils</b></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/__init__.py">\_\_init\_\_.py</a></td><td>6</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_accessor.py">_accessor.py</a></td><td>7</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_environment.py">_environment.py</a></td><td>26</td><td>0</td><td>97%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_environment.py#L29-L>34">29&ndash;>34</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_logger.py">_logger.py</a></td><td>21</td><td>4</td><td>84%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_logger.py#L14-L18">14&ndash;18</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_parallel.py">_parallel.py</a></td><td>38</td><td>3</td><td>88%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_parallel.py#L23-L33">23&ndash;33</a>, <a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_parallel.py#L124">124</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_patch.py">_patch.py</a></td><td>11</td><td>5</td><td>46%</td><td><a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_patch.py#L19-L35">19&ndash;35</a></td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_progress_bar.py">_progress_bar.py</a></td><td>31</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td>&nbsp; &nbsp;<a href="https://github.com/probabl-ai/skore/blob/5dfda17bf81c7b7dd39ae36a6111caeaae7f231f/venv/lib/python3.12/site-packages/skore/utils/_show_versions.py">_show_versions.py</a></td><td>31</td><td>0</td><td>100%</td><td>&nbsp;</td></tr><tr><td><b>TOTAL</b></td><td><b>2682</b></td><td><b>356</b></td><td><b>86%</b></td><td>&nbsp;</td></tr></tbody></table></details>

| Tests | Skipped | Failures | Errors | Time |
| ----- | ------- | -------- | -------- | ------------------ |
| 550 | 0 :zzz: | 0 :x: | 0 :fire: | 52.013s :stopwatch: |


Patch:
diff --git a/README.md b/README.md
index dfa9fca..6248eaa 100644
--- a/README.md
+++ b/README.md
@@ -33,7 +33,8 @@ skore is a Python open-source library designed to help data scientists apply rec
   - `train_test_split` supercharged with methodological guidance: the API is the same as scikit-learn's, but skore displays warnings when applicable. For example, it warns you against shuffling time series data or when you have class imbalance.
 - **Evaluate**: automated insightful reports.
   - `EstimatorReport`: feed your scikit-learn compatible estimator and dataset, and it generates recommended metrics and plots to help you analyze your estimator. All these are computed and generated for you in 1 line of code. Under the hood, we use efficient caching to make the computations blazing fast.
-  - `CrossValidationReport`: Get a skore estimator report for each fold of your cross-validation.
+  - `CrossValidationReport`: get a skore estimator report for each fold of your cross-validation.
+  - `ComparisonReport`: benchmark your skore estimator reports.
 
 ## What's next?
 
@@ -91,7 +92,7 @@ You can find information on the latest version [here](https://anaconda.org/conda
     ```python
     # Display the ROC curve that was generated for you:
     roc_plot = cv_report.metrics.roc()
-    roc_plot
+    roc_plot.plot()
     ```
 
 1. Store your results for safe-keeping.
@@ -109,7 +110,8 @@ You can find information on the latest version [here](https://anaconda.org/conda
 
     ```python
     # Get your results
-    df_get = my_project.put("df_cv_report_metrics")
+    df_get = my_project.get("df_cv_report_metrics")
+    df_get
     ```
 
 Learn more in our [documentation](https://skore.probabl.ai).
diff --git a/examples/getting_started/plot_skore_getting_started.py b/examples/getting_started/plot_skore_getting_started.py
index ce92bbf..1be7e1c 100644
--- a/examples/getting_started/plot_skore_getting_started.py
+++ b/examples/getting_started/plot_skore_getting_started.py
@@ -17,6 +17,8 @@ Skore: getting started
 #       *   :class:`skore.CrossValidationReport`: get an insightful report on your
 #           cross-validation results
 #
+#       *   :class:`skore.ComparisonReport`: benchmark your skore estimator reports
+#
 #       *   :func:`skore.train_test_split`: get diagnostics when splitting your data
 #
 # #.    Track your ML/DS results using skore's :class:`~skore.Project`
@@ -50,10 +52,10 @@ from skore import EstimatorReport
 X, y = make_classification(n_classes=2, n_samples=100_000, n_informative=4)
 X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
 
-clf = LogisticRegression(random_state=0)
+log_reg = LogisticRegression(random_state=0)
 
-est_report = EstimatorReport(
-    clf, X_train=X_train, X_test=X_test, y_train=y_train, y_test=y_test
+log_reg_report = EstimatorReport(
+    log_reg, X_train=X_train, X_test=X_test, y_train=y_train, y_test=y_test
 )
 
 # %%
@@ -61,14 +63,14 @@ est_report = EstimatorReport(
 # (skore detected that we are doing binary classification):
 
 # %%
-est_report.help()
+log_reg_report.help()
 
 # %%
 # We can get the report metrics that was computed for us:
 
 # %%
-df_est_report_metrics = est_report.metrics.report_metrics()
-df_est_report_metrics
+df_log_reg_report_metrics = log_reg_report.metrics.report_metrics()
+df_log_reg_report_metrics
 
 # %%
 # We can also plot the ROC curve that was generated for us:
@@ -76,7 +78,7 @@ df_est_report_metrics
 # %%
 import matplotlib.pyplot as plt
 
-roc_plot = est_report.metrics.roc()
+roc_plot = log_reg_report.metrics.roc()
 roc_plot.plot()
 plt.tight_layout()
 
@@ -97,7 +99,7 @@ plt.tight_layout()
 # %%
 from skore import CrossValidationReport
 
-cv_report = CrossValidationReport(clf, X, y, cv_splitter=5)
+cv_report = CrossValidationReport(log_reg, X, y, cv_splitter=5)
 
 # %%
 # We display the cross-validation report helper:
@@ -125,9 +127,9 @@ plt.tight_layout()
 # for example the first fold:
 
 # %%
-est_report_fold = cv_report.estimator_reports_[0]
-df_report_metrics_fold = est_report_fold.metrics.report_metrics()
-df_report_metrics_fold
+log_reg_report_fold = cv_report.estimator_reports_[0]
+df_log_reg_report_fold_metrics = log_reg_report_fold.metrics.report_metrics()
+df_log_reg_report_fold_metrics
 
 # %%
 # .. seealso::
@@ -135,6 +137,50 @@ df_report_metrics_fold
 #   For more information about the motivation and usage of
 #   :class:`skore.CrossValidationReport`, see :ref:`example_use_case_employee_salaries`.
 
+# %%
+# Comparing estimators reports
+# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+#
+# :class:`skore.ComparisonReport` enables users to compare several estimator reports
+# (corresponding to several estimators) on a same test set, as in a benchmark of
+# estimators.
+#
+# Apart from the previous ``log_reg_report``, let use define another estimator report:
+
+# %%
+from sklearn.ensemble import RandomForestClassifier
+
+rf = RandomForestClassifier(max_depth=2, random_state=0)
+rf_report = EstimatorReport(
+    rf, X_train=X_train, X_test=X_test, y_train=y_train, y_test=y_test
+)
+
+# %%
+# Now, let us compare these two estimator reports, that were applied to the exact
+# same test set:
+
+# %%
+from skore import ComparisonReport
+
+comparator = ComparisonReport(reports=[log_reg_report, rf_report])
+
+# %%
+# As for the :class:`~skore.EstimatorReport` and the
+# :class:`~skore.CrossValidationReport`, we have a helper:
+
+# %%
+comparator.help()
+
+# %%
+# Let us display the result of our benchmark:
+
+# %%
+benchmark_metrics = comparator.metrics.report_metrics()
+benchmark_metrics
+
+# %%
+# We have the result of our benchmark.
+
 # %%
 # Train-test split with skore
 # ^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/examples/use_cases/plot_employee_salaries.py b/examples/use_cases/plot_employee_salaries.py
index 64489cb..c133270 100644
--- a/examples/use_cases/plot_employee_salaries.py
+++ b/examples/use_cases/plot_employee_salaries.py
@@ -298,6 +298,11 @@ results = pd.concat(
 )
 results
 
+# %%
+# .. note::
+#   We could have also used the :class:`skore.ComparisonReport` to compare estimator
+#   reports.
+
 # %%
 #
 # Finally, we can even get the individual :class:`~skore.EstimatorReport` for each fold
diff --git a/skore/src/skore/__init__.py b/skore/src/skore/__init__.py
index c4c7d1c..b8d06f5 100644
--- a/skore/src/skore/__init__.py
+++ b/skore/src/skore/__init__.py
@@ -8,6 +8,7 @@ from rich.theme import Theme
 from skore._config import config_context, get_config, set_config
 from skore.project import Project, open
 from skore.sklearn import (
+    ComparisonReport,
     CrossValidationReport,
     EstimatorReport,
     PrecisionRecallCurveDisplay,
@@ -20,6 +21,7 @@ from skore.utils._show_versions import show_versions
 
 __all__ = [
     "CrossValidationReport",
+    "ComparisonReport",
     "EstimatorReport",
     "PrecisionRecallCurveDisplay",
     "PredictionErrorDisplay",
diff --git a/skore/src/skore/project/project.py b/skore/src/skore/project/project.py
index 4ae6b24..ebd731e 100644
--- a/skore/src/skore/project/project.py
+++ b/skore/src/skore/project/project.py
@@ -2,6 +2,8 @@
 
 from __future__ import annotations
 
+import functools
+import shutil
 from collections.abc import Iterator
 from logging import INFO, NullHandler, getLogger
 from pathlib import Path
@@ -16,6 +18,30 @@ logger.addHandler(NullHandler())  # Default to no output
 logger.setLevel(INFO)
 
 
+class ProjectDeletedError(Exception):
+    """A method of a Project was called but the Project is marked as deleted."""
+
+
+def _raise_if_deleted(method):
+    """Raise if the underlying Project has been deleted, otherwise execute `method`.
+
+    This wrapper makes it safe to "delete" a Project, even if the Project instance
+    still exists.
+    """
+
+    @functools.wraps(method)
+    def wrapper(self, *args, **kwargs):
+        if self._storage_initialized is not True:
+            raise ProjectDeletedError(
+                "This Project instance is marked as deleted. "
+                "Please re-create a Project and discard the current one."
+            )
+
+        return method(self, *args, **kwargs)
+
+    return wrapper
+
+
 class Project:
     """
     A collection of items persisted in a storage.
@@ -95,16 +121,38 @@ class Project:
         # Initialize repositories with dedicated storages
         self._item_repository = ItemRepository(DiskCacheStorage(item_storage_dirpath))
 
+        self._storage_initialized = True
+
         # Check if the project should rejoin a server
         from skore.project._launch import ServerInfo  # avoid circular import
 
         self._server_info = ServerInfo.rejoin(self)
 
-    def clear(self):
-        """Clear the project."""
+    @_raise_if_deleted
+    def clear(self, delete_project=False):
+        """Remove all items from the project.
+
+        .. warning::
+           Clearing the project with `delete_project=True` will invalidate the whole
+           `Project` instance, making it unusable.
+           A new Project instance can be created using the :class:`skore.Project`
+           constructor or the :func:`skore.open` function.
+
+        Parameters
+        ----------
+        delete_project : bool
+            If set, the project will be deleted entirely.
+        """
+        if delete_project:
+            self._storage_initialized = False
+            del self._item_repository
+            shutil.rmtree(self.path)
+            return
+
         for item_key in self._item_repository:
             self._item_repository.delete_item(item_key)
 
+    @_raise_if_deleted
     def put(
         self,
         key: str,
@@ -150,6 +198,7 @@ class Project:
             ),
         )
 
+    @_raise_if_deleted
     def get(
         self,
         key: str,
@@ -211,6 +260,7 @@ class Project:
 
         raise ValueError('`version` should be -1, "all", or an integer')
 
+    @_raise_if_deleted
     def keys(self) -> list[str]:
         """
         Get all keys of items stored in the project.
@@ -222,6 +272,7 @@ class Project:
         """
         return self._item_repository.keys()
 
+    @_raise_if_deleted
     def __iter__(self) -> Iterator[str]:
         """
         Yield the keys of items stored in the project.
@@ -233,6 +284,7 @@ class Project:
         """
         yield from self._item_repository
 
+    @_raise_if_deleted
     def delete(self, key: str):
         """Delete the item corresponding to ``key`` from the Project.
 
@@ -248,6 +300,7 @@ class Project:
         """
         self._item_repository.delete_item(key)
 
+    @_raise_if_deleted
     def set_note(self, key: str, note: str, *, version=-1):
         """Attach a note to key ``key``.
 
@@ -277,6 +330,7 @@ class Project:
         """
         return self._item_repository.set_item_note(key=key, note=note, version=version)
 
+    @_raise_if_deleted
     def get_note(self, key: str, *, version=-1) -> Union[str, None]:
         """Retrieve a note previously attached to key ``key``.
 
@@ -306,6 +360,7 @@ class Project:
         """
         return self._item_repository.get_item_note(key=key, version=version)
 
+    @_raise_if_deleted
     def delete_note(self, key: str, *, version=-1):
         """Delete a note previously attached to key ``key``.
 
@@ -333,6 +388,7 @@ class Project:
         """
         return self._item_repository.delete_item_note(key=key, version=version)
 
+    @_raise_if_deleted
     def shutdown_web_ui(self):
         """Shutdown the web UI server if it is running."""
         if self._server_info is None:
diff --git a/skore/src/skore/sklearn/__init__.py b/skore/src/skore/sklearn/__init__.py
index 0b58589..f1abb35 100644
--- a/skore/src/skore/sklearn/__init__.py
+++ b/skore/src/skore/sklearn/__init__.py
@@ -1,5 +1,6 @@
 """Enhance `sklearn` functions."""
 
+from skore.sklearn._comparison import ComparisonReport
 from skore.sklearn._cross_validation import CrossValidationReport
 from skore.sklearn._estimator import EstimatorReport
 from skore.sklearn._plot import (
@@ -13,6 +14,7 @@ __all__ = [
     "train_test_split",
     "CrossValidationReport",
     "EstimatorReport",
+    "ComparisonReport",
     "RocCurveDisplay",
     "PrecisionRecallCurveDisplay",
     "PredictionErrorDisplay",
diff --git a/skore/src/skore/sklearn/_base.py b/skore/src/skore/sklearn/_base.py
index d62963c..c23290b 100644
--- a/skore/src/skore/sklearn/_base.py
+++ b/skore/src/skore/sklearn/_base.py
@@ -124,7 +124,7 @@ class _BaseReport(_HelpMixin):
 
     def _create_help_tree(self):
         """Create a rich Tree with the available tools and accessor methods."""
-        tree = Tree("report")
+        tree = Tree(self.__class__.__name__)
 
         # Add accessor methods first
         for accessor_attr, config in self._ACCESSOR_CONFIG.items():
diff --git a/skore/src/skore/sklearn/_comparison/__init__.py b/skore/src/skore/sklearn/_comparison/__init__.py
new file mode 100644
index 0000000..eb72e33
--- /dev/null
+++ b/skore/src/skore/sklearn/_comparison/__init__.py
@@ -0,0 +1,7 @@
+from skore.externals._pandas_accessors import _register_accessor
+from skore.sklearn._comparison.metrics_accessor import _MetricsAccessor
+from skore.sklearn._comparison.report import ComparisonReport
+
+_register_accessor("metrics", ComparisonReport)(_MetricsAccessor)
+
+__all__ = ["ComparisonReport"]
diff --git a/skore/src/skore/sklearn/_comparison/metrics_accessor.py b/skore/src/skore/sklearn/_comparison/metrics_accessor.py
new file mode 100644
index 0000000..7f07a3d
--- /dev/null
+++ b/skore/src/skore/sklearn/_comparison/metrics_accessor.py
@@ -0,0 +1,1080 @@
+import joblib
+import numpy as np
+import pandas as pd
+from sklearn.metrics import make_scorer
+from sklearn.utils.metaestimators import available_if
+
+from skore.externals._pandas_accessors import DirNamesMixin
+from skore.sklearn._base import _BaseAccessor
+from skore.utils._accessor import _check_supported_ml_task
+from skore.utils._progress_bar import progress_decorator
+
+
+class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
+    """Accessor for metrics-related operations.
+
+    You can access this accessor using the `metrics` attribute.
+    """
+
+    _SCORE_OR_LOSS_INFO = {
+        "accuracy": {"name": "Accuracy", "icon": "(↗︎)"},
+        "precision": {"name": "Precision", "icon": "(↗︎)"},
+        "recall": {"name": "Recall", "icon": "(↗︎)"},
+        "brier_score": {"name": "Brier score", "icon": "(↘︎)"},
+        "roc_auc": {"name": "ROC AUC", "icon": "(↗︎)"},
+        "log_loss": {"name": "Log loss", "icon": "(↘︎)"},
+        "r2": {"name": "R²", "icon": "(↗︎)"},
+        "rmse": {"name": "RMSE", "icon": "(↘︎)"},
+        "custom_metric": {"name": "Custom metric", "icon": ""},
+        "report_metrics": {"name": "Report metrics", "icon": ""},
+    }
+
+    def __init__(self, parent):
+        super().__init__(parent)
+
+        self._parent_progress = None
+
+    def report_metrics(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        scoring=None,
+        scoring_names=None,
+        pos_label=None,
+        scoring_kwargs=None,
+    ):
+        """Report a set of metrics for the estimators.
+
+        Parameters
+        ----------
+        data_source : {"test", "train", "X_y"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        scoring : list of str, callable, or scorer, default=None
+            The metrics to report. You can get the possible list of strings by calling
+            `report.metrics.help()`. When passing a callable, it should take as
+            arguments ``y_true``, ``y_pred`` as the two first arguments. Additional
+            arguments can be passed as keyword arguments and will be forwarded with
+            `scoring_kwargs`. If the callable API is too restrictive (e.g. need to pass
+            same parameter name with different values), you can use scikit-learn scorers
+            as provided by :func:`sklearn.metrics.make_scorer`.
+
+        scoring_names : list of str, default=None
+            Used to overwrite the default scoring names in the report. It should be of
+            the same length as the ``scoring`` parameter.
+
+        pos_label : int, float, bool or str, default=None
+            The positive class.
+
+        scoring_kwargs : dict, default=None
+            The keyword arguments to pass to the scoring functions.
+
+        Returns
+        -------
+        pd.DataFrame
+            The statistics for the metrics.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.report_metrics(
+        ...     scoring=["precision", "recall"],
+        ...     pos_label=1,
+        ... )
+        Estimator       LogisticRegression  LogisticRegression
+        Metric
+        Precision (↗︎)              0.96...             0.96...
+        Recall (↗︎)                 0.97...             0.97...
+        """
+        return self._compute_metric_scores(
+            report_metric_name="report_metrics",
+            data_source=data_source,
+            X=X,
+            y=y,
+            scoring=scoring,
+            pos_label=pos_label,
+            scoring_kwargs=scoring_kwargs,
+            scoring_names=scoring_names,
+        )
+
+    @progress_decorator(description="Compute metric for each split")
+    def _compute_metric_scores(
+        self,
+        report_metric_name,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        **metric_kwargs,
+    ):
+        cache_key = (self._parent._hash, report_metric_name, data_source)
+
+        # we need to enforce the order of the parameter for a specific metric
+        # to make sure that we hit the cache in a consistent way
+        ordered_metric_kwargs = sorted(metric_kwargs.keys())
+
+        for key in ordered_metric_kwargs:
+            if isinstance(metric_kwargs[key], (np.ndarray, list, dict)):
+                cache_key += (joblib.hash(metric_kwargs[key]),)
+            else:
+                cache_key += (metric_kwargs[key],)
+
+        progress = self._progress_info["current_progress"]
+        main_task = self._progress_info["current_task"]
+
+        total_estimators = len(self._parent.estimator_reports_)
+        progress.update(main_task, total=total_estimators)
+
+        if cache_key in self._parent._cache:
+            results = self._parent._cache[cache_key]
+        else:
+            parallel = joblib.Parallel(
+                n_jobs=self._parent.n_jobs,
+                return_as="generator",
+                require="sharedmem",
+            )
+            generator = parallel(
+                joblib.delayed(getattr(report.metrics, report_metric_name))(
+                    data_source=data_source,
+                    X=X,
+                    y=y,
+                    **metric_kwargs,
+                )
+                for report in self._parent.estimator_reports_
+            )
+            results = []
+            for result in generator:
+                results.append(result)
+                progress.update(main_task, advance=1, refresh=True)
+
+            results = pd.concat(results, axis=1)
+            results.columns = pd.Index(self._parent.report_names_, name="Estimator")
+
+            self._parent._cache[cache_key] = results
+        return results
+
+    @available_if(
+        _check_supported_ml_task(
+            supported_ml_tasks=["binary-classification", "multiclass-classification"]
+        )
+    )
+    def accuracy(self, *, data_source="test", X=None, y=None):
+        """Compute the accuracy score.
+
+        Parameters
+        ----------
+        data_source : {"test", "train", "X_y"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        Returns
+        -------
+        pd.DataFrame
+            The accuracy score.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.accuracy()
+        Estimator      LogisticRegression  LogisticRegression
+        Metric
+        Accuracy (↗︎)              0.96...             0.96...
+        """
+        return self.report_metrics(
+            scoring=["accuracy"],
+            data_source=data_source,
+            X=X,
+            y=y,
+        )
+
+    @available_if(
+        _check_supported_ml_task(
+            supported_ml_tasks=["binary-classification", "multiclass-classification"]
+        )
+    )
+    def precision(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        average=None,
+        pos_label=None,
+    ):
+        """Compute the precision score.
+
+        Parameters
+        ----------
+        data_source : {"test", "train", "X_y"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        average : {"binary", "macro", "micro", "weighted", "samples"} or None, \
+                default=None
+            Used with multiclass problems.
+            If `None`, the metrics for each class are returned. Otherwise, this
+            determines the type of averaging performed on the data:
+
+            - "binary": Only report results for the class specified by `pos_label`.
+              This is applicable only if targets (`y_{true,pred}`) are binary.
+            - "micro": Calculate metrics globally by counting the total true positives,
+              false negatives and false positives.
+            - "macro": Calculate metrics for each label, and find their unweighted
+              mean.  This does not take label imbalance into account.
+            - "weighted": Calculate metrics for each label, and find their average
+              weighted by support (the number of true instances for each label). This
+              alters 'macro' to account for label imbalance; it can result in an F-score
+              that is not between precision and recall.
+            - "samples": Calculate metrics for each instance, and find their average
+              (only meaningful for multilabel classification where this differs from
+              :func:`accuracy_score`).
+
+            .. note::
+                If `pos_label` is specified and `average` is None, then we report
+                only the statistics of the positive class (i.e. equivalent to
+                `average="binary"`).
+
+        pos_label : int, float, bool or str, default=None
+            The positive class.
+
+        Returns
+        -------
+        pd.DataFrame
+            The precision score.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.precision()
+        Estimator                    LogisticRegression  LogisticRegression
+        Metric      Label / Average
+        Precision (↗︎)             0             0.96...             0.96...
+                                  1             0.96...             0.96...
+        """
+        return self.report_metrics(
+            scoring=["precision"],
+            data_source=data_source,
+            X=X,
+            y=y,
+            pos_label=pos_label,
+            scoring_kwargs={"average": average},
+        )
+
+    @available_if(
+        _check_supported_ml_task(
+            supported_ml_tasks=["binary-classification", "multiclass-classification"]
+        )
+    )
+    def recall(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        average=None,
+        pos_label=None,
+    ):
+        """Compute the recall score.
+
+        Parameters
+        ----------
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        average : {"binary","macro", "micro", "weighted", "samples"} or None, \
+                default=None
+            Used with multiclass problems.
+            If `None`, the metrics for each class are returned. Otherwise, this
+            determines the type of averaging performed on the data:
+
+            - "binary": Only report results for the class specified by `pos_label`.
+              This is applicable only if targets (`y_{true,pred}`) are binary.
+            - "micro": Calculate metrics globally by counting the total true positives,
+              false negatives and false positives.
+            - "macro": Calculate metrics for each label, and find their unweighted
+              mean.  This does not take label imbalance into account.
+            - "weighted": Calculate metrics for each label, and find their average
+              weighted by support (the number of true instances for each label). This
+              alters 'macro' to account for label imbalance; it can result in an F-score
+              that is not between precision and recall. Weighted recall is equal to
+              accuracy.
+            - "samples": Calculate metrics for each instance, and find their average
+              (only meaningful for multilabel classification where this differs from
+              :func:`accuracy_score`).
+
+            .. note::
+                If `pos_label` is specified and `average` is None, then we report
+                only the statistics of the positive class (i.e. equivalent to
+                `average="binary"`).
+
+        pos_label : int, float, bool or str, default=None
+            The positive class.
+
+        Returns
+        -------
+        pd.DataFrame
+            The recall score.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.recall()
+        Estimator                    LogisticRegression  LogisticRegression
+        Metric      Label / Average
+        Recall (↗︎)                0            0.944...            0.944...
+                                  1            0.977...            0.977...
+        """
+        return self.report_metrics(
+            scoring=["recall"],
+            data_source=data_source,
+            X=X,
+            y=y,
+            pos_label=pos_label,
+            scoring_kwargs={"average": average},
+        )
+
+    @available_if(
+        _check_supported_ml_task(supported_ml_tasks=["binary-classification"])
+    )
+    def brier_score(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+    ):
+        """Compute the Brier score.
+
+        Parameters
+        ----------
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        Returns
+        -------
+        pd.DataFrame
+            The Brier score.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.brier_score()
+        Estimator         LogisticRegression  LogisticRegression
+        Metric
+        Brier score (↘︎)              0.025...            0.025...
+        """
+        return self.report_metrics(
+            scoring=["brier_score"],
+            data_source=data_source,
+            X=X,
+            y=y,
+        )
+
+    @available_if(
+        _check_supported_ml_task(
+            supported_ml_tasks=["binary-classification", "multiclass-classification"]
+        )
+    )
+    def roc_auc(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        average=None,
+        multi_class="ovr",
+    ):
+        """Compute the ROC AUC score.
+
+        Parameters
+        ----------
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        average : {"auto", "macro", "micro", "weighted", "samples"}, \
+                default=None
+            Average to compute the ROC AUC score in a multiclass setting. By default,
+            no average is computed. Otherwise, this determines the type of averaging
+            performed on the data.
+
+            - "micro": Calculate metrics globally by considering each element of
+              the label indicator matrix as a label.
+            - "macro": Calculate metrics for each label, and find their unweighted
+              mean. This does not take label imbalance into account.
+            - "weighted": Calculate metrics for each label, and find their average,
+              weighted by support (the number of true instances for each label).
+            - "samples": Calculate metrics for each instance, and find their
+              average.
+
+            .. note::
+                Multiclass ROC AUC currently only handles the "macro" and
+                "weighted" averages. For multiclass targets, `average=None` is only
+                implemented for `multi_class="ovr"` and `average="micro"` is only
+                implemented for `multi_class="ovr"`.
+
+        multi_class : {"raise", "ovr", "ovo"}, default="ovr"
+            The multi-class strategy to use.
+
+            - "raise": Raise an error if the data is multiclass.
+            - "ovr": Stands for One-vs-rest. Computes the AUC of each class against the
+              rest. This treats the multiclass case in the same way as the multilabel
+              case. Sensitive to class imbalance even when `average == "macro"`,
+              because class imbalance affects the composition of each of the "rest"
+              groupings.
+            - "ovo": Stands for One-vs-one. Computes the average AUC of all possible
+              pairwise combinations of classes. Insensitive to class imbalance when
+              `average == "macro"`.
+
+        Returns
+        -------
+        pd.DataFrame
+            The ROC AUC score.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.roc_auc()
+        Estimator      LogisticRegression  LogisticRegression
+        Metric
+        ROC AUC (↗︎)               0.99...             0.99...
+        """
+        return self.report_metrics(
+            scoring=["roc_auc"],
+            data_source=data_source,
+            X=X,
+            y=y,
+            scoring_kwargs={"average": average, "multi_class": multi_class},
+        )
+
+    @available_if(
+        _check_supported_ml_task(
+            supported_ml_tasks=["binary-classification", "multiclass-classification"]
+        )
+    )
+    def log_loss(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+    ):
+        """Compute the log loss.
+
+        Parameters
+        ----------
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        Returns
+        -------
+        pd.DataFrame
+            The log-loss.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_breast_cancer
+        >>> from sklearn.linear_model import LogisticRegression
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_breast_cancer(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = LogisticRegression(max_iter=10000, random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = LogisticRegression(max_iter=10000, random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.log_loss()
+        Estimator      LogisticRegression  LogisticRegression
+        Metric
+        Log loss (↘︎)             0.082...            0.082...
+        """
+        return self.report_metrics(
+            scoring=["log_loss"],
+            data_source=data_source,
+            X=X,
+            y=y,
+        )
+
+    @available_if(_check_supported_ml_task(supported_ml_tasks=["regression"]))
+    def r2(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        multioutput="raw_values",
+    ):
+        """Compute the R² score.
+
+        Parameters
+        ----------
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        multioutput : {"raw_values", "uniform_average"} or array-like of shape \
+                (n_outputs,), default="raw_values"
+            Defines aggregating of multiple output values. Array-like value defines
+            weights used to average errors. The other possible values are:
+
+            - "raw_values": Returns a full set of errors in case of multioutput input.
+            - "uniform_average": Errors of all outputs are averaged with uniform weight.
+
+            By default, no averaging is done.
+
+        Returns
+        -------
+        pd.DataFrame
+            The R² score.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_diabetes
+        >>> from sklearn.linear_model import Ridge
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_diabetes(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = Ridge(random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = Ridge(random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.r2()
+        Estimator     Ridge    Ridge
+        Metric
+        R² (↗︎)      0.43...  0.43...
+        """
+        return self.report_metrics(
+            scoring=["r2"],
+            data_source=data_source,
+            X=X,
+            y=y,
+            scoring_kwargs={"multioutput": multioutput},
+        )
+
+    @available_if(_check_supported_ml_task(supported_ml_tasks=["regression"]))
+    def rmse(
+        self,
+        *,
+        data_source="test",
+        X=None,
+        y=None,
+        multioutput="raw_values",
+    ):
+        """Compute the root mean squared error.
+
+        Parameters
+        ----------
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        multioutput : {"raw_values", "uniform_average"} or array-like of shape \
+                (n_outputs,), default="raw_values"
+            Defines aggregating of multiple output values. Array-like value defines
+            weights used to average errors. The other possible values are:
+
+            - "raw_values": Returns a full set of errors in case of multioutput input.
+            - "uniform_average": Errors of all outputs are averaged with uniform weight.
+
+            By default, no averaging is done.
+
+        Returns
+        -------
+        pd.DataFrame
+            The root mean squared error.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_diabetes
+        >>> from sklearn.linear_model import Ridge
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_diabetes(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = Ridge(random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = Ridge(random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.rmse()
+        Estimator       Ridge       Ridge
+        Metric
+        RMSE (↘︎)    55.726...   55.726...
+        """
+        return self.report_metrics(
+            scoring=["rmse"],
+            data_source=data_source,
+            X=X,
+            y=y,
+            scoring_kwargs={"multioutput": multioutput},
+        )
+
+    def custom_metric(
+        self,
+        metric_function,
+        response_method,
+        *,
+        metric_name=None,
+        data_source="test",
+        X=None,
+        y=None,
+        **kwargs,
+    ):
+        """Compute a custom metric.
+
+        It brings some flexibility to compute any desired metric. However, we need to
+        follow some rules:
+
+        - `metric_function` should take `y_true` and `y_pred` as the first two
+          positional arguments.
+        - `response_method` corresponds to the estimator's method to be invoked to get
+          the predictions. It can be a string or a list of strings to defined in which
+          order the methods should be invoked.
+
+        Parameters
+        ----------
+        metric_function : callable
+            The metric function to be computed. The expected signature is
+            `metric_function(y_true, y_pred, **kwargs)`.
+
+        response_method : str or list of str
+            The estimator's method to be invoked to get the predictions. The possible
+            values are: `predict`, `predict_proba`, `predict_log_proba`, and
+            `decision_function`.
+
+        metric_name : str, default=None
+            The name of the metric. If not provided, it will be inferred from the
+            metric function.
+
+        data_source : {"test", "train"}, default="test"
+            The data source to use.
+
+            - "test" : use the test set provided when creating the report.
+            - "train" : use the train set provided when creating the report.
+            - "X_y" : use the provided `X` and `y` to compute the metric.
+
+        X : array-like of shape (n_samples, n_features), default=None
+            New data on which to compute the metric. By default, we use the validation
+            set provided when creating the report.
+
+        y : array-like of shape (n_samples,), default=None
+            New target on which to compute the metric. By default, we use the target
+            provided when creating the report.
+
+        **kwargs : dict
+            Any additional keyword arguments to be passed to the metric function.
+
+        Returns
+        -------
+        pd.DataFrame
+            The custom metric.
+
+        Examples
+        --------
+        >>> from sklearn.datasets import load_diabetes
+        >>> from sklearn.linear_model import Ridge
+        >>> from sklearn.metrics import mean_absolute_error
+        >>> from sklearn.model_selection import train_test_split
+        >>> from skore import ComparisonReport, EstimatorReport
+        >>> X, y = load_diabetes(return_X_y=True)
+        >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+        >>> estimator_1 = Ridge(random_state=42)
+        >>> estimator_report_1 = EstimatorReport(
+        ...     estimator_1,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> estimator_2 = Ridge(random_state=43)
+        >>> estimator_report_2 = EstimatorReport(
+        ...     estimator_2,
+        ...     X_train=X_train,
+        ...     y_train=y_train,
+        ...     X_test=X_test,
+        ...     y_test=y_test,
+        ... )
+        >>> comparison_report = ComparisonReport(
+        ...     [estimator_report_1, estimator_report_2]
+        ... )
+        >>> comparison_report.metrics.custom_metric(
+        ...     metric_function=mean_absolute_error,
+        ...     response_method="predict",
+        ...     metric_name="MAE (↗︎)",
+        ... )
+        Estimator      Ridge      Ridge
+        Metric
+        MAE (↗︎)     45.91...   45.91...
+        """
+        # create a scorer with `greater_is_better=True` to not alter the output of
+        # `metric_function`
+        scorer = make_scorer(
+            metric_function,
+            greater_is_better=True,
+            response_method=response_method,
+            **kwargs,
+        )
+        return self.report_metrics(
+            scoring=[scorer],
+            data_source=data_source,
+            X=X,
+            y=y,
+            scoring_names=[metric_name],
+        )
+
+    ####################################################################################
+    # Methods related to the help tree
+    ####################################################################################
+
+    def _sort_methods_for_help(self, methods):
+        """Override sort method for metrics-specific ordering.
+
+        In short, we display the `report_metrics` first and then the `custom_metric`.
+        """
+
+        def _sort_key(method):
+            name = method[0]
+            if name == "custom_metric":
+                priority = 1
+            elif name == "report_metrics":
+                priority = 2
+            else:
+                priority = 0
+            return priority, name
+
+        return sorted(methods, key=_sort_key)
+
+    def _format_method_name(self, name):
+        """Override format method for metrics-specific naming."""
+        method_name = f"{name}(...)"
+        method_name = method_name.ljust(22)
+        if name in self._SCORE_OR_LOSS_INFO and self._SCORE_OR_LOSS_INFO[name][
+            "icon"
+        ] in ("(↗︎)", "(↘︎)"):
+            if self._SCORE_OR_LOSS_INFO[name]["icon"] == "(↗︎)":
+                method_name += f"[cyan]{self._SCORE_OR_LOSS_INFO[name]['icon']}[/cyan]"
+                return method_name.ljust(43)
+            else:  # (↘︎)
+                method_name += (
+                    f"[orange1]{self._SCORE_OR_LOSS_INFO[name]['icon']}[/orange1]"
+                )
+                return method_name.ljust(49)
+        else:
+            return method_name.ljust(29)
+
+    def _get_methods_for_help(self):
+        """Override to exclude the plot accessor from methods list."""
+        methods = super()._get_methods_for_help()
+        return [(name, method) for name, method in methods if name != "plot"]
+
+    def _get_help_panel_title(self):
+        return "[bold cyan]Available metrics methods[/bold cyan]"
+
+    def _get_help_legend(self):
+        return (
+            "[cyan](↗︎)[/cyan] higher is better [orange1](↘︎)[/orange1] lower is better"
+        )
+
+    def _get_help_tree_title(self):
+        return "[bold cyan]report.metrics[/bold cyan]"
+
+    def __repr__(self):
+        """Return a string representation using rich."""
+        return self._rich_repr(
+            class_name="skore.ComparisonReport.metrics",
+            help_method_name="report.metrics.help()",
+        )
diff --git a/skore/src/skore/sklearn/_comparison/report.py b/skore/src/skore/sklearn/_comparison/report.py
new file mode 100644
index 0000000..ac4c3a3
--- /dev/null
+++ b/skore/src/skore/sklearn/_comparison/report.py
@@ -0,0 +1,170 @@
+from __future__ import annotations
+
+import time
+from collections.abc import Iterable
+from typing import Optional, Union
+
+import joblib
+import numpy as np
+
+from skore.externals._pandas_accessors import DirNamesMixin
+from skore.sklearn._base import _BaseReport
+from skore.sklearn._estimator.report import EstimatorReport
+
+
+class ComparisonReport(_BaseReport, DirNamesMixin):
+    """Report for comparison of instances of :class:`skore.EstimatorReport`.
+
+    Caution: reports passed to `ComparisonReport` are not copied. If you pass
+    a report to `ComparisonReport`, and then modify the report outside later, it will
+    affect the report stored inside the `ComparisonReport` as well, which can lead to
+    inconsistent results. For this reason, modifying reports after creation is strongly
+    discouraged.
+
+    Parameters
+    ----------
+    reports : list of :class:`~skore.EstimatorReport` instances or dict
+        Estimator reports to compare.
+
+        * If `reports` is a list, the class name of each estimator is used.
+        * If `reports` is a dict, it is expected to have estimator names as keys
+          and :class:`~skore.EstimatorReport` instances as values.
+          If the keys are not strings, they will be converted to strings.
+
+    n_jobs : int, default=None
+        Number of jobs to run in parallel. Training the estimators and computing
+        the scores are parallelized.
+        When accessing some methods of the `ComparisonReport`, the `n_jobs`
+        parameter is used to parallelize the computation.
+        ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context.
+        ``-1`` means using all processors.
+
+    Attributes
+    ----------
+    estimator_reports_ : list of `~skore.EstimatorReport`
+        The compared estimator reports.
+
+    report_names_ : list of str
+        The names of the compared estimator reports.
+
+    See Also
+    --------
+    skore.EstimatorReport
+        Report for a fitted estimator.
+
+    skore.CrossValidationReport
+        Report for the cross-validation of an estimator.
+
+    Examples
+    --------
+    >>> from sklearn.datasets import make_classification
+    >>> from sklearn.model_selection import train_test_split
+    >>> from sklearn.linear_model import LogisticRegression
+    >>> from skore import ComparisonReport, EstimatorReport
+    >>> X, y = make_classification(random_state=42)
+    >>> X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+    >>> estimator_1 = LogisticRegression()
+    >>> estimator_report_1 = EstimatorReport(
+    ...     estimator_1,
+    ...     X_train=X_train,
+    ...     y_train=y_train,
+    ...     X_test=X_test,
+    ...     y_test=y_test
+    ... )
+    >>> estimator_2 = LogisticRegression(C=2)  # Different regularization
+    >>> estimator_report_2 = EstimatorReport(
+    ...     estimator_2,
+    ...     X_train=X_train,
+    ...     y_train=y_train,
+    ...     X_test=X_test,
+    ...     y_test=y_test
+    ... )
+    >>> report = ComparisonReport([estimator_report_1, estimator_report_2])
+    ...
+    >>> report = ComparisonReport(
+    ...     {"model1": estimator_report_1, "model2": estimator_report_2}
+    ... )
+    ...
+    """
+
+    _ACCESSOR_CONFIG = {
+        "metrics": {"name": "metrics"},
+    }
+
+    def __init__(
+        self,
+        reports: Union[list[EstimatorReport], dict[str, EstimatorReport]],
+        *,
+        n_jobs: Optional[int] = None,
+    ):
+        """
+        ComparisonReport instance initializer.
+
+        Notes
+        -----
+        We check that the estimator reports can be compared:
+        - all reports are estimator reports,
+        - all estimators are in the same ML use case,
+        - all estimators have non-empty X_test and y_test,
+        - all estimators have the same X_test and y_test.
+        """
+        if not isinstance(reports, Iterable):
+            raise TypeError(f"Expected reports to be an iterable; got {type(reports)}")
+
+        if len(reports) < 2:
+            raise ValueError("At least 2 instances of EstimatorReport are needed")
+
+        report_names = (
+            list(map(str, reports.keys())) if isinstance(reports, dict) else None
+        )
+        reports = list(reports.values()) if isinstance(reports, dict) else reports
+
+        if not all(isinstance(report, EstimatorReport) for report in reports):
+            raise TypeError("Expected instances of EstimatorReport")
+
+        test_dataset_hashes = {
+            joblib.hash((report.X_test, report.y_test))
+            for report in reports
+            if not ((report.X_test is None) and (report.y_test is None))
+        }
+        if len(test_dataset_hashes) > 1:
+            raise ValueError("Expected all estimators to have the same testing data.")
+
+        ml_tasks = {report: report._ml_task for report in reports}
+        if len(set(ml_tasks.values())) > 1:
+            raise ValueError(
+                f"Expected all estimators to have the same ML usecase; "
+                f"got {ml_tasks}"
+            )
+
+        if report_names is None:
+            self.report_names_ = [report.estimator_name_ for report in reports]
+        else:
+            self.report_names_ = report_names
+
+        self.estimator_reports_ = reports
+
+        # NEEDED FOR METRICS ACCESSOR
+        self.n_jobs = n_jobs
+        self._rng = np.random.default_rng(time.time_ns())
+        self._hash = self._rng.integers(
+            low=np.iinfo(np.int64).min, high=np.iinfo(np.int64).max
+        )
+        self._cache = {}
+        self._ml_task = self.estimator_reports_[0]._ml_task
+
+    ####################################################################################
+    # Methods related to the help and repr
+    ####################################################################################
+
+    def _get_help_panel_title(self):
+        return "[bold cyan]Tools to compare estimators[/bold cyan]"
+
+    def _get_help_legend(self):
+        return (
+            "[cyan](↗︎)[/cyan] higher is better [orange1](↘︎)[/orange1] lower is better"
+        )
+
+    def __repr__(self):
+        """Return a string representation."""
+        return f"{self.__class__.__name__}(...)"
diff --git a/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py b/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py
index ce0b8d3..e1fcc44 100644
--- a/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py
+++ b/skore/src/skore/sklearn/_cross_validation/metrics_accessor.py
@@ -126,16 +126,15 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         cache_key = (self._parent._hash, report_metric_name, data_source)
         cache_key += (aggregate,) if aggregate is None else tuple(aggregate)
 
-        if metric_kwargs != {}:
-            # we need to enforce the order of the parameter for a specific metric
-            # to make sure that we hit the cache in a consistent way
-            ordered_metric_kwargs = sorted(metric_kwargs.keys())
-
-            for key in ordered_metric_kwargs:
-                if isinstance(metric_kwargs[key], (np.ndarray, list, dict)):
-                    cache_key += (joblib.hash(metric_kwargs[key]),)
-                else:
-                    cache_key += (metric_kwargs[key],)
+        # we need to enforce the order of the parameter for a specific metric
+        # to make sure that we hit the cache in a consistent way
+        ordered_metric_kwargs = sorted(metric_kwargs.keys())
+
+        for key in ordered_metric_kwargs:
+            if isinstance(metric_kwargs[key], (np.ndarray, list, dict)):
+                cache_key += (joblib.hash(metric_kwargs[key]),)
+            else:
+                cache_key += (metric_kwargs[key],)
 
         progress = self._progress_info["current_progress"]
         main_task = self._progress_info["current_task"]
diff --git a/skore/src/skore/sklearn/_cross_validation/report.py b/skore/src/skore/sklearn/_cross_validation/report.py
index a767e21..978a4a9 100644
--- a/skore/src/skore/sklearn/_cross_validation/report.py
+++ b/skore/src/skore/sklearn/_cross_validation/report.py
@@ -84,13 +84,12 @@ class CrossValidationReport(_BaseReport, DirNamesMixin):
 
     See Also
     --------
-    skore.sklearn.estimator.report.EstimatorReport :
+    skore.EstimatorReport
         Report for a fitted estimator.
 
     Examples
     --------
     >>> from sklearn.datasets import make_classification
-    >>> from sklearn.model_selection import train_test_split
     >>> from sklearn.linear_model import LogisticRegression
     >>> X, y = make_classification(random_state=42)
     >>> estimator = LogisticRegression()
diff --git a/skore/src/skore/sklearn/_estimator/metrics_accessor.py b/skore/src/skore/sklearn/_estimator/metrics_accessor.py
index 95795c8..b9e7462 100644
--- a/skore/src/skore/sklearn/_estimator/metrics_accessor.py
+++ b/skore/src/skore/sklearn/_estimator/metrics_accessor.py
@@ -352,18 +352,18 @@ class _MetricsAccessor(_BaseAccessor, DirNamesMixin):
         metric_params = inspect.signature(metric_fn).parameters
         if "pos_label" in metric_params:
             cache_key += (pos_label,)
-        if metric_kwargs != {}:
-            # we need to enforce the order of the parameter for a specific metric
-            # to make sure that we hit the cache in a consistent way
-            ordered_metric_kwargs = sorted(metric_kwargs.keys())
-            cache_key += tuple(
-                (
-                    joblib.hash(metric_kwargs[key])
-                    if isinstance(metric_kwargs[key], np.ndarray)
-                    else metric_kwargs[key]
-                )
-                for key in ordered_metric_kwargs
+
+        # we need to enforce the order of the parameter for a specific metric
+        # to make sure that we hit the cache in a consistent way
+        ordered_metric_kwargs = sorted(metric_kwargs.keys())
+        cache_key += tuple(
+            (
+                joblib.hash(metric_kwargs[key])
+                if isinstance(metric_kwargs[key], np.ndarray)
+                else metric_kwargs[key]
             )
+            for key in ordered_metric_kwargs
+        )
 
         if cache_key in self._parent._cache:
             score = self._parent._cache[cache_key]
diff --git a/sphinx/api/skore.config_context.rst b/sphinx/api/skore.config_context.rst
new file mode 100644
index 0000000..b64a9f9
--- /dev/null
+++ b/sphinx/api/skore.config_context.rst
@@ -0,0 +1,10 @@
+﻿config\_context
+===============
+
+.. currentmodule:: skore
+
+.. autofunction:: config_context
+
+.. minigallery:: skore.config_context
+   :add-heading: Gallery examples
+   :heading-level: -
\ No newline at end of file
diff --git a/sphinx/api/skore.get_config.rst b/sphinx/api/skore.get_config.rst
new file mode 100644
index 0000000..0f76d78
--- /dev/null
+++ b/sphinx/api/skore.get_config.rst
@@ -0,0 +1,10 @@
+﻿get\_config
+===========
+
+.. currentmodule:: skore
+
+.. autofunction:: get_config
+
+.. minigallery:: skore.get_config
+   :add-heading: Gallery examples
+   :heading-level: -
\ No newline at end of file
diff --git a/sphinx/api/skore.set_config.rst b/sphinx/api/skore.set_config.rst
new file mode 100644
index 0000000..de1045f
--- /dev/null
+++ b/sphinx/api/skore.set_config.rst
@@ -0,0 +1,10 @@
+﻿set\_config
+===========
+
+.. currentmodule:: skore
+
+.. autofunction:: set_config
+
+.. minigallery:: skore.set_config
+   :add-heading: Gallery examples
+   :heading-level: -
\ No newline at end of file
diff --git a/sphinx/index.rst b/sphinx/index.rst
index d09f653..56a592a 100644
--- a/sphinx/index.rst
+++ b/sphinx/index.rst
@@ -41,9 +41,11 @@ Key features
       All these are computed and generated for you in 1 line of code.
       Under the hood, we use efficient caching to make the computations blazing fast.
 
-   -  :class:`skore.CrossValidationReport`: Get a skore estimator report for each fold
+   -  :class:`skore.CrossValidationReport`: get a skore estimator report for each fold
       of your cross-validation.
 
+   -  :class:`skore.ComparisonReport`: benchmark your skore estimator reports.
+
 What's next?
 """"""""""""
 
diff --git a/sphinx/reference/report/comparison_report.rst b/sphinx/reference/report/comparison_report.rst
new file mode 100644
index 0000000..7158cc5
--- /dev/null
+++ b/sphinx/reference/report/comparison_report.rst
@@ -0,0 +1,50 @@
+Report for a comparison of :class:`EstimatorReport`
+===================================================
+
+.. currentmodule:: skore
+
+The class :class:`ComparisonReport` provides a report allowing to compare :class:`EstimatorReport` instances in an interactive way. The functionalities of the report are accessible through accessors.
+
+.. autosummary::
+    :toctree: ../api/
+    :template: base.rst
+
+    ComparisonReport
+
+.. autosummary::
+    :toctree: ../api/
+    :nosignatures:
+    :template: autosummary/accessor_method.rst
+
+    ComparisonReport.help
+
+.. autosummary::
+    :toctree: ../api/
+    :nosignatures:
+    :template: autosummary/accessor.rst
+
+    ComparisonReport.metrics
+
+Metrics
+-------
+
+The `metrics` accessor helps you to evaluate the statistical performance of the
+compared estimators. In addition, we provide a sub-accessor `plot`, to
+get the common performance metric representations.
+
+.. autosummary::
+    :toctree: ../api/
+    :nosignatures:
+    :template: autosummary/accessor_method.rst
+
+    ComparisonReport.metrics.help
+    ComparisonReport.metrics.report_metrics
+    ComparisonReport.metrics.custom_metric
+    ComparisonReport.metrics.accuracy
+    ComparisonReport.metrics.brier_score
+    ComparisonReport.metrics.log_loss
+    ComparisonReport.metrics.precision
+    ComparisonReport.metrics.r2
+    ComparisonReport.metrics.recall
+    ComparisonReport.metrics.rmse
+    ComparisonReport.metrics.roc_auc
diff --git a/sphinx/reference/report/index.rst b/sphinx/reference/report/index.rst
index 2a87d82..17fb633 100644
--- a/sphinx/reference/report/index.rst
+++ b/sphinx/reference/report/index.rst
@@ -42,6 +42,18 @@ scikit-learn estimators by cross-validation, and reporting the results.
 
    cross_validation_report
 
+Comparison Report
+-----------------------
+
+:class:`skore.ComparisonReport` provides comprehensive capabilities for comparing
+:class:`skore.EstimatorReport` instances, and reporting the results.
+
+.. toctree::
+   :maxdepth: 2
+   :hidden:
+
+   comparison_report
+
 Visualization Displays
 ----------------------
 


Test patch:
diff --git a/skore/tests/conftest.py b/skore/tests/conftest.py
index bfcce26..a7e00c3 100644
--- a/skore/tests/conftest.py
+++ b/skore/tests/conftest.py
@@ -43,6 +43,7 @@ def in_memory_project(monkeypatch):
     project.path = None
     project.name = "test"
     project._item_repository = ItemRepository(storage=InMemoryStorage())
+    project._storage_initialized = True
 
     return project
 
diff --git a/skore/tests/unit/project/test_project.py b/skore/tests/unit/project/test_project.py
index 145a642..c5c4202 100644
--- a/skore/tests/unit/project/test_project.py
+++ b/skore/tests/unit/project/test_project.py
@@ -12,6 +12,7 @@ from matplotlib.testing.compare import compare_images
 from PIL import Image
 from sklearn.ensemble import RandomForestClassifier
 from skore import Project
+from skore.project.project import ProjectDeletedError
 
 
 @pytest.fixture(autouse=True)
@@ -49,6 +50,21 @@ def test_clear(tmp_path):
     assert project.keys() == []
     assert project._item_repository.keys() == []
 
+    assert dirpath.exists()
+
+
+def test_clear_delete_project(tmp_path):
+    dirpath = tmp_path / "my-project.skore"
+    project = Project(dirpath)
+
+    project.clear(delete_project=True)
+    assert not dirpath.exists()
+
+    with pytest.raises(
+        ProjectDeletedError, match="This Project instance is marked as deleted"
+    ):
+        project.keys()
+
 
 def test_put_string_item(in_memory_project):
     in_memory_project.put("string_item", "Hello, World!")
diff --git a/skore/tests/unit/sklearn/test_comparison.py b/skore/tests/unit/sklearn/test_comparison.py
new file mode 100644
index 0000000..622d06f
--- /dev/null
+++ b/skore/tests/unit/sklearn/test_comparison.py
@@ -0,0 +1,536 @@
+import re
+from io import BytesIO
+
+import joblib
+import pandas as pd
+import pytest
+from sklearn.datasets import make_classification
+from sklearn.linear_model import LinearRegression, LogisticRegression
+from sklearn.model_selection import train_test_split
+from skore import ComparisonReport, EstimatorReport
+
+
+@pytest.fixture
+def binary_classification_model():
+    """Create a binary classification dataset and return fitted estimator and data."""
+    X, y = make_classification(random_state=42)
+    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+
+    return LogisticRegression(random_state=42), X_train, X_test, y_train, y_test
+
+
+@pytest.fixture
+def regression_model():
+    """Create a binary classification dataset and return fitted estimator and data."""
+    X, y = make_classification(random_state=42)
+    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
+
+    return LinearRegression(), X_train, X_test, y_train, y_test
+
+
+def test_comparison_report_init_wrong_parameters(binary_classification_model):
+    """If the input is not valid, raise."""
+
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    with pytest.raises(TypeError, match="Expected reports to be an iterable"):
+        ComparisonReport(estimator_report)
+
+    with pytest.raises(
+        ValueError, match="At least 2 instances of EstimatorReport are needed"
+    ):
+        ComparisonReport([estimator_report])
+
+    with pytest.raises(TypeError, match="Expected instances of EstimatorReport"):
+        ComparisonReport([None, estimator_report])
+
+
+def test_comparison_report_without_testing_data(binary_classification_model):
+    """If there is no test data (`None`) for some estimator report,
+    initialization works, but computing metrics can fail.
+    """
+    estimator, _, _, _, _ = binary_classification_model
+    estimator_report = EstimatorReport(estimator, fit=False)
+
+    report = ComparisonReport([estimator_report, estimator_report])
+
+    with pytest.raises(ValueError, match="No test data"):
+        report.metrics.report_metrics(data_source="test")
+
+
+def test_comparison_report_different_test_data(binary_classification_model):
+    """Raise an error if the passed estimators do not have the same testing data."""
+    estimator, X_train, X_test, y_train, y_test = binary_classification_model
+    estimator.fit(X_train, y_train)
+
+    # The estimators that have testing data, need to have the same testing data
+    # The estimators that do not have testing data do not count
+    with pytest.raises(
+        ValueError, match="Expected all estimators to have the same testing data"
+    ):
+        ComparisonReport(
+            [
+                EstimatorReport(estimator, X_test=X_test, y_test=y_test),
+                EstimatorReport(estimator, X_test=X_test[1:], y_test=y_test[1:]),
+            ]
+        )
+
+    # The estimators without testing data (i.e. no X_test and no y_test) do not count
+    ComparisonReport(
+        [
+            EstimatorReport(estimator, X_test=X_test, y_test=y_test),
+            EstimatorReport(estimator, X_test=X_test, y_test=y_test),
+            EstimatorReport(estimator),
+        ]
+    )
+
+    # If there is an X_test but no y_test, it counts
+    with pytest.raises(
+        ValueError, match="Expected all estimators to have the same testing data"
+    ):
+        ComparisonReport(
+            [
+                EstimatorReport(estimator, fit=False, X_test=X_test, y_test=y_test),
+                EstimatorReport(estimator, fit=False, X_test=X_test),
+            ]
+        )
+
+
+def test_comparison_report_init_different_ml_usecases(
+    binary_classification_model, regression_model
+):
+    """Raise an error if the passed estimators do not have the same ML usecase."""
+    linear_regression_estimator, _, X_test, _, y_test = regression_model
+    linear_regression_report = EstimatorReport(
+        linear_regression_estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    logistic_regression_estimator, _, X_test, _, y_test = binary_classification_model
+    logistic_regression_report = EstimatorReport(
+        logistic_regression_estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    with pytest.raises(
+        ValueError, match="Expected all estimators to have the same ML usecase"
+    ):
+        ComparisonReport([linear_regression_report, logistic_regression_report])
+
+
+def test_comparison_report_init_with_report_names(binary_classification_model):
+    """If the estimators are passed as a dict,
+    then the estimator names are the dict keys."""
+    estimator, X_train, X_test, y_train, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        X_train=X_train,
+        y_train=y_train,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    comp = ComparisonReport({"r1": estimator_report, "r2": estimator_report})
+
+    pd.testing.assert_index_equal(
+        comp.metrics.accuracy().columns,
+        pd.Index(["r1", "r2"], name="Estimator"),
+    )
+
+
+def test_comparison_report_init_without_report_names(binary_classification_model):
+    """If the estimators are passed as a list,
+    then the estimator names are the estimator class names."""
+    estimator, X_train, X_test, y_train, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        X_train=X_train,
+        y_train=y_train,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    comp = ComparisonReport([estimator_report, estimator_report])
+
+    pd.testing.assert_index_equal(
+        comp.metrics.accuracy().columns,
+        pd.Index(["LogisticRegression", "LogisticRegression"], name="Estimator"),
+    )
+
+
+def test_comparison_report_non_string_report_names(binary_classification_model):
+    """If the estimators are passed as a dict with non-string keys,
+    then the estimator names are the dict keys converted to strings."""
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    report = ComparisonReport({0: estimator_report, "1": estimator_report})
+    assert report.report_names_ == ["0", "1"]
+
+
+def test_comparison_report_help(capsys, binary_classification_model):
+    """Check the help menu works."""
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    ComparisonReport([estimator_report, estimator_report]).help()
+
+    captured = capsys.readouterr()
+    assert "Tools to compare estimators" in captured.out
+
+    # Check that we have a line with accuracy and the arrow associated with it
+    assert re.search(
+        r"\.accuracy\([^)]*\).*\(↗︎\).*-.*accuracy", captured.out, re.MULTILINE
+    )
+
+
+def test_comparison_report_repr(binary_classification_model):
+    """Check the `__repr__` works."""
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    repr_str = repr(ComparisonReport([estimator_report, estimator_report]))
+
+    assert "ComparisonReport" in repr_str
+
+
+def test_comparison_report_pickle(tmp_path, binary_classification_model):
+    """Check that we can pickle a comparison report."""
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    with BytesIO() as stream:
+        joblib.dump(ComparisonReport([estimator_report, estimator_report]), stream)
+
+
+def test_comparison_report_metrics_help(capsys, binary_classification_model):
+    """Check that the help method writes to the console."""
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+    report = ComparisonReport([estimator_report, estimator_report])
+
+    report.metrics.help()
+    captured = capsys.readouterr()
+    assert "Available metrics methods" in captured.out
+
+
+def test_comparison_report_metrics_repr(binary_classification_model):
+    """Check the repr method."""
+    estimator, _, X_test, _, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        fit=False,
+        X_test=X_test,
+        y_test=y_test,
+    )
+    report = ComparisonReport([estimator_report, estimator_report])
+
+    repr_str = repr(report.metrics)
+    assert "skore.ComparisonReport.metrics" in repr_str
+    assert "report.metrics.help()" in repr_str
+
+
+@pytest.mark.parametrize("data_source", ["test", "X_y"])
+@pytest.mark.parametrize(
+    "metric_name, expected",
+    [
+        (
+            "accuracy",
+            pd.DataFrame(
+                [[1.0, 1.0]],
+                columns=pd.Index(
+                    ["LogisticRegression", "LogisticRegression"],
+                    name="Estimator",
+                ),
+                index=pd.Index(["Accuracy (↗︎)"], dtype="object", name="Metric"),
+            ),
+        ),
+        (
+            "precision",
+            pd.DataFrame(
+                [[1.0, 1.0], [1.0, 1.0]],
+                columns=pd.Index(
+                    ["LogisticRegression", "LogisticRegression"],
+                    name="Estimator",
+                ),
+                index=pd.MultiIndex.from_tuples(
+                    [("Precision (↗︎)", 0), ("Precision (↗︎)", 1)],
+                    names=["Metric", "Label / Average"],
+                ),
+            ),
+        ),
+        (
+            "recall",
+            pd.DataFrame(
+                [[1.0, 1.0], [1.0, 1.0]],
+                columns=pd.Index(
+                    ["LogisticRegression", "LogisticRegression"],
+                    name="Estimator",
+                ),
+                index=pd.MultiIndex.from_tuples(
+                    [("Recall (↗︎)", 0), ("Recall (↗︎)", 1)],
+                    names=["Metric", "Label / Average"],
+                ),
+            ),
+        ),
+        (
+            "brier_score",
+            pd.DataFrame(
+                [[0.026684, 0.026684]],
+                columns=pd.Index(
+                    ["LogisticRegression", "LogisticRegression"],
+                    name="Estimator",
+                ),
+                index=pd.Index(["Brier score (↘︎)"], dtype="object", name="Metric"),
+            ),
+        ),
+        (
+            "roc_auc",
+            pd.DataFrame(
+                [[1.0, 1.0]],
+                columns=pd.Index(
+                    ["LogisticRegression", "LogisticRegression"],
+                    name="Estimator",
+                ),
+                index=pd.Index(["ROC AUC (↗︎)"], dtype="object", name="Metric"),
+            ),
+        ),
+        (
+            "log_loss",
+            pd.DataFrame(
+                [[0.113233, 0.113233]],
+                columns=pd.Index(
+                    ["LogisticRegression", "LogisticRegression"],
+                    name="Estimator",
+                ),
+                index=pd.Index(["Log loss (↘︎)"], dtype="object", name="Metric"),
+            ),
+        ),
+    ],
+)
+def test_comparison_report_metrics_binary_classification(
+    metric_name, expected, data_source, binary_classification_model
+):
+    """Check the metrics work."""
+    estimator, X_train, X_test, y_train, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        X_train=X_train,
+        y_train=y_train,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    comp = ComparisonReport([estimator_report, estimator_report])
+
+    # ensure metric is valid
+    if data_source == "X_y":
+        result = getattr(comp.metrics, metric_name)(
+            data_source=data_source, X=X_test, y=y_test
+        )
+    else:
+        result = getattr(comp.metrics, metric_name)(data_source=data_source)
+    pd.testing.assert_frame_equal(result, expected)
+
+    # ensure metric is valid even from the cache
+    if data_source == "X_y":
+        result = getattr(comp.metrics, metric_name)(
+            data_source=data_source, X=X_test, y=y_test
+        )
+    else:
+        result = getattr(comp.metrics, metric_name)(data_source=data_source)
+    pd.testing.assert_frame_equal(result, expected)
+
+
+@pytest.mark.parametrize("data_source", ["test", "X_y"])
+@pytest.mark.parametrize(
+    "metric_name, expected",
+    [
+        (
+            "rmse",
+            pd.DataFrame(
+                [[0.27699, 0.27699]],
+                columns=pd.Index(
+                    ["LinearRegression", "LinearRegression"],
+                    name="Estimator",
+                ),
+                index=pd.Index(["RMSE (↘︎)"], dtype="object", name="Metric"),
+            ),
+        ),
+        (
+            "r2",
+            pd.DataFrame(
+                [[0.680319, 0.680319]],
+                columns=pd.Index(
+                    ["LinearRegression", "LinearRegression"],
+                    name="Estimator",
+                ),
+                index=pd.Index(["R² (↗︎)"], dtype="object", name="Metric"),
+            ),
+        ),
+    ],
+)
+def test_comparison_report_metrics_linear_regression(
+    metric_name, expected, data_source, regression_model
+):
+    """Check the metrics work."""
+    estimator, X_train, X_test, y_train, y_test = regression_model
+    estimator_report = EstimatorReport(
+        estimator,
+        X_train=X_train,
+        y_train=y_train,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    comp = ComparisonReport([estimator_report, estimator_report])
+
+    # ensure metric is valid
+    if data_source == "X_y":
+        result = getattr(comp.metrics, metric_name)(
+            data_source=data_source, X=X_test, y=y_test
+        )
+    else:
+        result = getattr(comp.metrics, metric_name)()
+    pd.testing.assert_frame_equal(result, expected)
+
+    # ensure metric is valid even from the cache
+    if data_source == "X_y":
+        result = getattr(comp.metrics, metric_name)(
+            data_source=data_source, X=X_test, y=y_test
+        )
+    else:
+        result = getattr(comp.metrics, metric_name)()
+    pd.testing.assert_frame_equal(result, expected)
+
+
+def test_comparison_report_report_metrics_X_y(binary_classification_model):
+    """Check that `report_metrics` works with an "X_y" data source."""
+    estimator, X_train, X_test, y_train, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        X_train=X_train,
+        y_train=y_train,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    comp = ComparisonReport([estimator_report, estimator_report])
+
+    result = comp.metrics.report_metrics(
+        data_source="X_y",
+        X=X_train[:10],
+        y=y_train[:10],
+    )
+
+    expected = pd.DataFrame(
+        [
+            [1.0, 1.0],
+            [1.0, 1.0],
+            [1.0, 1.0],
+            [1.0, 1.0],
+            [1.0, 1.0],
+            [0.01514976, 0.01514976],
+        ],
+        columns=pd.Index(
+            ["LogisticRegression", "LogisticRegression"],
+            name="Estimator",
+        ),
+        index=pd.MultiIndex.from_tuples(
+            [
+                ("Precision (↗︎)", 0),
+                ("Precision (↗︎)", 1),
+                ("Recall (↗︎)", 0),
+                ("Recall (↗︎)", 1),
+                ("ROC AUC (↗︎)", ""),
+                ("Brier score (↘︎)", ""),
+            ],
+            names=["Metric", "Label / Average"],
+        ),
+    )
+    pd.testing.assert_frame_equal(result, expected)
+
+    assert len(comp._cache) == 1
+    cached_result = list(comp._cache.values())[0]
+    pd.testing.assert_frame_equal(cached_result, expected)
+
+
+def test_comparison_report_custom_metric_X_y(binary_classification_model):
+    """Check that `custom_metric` works with an "X_y" data source."""
+    from sklearn.metrics import mean_absolute_error
+
+    estimator, X_train, X_test, y_train, y_test = binary_classification_model
+    estimator_report = EstimatorReport(
+        estimator,
+        X_train=X_train,
+        y_train=y_train,
+        X_test=X_test,
+        y_test=y_test,
+    )
+
+    comp = ComparisonReport([estimator_report, estimator_report])
+
+    expected = pd.DataFrame(
+        [[0.0, 0.0]],
+        columns=pd.Index(
+            ["LogisticRegression", "LogisticRegression"], name="Estimator"
+        ),
+        index=pd.Index(["MAE (↗︎)"], name="Metric"),
+    )
+
+    # ensure metric is valid
+    result = comp.metrics.custom_metric(
+        metric_function=mean_absolute_error,
+        response_method="predict",
+        metric_name="MAE (↗︎)",
+        data_source="X_y",
+        X=X_test,
+        y=y_test,
+    )
+    pd.testing.assert_frame_equal(result, expected)
+
+    # ensure metric is valid even from the cache
+    result = comp.metrics.custom_metric(
+        metric_function=mean_absolute_error,
+        response_method="predict",
+        metric_name="MAE (↗︎)",
+        data_source="X_y",
+        X=X_test,
+        y=y_test,
+    )
+    pd.testing.assert_frame_equal(result, expected)
diff --git a/skore/tests/unit/sklearn/test_estimator.py b/skore/tests/unit/sklearn/test_estimator.py
index 24f0aae..a49c2e2 100644
--- a/skore/tests/unit/sklearn/test_estimator.py
+++ b/skore/tests/unit/sklearn/test_estimator.py
@@ -1,5 +1,6 @@
 import re
 from copy import deepcopy
+from io import BytesIO
 from numbers import Real
 
 import joblib
@@ -335,7 +336,7 @@ def test_estimator_report_cache_predictions(
     assert report._cache.keys() == stored_cache.keys()
 
 
-def test_estimator_report_pickle(tmp_path, binary_classification_data):
+def test_estimator_report_pickle(binary_classification_data):
     """Check that we can pickle an estimator report.
 
     In particular, the progress bar from rich are pickable, therefore we trigger
@@ -344,7 +345,9 @@ def test_estimator_report_pickle(tmp_path, binary_classification_data):
     estimator, X_test, y_test = binary_classification_data
     report = EstimatorReport(estimator, X_test=X_test, y_test=y_test)
     report.cache_predictions()
-    joblib.dump(report, tmp_path / "report.joblib")
+
+    with BytesIO() as stream:
+        joblib.dump(report, stream)
 
 
 ########################################################################################