UX: Enhance the HTML displays

### Describe the workflow you want to enable

When I interact when non-advanced users a recurrent difficulty for them is finding information and understanding what is going on.



### Describe your proposed solution

I think that we can guide users with better html displays.  In general, what would be desirable is to give ways to the users to access all the information that an estimator knows about itself, but avoiding to add any lengthy computation during fit. Of course the difficulty of any UX which is that adding more information leads to crowding, and thus the UX needs to be kept light and focused.

I propose to do changes in an iterative way, adding one feature after the other. Here are the ideas that I have in mind:

* [ ] Display the result of "get_params" (not visible by default, either folded or in a hover)
   * [x] #21266
* [x] Add a link to the API documentation. This link would be inferred from the version of the module, the import path and the name of the class. For instance sklearn.cluster._spectral.SpectralClustering would lead to https://scikit-learn.org/1.2/modules/generated/sklearn.cluster.SpectralClustering.html . Note that we will have to apply heuristics such as dropping the last modules in the path if they are private. Also, we will have to be careful to cater for non scikit-learn classes inheriting from our BaseEstimator, and thus define an override mechanism and probably check that the imported module corresponds to the one for which the path was defined
* [x] Display in a light way whether the estimator has been fit or not
* [ ] Display the estimator's parameters
  * [ ] Add a "?" symbol redirecting to the parameter documentation (cf. https://github.com/scikit-learn/scikit-learn/pull/30763#issuecomment-2737192403)
* [ ] Display the (public) fitted attributes
  * [ ] at least dtype and shape for array-valued attributes
  * [ ] maybe a few summary statistics for array-valued attributes.
* [ ] Display the methods of an estimator with a tooltip with a documentation portion
* [ ] Display the feature names
* [ ] Display the shape of outgoing data structures
* [ ] Reorganize the HTML diagram to have a more condensed view or a more vertical view
* [ ] Fix the string representation (not displays) using a more vertical appearance (cf. black style)

In terms of plan, I propose to first adapt our current display without changing its main philosophy. Hence we need to add light accessors of the information, and not a huge list of things (think "mac", design).

Another important thing to keep in mind is that, for users, the hardest things to comprehend are composite estimators, such as pipelines. Most users do not understand how they can access internal objects in these.

cc @amueller who, if I understand correctly, has been pushing these ideas for a long time. Also cc @thomasjpfan who has always shown impressive skills at html.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

We should make sure that the displays work and are easy to view in all the relevant environments: jupyter notebooks, vscode.

This means that avoid javascript. If needed, we can consider using https://purecss.io/ for buttons, tabs, ...

COMMENT:
> Is this display enhancement only for using sklearn in notebooks?

Or in vscode, or any other environment supporting html displays, which has become quite standard these days.

> how some of those things you mentioned could be implemented for those running sklearn in scripts on their terminal.

Scripts run in terminal are much less interactive than an html display and we cannot develop rich visual UX in them. The users running in such environments can always rely on the classic object-oriented API of scikit-learn to access all these information.

In addition, if it is an actual script, running mostly without user interaction, the problem is more related to logging, on which @jeremiedbb is working to enable much more powerful tracking of execution.

> —
> Reply to this email directly, view it on GitHub, or unsubscribe.
> You are receiving this because you authored the thread.*Message ID: 
> ***@***.***>


-- 
    Gael Varoquaux
    Research Director, INRIA
    http://gael-varoquaux.info            http://twitter.com/GaelVaroquaux

Labels: New Feature, RFC