Repository: scikit-learn/scikit-learn
Pull request: DOC Standardizing the documentation for `alpha` in `Ridge` and `Lasso`
#### Reference Issues/PRs
Fixes #18692
Resolves #18781 (stalled)

#### What does this implement/fix? Explain your changes.
PR #18781: Makes documentation for `alpha` parameter consistent between `Ridge` and `Lasso` classes.
This PR: Addresses review comments, updates w.r.t `main`, completes formatting, minor wording changes
in the shared `alpha` description

#### Any other comments?

Problem statement:
Standardizing the documentation for `alpha` in `Ridge` and `Lasso`
#### Describe the issue linked to the documentation


I'm helping to teach a machine learning course using sklearn.  The documentation for `alpha` in  `Ridge` is as follows:

>Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to 1 / (2C) in other linear models such as LogisticRegression or sklearn.svm.LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number.

The documentation for `alpha` in `Lasso` is as follows
>Constant that multiplies the L1 term. Defaults to 1.0. alpha = 0 is equivalent to an ordinary least square, solved by the LinearRegression object. For numerical reasons, using alpha = 0 with the Lasso object is not advised. Given this, you should use the LinearRegression object.

Between the different documentation for alpha in these models, plus the `C` being the inverse regularization in the closely related `LogisticRegression` there has understandably been some confusion for myself and some students.


#### Suggest a potential alternative/fix


The two models are closely related, and I think standardizing the description of `alpha` to be the same in `Ridge` and `Lasso` (making appropriate L2/L1 changes respectively) would go a long way to preventing some confusion students have been encountering.  The documentation for alpha in `Lasso` seems to be most straightforward to me so I would propose using that description in both `Lasso` and `Ridge`.

Hints from issue discussion:
Agreed, the `alpha` definition in `Lasso` seems to be much more straightforward, however, the `alpha` definition in`Ridge` seems to be interesting for a kind of why the value of `alpha` defaults to `1.0`. I would suggest we make the `Lasso` definition in both places, and below that, at the bottom of the page, under `Notes`, the `Ridge` definition for `alpha` be added as a `Note`.

So that means, 

1. Change the `Ridge` alpha definition to that of `Lasso`'s
2. Add at the end of both the pages, as a note, under `Notes` about why we want `alpha` to be a positive float.

Does that sound good? Let me know so I can open a PR for that.
That sounds like a good approach to me
I'm not exactly familiar with .rst files under the `docs/` folder, however, I was able to find this, https://github.com/scikit-learn/scikit-learn/blob/master/doc/modules/linear_model.rst, it seems to contain some part of `Lasso`, and `Ridge`, but I can't find the definitions of `alpha` here. Let me take a take a look at Sphinx and RST files, and I'll get back to this. 
The doc is rendered on the scikit-learn website: https://scikit-learn.org/dev/modules/linear_model.html
Yes, @TomDLT ! I was assuming that it is how it should be built and routed to on the doc website.

The link for Lasso is here, https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html
The link for Ridge is here, https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Ridge.html

But when I go to https://github.com/scikit-learn/scikit-learn/tree/master/doc/modules, there is no folder named 'generated', as I was initially expecting. I then search for the word `lasso`, and did not find any files with that string in them. For `ridge`, I found kernel_ridge.rst, but that is unrelated to the issue.
The API page for each class is automatically generated from the docstring:
https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/_ridge.py#L603-L617
https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_model/_coordinate_descent.py#L904-L915
Thank you, @TomDLT , I'll look into them. 
@Dpananos please take a look at the PR. Thanks!

Commit message: Update sklearn/linear_model/_coordinate_descent.py

Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>

Patch to review:
diff --git a/sklearn/linear_model/_coordinate_descent.py b/sklearn/linear_model/_coordinate_descent.py
index 1813963717f7d..ecc2bb8c55f3f 100644
--- a/sklearn/linear_model/_coordinate_descent.py
+++ b/sklearn/linear_model/_coordinate_descent.py
@@ -1142,10 +1142,10 @@ class Lasso(ElasticNet):
     ----------
     alpha : float, default=1.0
         Constant that multiplies the L1 term. Defaults to 1.0.
-        ``alpha = 0`` is equivalent to an ordinary least square, solved
-        by the :class:`LinearRegression` object. For numerical
-        reasons, using ``alpha = 0`` with the ``Lasso`` object is not advised.
-        Given this, you should use the :class:`LinearRegression` object.
+        When `alpha = 0`, the objective is equivalent to ordinary least
+        squares, solved by the :class:`LinearRegression` object. For numerical
+        reasons, using `alpha = 0` with the `Lasso` object is not advised.
+        Instead, you should use the :class:`LinearRegression` object.
 
     fit_intercept : bool, default=True
         Whether to calculate the intercept for this model. If set
@@ -1249,6 +1249,15 @@ class Lasso(ElasticNet):
     To avoid unnecessary memory duplication the X argument of the fit method
     should be directly passed as a Fortran-contiguous numpy array.
 
+    Alpha must be a positive float for regularization strength.
+    Regularization improves the conditioning of the problem and
+    reduces the variance of the estimates. Larger values specify stronger
+    regularization. Alpha corresponds to `1 / (2C)` in other linear
+    models such as :class:`~sklearn.linear_model.LogisticRegression` or
+    :class:`~sklearn.svm.LinearSVC`. If an array is passed, penalties are
+    assumed to be specific to the targets. Hence they must correspond in
+    number.
+
     Examples
     --------
     >>> from sklearn import linear_model
diff --git a/sklearn/linear_model/_ridge.py b/sklearn/linear_model/_ridge.py
index b0312b2d7cd58..ed8621619de86 100644
--- a/sklearn/linear_model/_ridge.py
+++ b/sklearn/linear_model/_ridge.py
@@ -335,14 +335,11 @@ def ridge_regression(
         Target values
 
     alpha : float or array-like of shape (n_targets,)
-        Regularization strength; must be a positive float. Regularization
-        improves the conditioning of the problem and reduces the variance of
-        the estimates. Larger values specify stronger regularization.
-        Alpha corresponds to ``1 / (2C)`` in other linear models such as
-        :class:`~sklearn.linear_model.LogisticRegression` or
-        :class:`~sklearn.svm.LinearSVC`. If an array is passed, penalties are
-        assumed to be specific to the targets. Hence they must correspond in
-        number.
+        Constant that multiplies the L2 term. Defaults to 1.0.
+        When `alpha = 0`, the objective is equivalent to ordinary least
+        squares, solved by the :class:`LinearRegression` object. For numerical
+        reasons, using `alpha = 0` with the `Ridge` object is not advised.
+        Instead, you should use the :class:`LinearRegression` object.
 
     sample_weight : float or array-like of shape (n_samples,), default=None
         Individual weights for each sample. If given a float, every sample
@@ -450,6 +447,15 @@ def ridge_regression(
     Notes
     -----
     This function won't compute the intercept.
+
+    Alpha must be a positive float for regularization strenght.
+    Regularization improves the conditioning of the problem and
+    reduces the variance of the estimates. Larger values specify stronger
+    regularization. Alpha corresponds to ``1 / (2C)`` in other linear
+    models such as :class:`~sklearn.linear_model.LogisticRegression` or
+    :class:`~sklearn.svm.LinearSVC`. If an array is passed, penalties are
+    assumed to be specific to the targets. Hence they must correspond in
+    number.
     """
     return _ridge_regression(
         X,


Code review comments:
diff_hunk: @@ -1249,6 +1249,15 @@ class Lasso(ElasticNet):
     To avoid unnecessary memory duplication the X argument of the fit method
     should be directly passed as a Fortran-contiguous numpy array.
 
+    Alpha must be a positive float for regularization strength.
path: sklearn/linear_model/_coordinate_descent.py
text: @user1:
This sentence should be added in the `alpha` section instead I think. Now, we tend to provide the interval `[0, inf)` in this case.