Repository: scikit-learn/scikit-learn
Pull request: FIX Fix warning for ElasticNetCV l1_ratio=0
#### Reference Issues/PRs
Fixes #21644.

#### What does this implement/fix? Explain your changes.
The warning for `alpha=0` is modified in `cd_fast.enet_coordinate_descent_gram`.

#### Any other comments?
Maybe another PR can be done to modify the parameter names in `cd_fast` enet gradient descent methods from `alpha` and `beta` into `l1_reg` and `l2_reg` which will be more concise.


Problem statement:
ElasticNetCV call with l1_ratio=0 produces warning incorrectly identifying alpha=0 as the issue
Run the following:

```python
from sklearn.linear_model import ElasticNetCV
from sklearn.datasets import make_regression
X, y = make_regression(n_features=2, random_state=0)
test = ElasticNetCV(l1_ratio=[0, 0], alphas=[1, 1])
test.fit(X, y)
```

This results in the warning:

```pytb
Warning (from warnings module):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py", line 526
    model = cd_fast.enet_coordinate_descent_gram(
UserWarning: Coordinate descent with alpha=0 may lead to unexpected results and is discouraged.
```
System information:

```bash
System:
    python: 3.9.6 (v3.9.6:db3ff76da1, Jun 28 2021, 11:49:53)  [Clang 6.0 (clang-600.0.57)]
executable: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9
   machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.2
   setuptools: 56.0.0
      sklearn: 0.24.2
        numpy: 1.21.1
        scipy: 1.7.1
       Cython: None
       pandas: 1.3.1
   matplotlib: None
       joblib: 1.0.1
threadpoolctl: 2.2.0

Built with OpenMP: True
```

Hints from issue discussion:
I can reproduce with `main`.
### Potential problem

I think the problem is that for the gradient descent of LassoCV and ElasticNetCV, `enet_coordinate_descent_gram()` is called which takes in `alpha` and `beta` to minimize `(1/2) * ||y - Xw||^2_2 + alpha * ||w||_1 + (beta/2) * ||w||^2_2 `.

https://github.com/scikit-learn/scikit-learn/blob/226da0d7c458816776549c2580abaa4782dc4c48/sklearn/linear_model/_cd_fast.pyx#L478-L495

But the `enet_path()` used in ElasticNetCV and LassoCV is trying to minimize `1 / (2 * n_samples) * ||y - Xw||^2_2 + alpha * l1_ratio * ||w||_1 + 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2`. (According to [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.ElasticNet.html)). It achieves this by accounting for `l1_ratio` and `n_samples` with the calculations shown below.

https://github.com/scikit-learn/scikit-learn/blob/226da0d7c458816776549c2580abaa4782dc4c48/sklearn/linear_model/_coordinate_descent.py#L604-L607

https://github.com/scikit-learn/scikit-learn/blob/226da0d7c458816776549c2580abaa4782dc4c48/sklearn/linear_model/_coordinate_descent.py#L633-L645

The problem does not occur in LassoCV as `l1_ratio=1` is always passed.

### Proposed Fix

We can follow the function parameters of `enet_coordinate_descent_multi_task()` and allow `enet_coordinate_descent_gram()` to take in `l1_reg` and `l2_reg` instead of `alpha` and `beta`? It also has this warning `"Coordinate descent with l1_reg=0 may lead to unexpected results and is discouraged."` which will be suitable for this case too.

Commit message: Apply suggestions from code review

Co-authored-by: Julien Jerphanion <git@jjerphan.xyz>

Patch to review:
diff --git a/doc/whats_new/v1.1.rst b/doc/whats_new/v1.1.rst
index 61d5c64255f71..c30ee4b2fc239 100644
--- a/doc/whats_new/v1.1.rst
+++ b/doc/whats_new/v1.1.rst
@@ -319,6 +319,10 @@ Changelog
   :pr:`21481` by :user:`Guillaume Lemaitre <glemaitre>` and
   :user:`Andrés Babino <ababino>`.
 
+- |Fix| :class:`linear_model.ElasticNetCV` now produces correct
+  warning when `l1_ratio=0`.
+  :pr:`21724` by :user:`Yar Khine Phyo <yarkhinephyo>`.
+
 :mod:`sklearn.metrics`
 ......................
 
diff --git a/sklearn/linear_model/_cd_fast.pyx b/sklearn/linear_model/_cd_fast.pyx
index 338ca0f0cae7e..19b4d0a6abd10 100644
--- a/sklearn/linear_model/_cd_fast.pyx
+++ b/sklearn/linear_model/_cd_fast.pyx
@@ -532,8 +532,9 @@ def enet_coordinate_descent_gram(floating[::1] w,
     tol = tol * y_norm2
 
     if alpha == 0:
-        warnings.warn("Coordinate descent with alpha=0 may lead to unexpected"
-            " results and is discouraged.")
+        warnings.warn("Coordinate descent without L1 regularization may "
+            "lead to unexpected results and is discouraged. "
+            "Set l1_ratio > 0 to add L1 regularization.")
 
     with nogil:
         for n_iter in range(max_iter):
diff --git a/sklearn/linear_model/tests/test_coordinate_descent.py b/sklearn/linear_model/tests/test_coordinate_descent.py
index a94954f089148..03dd3919ae5e6 100644
--- a/sklearn/linear_model/tests/test_coordinate_descent.py
+++ b/sklearn/linear_model/tests/test_coordinate_descent.py
@@ -1280,6 +1280,15 @@ def test_enet_l1_ratio():
     with pytest.raises(ValueError, match=msg):
         MultiTaskElasticNetCV(l1_ratio=0, random_state=42).fit(X, y[:, None])
 
+    # Test that l1_ratio=0 with alpha>0 produces user warning
+    warning_message = (
+        "Coordinate descent without L1 regularization may "
+        "lead to unexpected results and is discouraged. "
+        "Set l1_ratio > 0 to add L1 regularization."
+   )
+    with pytest.warns(UserWarning, match=warning_message):
+        ElasticNetCV(l1_ratio=[0], alphas=[1]).fit(X, y)
+
     # Test that l1_ratio=0 is allowed if we supply a grid manually
     alphas = [0.1, 10]
     estkwds = {"alphas": alphas, "random_state": 42}


Code review comments:
diff_hunk: @@ -1280,6 +1280,15 @@ def test_enet_l1_ratio():
     with pytest.raises(ValueError, match=msg):
         MultiTaskElasticNetCV(l1_ratio=0, random_state=42).fit(X, y[:, None])
 
+    # Test that l1_ratio=0 with alpha>0 produces user warning
+    warning_message = (
+        "Coordinate descent without L1 regularization may "
+        "lead to unexpected results and is discouraged. "
+        "Set l1_ratio > 0 to add L1 regularization."
+   )
path: sklearn/linear_model/tests/test_coordinate_descent.py
text: @user1:
This should fix the linting.
```suggestion
    )
```