Ridge replacement for normalize=True gives different results

> I will look more closely next week but even this breaks:
> 
> ```python
> from sklearn.datasets import make_regression
> from sklearn import linear_model
> from sklearn.pipeline import make_pipeline
> from sklearn.preprocessing import StandardScaler
> 
> X, y, w = make_regression(
>     n_samples=50, n_features=10, coef=True, random_state=1, bias=3.5
> )
> 
> reg = linear_model.Ridge(normalize=True, fit_intercept=True)
> 
> reg.fit(X, y)
> print('old method: ', reg.predict(X))
> 
> model = make_pipeline(
>     StandardScaler(),
>     linear_model.Ridge(fit_intercept=True)
> )
> model.fit(X, y)
> print('new method: ', model.predict(X))
> ```
> 
> now what is clear is that you cannot compare the coef_ of before and after as the new coef_ are now defined in the "scaled" space.

@agramfort @ogrisel 

Sorry to bother you, but recently I met the same question with this. Running this code with version 1.0.1 (I know scikit-learn version has been updated yet the result cannot replace the same while using the newest scikit-learn version) and gives the different predict result.


```
old method:  [  26.80677682    5.32421324   56.87586471   25.09288338  184.57441649
  115.28569814  -38.17096818  -65.48997965  -17.52520599 -226.23820926
   75.52428636   88.96624229   52.21631343  202.57225844  -10.37444071
    8.3514785    42.78798663  128.68167862  -77.69401655  -20.30215602
  100.45923692   63.45275614   16.66907894   13.352617     38.13886239
  160.41741094   60.77599418   27.49979846 -164.31228182  139.03996039
 -108.97657639  -85.44550151  -32.54404793   98.68209203   -3.78650194
   76.60412551   30.83213859  -57.04303337  -36.70378939   19.87053059
   10.46160413   59.68552735  -84.43707144  161.00989838  130.01474286
  142.26922884  -21.67004408   67.01494093   10.52746422   60.66652568]

new method:  [ -22.09709242  -55.21511813   88.13620228   12.6803576   265.56532329
  130.91815509  -47.89521848  -99.78794297  -34.20758396 -443.26431438
  127.84993514   83.74235635   54.93256409  323.09817918  -78.8014361
   -3.68818136   74.84222868  177.58494986 -101.26769575  -63.49599312
  159.94128969   99.99366425    6.24434524    9.08707909   22.20588518
  238.30879086   92.65443547   -1.4919619  -255.01677008  184.80310446
 -175.76075297 -153.55894015  -20.04319014  139.13339377  -12.92968222
  103.74804195   13.50686135 -155.65339945  -96.7916892   -37.25656423
   23.51943588   72.62895377 -156.84841568  276.06897426  268.15890225
  302.98621835  -84.12814864  102.74257583   34.30762102   59.60107436]
```
I tried to print 'reg.coef_', 'model["ridge"].coef_', 'model["standardscaler"].transform(model["ridge"].coef_.reshape(1, -1))', they are all different.

```
reg.coef_
Out[174]: 
array([-7.01140399,  4.3090024 , 45.44280547,  0.6417098 , 43.19423148,
       14.59269402, 16.13499612,  2.93863391, -1.2163097 , 25.35947689])

model["ridge"].coef_
Out[175]: 
array([ 6.43722048,  9.55541876, 92.12396593, 16.71544369, 78.87276786,
       34.76539027, 11.92974529, 16.35566485, 22.40027456, 50.58052386])

model["standardscaler"].transform(model["ridge"].coef_.reshape(1, -1))
Out[176]: 
array([[ 6.23213176, 10.29425352, 95.4903577 , 16.55780647, 77.38690859,
        34.21853949, 11.55574159, 19.25708867, 23.64343494, 49.54074724]])
```

So what's the reason for this? Or does this only appear on 'Ridge'? 

Thank you for your time and support.

_Originally posted by @Accelerator086 in https://github.com/scikit-learn/scikit-learn/issues/22540#issuecomment-1826847040_

COMMENT:
> Why are you not centering the data (i.e. `with_mean=False`)?

Thank you for your suggestion! I've written `with_mean=False` in the code above:

> Thank you for your guidance! I've made it and still I find that for the intercept and coefficient, codes for `Ridge()` should also change:
> 
> ```
> from sklearn.datasets import make_regression
> from sklearn import linear_model
> from sklearn.pipeline import make_pipeline
> from sklearn.preprocessing import StandardScaler
> 
> X, y, w = make_regression(
>     n_samples=50, n_features=10, coef=True, random_state=1, bias=3.5
> )
> 
> reg = linear_model.Ridge(alpha=1.0, normalize=True) # , fit_intercept=True
> reg.fit(X, y)
> print('old method: ', reg.predict(X) ,reg.coef_, reg.intercept_)
> 
> model = make_pipeline(
>     **StandardScaler(with_mean=False)**,
>     linear_model.Ridge(alpha=1.0 * X.shape[0]) # , fit_intercept=True
> )
> model.fit(X, y)
> print('new method: ', model.predict(X), model["ridge"].coef_/model['standardscaler'].scale_, model['ridge'].intercept_)
> ```
> 
> See [StackOverflow: How to reproduce the behaviour of Ridge(normalize=True)?](https://stackoverflow.com/questions/73583687/how-to-reproduce-the-behaviour-of-ridgenormalize-true)

And it really works for me.

Labels: Needs Triage