BUG loss of precision in LogisticRegression as of version 1.4

### Describe the bug

Between version 1.3.2 and 1.4.0, LogisticRegression became less accurate.

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd

import sklearn.pipeline
import sklearn.preprocessing
import sklearn.linear_model

df = pd.DataFrame({
    'age': [0, 1, 0, 1, 2, 3, 4, 5, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 0, 1, 2, 0, 1, 0, 1, 2, 0, 1, 2, 3, 4, 5, 6, 0, 1, 2, 0, 1, 2, 0, 1, 2, 3, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
    'exiting':  [False, False, False, False, False, False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, True, False, False, False, False, False, False, False, False, False, False, False, True, False, False, True, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False, False]
})

data_timeline = np.arange(df['age'].max() + 1)

estimator = sklearn.pipeline.Pipeline(
    [
        (
            "onehot",
            sklearn.preprocessing.OneHotEncoder(
                categories=[data_timeline], sparse_output=False
            )
        ),
        (
            "logistic",
            sklearn.linear_model.LogisticRegression(
                fit_intercept=False, C=1e6, max_iter=1000, tol=1e-7, solver="newton-cg"
            ),
        ),
    ]
)

estimator.fit(df[['age']], df["exiting"])
def logistic_regression_gradient(X, y, y_hat, coef, C):
    return coef + C * (y_hat - y) @ X

gradient = logistic_regression_gradient(
    X=estimator.named_steps['onehot'].transform(df[['age']]),
    y=df["exiting"].astype(float).to_numpy(),
    y_hat=estimator.predict_proba(df[['age']])[:, 1],
    coef=estimator.named_steps["logistic"].coef_,
    C=estimator.named_steps['logistic'].C
)
print(sklearn.__version__)
print(gradient)
```

### Expected Results

```
1.3.2
[[ 6.38057971e-06 -1.47032373e-02  3.23092176e-06 -1.86882858e-02
  -1.85588401e-02 -7.10504243e-05  8.47822321e-05 -1.58804369e-02
  -1.58804369e-02 -1.58804369e-02 -1.58804369e-02 -1.18021221e-02
  -2.76625890e-03 -2.76625890e-03 -2.76625890e-03 -2.76625890e-03
  -2.76625890e-03 -2.76625890e-03 -2.76625890e-03  0.00000000e+00
   2.22974349e-02  2.22974349e-02  2.22974349e-02  2.22974349e-02
   2.22974349e-02  2.22974349e-02  2.22974349e-02  2.22974349e-02
   2.22974349e-02  2.22974349e-02  2.22974349e-02  2.22974349e-02
   2.22974349e-02]]
```

### Actual Results

```
1.4.0
[[ 2.68892818e-03  3.64025037e+00  6.76589138e-04  3.77020823e+00
   3.80372303e+00 -1.41805879e-04  1.51207953e-02  3.89408378e+00
   3.89408378e+00  3.89408378e+00  3.89408378e+00  3.96014198e+00
   4.05600402e+00  4.05600402e+00  4.05600402e+00  4.05600402e+00
   4.05600402e+00  4.05600402e+00  4.05600402e+00  0.00000000e+00
   4.22741992e+00  4.22741992e+00  4.22741992e+00  4.22741992e+00
   4.22741992e+00  4.22741992e+00  4.22741992e+00  4.22741992e+00
   4.22741992e+00  4.22741992e+00  4.22741992e+00  4.22741992e+00
   4.22741992e+00]]
```

### Versions

```shell
System:
    python: 3.12.2 | packaged by conda-forge | (main, Feb 16 2024, 20:50:58) [GCC 12.3.0]
executable: ~/miniforge3/envs/sklearn-env/bin/python
   machine: Linux-6.8.1-arch1-1-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.4.0
          pip: 24.0
   setuptools: 69.2.0
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: 3.0.9
       pandas: 2.2.1
   matplotlib: None
       joblib: 1.3.2
threadpoolctl: 3.4.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: ~/miniforge3/envs/sklearn-env/lib/libopenblasp-r0.3.26.so
        version: 0.3.26
threading_layer: pthreads
   architecture: Zen

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libgomp
       filepath: ~/miniforge3/envs/sklearn-env/lib/libgomp.so.1.0.0
        version: None
```
```

COMMENT:
Thank you guys for looking into this!

I've decreased the tolerance and that solved the issue.

Sorry for not looking at the release notes first.

Would it make sense to add this to the documentation? I'm happy to contribute if you think it's worthwhile.

Labels: Bug, module:linear_model, Numerical Stability