MLPR with solver='lbfgs', nonzero alpha doesn't make the same result from multiple run.

### Describe the bug

I have tested the simple regression modeling with MLPRegressor(solver='lbfgs', alpha=0.01, tol=0.0001, random_state=42) with sample data with three input parameters and three targets.
I tested multiple run in Linux OS. But, sometimes I got different regression.score(X_test, y_test).
But, if I change the solver to 'adam' or set alpha to zero, it makes the same value all the time.
So, I am suspicious that there is randomness with respect to solver, alpha, and tol. But, it's hard to find the cause from the source code.
I wonder if anybody has the same experience.

### Steps/Code to Reproduce

```py
import warnings
warnings.simplefilter('ignore')


for k in range(100):

    import numpy as np
    import pandas as pd
    from sklearn.model_selection import train_test_split
    from sklearn.preprocessing import StandardScaler
    from sklearn.neural_network import MLPRegressor

    data = np.array(
            [
                [1.3436424, 8.4743374, 5.0, 67.34851990996208, 130.0, 126.0],
                [1.3436424, 8.4743374, 1.0, 63.34851990996208, 130.0, 126.0],
                [7.6377462, 2.5506903, 5.0, 277.4131267778526, 130.0, 126.0],
                [7.6377462, 2.5506903, 1.0, 273.4131267778526, 130.0, 126.0],
                [4.9543509, 4.4949106, 5.0, 189.98360040364014, 130.0, 126.0],
                [4.9543509, 4.4949106, 1.0, 85.98360040364014, 130.0, 126.0],
                [6.5159297, 7.8872335, 5.0, 389.335314633451, 130.0, 126.0],
                [6.5159297, 7.8872335, 1.0, 385.335314633451, 130.0, 126.0],
                [0.93859587, 0.28347477, 5.0, 8.97322786339917, 130.0, 126.0],
                [0.93859587, 0.28347477, 1.0, 4.9732278633991704, 130.0, 126.0],
                [8.357651, 4.3276707, 5.0, 395.3967974810315, 130.0, 126.0],
                [8.357651, 4.3276707, 1.0, 391.3967974810315, 130.0, 126.0],
                [7.6228008, 0.021060534, 5.0, 180.12397738656006, 130.0, 126.0],
                [7.6228008, 0.021060534, 1.0, 176.12397738656006, 130.0, 126.0],
                [4.4538719, 7.2154003, 5.0, 225.1932679219367, 130.0, 126.0],
                [4.4538719, 7.2154003, 1.0, 221.1932679219367, 130.0, 126.0],
                [2.2876222, 9.452707, 5.0, 28.8207579062755, 130.0, 126.0],
                [2.2876222, 9.452707, 1.0, 124.82075790627552, 130.0, 126.0],
                [9.0142746, 0.30589983, 5.0, 262.5587650311821, 130.0, 126.0],
                [9.0142746, 0.30589983, 1.0, 258.5587650311821, 130.0, 126.0],
                [0.25445861, 5.4141247, 5.0, 12.082600780252731, 130.0, 126.0],
                [0.25445861, 5.4141247, 1.0, 8.082600780252731, 130.0, 126.0],
                [9.3914916, 3.8120424, 5.0, 448.60416431083087, 130.0, 126.0],
                [9.3914916, 3.8120424, 1.0, 444.60416431083087, 130.0, 126.0],
                [2.165994, 4.2211658, 5.0, 64.78968900313399, 130.0, 126.0],
            ],
            dtype=object,
        )

    df = pd.DataFrame(
        data, columns=["a", "b", "c", "t0", "t1", "t2"]
    )

    X = df[["a", "b", "c"]]
    y = df[["t0", "t1", "t2"]]

    scaler_x = StandardScaler()
    X = scaler_x.fit_transform(X)
    scaler_y = StandardScaler()
    y = scaler_y.fit_transform(y)

    random_state = 42

    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=random_state)

    reg = MLPRegressor(max_iter=200, solver='lbfgs', alpha=0.01, tol=0.0001, random_state=42)
    reg.fit(X_train, y_train)
    scores = reg.score(X_train, y_train), reg.score(X_test, y_test)
    print(k, scores[0], scores[1])
```

### Expected Results

0 0.33326240015324454 0.30623113513186795
1 0.33326240015324454 0.30623113513186795
2 0.33326240015324454 0.30623113513186795
3 0.33326240015324454 0.30623113513186795
4 0.33326240015324454 0.30623113513186795
5 0.33326240015324454 0.30623113513186795
...
98 0.33326240015324454 0.30623113513186795
99 0.33326240015324454 0.30623113513186795

### Actual Results

It makes two different output values randomly.

### Versions

```shell
1.1.3
```

COMMENT:
I can't reproduce with the recent version and also do not see any change in scores. I only ever get `train score = 0.33327103159867577` `test score = 0.3078730715851509`.

Labels: Bug, Needs Info