Importing BaseEstimator leads to unnecessary memory usage

### Describe the bug

Importing `BaseEstimator` from `sklearn.base` causes a cascade of imports that leads to unnecessary memory usage (500MiB of stuff at peak, see screenshot below).

![Screenshot from 2023-02-10 19-23-53](https://user-images.githubusercontent.com/1296726/218169206-d8e940e9-40ba-458d-8047-deacf2f30521.png)


### Steps/Code to Reproduce

```python
from sklearn.base import BaseEstimator
```

### Expected Results

I would expect such a basic class to import some utilities around, but nothing major.

### Actual Results

The code ends up importing a large amount of unnecessary and large modules, the full chain to some point is:
 - code -> `from sklearn.base import BaseEstimator`
 - sklearn/\_\_init\_\_.py:82 -> `from .base import clone`
 - sklearn/base.py:17 -> `from .utils import _IS_32BIT`
 - sklearn/utils/\_\_init\_\_.py:21 -> `from scipy.sparse import issparse` (This is the most useless import, as the issparse function is a oneliner that calls isinstance)
 - scipy/sparse/\_\_init\_\_.py:283 -> `from . import csgraph`
 - scipy/sparse/csgraph/\_\_init\_\_.py:185 -> `from ._laplacian import laplacian`
 - scipy/sparse/csgraph/_laplacian.py:7 -> `from scipy.sparse.linalg import LinearOperator`
 - scipy/sparse/linalg/\_\_init\_\_.py:120 -> `from ._isolve import *`
 - scipy/sparse/linalg/_isolve/\_\_init\_\_.py:4 -> `from .iterative import *`
 - scipy/sparse/linalg/_isolve/iterative.py:9 -> `from . import _iterative`

I guess the issue I am reporting is that importing a simple estimator base class should not cause so many unrelated imports and spike memory usage by doing so.

### Versions

```shell
System:
    python: 3.10.8 (main, Nov  1 2022, 14:18:21) [GCC 12.2.0]
executable: /home/jjan/dev/sec-certs-page/virt/bin/python
   machine: Linux-6.0.12-arch1-1-x86_64-with-glibc2.36

Python dependencies:
      sklearn: 1.2.1
          pip: 23.0
   setuptools: 61.2.0
        numpy: 1.22.3
        scipy: 1.10.0
       Cython: None
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /home/jjan/dev/sec-certs-page/virt/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/jjan/dev/sec-certs-page/virt/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-2f7c42d4.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: Zen
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /home/jjan/dev/sec-certs-page/virt/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-41284840.3.18.so
        version: 0.3.18
threading_layer: pthreads
   architecture: Zen
    num_threads: 16
```

COMMENT:
Hmm, I guess it might be a version thing or a platform thing? I installed both scipy and scikit-learn via pip into a fresh virtualenv.

Labels: Bug, Needs Investigation