PCA hangs occasionally when applied to data that fits in memory

### Describe the bug

When trying to `fit` PCA with a large enough array that fits in memory (approximately `300 000 x 750`) with an aim to make about 3 times reduction, unstable results are obtained: in most cases the *fit succeeds*, but occasionaly it *hangs "forever"*.

Random seed is fixed, but unfortunalely it does not help in this case.
The `svd_solver` option is left at the default. Looks like explicitly specifying `svd_solver='arpack'` does not cause this issue.

### Steps/Code to Reproduce

```python
import numpy as np
import random

np.random.seed(42)
random.seed(42)

from sklearn.decomposition import PCA


if __name__ == '__main__':
    embeddings = np.random.RandomState(42).normal(size=(328039, 768))
    embeddings = embeddings.astype(np.float32)

    for i in range(20):
        print('Iteration', i)
        pca = PCA(n_components=260, random_state=42)
        pca = pca.fit(embeddings)

```

### Expected Results

Stable results: consistently successful fits over a finite time.

### Actual Results

Sometimes the process hangs, in most cases in the first 10 iterations. I haven't found any consistency pattern at what time it fails.

### Versions

```shell
System:
   python: 3.9.7 | packaged by conda-forge 
   machine: Windows-10-10.0.19043-SP0
   RAM: 32Gb

Python dependencies:
          pip: 21.3.1
   setuptools: 60.8.1
      sklearn: 1.0.1
        numpy: 1.20.1
        scipy: 1.7.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I am convinced that this is a threading problem for two reasons. When the deadlock happens, it looks like all cores are being used off and on but they are fluctuating up and down constantly. Also, the problem occurs on the first call to `.fit()` for me if there is other multi-threaded code before the fit. 

This is hard to diagnose because there is no explicit `n_jobs` for the PCA module and so the parallelization must be happening somewhere out of view. 

```
System:
    python: 3.9.16 (main, Jan 11 2023, 16:05:54)  [GCC 11.2.0]
executable: /home/jordan/miniconda3/envs/test/bin/python
   machine: Linux-5.19.0-43-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.1
          pip: 23.0.1
   setuptools: 65.6.3
        numpy: 1.23.5
        scipy: 1.10.0
       Cython: None
       pandas: 1.5.2
   matplotlib: 3.6.2
       joblib: 1.2.0
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: /home/jordan/miniconda3/envs/test/lib/libmkl_rt.so.1
         prefix: libmkl_rt
       user_api: blas
   internal_api: mkl
        version: 2021.4-Product
    num_threads: 10
threading_layer: intel

       filepath: /home/jordan/miniconda3/envs/test/lib/libiomp5.so
         prefix: libiomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 20

       filepath: /home/jordan/miniconda3/envs/test/lib/libgomp.so.1.0.0
         prefix: libgomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 20
```

Labels: Bug, module:decomposition, Needs Investigation