PCA hangs occasionally when applied to data that fits in memory

### Describe the bug

When trying to `fit` PCA with a large enough array that fits in memory (approximately `300 000 x 750`) with an aim to make about 3 times reduction, unstable results are obtained: in most cases the *fit succeeds*, but occasionaly it *hangs "forever"*.

Random seed is fixed, but unfortunalely it does not help in this case.
The `svd_solver` option is left at the default. Looks like explicitly specifying `svd_solver='arpack'` does not cause this issue.

### Steps/Code to Reproduce

```python
import numpy as np
import random

np.random.seed(42)
random.seed(42)

from sklearn.decomposition import PCA


if __name__ == '__main__':
    embeddings = np.random.RandomState(42).normal(size=(328039, 768))
    embeddings = embeddings.astype(np.float32)

    for i in range(20):
        print('Iteration', i)
        pca = PCA(n_components=260, random_state=42)
        pca = pca.fit(embeddings)

```

### Expected Results

Stable results: consistently successful fits over a finite time.

### Actual Results

Sometimes the process hangs, in most cases in the first 10 iterations. I haven't found any consistency pattern at what time it fails.

### Versions

```shell
System:
   python: 3.9.7 | packaged by conda-forge 
   machine: Windows-10-10.0.19043-SP0
   RAM: 32Gb

Python dependencies:
          pip: 21.3.1
   setuptools: 60.8.1
      sklearn: 1.0.1
        numpy: 1.20.1
        scipy: 1.7.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Huh, I'm actually able to reproduce this error semi-reliably. When it occurs still varies -- in one instance it started hanging on iteration 3 while in another it was on iteration 7 -- but I think there's a genuine issue here. On average one iteration would take ~25sec but after letting the stuck fit proceed for a few minutes I noticed no significant change in my system's resource usages (memory or cpu).

I have no clue what could be causing this but I'll try to take a look.

Here are my versions:
```
System:
    python: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)]
executable: R:\ProgramFiles\anaconda3\envs\scikit-dev\python.exe
   machine: Windows-10-10.0.19043-SP0

Python dependencies:
      sklearn: 1.1.dev0
          pip: 21.2.4
   setuptools: 58.0.4
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: 0.29.26
       pandas: 1.3.5
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.0.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: R:\ProgramFiles\anaconda3\envs\scikit-dev\Lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
        version: 0.3.17
threading_layer: pthreads
   architecture: Zen
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\Windows\System32\vcomp140.dll
        version: None
    num_threads: 12

```

Labels: Bug, module:decomposition, Needs Investigation