torch.randperm uses too much cpu, but not efficient.
Labels: module: performance, triaged, module: random, module: multithreading

### 🐛 Describe the bug

I found that when running the torch.randperm function on the cpu, it uses more threads by default, but is rather less fast than using only one thread.

A reproduction code is as follows:
```python
import torch
from tqdm import tqdm

N_iter = 10000

for _ in tqdm(range(N_iter)):
    idxs = torch.randperm(65536)
```

using default OMP_NUM_THREADS, the cpu usage is ~3800%, and produce more subthreads:
```
python test_cpu.py
100%|████████████████████████████████████████████████████████████| 10000/10000 [00:18<00:00, 539.00it/s]
```

set OMP_NUM_THREADS=1, the cpu usage is ~100%, and it's more efficient:
```
OMP_NUM_THREADS=1 python test_cpu.py
100%|████████████████████████████████████████████████████████████| 10000/10000 [00:12<00:00, 770.86it/s]
```

Although I can avoid this problem by setting OMP_NUM_THREADS or torch.set_num_threads() globally, I don't want to force a limit on the number of threads because I don't know if this will run other code that really needs multiple threads.
Is there a better way to avoid this unnecessary use of multiple threads?


### Versions

PyTorch version: 1.10.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.20.3
Libc version: glibc-2.27

Python version: 3.8.13 (default, Mar 28 2022, 11:38:47)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.15.0-151-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration:
GPU 0: Tesla V100-PCIE-16GB
GPU 1: Tesla V100-PCIE-16GB
GPU 2: Tesla V100-PCIE-16GB
GPU 3: Tesla V100-PCIE-16GB
GPU 4: Tesla V100-PCIE-16GB
GPU 5: Tesla V100-PCIE-16GB
GPU 6: Tesla V100-PCIE-16GB
GPU 7: Tesla V100-PCIE-16GB

Nvidia driver version: 460.32.03
cuDNN version: Probably one of the following:
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-10.2/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8
/usr/local/cuda-11.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.10.0
[pip3] torch-tb-profiler==0.4.0
[pip3] torchaudio==0.10.0
[pip3] torchsearchsorted==1.1
[pip3] torchvision==0.11.0
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               10.2.89              hfd86e86_1
[conda] cudatoolkit-dev           11.4.0               h5e8e339_5    conda-forge
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py38h7f8727e_0
[conda] mkl_fft                   1.3.1            py38hd3c417c_0
[conda] mkl_random                1.2.2            py38h51133e4_0
[conda] numpy                     1.21.5           py38he7a7128_1
[conda] numpy-base                1.21.5           py38hf524024_1
[conda] pytorch                   1.10.0          py3.8_cuda10.2_cudnn7.6.5_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torch-tb-profiler         0.4.0                    pypi_0    pypi
[conda] torchaudio                0.10.0               py38_cu102    pytorch
[conda] torchsearchsorted         1.1                      pypi_0    pypi
[conda] torchvision               0.11.0               py38_cu102    pytorch

cc @VitalyFedyunin @ngimel @pbelevich