unexpected OOM error when creating pipeline
Author: hcscctv
Pull request: no

### System Info

```
from transformers import pipeline, set_seed

generator = pipeline('text-generation', model='gpt2',device=0)
```

When i run these code, an unexpected error occured.

```
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 47.46 GiB total capacity; 148.00 MiB already allocated; 22.31 MiB free; 148.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```

the gpu has enough space
```
Fri Oct 28 05:53:34 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.141.03   Driver Version: 470.141.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 8000     Off  | 00000000:1A:00.0 Off |                  Off |
| 33%   46C    P2    74W / 260W |   2504MiB / 48601MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     15308      C   python                            833MiB |
|    0   N/A  N/A     16307      C   python                            833MiB |
|    0   N/A  N/A     17441      C   python                            833MiB |
+-----------------------------------------------------------------------------+

```


it is interesting that 47.46 GiB total capacity; 148.00 MiB already allocated; 22.31 MiB free; 148.00 MiB reserved in total by PyTorch, and i cannot allocate 20Mb.

i use slurm to upload the job, i did not know whether it has effect

### Who can help?

@Narsil

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import pipeline, set_seed
generator = pipeline('text-generation', model='gpt2',device=0)

# upload job by slurm

### Expected behavior

successfully use pipeline, which is a quite cool things.

Comments:
sorry, there are some errors on the server in fact

Ok, if you could share what happened it could help potential readers. But glad you fixed it !