MusicGen small model is not using GPU
Author: mepc36
Pull request: no

### System Info

```
root@redacted:/# transformers-cli env

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.31.0
- Platform: Linux-5.4.0-156-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.2
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: trying to, but it's not working
- Using distributed or parallel set-up in script?: no
```

I'm trying to use the MusicGen model on a GPU. I've got the results of `nvidia-smi` during the run. Here they are. As you can see, the amount of GPU utilized is not increasing:

```
root@85795c5b68bd:/# nvidia-smi -l 1
Wed Aug 16 13:22:55 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.05              Driver Version: 535.86.05    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX A4500               On  | 00000000:C1:00.0 Off |                  Off |
| 30%   29C    P8              13W / 200W |   5533MiB / 20470MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+
Wed Aug 16 13:22:56 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.05              Driver Version: 535.86.05    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA RTX A4500               On  | 00000000:C1:00.0 Off |                  Off |
| 30%   29C    P8              13W / 200W |   5533MiB / 20470MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+
```

I'm wondering how I can get the MusicGen small model to utilize GPU. Do I have to pass `device='cuda'` as an argument somewhere?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The machine I'm testing on has the following specs:

```
1 x RTX A4500
12 vCPU 62 GB RAM
```

Here is the code I'm using to generate audio:

```
    from transformers import AutoProcessor, MusicgenForConditionalGeneration

    processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
    model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")

    inputs = processor(
        text=[prompt],
        padding=True,
        return_tensors="pt",
    )

    audio_values = model.generate(**inputs, max_new_tokens=max_new_tokens)
```

### Expected behavior

I expect that the GPU would be utilized.

Comments:
A reproducer on which code you are using is required according to the contribution guidelines. Otherwise yes, you are the onde that should specify the device. 

Sorry about that @ArthurZucker , I just added the code I'm using to generate audio.

So I just re-wrote the reproduction code by trying to pass a `device='cuda'` argument like this...

```
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small", device='cuda')
```

...but got the following `unexpected keyword argument` error when doing so:

```
Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/Users/martinconnor/Desktop/rapbot-riffusion/riffusion/server.py", line 269, in infer_musicgen
    model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small", device='cuda')
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/transformers/models/musicgen/modeling_musicgen.py", line 1599, in from_pretrained
    return super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/riffusion/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2700, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
TypeError: __init__() got an unexpected keyword argument 'device'
```

So then I tried passing the `device` argument to the processor, like so...

```
 processor = AutoProcessor.from_pretrained("facebook/musicgen-small", device='cuda')
```

...and that did not error, but looking through the specs for the `from_pretrained` function definition, I think it just ignored the `device` argument.

Anyone help you can provide would be greatly appreciated, thanks again! And sorry for the crappy initial bug report, that was just plain laziness on my part. My apologies

Inviting you to ask this on the forum, as there are some similar [questions](https://discuss.huggingface.co/t/is-transformers-using-gpu-by-default/8500).
TLDR; the device cannot be specify this way you need to call `module.to('cuda')`. The device argument is for the `pipelines`. 

Thanks @ArthurZucker ! I got it working. Here is my reproduction code with a fix so that it uses GPU. You're right, call `.to('cuda')` on both the model AND the inputs got it working:

```
from transformers import AutoProcessor, MusicgenForConditionalGeneration

prompt = '80s EDM music'
processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
model = model.to('cuda:0')

inputs = processor(
    text=[prompt],
    padding=True,
    return_tensors="pt"
)
inputs = inputs.to('cuda:0')

audio_values = model.generate(**inputs, max_new_tokens=256)
print('audio_values:', audio_values)
```

Closing! Thanks again.