Error when running wav2vec2 embeddings
Author: BirgerMoell
Pull request: no

While trying to extract wav2vec2 embeddings I get the following errors.

e "feature_extractor.py", line 80, in <module>
    feature_extractor("/home/bmoell/data/media.talkbank.org/dementia/English/Pitt/Control/cookie")
  File "feature_extractor.py", line 36, in feature_extractor
    get_wav2vecembeddings_from_audiofile(wav_file)
  File "feature_extractor.py", line 57, in get_wav2vecembeddings_from_audiofile
    input_values = processor(resampled, return_tensors="pt", padding=True, sampling_rate=new_sample_rate) # there is no truncation param anymore
  File "/home/bmoell/hubert-dementia-screening/dementia/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2284, in __call__
    raise ValueError(
ValueError: text input must of type `str` (single example), `List[str]` (batch or single pretokenized example) or `List[List[str]]` (batch of pretokenized examples).

I'm using the following script to extract wav2vec2 embeddings from .wav files.

```python
def get_wav2vecembeddings_from_audiofile(wav_file):
    print("the file is", wav_file)
    speech, sample_rate = sf.read(wav_file)

    if len(speech.shape) > 1:
        speech = stereo_to_mono(speech)
     # change sample rate to 16 000 hertz
    resampled = change_sample_rate(speech, sample_rate, new_sample_rate)
    print("the speech is", speech)
    input_values = processor(resampled, return_tensors="pt", padding=True, sampling_rate=new_sample_rate) # there is no truncation param anymore
    print("input values", input_values)
    # import pdb
    # pdb.set_trace()

    with torch.no_grad():
        encoded_states = model(
            **input_values, 
            # attention_mask=input_values["attention_mask"], 
            output_hidden_states=True
        )
        last_hidden_state = encoded_states.hidden_states[-1] # The last hidden-state is the first element of the output tuple
        print("getting wav2vec2 embeddings")
        print(last_hidden_state)
        torch.save(last_hidden_state, wav_file + '.wav2vec2.pt')
```




Updated script that takes in the file_path to processor. Now I get a different error.

```python

def get_wav2vecembeddings_from_audiofile(wav_file):
    print("the file is", wav_file)
    speech, sample_rate = sf.read(wav_file)

    if len(speech.shape) > 1:
        speech = stereo_to_mono(speech)
     # change sample rate to 16 000 hertz
    resampled = change_sample_rate(speech, sample_rate, new_sample_rate)
    print("the speech is", speech)
    input_values = processor(wav_file, return_tensors="pt", padding=True, sampling_rate=new_sample_rate) # there is no truncation param anymore
    print("input values", input_values)
    # import pdb
    # pdb.set_trace()

    with torch.no_grad():
        encoded_states = model(
            input_values=input_values["input_ids"], 
            # attention_mask=input_values["attention_mask"], 
            output_hidden_states=True
        )
        last_hidden_state = encoded_states.hidden_states[-1] # The last hidden-state is the first element of the output tuple
        print("getting wav2vec2 embeddings")
        print(last_hidden_state)
        torch.save(last_hidden_state, wav_file + '.wav2vec2.pt')

  ```    
  File "/home/bmoell/hubert-dementia-screening/dementia/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 294, in _conv_forward
    return F.conv1d(input, weight, bias, self.stride,
RuntimeError: expected scalar type Long but found Float










Comments:
Can you copy-paste a reproducible code snippet here (create dummy data if necessary) ? :-)  

Made a colab that reproduced the error.
https://colab.research.google.com/drive/1JpZ33M3tCKJBK6XZeDhj4u30roWKQ63s?usp=sharing

I can't run the colab as `processor` is commented out

The main problem is the following: 

We should not use a tokenizer to process wav files -> we should use the processor for that. So `AutoTokenizer` should be replaced by `Wav2Vec2Processor`. If you settle on using `HubertForCTC`, it's a good idea to first look into the examples of the docs to check how the model should be used. E.g. here we should an example for `HubertForCTC`: https://huggingface.co/transformers/master/model_doc/hubert.html#hubertforctc

=> so from this example you can see that you should load the wav file yourself and then use the `Wav2Vec2Processor` to process the input. This will return `input_values` that you can pass to the model.

Also, just a note on how to write issue for the future ;-):

It's always good to aim for a *minimal* reproducible code example. E.g. for this error it should be relatively simple to figure out that the error is produced by the following lines: 

```python
input_values = processor(wav_file, return_tensors="pt", padding=True, sampling_rate=new_sample_rate) # there is no truncation param anymore
encoded_states = model(
    input_values=input_values["input_ids"], 
    # attention_mask=input_values["attention_mask"], 
    output_hidden_states=True
)
```

in your code. So to make debugging easier it would be good to create a dumy `wav_array` (this can be just a random 1-D np.float32 array) and then post 3,4 lines here that show that there is a bug. E.g.:

```python
import numpy as np
from transformers import AutoTokenizer, HubertForCTC

tokenizer = AutoTokenizer.from_pretrained("facebook/hubert-large-ls960-ft")
model = HubertForCTC.from_pretrained("facebook/hubert-large-ls960-ft")

wav_file = np.random.random((1, 1024))

input_values = processor(wav_file, return_tensors="pt", padding=True)
encoded_states = model(input_values=input_values["input_ids"])
```

=> It takes much less time to run these 5 lines then going through the colab (which sadly doesn't even run correctly).


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.