AttributeError:` 'str' object `has no attribute 'dtype'
Author: flckv
Pull request: no

### System Info

- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.4.204-ql-generic-12.0-19-x86_64-with-glibc2.31
- Python version: 3.11.3
- Huggingface_hub version: 0.14.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1 (True)
Versions of relevant libraries:
[pip3] numpy==1.23.0
[pip3] torch==2.0.1
[pip3] torchaudio==2.0.2
[pip3] torchvision==0.15.2
[conda] blas                      1.0                         mkl  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2023.1.0         h6d00ec8_46342  
[conda] mkl-service               2.4.0           py311h5eee18b_1  
[conda] mkl_fft                   1.3.6           py311ha02d727_1  
[conda] mkl_random                1.2.2           py311ha02d727_1  
[conda] numpy                     1.23.0                   pypi_0    pypi
[conda] pytorch                   2.0.1           py3.11_cuda11.8_cudnn8.7.0_0    pytorch
[conda] pytorch-cuda              11.8                 h7e8668a_5    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchaudio                2.0.2               py311_cu118    pytorch
[conda] torchtriton               2.0.0                     py311    pytorch
[conda] torchvision               0.15.2              py311_cu118    pytorch

### Who can help?

@sgugger @sanchit-gandhi 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction


## ERROR:
 train_result = `trainer.train`(resume_from_checkpoint=checkpoint)
...
python3.11/site-packages/transformers/`feature_extraction_sequence_utils.py`", line 220, in pad
    if value.dtype is np.dtype(np.float64):
       ^^^^^^^^^^^
AttributeError:` 'str' object `has no attribute 'dtype'

I am not sure which element of the dataset is read as 'str'


###  1. OFFICIAL SCRIPT: [transformers/examples/pytorch/audio-classification/run_audio_classification.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/audio-classification/README.md)


### 2. LOADED DATASET: 

DatasetDict({
    train: Dataset({
        features: [`'audio',` `'label'`],
        num_rows: 1280
    })
    validation: Dataset({
        features: ['audio', 'label'],
        num_rows: 160
    })
    test: Dataset({
        features: ['audio', 'label'],
        num_rows: 160
    })


### 3. logger.info(raw_datasets['train'][0])
 {'audio': {`'path'`: '/transformers/examples/pytorch/audio-classification/s/data/s/s/train/audio1.wav', `'array'`: array([0.02072144, 0.02767944, 0.03274536, ..., 0.00079346, 0.00088501,
       0.00149536]), 'sampling_rate': 16000}, `'label'`: 'happy'}







### Expected behavior

load the dataset to model for training  in  train_result = `trainer.train`(resume_from_checkpoint=checkpoint)

Comments:
Hi @flckv, 

In the issue info above, you mention using the official script, however it appears a custom dataset is being used. So that we can best help you, could you share a reproducible code snippet and full traceback of the error encountered? 

As mentioned in a previous issue - #24143 - for general questions on how to adapt a script to a custom use case, please use the [forums](https://discuss.huggingface.co/). We try to reserve github issues for bugs and feature requests. 

@amyeroberts thanks for the reply

## 1. Reproducible code snippet

is [here](https://gist.github.com/flckv/0e01c9ee1167f2d1af18b811e98194c6#file-run_audio_classification-py-L258), the highlighted line shows the approach I used to load audio files with metadata csvs 

**Dataset structure:**
![image](https://github.com/huggingface/transformers/assets/103381497/ceb7daf1-5216-44e8-a66c-7ebab8a35410)


command:

> python run_audio_classification.py \
>     --model_name_or_path facebook/wav2vec2-base \
>     --output_dir l/users/flck/outputs/wav2vec2-base-s \
>     --overwrite_output_dir \
>     --remove_unused_columns False \
>     --do_train \
>     --do_eval \
>     --fp16 \
>     --learning_rate 3e-5 \
>     --max_length_seconds 1 \
>     --attention_mask False \
>     --warmup_ratio 0.1 \
>     --num_train_epochs 5 \
>     --per_device_train_batch_size 32 \
>     --gradient_accumulation_steps 4 \
>     --per_device_eval_batch_size 32 \
>     --dataloader_num_workers 4 \
>     --logging_strategy steps \
>     --logging_steps 10 \
>     --evaluation_strategy epoch \
>     --save_strategy epoch \
>     --load_best_model_at_end True \
>     --metric_for_best_model accuracy \
>     --save_total_limit 3 \
>     --seed 0 \
>     --push_to_hub \
>     --use_auth_token True







## 2. Full traceback of the error encountered

is [here](https://gist.github.com/flckv/3cb6179b15e3571ea7421002ab65c5c2#file-error-py-L206 ), the highlighted lines show the log of the loaded dataset example 



Hi @flckv, thanks for sharing that information.

Looking at this it doesn't look like a bug in our code, but rather the dataset creation. As mentioned above, this is really a question for our forums. I'll give a suggestion below of where to begin, but unfortunately we don't have time to help you debug your own custom code.  

To get things working I suggest two things: 

* Checking that the unaltered script runs with the default values as shown [in the README](https://github.com/huggingface/transformers/tree/main/examples/pytorch/audio-classification#single-gpu). If this doesn't work, then there might be an issue on our side. In which case, report that in a new issue please. 

* Check that your loaded dataset and the dataset used in the example are comparable e.g. 

```python
from datasets import load_dataset

# Load in the default dataset from the example
example_dataset = load_dataset("superb", "asr", split="train")

# Load in my dataset
my_dataset = load_dataset("audiofolder", data_dir="/home/flck/hf38/transformers/examples/pytorch/audio-classification/s/data/s/s/", split="train")

# Inspect the datasets
print(example_dataset)
print(my_dataset)

# Check the values for a specific column that will be used during training
print(example_dataset['audio'])
print(my_dataset['audio'])
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

What worked for me was casting the labels:

```
    data_files = {'train': 'train/train.csv', 'eval': 'test/test.csv'}
    raw_datasets = load_dataset('dataset', data_files=data_files)
    raw_datasets = raw_datasets.class_encode_column("sentiment")
```

1. I call the test set "eval" in order to match what the script expects.

3. Then I encode my label column ("sentiment) as class.