unable to pass create vocabulary_from_data when using concatenate_datasets() method to train a combined dataset model 
Author: StephennFernandes
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

 `transformers` version: 4.17.0.dev0
- Platform:Ubuntu 20.04.3 LTS
- Python version: 3.8.10
- PyTorch version (GPU?):1.10.2+cu113
- Tensorflow version (GPU?): not installed 
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
@patrickvonplaten @anton-l 

Models:

- Wav2Vec2 

Library:

- Speech

## Information

Model I am using : facebook/wav2vec2-xls-r-300m

The problem arises when using:
* [ ] the official example script: https://github.com/huggingface/transformers/blob/master/examples/pytorch/speech-recognition/run_speech_recognition_ctc.py


The tasks I am working on is:
* [ ]  dataset: using the .concatenate_dataset([common_voice_dataset, openslr_dataset]) method to concatenate common_voice and openslr datasets from huggingface


## To reproduce

Steps to reproduce the behavior:
download multiple marathi or tamil datasets and use the .concatenate_dataset() method to concatenate them together, and further feed them into the raw_dataset['train']  in the official training script 

 I have been running the official script `run_speech_recognition_ctc.py` as provided in
 [https://github.com/huggingface/transformers/tree/master/examples/pytorch/speech-recognition](https://github.com/huggingface/transformers/tree/master/examples/pytorch/speech-recognition)

The same code has been tested for tamil dataset concatenating openslr and common_voice dataset and the same problem persists. 

1. `python run_speech_recognition_ctc.py \
	--dataset_name="common_voice" \
	--model_name_or_path="facebook/wav2vec2-large-xlsr-53" \
	--dataset_config_name="mr" \
	--output_dir="./wav2vec2-common_voice-mr" \
	--overwrite_output_dir \
	--num_train_epochs="15" \
	--per_device_train_batch_size="16" \
	--gradient_accumulation_steps="2" \
	--learning_rate="3e-4" \
	--warmup_steps="500" \
	--evaluation_strategy="steps" \
	--text_column_name="sentence" \
	--length_column_name="input_length" \
	--save_steps="400" \
	--eval_steps="100" \
	--layerdrop="0.0" \
	--save_total_limit="3" \
	--freeze_feature_encoder \
	--gradient_checkpointing \
	--chars_to_ignore , ? . ! - \; \: \" “ % ‘ ” � \
	--fp16 \
	--group_by_length \
	--push_to_hub \
	--do_train --do_eval `




## Error Message
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.64ba/s]
  0%|                                                                                                                                            | 0/1 [00:00<?, ?ba/s]
Traceback (most recent call last):
  File "bnb.py", line 751, in <module>
    main()
  File "bnb.py", line 480, in main
    vocab_dict = create_vocabulary_from_data(
  File "bnb.py", line 299, in create_vocabulary_from_data
    vocabs = datasets.map(
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/dataset_dict.py", line 494, in map
    {
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/dataset_dict.py", line 495, in <dictcomp>
    k: dataset.map(
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 2102, in map
    return self._map_single(
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 518, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 485, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/fingerprint.py", line 413, in wrapper
    out = func(self, *args, **kwargs)
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 2498, in _map_single
    writer.write_batch(batch)
  File "/home/stephen/anaconda3/lib/python3.8/site-packages/datasets/arrow_writer.py", line 498, in write_batch
    pa_table = pa.Table.from_arrays(arrays, schema=schema)
  File "pyarrow/table.pxi", line 1702, in pyarrow.lib.Table.from_arrays
  File "pyarrow/table.pxi", line 1314, in pyarrow.lib.Table.validate
  File "pyarrow/error.pxi", line 99, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Column 9 named vocab expected length 306 but got length 1

the same script works if you do .train_test_split() on the concatenated dataset and pass the training dataset to to ['train'] and ['eval'] 



Comments:
The issue was caused because i the columns in train and eval didnt align, closing this issue.