ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length
Author: kamil-bentounes
Pull request: no

## Environment info

- `transformers` version: 4.4.0
- Ubuntu 18.04
- Python version: 3.7.4
- PyTorch version: 1.8.1
- Using Colab

@Nithin-Holla @LysandreJik 

Models: Wav2vec 2.0 

## Information

Model I am using : Wav2vec 2.0

The problem arises when using:
* [Theses steps](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_Tune_XLSR_Wav2Vec2_on_Turkish_ASR_with_%F0%9F%A4%97_Transformers.ipynb#scrollTo=e7cqAWIayn6w) 
* on my own data, changing all .map functions with mine:

```
import pandas as pd
import os 
from tqdm import tqdm
import random

train_data = pd.read_csv(os.path.join(os.getcwd(), "data", "wav_file", "train.tsv"))
test_data = pd.read_csv(os.path.join(os.getcwd(), "data", "wav_file", "valid.tsv"))


from datasets import ClassLabel
import random
import pandas as pd
from IPython.display import display, HTML

def show_random_elements(dataset, num_examples=10):
    assert num_examples <= len(dataset), "Can't pick more elements than there are in the dataset."
    picks = []
    for _ in range(num_examples):
        pick = random.randint(0, len(dataset)-1)
        while pick in picks:
            pick = random.randint(0, len(dataset)-1)
        picks.append(pick)
    
    df = pd.DataFrame(dataset["sentence"][picks])
    display(HTML(df.to_html()))
    print(df)

show_random_elements(train_data, num_examples=20)
show_random_elements(test_data, num_examples=2)

import re
chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\%\‘\”\�]'

for element in train_data.index:
    train_data["sentence"][element] = re.sub(chars_to_ignore_regex, '', train_data["sentence"][element]).lower() + " "

for element in test_data.index:
    test_data["sentence"][element] = re.sub(chars_to_ignore_regex, '', test_data["sentence"][element]).lower() + " "


show_random_elements(train_data, num_examples=20)
show_random_elements(test_data, num_examples=2)

all_text = ""
for element in train_data.index:
    all_text += train_data["sentence"][element].replace("î", "i")
vocab_train = list(set(all_text))

all_text = ""
for element in test_data.index:
    all_text += test_data["sentence"][element].replace("î", "i")
vocab_test = list(set(all_text))

vocab_list = list(set(vocab_train) | set(vocab_test))

vocab_dict = {v: k for k, v in enumerate(vocab_list)}
vocab_dict

vocab_dict["|"] = vocab_dict[" "]
del vocab_dict[" "]
vocab_dict

vocab_dict["[UNK]"] = len(vocab_dict)
vocab_dict["[PAD]"] = len(vocab_dict)
len(vocab_dict)

import json
with open(os.path.join(os.getcwd(), "data", "wav_file", "vocab.json"), 'w') as vocab_file:
    json.dump(vocab_dict, vocab_file)



from transformers import Wav2Vec2CTCTokenizer

tokenizer = Wav2Vec2CTCTokenizer("/content/drive/MyDrive/wav2vec_commonVoice (copy)/vocab.json", unk_token="[UNK]", pad_token="[PAD]", word_delimiter_token="|", truncation=True, padding=True)


from transformers import Wav2Vec2FeatureExtractor

feature_extractor = Wav2Vec2FeatureExtractor(feature_size=1, sampling_rate=16000, padding=True, padding_value=0.0, do_normalize=True, return_attention_mask=True)


from transformers import Wav2Vec2Processor

processor = Wav2Vec2Processor(feature_extractor=feature_extractor, tokenizer=tokenizer)

train = []
for element in train_data.index:
    dico = dict()
    dico["path"] = train_data["path"][element]
    dico["sentence"] = train_data["sentence"][element]
    train.append(dico)

train[0]
    
test = []
for element in test_data.index:
    dico = dict()
    dico["path"] = test_data["path"][element]
    dico["sentence"] = test_data["sentence"][element]
    test.append(dico)
test[0]

import torchaudio

def speech_file_to_array_fn(batch):
    speech_array, sampling_rate = torchaudio.load(os.path.join(os.getcwd(), "data", "wav_file", batch["path"]))
    batch["speech"] = speech_array[0].numpy()
    batch["sampling_rate"] = sampling_rate
    batch["target_text"] = batch["sentence"]
    del batch['path']
    del batch["sentence"]
    return batch


train_batch = []
for audio in tqdm(train):
    train_batch.append(speech_file_to_array_fn(audio))

test_batch = []
for audio in tqdm(test):
    test_batch.append(speech_file_to_array_fn(audio))

    
print(len(train_batch))
print(len(test_batch))

import IPython.display as ipd
import numpy as np
import random

rand_int = random.randint(0, len(train_batch)-1)

ipd.Audio(data=np.asarray(train_batch[rand_int]["speech"]), autoplay=True, rate=16000)


rand_int = random.randint(0, len(train_batch)-1)

print("Target text:", train_batch[rand_int]["target_text"])
print("Input array shape:", np.asarray(train_batch[rand_int]["speech"]).shape)
print("Sampling rate:", train_batch[rand_int]["sampling_rate"])

train_final = []
batch = dict()

for element in range(0, len(train_batch) - 8, 8):

    batch["speech"] = []
    batch["sampling_rate"] = []
    batch["target_text"] = []

    for batch_size in range(8):
        batch["speech"].append(train_batch[element + batch_size]["speech"])
        batch["target_text"].append(train_batch[element + batch_size]["target_text"])
        batch["sampling_rate"].append(train_batch[element + batch_size]["sampling_rate"])
        
    batch["input_values"] = processor(batch["speech"], sampling_rate=batch["sampling_rate"][0]).input_values    
    
    with processor.as_target_processor():
        batch["labels"] = processor(batch["target_text"]).input_ids
    train_final.append(batch)

        
        
batch_test = dict()
test_final = []
for element in range(0, len(test_batch) - 8, 8):

    batch_test["speech"] = []
    batch_test["sampling_rate"] = []
    batch_test["target_text"] = []

    for batch_size in range(8):
        batch_test["speech"].append(test_batch[element + batch_size]["speech"])
        batch_test["target_text"].append(test_batch[element + batch_size]["target_text"])
        batch_test["sampling_rate"].append(test_batch[element + batch_size]["sampling_rate"])
        
    batch_test["input_values"] = processor(batch["speech"], sampling_rate=batch_test["sampling_rate"][0]).input_values    
    
    with processor.as_target_processor():
        batch_test["labels"] = processor(batch_test["target_text"]).input_ids
    test_final.append(batch_test)

import torch

from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Union

@dataclass
class DataCollatorCTCWithPadding:
    """
    Data collator that will dynamically pad the inputs received.
    Args:
        processor (:class:`~transformers.Wav2Vec2Processor`)
            The processor used for proccessing the data.
        padding (:obj:`bool`, :obj:`str` or :class:`~transformers.tokenization_utils_base.PaddingStrategy`, `optional`, defaults to :obj:`True`):
            Select a strategy to pad the returned sequences (according to the model's padding side and padding index)
            among:
            * :obj:`True` or :obj:`'longest'`: Pad to the longest sequence in the batch (or no padding if only a single
              sequence if provided).
            * :obj:`'max_length'`: Pad to a maximum length specified with the argument :obj:`max_length` or to the
              maximum acceptable input length for the model if that argument is not provided.
            * :obj:`False` or :obj:`'do_not_pad'` (default): No padding (i.e., can output a batch with sequences of
              different lengths).
        max_length (:obj:`int`, `optional`):
            Maximum length of the ``input_values`` of the returned list and optionally padding length (see above).
        max_length_labels (:obj:`int`, `optional`):
            Maximum length of the ``labels`` returned list and optionally padding length (see above).
        pad_to_multiple_of (:obj:`int`, `optional`):
            If set will pad the sequence to a multiple of the provided value.
            This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >=
            7.5 (Volta).
    """

    processor: Wav2Vec2Processor
    padding: Union[bool, str] = True
    truncated: Union[bool, str] = True
    max_length: Optional[int] = None
    max_length_labels: Optional[int] = None
    pad_to_multiple_of: Optional[int] = None
    pad_to_multiple_of_labels: Optional[int] = None

    def __call__(self, features: List[Dict[str, Union[List[int], torch.Tensor]]]) -> Dict[str, torch.Tensor]:
        # split inputs and labels since they have to be of different lenghts and need
        # different padding methods
        input_features = [{"input_values": feature["input_values"]} for feature in features]
        label_features = [{"input_ids": feature["labels"]} for feature in features]

        batch = self.processor.pad(
            input_features,
            padding=self.padding,
            max_length=self.max_length,
            pad_to_multiple_of=self.pad_to_multiple_of,
            return_tensors="pt",
        )
        with self.processor.as_target_processor():
            labels_batch = self.processor.pad(
                label_features,
                padding=self.padding,
                max_length=self.max_length_labels,
                pad_to_multiple_of=self.pad_to_multiple_of_labels,
                return_tensors="pt",
            )

        # replace padding with -100 to ignore loss correctly
        labels = labels_batch["input_ids"].masked_fill(labels_batch.attention_mask.ne(1), -100)

        batch["labels"] = labels

        return batch

data_collator = DataCollatorCTCWithPadding(processor=processor, padding=True)

from jiwer import wer

def compute_metrics(pred):
    pred_logits = pred.predictions
    pred_ids = np.argmax(pred_logits, axis=-1)

    pred.label_ids[pred.label_ids == -100] = processor.tokenizer.pad_token_id

    pred_str = processor.batch_decode(pred_ids)
    # we do not want to group tokens when computing the metrics
    label_str = processor.batch_decode(pred.label_ids, group_tokens=False)

    wer = wer(predictions=pred_str, references=label_str)

    return {"wer": wer}

from transformers import Wav2Vec2ForCTC

model = Wav2Vec2ForCTC.from_pretrained(
    "facebook/wav2vec2-base", 
    attention_dropout=0.1,
    hidden_dropout=0.1,
    feat_proj_dropout=0.0,
    mask_time_prob=0.05,
    layerdrop=0.1,
    gradient_checkpointing=True, 
    ctc_loss_reduction="mean", 
    pad_token_id=processor.tokenizer.pad_token_id,
    vocab_size=len(processor.tokenizer)
)

model.freeze_feature_extractor()

from transformers import TrainingArguments

training_args = TrainingArguments(
  output_dir="/home/kamil/wav2vec_commonVoice/Checkpoints",
  # output_dir="./wav2vec2-large-xlsr-turkish-demo",
  group_by_length=True,
  per_device_train_batch_size=16,
  gradient_accumulation_steps=2,
  evaluation_strategy="steps",
  num_train_epochs=1000,
  fp16=True,
  save_steps=400,
  eval_steps=400,
  logging_steps=400,
  learning_rate=0.00005,
  warmup_steps=500,
  save_total_limit=2
)

from transformers import Trainer

trainer = Trainer(
    model=model,
    data_collator=data_collator,
    args=training_args,
    compute_metrics=compute_metrics,
    train_dataset=train_final,
    eval_dataset=test_final,
    tokenizer=processor.feature_extractor,
    
)

trainer.train()

```

The tasks I am working on is:
* Finetuning the model my own task or dataset: (give details below)

## To reproduce

Since Im fintuning on my own data, I can't use load_dataset method from datasets. So I changed some .map functions (as shown above) so I can have the same data structure to finetune the model. 

I had the following error: `ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.` occured chen the train part is launched. Any ideas ?

Thanks    


Comments:
@Kamilbentounes Were you able to resolve this error? If so, how did you do it? I'm also having the same error.

I am having the same error again in the following notebook. It occurs on cell # 3
https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/chapter7/section5_pt.ipynb

You could load your local 'csv' files with 🤗 [datasets load_dataset](https://huggingface.co/docs/datasets/loading.html#local-and-remote-files). I hope this helps.

Hi! I am facing the same issue, can you tell us how do you fix it?