ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.
Author: anum94
Pull request: no

Maybe  @SaulLu can help?

## Information

I am following the [text summarization](https://huggingface.co/course/chapter7/5) tutorial on hugging face website which uses the mt5-small model. It explains step by step on how to perform a text summarization task. 



## To reproduce

Steps to reproduce the behavior:

1. Run the [following notebook](https://colab.research.google.com/github/huggingface/notebooks/blob/master/course/chapter7/section5_pt.ipynb)
2. cell # 32 should reproduce the following error. (it did for me)

```
ValueError                                Traceback (most recent call last)
File ~/PycharmProjects/nlp-env/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:707, in BatchEncoding.convert_to_tensors(self, tensor_type, prepend_batch_axis)
    706 if not is_tensor(value):
--> 707     tensor = as_tensor(value)
    709     # Removing this for now in favor of controlling the shape with `prepend_batch_axis`
    710     # # at-least2d
    711     # if tensor.ndim > 2:
    712     #     tensor = tensor.squeeze(0)
    713     # elif tensor.ndim < 2:
    714     #     tensor = tensor[None, :]

ValueError: too many dimensions 'str'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Input In [72], in <module>
----> 1 data_collator(features)

File ~/PycharmProjects/nlp-env/lib/python3.9/site-packages/transformers/data/data_collator.py:586, in DataCollatorForSeq2Seq.__call__(self, features, return_tensors)
    583         else:
    584             feature["labels"] = np.concatenate([remainder, feature["labels"]]).astype(np.int64)
--> 586 features = self.tokenizer.pad(
    587     features,
    588     padding=self.padding,
    589     max_length=self.max_length,
    590     pad_to_multiple_of=self.pad_to_multiple_of,
    591     return_tensors=return_tensors,
    592 )
    594 # prepare decoder_input_ids
    595 if (
    596     labels is not None
    597     and self.model is not None
    598     and hasattr(self.model, "prepare_decoder_input_ids_from_labels")
    599 ):

File ~/PycharmProjects/nlp-env/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:2842, in PreTrainedTokenizerBase.pad(self, encoded_inputs, padding, max_length, pad_to_multiple_of, return_attention_mask, return_tensors, verbose)
   2839             batch_outputs[key] = []
   2840         batch_outputs[key].append(value)
-> 2842 return BatchEncoding(batch_outputs, tensor_type=return_tensors)

File ~/PycharmProjects/nlp-env/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:212, in BatchEncoding.__init__(self, data, encoding, tensor_type, prepend_batch_axis, n_sequences)
    208     n_sequences = encoding[0].n_sequences
    210 self._n_sequences = n_sequences
--> 212 self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis)

File ~/PycharmProjects/nlp-env/lib/python3.9/site-packages/transformers/tokenization_utils_base.py:723, in BatchEncoding.convert_to_tensors(self, tensor_type, prepend_batch_axis)
    718         if key == "overflowing_tokens":
    719             raise ValueError(
    720                 "Unable to create tensor returning overflowing tokens of different lengths. "
    721                 "Please see if a fast version of this tokenizer is available to have this feature available."
    722             )
--> 723         raise ValueError(
    724             "Unable to create tensor, you should probably activate truncation and/or padding "
    725             "with 'padding=True' 'truncation=True' to have batched tensors with the same length."
    726         )
    728 return self

ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length.

```




Comments:
Ah actually this is linked to the summarization example in the course @lewtun 

Hi @anum94, thanks for reporting this bug! The cause of the error is that the `tokenized_datasets` object has columns with strings, and the data collator doesn't know how to pad these.

The fix is to add the following line before the data collator:

```python
tokenized_datasets = tokenized_datasets.remove_columns(books_dataset["train"].column_names)
``` 

I'll post a fix in the website and Colab too - thanks!

Thank you.
I think I have an older version of transformers so it worked for me when I used 

`tokenized_datasets = tokenized_datasets.remove_columns_(books_dataset["train"].column_names)`

instead of 
`tokenized_datasets = tokenized_datasets.remove_columns(books_dataset["train"].column_names)`

Thanks for your help and the prompt response. 
cheers

When I used `tokenized_datasets = tokenized_datasets.remove_columns(books_dataset["train"].column_names)` it gives `ZeroDivisionError: integer division or modulo by zero` because it can't access rows.

I'm following [this tutorial ](https://huggingface.co/blog/fine-tune-xlsr-wav2vec2)and facing this same error of _Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length._ during model training. I'm not sure how to implement the solution which worked in the case above. Any help will be appreciated. 

I am having the same issue^

Upgrading numpy to 1.24 resolved this issue:
```
pip install numpy --upgrade
```

For my particular case, setting `packing = True` in my SFTTrainer initianlization resolved the issue. Maybe it also works for other Trainer instances as well.

```
trainer = SFTTrainer(
    model=WrappedModel(model),
    tokenizer=tokenizer,
    max_seq_length=max_tokens,
    train_dataset=dataset_train,
    eval_dataset=None,
    peft_config=None,
    packing=True,
    args=training_args,
    dataset_text_field="text",
)
```

inputs = processor(prompts, images=[image1, image2], padding=True, return_tensors="pt").to("cuda")

for above loc i am getting same error 

raise ValueError(
ValueError: Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.

what could be the possible solution

Hi, I have same issues, how can we fix it?
import gradio as gr
from transformers import BlipProcessor, BlipForConditionalGeneration
from PIL import Image
processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-base")
model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-base")
def generate_caption(image):
    # Now directly using the PIL Image object
    inputs = processor(images=image, return_tensors="pt")
    outputs = model.generate(**inputs)
    caption = processor.decode(outputs[0], skip_special_tokens=True)
    return caption
def caption_image(image):
    """
    Takes a PIL Image input and returns a caption.
    """
    try:
        caption = generate_caption(image)
        return caption
    except Exception as e:
        return f"An error occurred: {str(e)}"
iface = gr.Interface(
    fn=caption_image,
    inputs=gr.Image(type="pil"),
    outputs="text",
    title="Image Captioning with BLIP",
    description="Upload an image to generate a caption."
)
iface.launch()
Error: An error occurred: Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.

Hey @Tkag0001 have you tried activating `padding=True` as suggested in your error message?

@LysandreJik Hi sir, where can I add it? I have enabled padding = True in processor.
inputs = processor(images=image, return_tensors="pt", padding = True)
But it's still error.

In this case do you mind opening a new issue with the entire code and full traceback? It seems like there's a single error, seeing the full traceback would help debug the issue. Thanks!

I had similar error with image-to-text models processor. This fixed it for me:
https://discuss.huggingface.co/t/unable-to-create-tensor-you-should-probably-activate-padding-with-padding-true-to-have-batched-tensors-with-the-same-length-paligemma/94413/2
@LysandreJik  Could there be a better more permanent fix?



Ran into this error - frustration was that I had actually set padding=True with my tokenizer. What gave me a clue was an error I got earlier when actually importing the transformers package, it was complaining about numpy version mismatches, but it wasn’t a breaking error, more of a wanring. Want to implore huggingface to improve their error catching - since this just raises a ValueError in the try catch in feature_extraction_utils, whereas the error that actually occurred was a RuntimeError when attempting to do the torch.from_numpy when converting the array of values to a tensor in the convert_to_tensors method. Long story short, check your numpy version - transformers only works with numpy<2.0 as at time of writing, so a quick
`pip install "numpy<2.0"`
fixed it for me

I am also getting similar issue here.

```
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 
'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features
(`output` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
  0% 0/20 [00:05<?, ?it/s]
```

here are my fine-tuning step  details.

Model load using unsloth not Huggingface Transformers directly
```
from unsloth import FastLanguageModel, is_bfloat16_supported

max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Llama-3.2-3B", # or choose "unsloth/Llama-3.2-1B"
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)
```

Dataset preparation 
```
def prepare_dataset(tokenizer_data: dict) -> dict:
    alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

    def formatting_prompts_func(examples):
        return {"text": [alpaca_prompt.format(inst, inp, out) + tokenizer_data['tokenizer'].eos_token
                         for inst, inp, out in zip(examples["instruction"], examples["input"], examples["output"])]}

    # Load the dataset
    dataset = load_dataset("yahma/alpaca-cleaned")

    # Apply formatting
    dataset = dataset.map(formatting_prompts_func, batched=True)

    # Split the dataset into train, validation, and test sets
    train_valid_test_split = dataset['train'].train_test_split(test_size=0.1, seed=42)
    train_valid_dataset = train_valid_test_split['train']
    test_dataset = train_valid_test_split['test']

    train_valid_split = train_valid_dataset.train_test_split(test_size=0.1, seed=42)
    train_dataset = train_valid_split['train']
    val_dataset = train_valid_split['test']

    return {
        'train_dataset': train_dataset,
        'val_dataset': val_dataset,
        'test_dataset': test_dataset
    }
```

Here passing train_dataset and eval_dataset for eval_loss metrics.
```
    training_args = TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=5,
        # num_train_epochs = 1, # Set this for 1 full training run.
        max_steps = 20,
        learning_rate=2e-4,
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        lr_scheduler_type="linear",
        seed=3407,
        output_dir="outputs",
        report_to="mlflow",
        evaluation_strategy=IntervalStrategy.STEPS,
        eval_steps=20,
        save_total_limit=5,
        load_best_model_at_end=True,
        metric_for_best_model="eval_loss",
        greater_is_better=False,
        remove_unused_columns=False
    )

    trainer = SFTTrainer(
        model=model,
        tokenizer=tokenizer,
        train_dataset=train_dataset,
        eval_dataset=val_dataset,
        dataset_text_field="text",
        max_seq_length=2048,
        dataset_num_proc=2,
        packing=False,
        callbacks=[EarlyStoppingCallback(early_stopping_patience=3)],
        args=training_args
    )
```

> For my particular case, setting `packing = True` in my SFTTrainer initianlization resolved the issue. Maybe it also works for other Trainer instances as well.
> 
> ```
> trainer = SFTTrainer(
>     model=WrappedModel(model),
>     tokenizer=tokenizer,
>     max_seq_length=max_tokens,
>     train_dataset=dataset_train,
>     eval_dataset=None,
>     peft_config=None,
>     packing=True,
>     args=training_args,
>     dataset_text_field="text",
> )
> ```

Yeah, It worked for me as well. Thanks

In my case with the SFTTrainer, problem actually stem from the structure of the dataset. a "label" column when the task was generation and not classification. Just changing columns from ["input", "label", "text"] to ["input", "output", "instruction", "text"] did the trick. Nothing else worked (packing is currently disabled by unsloth "because it is unstable")