ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length
Author: velu1122
Pull request: no
Labels: name: bug

name: Vision

name: Processing

### System Info

**Description**
I am experiencing an issue when using the transformers library version 4.36.1 with a custom model serving endpoint that utilizes mlflow. The model is based on the ResNetForImageClassification class, and I am using the AutoImageProcessor for image preprocessing.

**Error Details**
During the inference process, I encountered the following error message:

ValueError: **Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length**
This error suggests that the input images may not be processed consistently, leading to issues with tensor creation.

Environment Details
mlflow: 2.16.2
transformers: 4.36.1
torch: 2.0.1
pillow: 10.4.0

The below is my custom model and served it is as an Serving endpoint in Databricks. While accessing the endpoint using the invocation URL, getting the issue. I have tried it mentioning "self.processor(image, return_tensors="pt", padding=True, truncation=True)", even after I get the error.


```
from mlflow.pyfunc.model import PythonModelContext
import torch
import pandas as pd
from transformers import AutoImageProcessor, ResNetForImageClassification
from PIL import Image
import logging
import os

class MicrosoftResnet50Model(mlflow.pyfunc.PythonModel):

    def load_context(self, context: PythonModelContext):
        model_path = context.artifacts["model_path"]
        self.data_path = context.artifacts["data_path"]
        self.device = 'cuda' if torch.cuda.is_available() else 'cpu'

        # Load processor and model from the specified path
        self.processor = AutoImageProcessor.from_pretrained(model_path)
        self.model = ResNetForImageClassification.from_pretrained(model_path)
        self.model.eval()
        self.model.to(self.device)

    def predict(self, context: PythonModelContext, model_input: pd.DataFrame, params=None):
        predictions = []

        for index, row in model_input.iterrows():
            image_name = row['filepath'].split('/')[-1]
            full_image_path = os.path.join(self.data_path, image_name)

            # Load and preprocess the image
            image = Image.open(full_image_path).convert("RGB")
            inputs = self.processor(image, return_tensors="pt", padding=True, truncation=True)

            # Perform inference
            with torch.no_grad():
                outputs = self.model(**inputs)
                logits = outputs.logits
                predicted_class_idx = logits.argmax(-1).item()
                predicted_class_name = self.model.config.id2label[predicted_class_idx]
                predictions.append(predicted_class_name)

        return {'predictions': predictions}
```


I could see multiple posts regarding LLM/text classification model issues related to this error, but it is related to image models.
Could you please help in resolving it.
Thanks.


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. create a custom class MicrosoftResnet50Model as metioned .
2. Logged the custom model in databricks unity catalog as show below using mlflow.pyfunc.log_model.

```
import transformers
import torch
import PIL

with mlflow.start_run():
    model_info = mlflow.pyfunc.log_model(
        artifact_path=REGISTERED_MODEL_NAME,
        python_model=MicrosoftResnet50Model(),
        input_example=input_example,
        artifacts={
            "model_path": MODEL_PATH,  
            "data_path": "/Volumes/<catalog>/<schema>/models--microsoft--resnet-50/data/" 
        },
        pip_requirements=[
            f"transformers=={transformers.__version__}",
            "torch==2.0.1",
            f"pillow=={PIL.__version__}"
        ],
        signature=signature,
        registered_model_name=f"{CATALOG}.{SCHEMA}.{REGISTERED_MODEL_NAME}"
    )
```

3. Created the Databricks endpoint with the logged MLFLOW custom model.
4. Access the custom model endpoint using,

```
import os
import json
import requests
import pandas as pd

def score_model(image_path):
    """Send the image to the model and get the prediction."""
    # Replace with your actual serving endpoint URL
    url = 'https://<_url_>/serving-endpoints/test_resnet_50_cpu/invocations'
    
    # Define the headers for the request
    headers = {
        'Authorization': f'Bearer {os.environ.get("DATABRICKS_TOKEN")}',  # Ensure you have set this environment variable
        'Content-Type': 'application/json'
    }

    # # Create the JSON payload using the dataframe_split structure
    input_payload = {
        "dataframe_split": {
            "columns": ["filepath", "filepath_type"],  # Define columns to match the model input schema
            "data": [
                [image_path, "UC_VOLUME"]  # Insert the image path and file type into the data section
            ]
        }
    }
    # Convert the payload to JSON and send the request
    response = requests.post(url, headers=headers, json=input_payload)

    # Check if the response is successful
    if response.status_code == 200:
        predictions = response.json()
        print("Predictions:", predictions)
        return predictions
    else:
        print("Error:", response.status_code, response.text)
        return None

# Example usage
score_model(f"{MODEL_PATH}/data/sample.jpg")
```

5. {'predictions': {'predictions': ["Error! Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length."]}}

### Expected behavior

Shouldn't get the error "Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length." if we set to "self.processor(image, return_tensors="pt", padding=True, truncation=True)"

Or any other way, we can resolve the issue "Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length".


Comments:
cc @qubvel 

Hi @velu1122! Thanks for opening the issue! 

Can you please try to create a minimum reproducing example without external dependencies such as databricsk and mlflow?

Arguments `padding=True, truncation=True` look unrelated to the resnet50 model, they are usually passed to format text inputs, so you can probably not pass them to the processor.

I implore huggingface to please consider improved error catching - with a generic try catch, you get the raised ValueError exception, whereas in my case it was actually a RuntimeError that caused the exception (numpy not found with the torch.from_numpy(values) in feature_extraction_utils when doing the as_tensor(value) in the try catch in convert_to_tensors method.  With mine the issue was with a numpy version mismatch (had numpy>2.0) - transformers only works with numpy<2.0.  After a pip install "numpy<2.0" the issue was fixed - had nothing to do with the array of values I was passing to the processor (I actually attempted a few models, among them the resnet ones and with some of them I could even pass the argument padding=True to the tokenizer/processor)

Hi @pwjvr! Thank you for sharing feedback on the error-catching issue, it’s definitely an area worth improving. Would you be interested in contributing a PR to address this?

Hi @qubvel - happy too, will share PR for it here

Hi @qubvel - I do not have permission to contribute to huggingface/transformers - can you please add me?

Hi @pwjvr, you can make a fork, push to the branch in the fork, and then open a PR 

Thanks for the guidance @qubvel - forked and PR created: https://github.com/huggingface/transformers/pull/34655
Lemme know if you have any suggestions for improvements - keen to learn and contribute.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.