InternVL3-8B quantize got TypeError: can only concatenate str (not "list") to str
Author: BigFaceBoy
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.56.2
- Platform: Linux-5.4.0-144-generic-x86_64-with-glibc2.31
- Python version: 3.12.4
- Huggingface_hub version: 0.35.3
- Safetensors version: 0.6.2
- Accelerate version: 1.10.1
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA L40

### Who can help?

@zucchini-nlp 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

I try to quantize InterVL3-8B with llmcompressor.
The quantize code is:
```
# W4A16 
import base64
from io import BytesIO
import torch
from datasets import load_dataset
from transformers import AutoProcessor, AutoModel, AutoTokenizer
from qwen_vl_utils import process_vision_info
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import GPTQModifier
from llmcompressor.utils import dispatch_for_generation

# Load model.
model_id = "/root/ezviz/models/InternVL3-8B"
model = AutoModel.from_pretrained(model_id, torch_dtype=torch.bfloat16, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

DATASET_ID = "/root/ezviz/datasets/flickr30k"
DATASET_SPLIT = "test[:512]"
NUM_CALIBRATION_SAMPLES = 512
MAX_SEQUENCE_LENGTH = 2048
ds = load_dataset(DATASET_ID, split=DATASET_SPLIT)
ds = ds.shuffle(seed=42)

def preprocess_and_tokenize(example):
    # preprocess
    buffered = BytesIO()
    example["image"].save(buffered, format="PNG")
    encoded_image = base64.b64encode(buffered.getvalue())
    encoded_image_text = encoded_image.decode("utf-8")
    base64_qwen = f"data:image;base64,{encoded_image_text}"
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": base64_qwen},
                {"type": "text", "text":  "What does the image show?"},
            ],
        }
    ]
    text = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    image_inputs, video_inputs = process_vision_info(messages)

    # tokenize
    return processor(
        text=[text],
        images=image_inputs,
        videos=video_inputs,
        padding=False
    )
ds = ds.map(preprocess_and_tokenize, remove_columns=ds.column_names)


def data_collator(batch):
    assert len(batch) == 1
    return {key: torch.tensor(value) for key, value in batch[0].items()}


# Recipe
recipe = [
    GPTQModifier(
        targets="Linear",
        scheme="W4A16",
        ignore=["lm_head"]
    ),
]

# Perform oneshot
oneshot(
    model=model,
    tokenizer=model_id,
    dataset=ds,
    recipe=recipe,
    max_seq_length=MAX_SEQUENCE_LENGTH,
    num_calibration_samples=NUM_CALIBRATION_SAMPLES,
    trust_remote_code_model=True,
    data_collator=data_collator,
    sequential_targets=["InternLM2ForCausalLM"],
)


SAVE_DIR = "/root/ezviz/models/InternVL3-8B-W4A16"
model.save_pretrained(SAVE_DIR, save_compressed=True)
processor.save_pretrained(SAVE_DIR)
```
I run the code and got the error:
```
Traceback (most recent call last):
  File "/root/ezviz/src/img_calib/w4a16.py", line 54, in <module>
    ds = ds.map(preprocess_and_tokenize, remove_columns=ds.column_names)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 562, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 3327, in map
    for rank, done, content in Dataset._map_single(**unprocessed_kwargs):
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 3659, in _map_single
    for i, example in iter_outputs(shard_iterable):
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 3633, in iter_outputs
    yield i, apply_function(example, i, offset=offset)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/datasets/arrow_dataset.py", line 3556, in apply_function
    processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/ezviz/src/img_calib/w4a16.py", line 42, in preprocess_and_tokenize
    text = processor.apply_chat_template(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 1640, in apply_chat_template
    rendered_chat, generation_indices = render_jinja_template(
                                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/transformers/utils/chat_template_utils.py", line 521, in render_jinja_template
    rendered_chat = compiled_template.render(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/jinja2/environment.py", line 1295, in render
    self.environment.handle_exception()
  File "/root/miniforge3/envs/quantize/lib/python3.12/site-packages/jinja2/environment.py", line 942, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 23, in top-level template code
TypeError: can only concatenate str (not "list") to str
```


### Expected behavior

Expect quantize success

Comments:
I believe you are using the official checkpoint before conversion. The HF-compatible checkpoints contain an `-HF` suffix and should be loaded with `trust_remote_code=False`

If you want to use un-converted weights for a specific reason, then I suggest to open discussion on the hub since the remote code is maintained by the authors

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.