Unable to use plain text for chat or eval of multimodal model (ValueError: Invalid input type. Must be a single image, a list of images, or a list of batches of images.)

### Reminder

- [x] I have read the above rules and searched the existing issues.

### System Info

```
- `llamafactory` version: 0.9.2.dev0
- Platform: Linux-5.15.0-92-generic-x86_64-with-glibc2.35
- Python version: 3.12.8
- PyTorch version: 2.5.1+cu124 (GPU)
- Transformers version: 4.49.0
- Datasets version: 3.1.0
- Accelerate version: 1.0.1
- PEFT version: 0.12.0
- TRL version: 0.9.6
- GPU type: NVIDIA RTX 6000 Ada Generation
- GPU number: 1
- GPU memory: 44.53GB
- vLLM version: 0.7.3
```

### Reproduction

```text
[INFO|2025-02-22 21:04:43] llamafactory.model.model_utils.attention:157 >> Using torch SDPA for faster training and inference.
[INFO|2025-02-22 21:04:43] llamafactory.model.loader:157 >> all params: 10,670,220,835
Traceback (most recent call last):
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/queueing.py", line 536, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/route_utils.py", line 322, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/blocks.py", line 1935, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/blocks.py", line 1532, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/utils.py", line 671, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/utils.py", line 664, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 962, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/utils.py", line 647, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/gradio/utils.py", line 809, in gen_wrapper
    response = next(iterator)
               ^^^^^^^^^^^^^^
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/webui/chatter.py", line 188, in stream
    for new_text in self.stream_chat(
                    ^^^^^^^^^^^^^^^^^
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 112, in stream_chat
    yield task.result()
          ^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/chat/chat_model.py", line 129, in astream_chat
    async for new_token in self.engine.stream_chat(
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 390, in stream_chat
    stream = self._stream_chat(*input_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 275, in _stream_chat
    gen_kwargs, _ = HuggingfaceEngine._process_args(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/chat/hf_engine.py", line 179, in _process_args
    mm_inputs = template.mm_plugin.get_mm_inputs(**mm_input_dict, batch_ids=[prompt_ids], processor=processor)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/data/mm_plugin.py", line 815, in get_mm_inputs
    mm_inputs = self._get_mm_inputs(images, videos, audios, processor, imglens)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/LLaMA-Factory/src/llamafactory/data/mm_plugin.py", line 800, in _get_mm_inputs
    return image_processor(batch_images, return_tensors="pt")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/transformers/image_processing_utils.py", line 42, in __call__
    return self.preprocess(images, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/transformers/models/mllama/image_processing_mllama.py", line 693, in preprocess
    images_list = make_nested_list_of_images(images)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/23132294r/.conda/envs/llm/lib/python3.12/site-packages/transformers/image_utils.py", line 310, in make_nested_list_of_images
    raise ValueError("Invalid input type. Must be a single image, a list of images, or a list of batches of images.")
ValueError: Invalid input type. Must be a single image, a list of images, or a list of batches of images.
```


### Others

I want to fine-tune Llama3.2-11B-vision-instruct on my customized text dataset in 'identify.json' format. It encounters an error `ValueError: Invalid input type. Must be a single image, a list of images, or a list of batches of images.` when I send pure text without any image in chatbox. Although I loaded untrained raw Llama model for chat using 'mllama' template, this error still appears. Text-only chats only work with the 'llama3' template.

I find some similar issues #6635 #5918 #5958, but they don't work for me. Many thanks for your time and attention!

Labels: solved
State: closed