`transformers serve` Fails to Handle Messages with Nested Content
Author: jakeret
Pull request: no
Labels: name: bug

### System Info

## Bug Description

When using the `transformers serve` command to serve a model, it fails to process chat messages containing nested content, raising a `TypeError`. Specifically, when integrating the served model endpoint with a chat interface such as Gradio, and the messages sent have a nested structure (e.g., lists instead of strings or dictionaries as expected), the server encounters an unhandled exception. This results in the inability to properly process the incoming request.

The issue stems from an assumption in the code that the `content` field of each message is either a string or a dictionary. However, if `content` is a list (as might occur with certain chat interfaces or user-provided inputs), the server attempts to access it using a string key (`message["content"]["text"]`), which leads to the following error:
````
TypeError: list indices must be integers or slices, not str
````

## Environment Information
- transformers version: 4.55.0.dev0
- Platform: macOS-15.5-x86_64-i386-64bit-Mach-O
- Python version: 3.13.5
- Gradio version: 5.38.2

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

## Steps to Reproduce

1. Serve a model using `transformers serve`. 

The server will be available at http://localhost:8000/v1/.

2. Use Gradio to launch a chat interface connected to the endpoint:
```python
import gradio as gr
gr.load_chat('http://localhost:8000/v1/', model='Qwen/Qwen2.5-0.5B-Instruct', token='').launch()
````

3. Send a message where the content field is nested (e.g., a list). For example:
```json
{"role": "user", "content": [{"text": "Can you help me write tests?", "type": "text"}]}
```

4. Observe the `TypeError` raised by the server.


### Expected behavior

## Expected Behaviour
The server should either:

1. Handle cases where content is a list, or
2. Gracefully return a meaningful error message to the client indicating that the input structure is invalid, rather than raising an internal exception.

## Actual Behaviour
The server crashes with the following traceback:
```python
File "/path/to/transformers/commands/serving.p", line 828, in get_processor_inputs_from_inbound_messages
    content = message["content"] if isinstance(message["content"], str) else message["content"]["text"]
                                                                             ~~~~~~~~~~~~~~~~~~^^^^^^^^
TypeError: list indices must be integers or slices, not str
```



Comments:
### Temporary workaround

Until the bug is fixed in the transformers repo, you can intercept or preprocess messages in your Gradio code before sending them:

```python
def sanitize_messages(messages):
    for msg in messages:
        if isinstance(msg.get("content"), list):
            msg["content"] = " ".join([m.get("text", "") for m in msg["content"] if isinstance(m, dict)])
    return messages
```
And use this in a wrapper around your Gradio app.



cc @zucchini-nlp since I think this will mostly apply to VLMs where `content` contains strings/images, but do you know who works on `transformers serve`?

@LysandreJik added VLMs support and cc @gante for general `serving` issues

@jakeret thank you for opening the issue 🤗 

#39829 fixes it!

@gante sure, happy to help. 

Seems like the PR complements the one I had opened (https://github.com/huggingface/transformers/pull/39792)