`chat_template` of tokenizers and in `apply_chat_template` behave differently
Author: Kamichanw
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.40.2
- Platform: Linux-4.15.0-76-generic-x86_64-with-glibc2.27
- Python version: 3.10.14
- Huggingface_hub version: 0.23.0
- Safetensors version: 0.4.3
- Accelerate version: 0.30.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@zucchini-nlp   @ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
processor = Idefics2Processor.from_pretrained(
    model_root,
    torch_dtype=torch.bfloat16,
    do_image_splitting=False,
)
processor.tokenizer.chat_template = self.chat_template

messages = [
    {
        "role": "example",
        "query": [
            {"type": "image"},
            {"type": "text", "text": "What is this photo taken looking through?"},
        ],
        "answer": [{"type": "text", "text": "net"}],
    },
    {
        "role": "example",
        "query": [
            {"type": "image"},
            {"type": "text", "text": "What position is this man playing?"},
        ],
        "answer": [{"type": "text", "text": "pitcher"}],
    },
    {
        "role": "question",
        "query": [
            {"type": "image"},
            {"type": "text", "text": "What color is the players shirt?"},
        ],
    },
]
# fmt: off
processor.tokenizer.chat_template = (
            "{% if messages[0]['role'] == 'instruction' %}"
                "Instruction: {{- messages[0]['content'] }}\n"
                "{% set messages = messages[1:] %}"
            "{% endif %}"
            "{% for message in messages %}"
                "Question:" 
                "{% for line in message['query'] %}"
                    "{% if line['type'] == 'text' %}"
                        "{{- line['text'] }}"
                    "{% elif line['type'] == 'image' %}"
                        "{{ '<image>' }}"
                    "{% endif %}"
                "{% endfor %}"
                "<end_of_utterance>\n"
                "{% if 'answer' in message %}"
                    "Short answer: " 
                    "{% for line in message['answer'] %}"
                        "{% if line['type'] == 'text' %}"
                            "{{- line['text'] }}"
                        "{% elif line['type'] == 'image' %}"
                            "{{ '<image>' }}"
                        "{% endif %}"
                    "{% endfor %}"
                    "<end_of_utterance>\n"
                "{% endif %}"
                "\n"
            "{% endfor %}"
            "{% if add_generation_prompt %}"
                "Short answer: " 
            "{% endif %}"
        )
# fmt: on
prompt = model.processor.apply_chat_template(
    messages, add_generation_prompt=True
)
print(prompt)
```
With code above, it yields error:
```
UndefinedError: 'dict object' has no attribute 'content'
```
But everything works fine when I add chat template to argument of `apply_chat_template` with following code snippet:
```
prompt = model.processor.apply_chat_template(
    messages, chat_template=chat_template, add_generation_prompt=True
)
```

### Expected behavior

It should output:
```
Question:<image>What is this photo taken looking through?<end_of_utterance>
Short answer:net<end_of_utterance>
Question:<image>What position is this man playing?<end_of_utterance>
Short answer:pitcher<end_of_utterance>
Question:<image>What color is the players shirt?<end_of_utterance>
Short answer:
```

Comments:
Hey! Feel free to share what solved it for you! 🤗