Repository: langchain
Issue: ChatPromtpTemplate doesn't accept PDF data as bytes
Labels: 🤖:bug

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in LangChain rather than my code.
- [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

### Example Code

```
pdf_data = base64.b64encode(httpx.get("https://dagrs.berkeley.edu/sites/default/files/2020-01/sample.pdf").content).decode("utf-8")

prompt = ChatPromptTemplate([
    ("system", "You are a helpful assistant "),
    ("human", [
        {"type": "media", "mime_type": "application/pdf", "data": pdf_data},
        {"type": "text", "text": "{user_input}"}
    ])
])
```

### Error Message and Stack Trace (if applicable)

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], [line 12](vscode-notebook-cell:?execution_count=4&line=12)
      [1](vscode-notebook-cell:?execution_count=4&line=1) model = ChatGoogleGenerativeAI(
      [2](vscode-notebook-cell:?execution_count=4&line=2)     model="gemini-1.5-flash",
      [3](vscode-notebook-cell:?execution_count=4&line=3)     temperature=0.1,
   (...)
      [6](vscode-notebook-cell:?execution_count=4&line=6)     max_retries=2
      [7](vscode-notebook-cell:?execution_count=4&line=7) )
      [9](vscode-notebook-cell:?execution_count=4&line=9) pdf_data = base64.b64encode(
     [10](vscode-notebook-cell:?execution_count=4&line=10)     httpx.get("http://www.dagrs.berkley.edu/sites/default/files/2020-01/sample.pdf").content).decode("utf-8")
---> [12](vscode-notebook-cell:?execution_count=4&line=12) prompt = ChatPromptTemplate([
     [13](vscode-notebook-cell:?execution_count=4&line=13)     ("system", "You are a helpful assistant "),
     [14](vscode-notebook-cell:?execution_count=4&line=14)     ("human", [
     [15](vscode-notebook-cell:?execution_count=4&line=15)         {"type": "media", "mime_type": "application/pdf", "data": pdf_data},
     [16](vscode-notebook-cell:?execution_count=4&line=16)         {"type": "text", "text": "{user_input}"}
     [17](vscode-notebook-cell:?execution_count=4&line=17)     ])
     [18](vscode-notebook-cell:?execution_count=4&line=18) ])

File c:\Users\alan-\miniconda3\Lib\site-packages\langchain_core\prompts\chat.py:992, in ChatPromptTemplate.__init__(self, messages, template_format, **kwargs)
    [938](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:938) def __init__(
    [939](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:939)     self,
    [940](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:940)     messages: Sequence[MessageLikeRepresentation],
   (...)
    [943](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:943)     **kwargs: Any,
    [944](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:944) ) -> None:
    [945](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:945)     """Create a chat prompt template from a variety of message formats.
    [946](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:946) 
    [947](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:947)     Args:
   (...)
    [990](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:990) 
    [991](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:991)     """
--> [992](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:992)     _messages = [
    [993](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:993)         _convert_to_message(message, template_format) for message in messages
    [994](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:994)     ]
    [996](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:996)     # Automatically infer input variables from messages
    [997](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:997)     input_vars: set[str] = set()

File c:\Users\alan-\miniconda3\Lib\site-packages\langchain_core\prompts\chat.py:993, in <listcomp>(.0)
    [938](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:938) def __init__(
    [939](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:939)     self,
    [940](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:940)     messages: Sequence[MessageLikeRepresentation],
   (...)
    [943](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:943)     **kwargs: Any,
    [944](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:944) ) -> None:
    [945](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:945)     """Create a chat prompt template from a variety of message formats.
    [946](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:946) 
    [947](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:947)     Args:
   (...)
    [990](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:990) 
    [991](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:991)     """
    [992](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:992)     _messages = [
--> [993](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:993)         _convert_to_message(message, template_format) for message in messages
    [994](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:994)     ]
    [996](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:996)     # Automatically infer input variables from messages
    [997](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:997)     input_vars: set[str] = set()

File c:\Users\alan-\miniconda3\Lib\site-packages\langchain_core\prompts\chat.py:1454, in _convert_to_message(message, template_format)
   [1452](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1452) message_type_str, template = message
   [1453](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1453) if isinstance(message_type_str, str):
-> [1454](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1454)     _message = _create_template_from_message_type(
   [1455](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1455)         message_type_str, template, template_format=template_format
   [1456](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1456)     )
   [1457](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1457) else:
   [1458](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1458)     _message = message_type_str(
   [1459](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1459)         prompt=PromptTemplate.from_template(
   [1460](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1460)             cast(str, template), template_format=template_format
   [1461](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1461)         )
   [1462](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1462)     )

File c:\Users\alan-\miniconda3\Lib\site-packages\langchain_core\prompts\chat.py:1365, in _create_template_from_message_type(message_type, template, template_format)
   [1351](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1351) """Create a message prompt template from a message type and template string.
   [1352](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1352) 
   [1353](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1353) Args:
   (...)
   [1362](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1362)     ValueError: If unexpected message type.
   [1363](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1363) """
   [1364](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1364) if message_type in ("human", "user"):
-> [1365](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1365)     message: BaseMessagePromptTemplate = HumanMessagePromptTemplate.from_template(
   [1366](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1366)         template, template_format=template_format
   [1367](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1367)     )
   [1368](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1368) elif message_type in ("ai", "assistant"):
   [1369](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1369)     message = AIMessagePromptTemplate.from_template(
   [1370](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1370)         cast(str, template), template_format=template_format
   [1371](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:1371)     )

File c:\Users\alan-\miniconda3\Lib\site-packages\langchain_core\prompts\chat.py:565, in _StringImageMessagePromptTemplate.from_template(cls, template, template_format, partial_variables, **kwargs)
    [563](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:563)             prompt.append(img_template_obj)
    [564](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:564)         else:
--> [565](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:565)             raise ValueError(f"Invalid template: {tmpl}")
    [566](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:566)     return cls(prompt=prompt, **kwargs)
    [567](file:///C:/Users/alan-/miniconda3/Lib/site-packages/langchain_core/prompts/chat.py:567) else:

ValueError: Invalid template: {'type': 'media', 'mime_type': 'application/pdf', 'data':
```

### Description

I'm trying to send a PDF file as bytes to Gemini using the ChatPromptTemplate method.

Sending the PDF to Gemini seems to works fine as per [#4589](https://github.com/BerriAI/litellm/issues/4589) and [#215](https://github.com/langchain-ai/langchain-google/issues/215)

But when using the ChatPromptTemplate, it appears that the code [here](https://github.com/langchain-ai/langchain/blob/8dc4bec9477e25df5149e68124eb19c3ec2494d0/libs/core/langchain_core/prompts/chat.py#L532) is not ready to deal with PDFs type in the prompt template.

### System Info

langchain==0.3.2
langchain-community==0.3.1
langchain-core==0.3.9
langchain-google-genai==2.0.0
langchain-text-splitters==0.3.0
langgraph==0.2.34
langgraph-checkpoint==2.0.0
langsmith==0.1.131